Using Fody to provide common parts for structs

The RampUp library is meant to provide low-latency, low/no-alloc environment for building fast systems in .NET. As it’s based on messaging in an actor-like/SEDA fashion, the messages are the first class citizen in its environment. Because of these requirements, unlike in other frameworks/systems, they’ve been built on structs. Yes, good old fashioned value types that has no virtual method tables, no object overhead. They’re just pure data. But even in the world of pure data sometimes you need a common denominator, which provides some basic information. Let me share my RampUp approach to this problem.

Envelope

In case of RampUp and its messages, the part that should be attachable to every message is an envelope. You probably want to now the sender of the message and maybe a few more facts. We can’t derive as structure types cannot derive one from another. How can this be done, how to introduce at least one common field in all the messages? Having one field of type Envelope would be sufficient as we could use this field to store all the needed information.

Fody

There’s a tool created by Simon Cropp called Fody. It’s a AOP tool, a weaver, a post compiler. With this you can create ModuleWeavers, that are reusable (there’s a lot of them) and/or applied only in the solution they were created. Using this tool I’ve been able to deliver a weaver that scans for messages in a project and adds a specific envelope field. For each message a metadata is created describing the offset to the Envelope field. Additionally, on the basis of the metadata, a message reader and a writer are emitted so that the final user of RampUp does not need to access this field manually.

Summary

Using a post compiler is often seen as an overkill. On the other hand, being able to introduce a common denominator for a set of value types is impossible without either manual copy-paste techniques or weaving it in a post compilation process. I prefer the latter.

OpCodes.Ldtoken

You probably used typeof operator a few times. It’s quite funny, that an operator like this actually has no MSIL counterpart. Using it emits TWO OpCodes.

The first emitted opcode is OpCodes.Ldtoken with the type. It consumes the token of the type, pushing the RuntimeTypeHandle structure onto the stack as the result of its operation. The second emitted code is a call to the Type.GetTypeFromHandle(RuntimeTypeHandle) which consumes the structure pushed by the previous code, returning the runtime type. The interesting thing is that you can’t use just OpCodes.Ldtoken from C#. You need to load the runtime type first and then you can access the handle by a property. You can emit IL with just OpCodes.Ldtoken though to remove overhead of calling a method and use the structure as a key for a lookup. It will be a bit faster for sure.

You can see the example of emitting this in RampUp code of the message writer.

Replacing a generic dictionary

There is a moment, when you profile your high-throughput system and you hit the wall. And it’s not your code but some BCL elements. That’s what happened in RampUp when I was profiling Write part of the buffer.

The writer is emitted, but as the very foundation it uses a dictionary of metadata stored per message type. The metadata are simple:

  • the message size
  • the offset of the message envelope

Before optimization it was using the generic Dictionary specified with the message type and the message metadata. It was horribly slow. As the set of messages does not change, you could easily provide a set of message types up-front. For each message type one can obtain RuntimeTypeHandle, which can be easily converted to long. With a set of longs, you could select minimal long and just subtract it from all the values. This would reduce the value to int. So here you are. You have a way of turning a type into int and ints are much easier to compare & to handle. One could even use a simple hash-map just to map between ints and metadata. This was the way to reduce the overhead of obtaining metadata with IntLookup<TValue>. After applying the change, write performance has increased by 10%.

IS vs HAS relationship for your API

There is an urge of making things automagically. For instance, when you have a DDD Aggregate, one could consider automatic publishing all the commands as the service API. As The Aggregate is a part of a model & a language which you agreed to use, that seems to be a perfect match to be your API. Is it?

IS vs HAS

There is a rule of Composition over inheritance. It says that instead of deriving from different components, one should compose bigger parts from already existing by using them, but no deriving. A good example might be a user and an employee. As an employee, you are given a user in the system. You might try to model it with the derivation in mind. Is an user an employee as well or the other way around? There’s no good answer to this.

You could model it in another way. There an employee, that a user has access to. When a user logs in he/she can access data of an employee is attached to him/her. You can see where is it going. Keep things minimal, use other elements but do not introduce a relation of being something.

A bit abusive allegory

Now ask yourself a question. Is your API using the model or is it the model? In the majority of cases, the interfaces of your API & your model may be aligned, but they are not the same! Even if you publish operations named after a model that you established, you’d like your API to use the model just in case of remodeling the domain. It’s good to automate and do not write much code. On the other hand, it’s good to have proper abstractions separating concerns of two different worlds.

Producer – consumer relationship

In the last post about the RampUp library I covered on of the foundations: IRingBuffer. Now I’d like to describe the contract it fulfills.

Producer consumer

If you take a look at the IRingBuffer you’ll see Write/Read methods. These two are responsible for producing/consuming or writing/reading messages to the buffer in FIFO way. What are the guarantees behind such an interface? What about concurrent accessing this structure?

Multimulti

The easiest approach for distinguishing accessing patterns is considering whether the structure could be accessed by one or more threads. If you consider producer/consumer you’ll see that there are four options:

  1. SPSC – Single Producer Single Consumer – only one thread produces items, and another consumes them
  2. MPSC – Multi Producer Single Consumer – multiple threads may produce items in a safe manner, again there’s a single consuming thread
  3. SPMC – Single Producer Multi Consumer – this could be treated as a distributor of work
  4. MPMC – Multi Producer Multi Consumer – multi/multi, ConcurrentQueue is a good example of it

Unfortunately nothing comes for free. If you want to get multi, you’ll pay the price of handling friction on that end. On the other hand, if you want to design a system where queues provide transport between different parts of the system, you’ll need to enable multiple producers for sure, as there’s going to be more than one system element. If you want to process items in an order they appeared and leave the locking issues, just write a fast single threaded code, a single consumer with a single worker thread would be the way to go. Of course this worker thread may access other queues and produce items for them (hence, multi producer is needed).

The ring buffer implementation in RampUp provides exactly MPSC behavior, as it’s prepared to handle items in order, by a single thread.

REST subresources as commands

This post is written write after reading the @liveweird entry about How not to hurt yourself while building a RESTful API. If you don’t know it, it’s probably a good time to read it.

Sebastian mentions that POSTs can be used for any non-crud like operation. I agree with that as the POSTs have a notion of general commands, they are not idempontent and they actually cause a change in a resource. They’re much smarter than DELETEs (just trash it) or PUTs (just replace it with my content). How do you distinguish different operations though? Is a header a good place to put the name? Or maybe a property in a JSON payload?

Subresources to the rescue

The easiest way to model actual non-CRUD commands I’ve found so far is modelling the commands as subresources specialized in handling them. For example consider the following resource:

/car/1

One could argue how to model operations like rent, return, etc. My take on this would be POSTs against the following subresources:

/car/1/rent

/car/1/return

This modelling is easy to navigate (either you want to HATEOAS or simply Swagger your API) and easy to follow. It has an additional advantage as well. It’s very easy to bind specific commands to an MVC/WebAPI controller methods as every operation can be handle with a separate method. Even if you write a custom class for each of this operations (a command object), leveraging this, can spare you some time on writing your own routing/dispatching of the request.

Unsafe buffer in RampUp

As we’re moving towards the core of RampUp, before visiting the most important parts we need to discuss one abstraction provided in the library named IUnsafeBuffer. The abstraction provides an interface a’la Stream wrapping a set of operations over a stream of an unmanaged memory. Currently, there’s only one implementation using VirtualAlloc, but it’s highly probable that in the future memory mapped files will be used as it’s the easiest way of providing a cross-process visible memory. Now let’s take a look at the following members of IUnsafeBuffer


public interface IUnsafeBuffer : IDisposable
{
    int Size { get; }
    unsafe byte* RawBytes { get; }
    AtomicLong GetAtomicLong(long index);
    AtomicInt GetAtomicInt(long index);
    void Write(int offset, ByteChunk chunk);
    void ZeroMemory(int start, int length);
}

As you can see the provided methods are a bit similar to the operations provided by the Stream. Currently pointer is leaked with RawBytes, but this is a subject to change. What’s important, the unsafe buffer provides way to get atomic wrappers when needed. The atomics are designed to be used with any unmanaged memory provider, especially with IUnsafeBuffer. This API will be needed in creating the next data structure in RampUp