A pointer to a generic method argument

Let’s consider a following method signature of an interface taken from a RampUp interface.


bool Write<TMessage>(ref Envelope envelope, 
    ref TMessage message, IRingBuffer bufferToWrite) 
    where TMessage : struct;

It’s a fairly simple signature, enabling to pass a struct of any size using just a reference to it, without copying it. Now let’s consider the need of obtaining a pointer to this message. Taking a pointer could be needed for various reasons. One could be getting fields by offset, another could be using memcpy for copying the value to any given address. Is it possible to get this pointer in C# code?

No pointers for generic parameters

Unfortunately, you can’t do it in C#. If you try to obtain a pointer to a generic parameter, you’ll be informed about the compiler error. If you can’t do it in C#, is there any other .NET language one could use to get it? Yes, there is. It’s the foundation of .NET programs, the MSIL itself and if it’s MSIL, it means emitting code dynamically.

Ref looks like a pointer

What is a reference to a struct? It looks like a pointer to me. What if we could load it and just assume that it is a pointer? Would CLR accept this program? It occurs that it would. I won’t cover the whole implementation which can be found in here, but want to accent some points.

  • CLR uses the argument with index 0 to passing this. If you want to load a field you need to use the following sequence of operations:
    • Ldloc_0; // load this on the stack
    • Ldfld, “Field1” // pops this loading the value named “Field1” on the stack
  • For Write method, getting a pointer to a message is nothing more than calling an op code: Ldarg_2. As the struct is passed by reference, it can be treated as a pointer by CLR and it will.

I encourage you to download the RampUp codebase and play a little bit with an emitted implementation of the IMessageWriter. Maybe you’ll never need to take the pointer to a generic method parameter (I did), but it’s a good starter to learn a little about emitting code.

Using Fody to provide common parts for structs

The RampUp library is meant to provide low-latency, low/no-alloc environment for building fast systems in .NET. As it’s based on messaging in an actor-like/SEDA fashion, the messages are the first class citizen in its environment. Because of these requirements, unlike in other frameworks/systems, they’ve been built on structs. Yes, good old fashioned value types that has no virtual method tables, no object overhead. They’re just pure data. But even in the world of pure data sometimes you need a common denominator, which provides some basic information. Let me share my RampUp approach to this problem.

Envelope

In case of RampUp and its messages, the part that should be attachable to every message is an envelope. You probably want to now the sender of the message and maybe a few more facts. We can’t derive as structure types cannot derive one from another. How can this be done, how to introduce at least one common field in all the messages? Having one field of type Envelope would be sufficient as we could use this field to store all the needed information.

Fody

There’s a tool created by Simon Cropp called Fody. It’s a AOP tool, a weaver, a post compiler. With this you can create ModuleWeavers, that are reusable (there’s a lot of them) and/or applied only in the solution they were created. Using this tool I’ve been able to deliver a weaver that scans for messages in a project and adds a specific envelope field. For each message a metadata is created describing the offset to the Envelope field. Additionally, on the basis of the metadata, a message reader and a writer are emitted so that the final user of RampUp does not need to access this field manually.

Summary

Using a post compiler is often seen as an overkill. On the other hand, being able to introduce a common denominator for a set of value types is impossible without either manual copy-paste techniques or weaving it in a post compilation process. I prefer the latter.

OpCodes.Ldtoken

You probably used typeof operator a few times. It’s quite funny, that an operator like this actually has no MSIL counterpart. Using it emits TWO OpCodes.

The first emitted opcode is OpCodes.Ldtoken with the type. It consumes the token of the type, pushing the RuntimeTypeHandle structure onto the stack as the result of its operation. The second emitted code is a call to the Type.GetTypeFromHandle(RuntimeTypeHandle) which consumes the structure pushed by the previous code, returning the runtime type. The interesting thing is that you can’t use just OpCodes.Ldtoken from C#. You need to load the runtime type first and then you can access the handle by a property. You can emit IL with just OpCodes.Ldtoken though to remove overhead of calling a method and use the structure as a key for a lookup. It will be a bit faster for sure.

You can see the example of emitting this in RampUp code of the message writer.

Size of Nullable

What’s the size of nullable structure in C#? Does it depends on anything? How fields are held in the memory. Let’s consider following cases: bool?, int?, long?, Guid? One way to get the structure size is use of Marshal.SizeOf(Type). Unfortunately this method performs a check whether the passed type is generic. Every nullable is generic, hence we cannot use it. If you take a look at the implementation of this method, there is a call to the private method of the Marshal class, named SizeOfHelper. This method does not perform a check and can be easily used to calculate the size of the struct.

Nullable consists of two fields. The first is hasValue which answers to the question if the nullable has a value assigned. The other is the value. If a nullable has no value assigned, the value field will held default(T) value. How this members are aligned in the memory, does it depend on anything? What is the offset from the start of the structure of these specific fields?

To answer the two questions above (size and alignment) please take a look at the following table:

Type Size hasValue offset value offset
bool? 8 0 4
int? 8 0 4
long? 16 0 8
Guid? 20 0 4

The first two: bool? and int? are easy to come up with. Bool is equal to int, it takes 4 bytes to store one, so the offset of the value is 4.

What about long? Why does it take 16 bytes, not 12? Why the value starts at 8, not 4? That’s because of the struct alignment, which CLR performs to enable nice packing up the given struct. In other way, the struct is aligned to the length of the 64bit CPU registries.

The final example with Guid breaks the rule for long. Or maybe not? The struct size is multiplication of 8 bytes, so it’s totally ok for CLR to use 24 bytes as it is already aligned.

If you want to do some checks on your own, you may use the gist I created.

Enhancing log4net exception logging

Using a library like log4net is surely helpful in your day to day app development. One of the most important cases where an entry should be logged is an application exception occurring on the production environment. It’s quite common to use Windows’ event logs as a storage for log entries and log4net has a special appender which is used in this cases called simply EventLogAppender. All it does by default is appending an entry to the Application event log using a parametrized event source name. It does not set an event id or a category of the entry, disabling a simple querying of entries by this param. There are imperative ways of adding one, like by adding an element EventID or Category to a dictionary log4net.ThreadContext.Properties. This makes you either wrap each call to log4net with a proxy (it’s useful and may be used in your project) or pollute your code with plenty of a logging logic. Can it be done in another way?

Filter to the rescue
It’s not quite hard to provide a better, separated way of enhancing your log entries on the information based on the exception object being thrown. One of them is using a filter, which has an ability to access an exception by a LoggingEvent instance. A simple code of doing it can be found in the following gist.

As you can see, the consts of EventID and Category are stored in the filter’s implementation as the original EventLogAppender uses inline consts to check them. Once their added they will be used by the mentioned EventLogAppender to mark the given entries with EventId and Category. The logic of determining them is up to you. You can hash the exception stacktrace, use an exception’s source property or something else. At the very end remember to add this filter to your appender. Happy logging!😉

Overvalidated design

Imagine, that you’re requested to allow adding additional validation in an ASP MVC application to properties of some models. The set of properties’ types is given and contains following types: string, int, DateTime, a reference to a glossary entity. The business (why?) scenario is irrelevant and will be not exposed here.
To allow MVC work nicely there are multiple ways to resolve this case. The following ways may be used:

  • a custom ModelValidatorProvider implementation can be provided, which takes the validation information saved in some storage and applies to
  • an extension to TypeDescriptor facilities, to add attributes to the specified properties

But how to store this information, being given a meta description (types and their properties already mapped to entities in a database). The first take was to provide an abstract class Validator and try to subclass it with a fully descriptive design, allowing saving any type of parameter, with any kind of operator. You can imagine how many enums, objects (not so strongly typed API) it created.

The “what for?” question arose. Being given a set of types and possibilities of their validation why not to provide validators tightly coupled to those types? If the set of validated types is frozen, the switches can be easily replaced with visitors (very stable hierarchy), which can easily transform the given data into sth which may be used by MVC.

Validators data new design

Having your information about validators correlated with types, which should not be changed in a while, allows you for easier editing and storing (no more operators, objectly typed properties). The transformed values can be easily applied via ModelValidator or added as attributes to the TypeDescriptor of the given model. This approach creates a simple pipeline with a possibility of getting the procession result in any moment and inject it in the framework (ASP MVC) in a preferable way.

A CRUD action provider

As it was described in the previous entry, the idea of the action providers is to allow to gather cross-cutting concerns into more generic (not in the C# terms, but functionalities/features) bag than a derivation from a generic controller. To provide a working example, a standard CRUD operations can be taken into the consideration. Let’s take a look into another gist.
I do hate static code, but for sake of a simplicity all the CRUD operations were moved to a static class, CrudActions. All of them are pretty simple, as they use an entity model binder introduced in the former blog entries. For instance, when an entity edit result is posted to the server, a model binder will take care of updating the entity, so the entity being passed to the [HttpPost]Edit will be already altered with the changes made by a user.
The very second step was discovering all of the CrudActions. It is made by a DetachedActionProvider which caches information in a ReadWriteCache taken from the MVC internals. To close the open generic of CrudActions, an entity type with an id’s type is needed. The entity type information is retrieved from a IEntityController, which is a simple markup convention (you can come up with a name convention, or delegate it somewhere else), for attaching to a controller a handled entity type. The id metadata are gathered from the NHibernate’s ISessionFactory. Once the data are retrieved, the CrudActions generic class definition can be turned into a closed generic and the specific method can be found.
Wait a second… how do we transform a MethodInfo into an ActionDescriptor? First, the method infos are wrapped with an ActionInfo instance (it caches attributes as well as the method info itself). Next, the ActionInfos, those who applies (HttpGet, HttpPost, etc.), are passed to the ReflectedMvc to create a ReflectedActionDescriptor.
At the end I left some bitterness about MVC internalization policy. I must admit, that I don’t understand internalizing types or constructors or any members which could be helpful for future app development. That case applies to the ReflectedActionDescriptor ctor, which (the public one) throws an exception if the method passed to it is a static method. The other, non-checking one, is of course internalized. I’m asking: why? The internalization should be used as a last resort, no as a ‘hide it, because I cannot predict if someone doesn’t want to crush MVC in his project’ rule.

Hope you like the action provider extension point and will use it for cross-cutting, ‘I can compose, not only derive from a generic class’ actions in your controllers.