Pearls: EventStore transaction log

I thought for a while about presenting a few projects which are in my opinion real pearls. Let’s start with the EventStore and one in one of its aspects: the transaction log.
If you’re not familiar with this project, EventStore is a stream database providing complex event processing. It’s oriented around streams of events, which can be easily aggregated or repartitioned with projections. Based on ever appended streams and projections chasing the streams one can build a truly powerful logic around processing events.
One of the interesting aspects of EventStore is its storage engine. You can find a bit of description in here. ES does not abstract a storage away, the storage is a built-in part of the database itself. Let’s take a look at its parts before discussing its further:

Appending to the log
One the building blocks of ES is SEDA architecture – the communication within db is based on publishing and consuming messages, which one can notice reviewing StorageWriterService. The service subscribes to multiple messages, mentioned in implementations of the IHandle interface. The arising question is how often does the service flushed it’s messages to disk. One can notice, that method EnqueueMessage beside enqueuing incoming messages counts ones marked by interface IFlushableMessage. What is it for?
Each Handle method call Flush at its very end. Additionally, as the EnqueueMessage increases the counter of messages requiring flush, each Handle method decreases the counter when it handles a flushable message. This brings us to the conclusion that the mentioned counter is equal 0 iff there are no more flushable messages in the queue.

Flushing the log
Once the Flush is called a condition is checked whether:

  • the call was made with force=true (this never happens) or
  • there are no more flush messages in the queue or
  • the given time from the last time has passed

This provides a very powerful batching behavior. Under stress, the flush-to-be counter will be constantly greater than 0, providing flushing every given period of time. Under less stress, with no more flushables in the queue, ES will flush every message which needs to flush the log file.

Acking the client
The final part of the processing is the acknowledgement part. The client should be informed about persisting a transaction to disk. I spent a bit of time (with help of Greg Young and James Nugent) of chasing the place where the ack is generated. It does not happen in the StorageWriterService. What’s responsible for considering the message written then? Here comes the second part of the solution, the StorageChaser. In a dedicated thread, in an infinite loop, a method ChaserIteration is called. The method tries to read a next record from a chunk of unmanaged memory, that was ensured to be flushed by the StorageWriterService. Once the chaser finds CommitRecord, written when a transaction is commited, it acks the client by publishing the StorageMessage.CommitAck in ProcessCommitRecord method. The message will be translated to a client message, confirming the commit and sent back to the client.

Sum up
One cannot deny the beauty and simplicity of this solution. One component tries to flush as fast as possible, or batches a few messages if it cannot endure the pressure. Another one waits for the position to which a file is flushed to be increased. Once it changes, it reads the record (from the in-memory chunk matched with the file on disk) processes it and sends acks. Simple and powerful.

Composition vs derivation

Assume you’re writing some reports in your application. You’ve just created the third controller covering some kind of reporting and it seems to be, that all the three controllers have a very similar code, modulo type passed as the entity type to your NH ISession.QueryOver() or another data source. It seems that, if the method creating report base was generic, it could be used in all the cases. You want to extract it, make it clearer and to stop repeating yourself. What would you do?

Derivation
The very first thing is to extract method in each of the controllers. Now they seem almost the same. A place is needed where the method can be easily moved. How about a super type? Let’s create a controller, call it in a fashionable way, for instance: ReportControllerBase, and move the method in there. Now you can easily remove the methods in the deriving controllers. Yeah! It’s reusable, everyone writing his/her report can derive from the ReportControllerBase and use its methods to speed up his/her task.

Composition
The very first step is exactly the same: the extract method must be done, to see the common code. Once it’s done, you notice that the whole method has only a few dependencies which can be easily pushed to parameters, for instance: isession, the entity type passed to query, the value used in some complex where clause, etc. You change all the field and properties usages to parameters which allows this method to be static. You create a static method and turn it into an extension method of a session. The refactorization is done, you can easily call this method in all the controllers, by simply calling an extension onto a session.

What’s the difference and why composition should be preferred
If you use C# or Java you should always be aware of one limitation: you can derive from only one type. Spending this ‘once in a lifetime’ for a simple functionality extraction, for me – that’s the wrong way. Using composition, and deriving only when you truly see that one type is another type, that’s the right way to go. In the next post I’ll write about ASP MVC Detached Actions, a simple mechanism you can use, to derive your controllers only, when it is needed and delegating common actions, without it.

ASP MVC Model binders

Recently I had to create a model for ASP MVC, which would be bound by the model binder and had its properties set according to the request. It’s the most common case you can imagine, but there was one ‘but’. The model, because of its nature had to have non-default constructor. It was only a few parameters but it broke the DefaultModelBinder, which simply couldn’t find all the needed values. My response was quite fast, specially having the controller factory implemented in the following way:

/// <summary>
/// The controller factory building up the controllers with the unity container.
/// </summary>
public class ControllerFactory : DefaultControllerFactory
{
    private readonly IUnityContainer _container;

    public ControllerFactory(IUnityContainer container)
    {
        _container = container;
    }

    protected override IController GetControllerInstance(RequestContext requestContext, 
        Type controllerType)
    {
        return (IController)_container.Resolve(controllerType, null);
    }
}

The very next step was to replace the default binder stored in ModelBinders.Binders.DefaultBinder with another, controller-based implementation:

/// <summary>
/// The model binder building up the model objects with the unity container.
/// </summary>
public class ModelBinder : DefaultModelBinder
{
    private readonly IUnityContainer _container;

    public ModelBinder(IUnityContainer container)
    {
        _container = container;
    }

    protected override object CreateModel(ControllerContext controllerContext, 
                                          ModelBindingContext bindingContext,
                                          Type modelType)
    {
        var type = modelType;
        if (modelType.IsGenericType)
        {
            var genericTypeDefinition = modelType.GetGenericTypeDefinition();
            if (genericTypeDefinition == typeof (IDictionary<,>))
            {
                type = typeof (Dictionary<,>).MakeGenericType(modelType.GetGenericArguments());
            }
            else if (((genericTypeDefinition == typeof (IEnumerable<>)) ||
                      (genericTypeDefinition == typeof (ICollection<>))) ||
                     (genericTypeDefinition == typeof (IList<>)))
            {
                type = typeof (List<>).MakeGenericType(modelType.GetGenericArguments());
            }
        }

        // in the base method: return Activator.CreateInstance(type);
        return _container.Resolve(type, null); 
    }
}

As you can compare, the method simply delegates creating model to the container. Having this binder set allowed passing models with non-default constructors, so the problem was solved:] After a while, I considered following case: if the type is constructed with a container, the controller can also be passed an object implementing an interface registered in container. Hence, if a method needs a NHibernate ISession, this can be expressed not only as a parameter in Controller constructor, but also as a method parameter. This paradigm creates controllers with no parameters needed in constructor (or only common parameters, used in all actions) and all the others passed as action parameters. The result is a method, which can be perfectly tested in an environment where the only smallest, needed set of dependencies is passed (no more constructors with not needed in the current method parameters). What do you think about it?

An example of a controller:

public class CarController : Controller
{
    private readonly ISession _session;
    private const int PageSize = 10;

    public HomeController(ISession session)
    {
        _session = session;
    }

    public ActionResult Rent(IUserService userService, Guid carId)
    {
        using (var tx = _session.BeginTransaction())
        {
            var car = _session.Load<Car>(carId);
            var user = userService.GetCurrentUser();

            car.RentBy(user);
            
            tx.Commit();
        }

        return View();
    }

    public ActionResult Index( int? pageNo)
    {
        var page = pageNo ?? 0;

        var cars = _session.CreateQuery("from Car")
            .SetFirstResult(page*PageSize)
            .SetMaxResults(PageSize)
            .Future<Car>();

        return View(cars);
    }
}

Themis wants you to hibernate her!

Recently, I’ve been working on project which in the near future will have a quite complex authorization rules. Additionally, these rules will affect the display, simply filtering data sets, which one can view. Instantly I thought about Themis and it’s ‘future feature’ allowing to integrate with NH. What I’d like to have is simple Themis’ role definition:

public class AnalystRoleDefinition : BaseRoleDefinition
{
    public AnalysRoleDefinition( )
    {
        // CanView - helper method introduced in the application, wrapping Themis': Add blah blah
        CanView<IProtectedDocument>((document, analyst)=> document.ProtectionLevel <= analyst.AllowedProtectionLevel);
    }
}

Ok… As I can see, each analyst is given permission to view protected documents ONLY when he/she has AllowedProtectionLevel greater than document. According to Themis’ functionality it’s simple to create a service configured this way and check, whether in the context of a specified document this permission is granted. But what about filtering? If the analyst is disallowed to view such a document, shouldn’t it be hidden?

NHibernate filters to the rescue

NHibernate filters can be used to add specific conditions to querying classes and their collections. Being given the role definition mentioned earlier I could add filter with an easy condition (the analyst property would be a parameter got from the context, the real condition is based on a document ProtectionLevel column) for each class implementing the IProtectedDocument and activate them with one call based on the context. My very first proposal for API would be:

var roles = _roleService.GetCurrentUserRoles();
using(var filter = _nhAuthorizationService.ApplySessionFilters(_session, roles)
{
    var q = _session.CreateQuery("from IProtectedDocument");
    // other query stuff
}

The filter wrapped in using is an applier of filters which uses ISession.EnableFilter during creation and ISession.DisableFilter to undo filtering. It leave the session in a untouched filtering state.

If you bothered with explicitness and you want something implicit, you can easily add this behavior to your DI and never thing about it again.

Any thoughts about it?

Domain events with Unity Container extension

I do like domain events perfectly described by Udi. I’d like to share my implementation of this pattern, using Unity (my preferable container), which seems to be simple, short and still powerful. I assume, that you read the Udi’s article carefully. First and foremost: the domain interfaces

    /// <summary>
    /// The markup interface for any system event.
    /// </summary>
    public interface IEvent
    {
    }

    /// <summary>
    /// The interface of a handler for events of type <typeparamref name="T"/>.
    /// </summary>
    /// <typeparam name="T">The type of the event to handle.</typeparam>
    public interface IEventHandler
        where T : IEvent
    {
        /// <summary>
        /// Handles the specified @event.
        /// </summary>
        /// <param name="event">The @event.</param>
        void Handle(T @event);
    }

    /// <summary>
    /// The interface of event manager, allowing raising events based on the <see cref="IEvent"/>.
    /// </summary>
    public interface IEventManager
    {
        /// <summary>
        /// Raises the specified event, not requiring, 
        /// that it is handled by any handler.
        /// </summary>
        /// <typeparam name="T">The type of the event.</typeparam>
        /// <param name="event">The @event.</param>
        /// <remarks>
        /// The method iterates through all of the registered handlers for the event of type <typeparamref name="T"/>.
        /// </remarks>
        void Raise(T @event)
            where T : IEvent;
    }

I assume usage of dependency injection and having the IEventManager implementation injected. Speaking about the IEventManager implementation it’s fairly simple, located in a project knowing the Unity assembly (Infrastructure in my case). It’s worth to mention, that I consider handlers to be transient objects, constructed and discarded immediately after event handling.

    /// <summary>
    /// The implementation of event manager using the container to resolve handlers.
    /// </summary>
    public class EventManager : IEventManager
    {
        private readonly IUnityContainer _container;

        [DebuggerStepThrough]
        public EventManager(IUnityContainer container)
        {
            _container = container;
        }

        [DebuggerStepThrough]
        public void Raise(T @event)
            where T : IEvent
        {
            var handlers = _container.&lt;ResolveAll&gt;();

            foreach (var handler in handlers)
            {
                try
                {
                    handler.Handle(@event);
                }
                finally
                {
                    _container.Teardown(handler);
                }
            }
        }
    }

The last but not least: how to register all the handlers in the container? I wrote a small unity extension providing assembly scan functionality which can be found below. The are two mouthful method names in the code:

  • GetFirstClosedGenericInterfaceBasedOnOpenGenericInterface, which tries to do what is written it does :P
  • RegisterInstanceWithSingletonLifetimeManager,an extension methods using a I-have-nothing-to-do-with-monitors LifetimeManager for setting up the singletons

The code itself:

    /// <summary>
    /// The <see cref="UnityContainer"/> extension registering all the event handlers
    /// and unity-based implementation of <see cref="IEventManager"/>.
    /// </summary>
    /// <remarks>
    /// The event handlers are regsitered as classes with transient lifetime. 
    /// Their instances are tear down immediately after usage.
    /// </remarks>
    public class EventUnityContainerExtension : UnityContainerExtension
    {
        protected override void Initialize()
        {
            var eventManager = new EventManager(Container);
            Container.RegisterInstanceWithSingletonLifetimeManager(eventManager);
        }
        /// <summary>
        /// Registers all the event handlers from assembly of the passed type <typeparamref name="T"/>.
        /// </summary>
        /// <typeparam name="T">The type which assembly should be scanned.</typeparam>
        /// <returns>This instance.</returns>
        public EventUnityContainerExtension RegisterHandlersFromAssemblyOf<T>()
        {
            return RegisterHandlersFromAssembly(typeof(T).Assembly);
        }

        /// <summary>
        /// Registers all the event handlers from all <paramref name="eventHandlersAssemblies"/>.
        /// </summary>
        /// <param name="eventHandlersAssemblies">List of assemblies to scan.</param>
        /// <returns>This instance.</returns>
        public EventUnityContainerExtension RegisterHandlersFromAssemblies(params Assembly[] eventHandlersAssemblies)
        {
            foreach (var eventHandlersAssembly in eventHandlersAssemblies)
            {
                RegisterHandlersFromAssembly(eventHandlersAssembly);
            }

            return this;
        }

        private EventUnityContainerExtension RegisterHandlersFromAssembly(Assembly assembly)
        {
            foreach (var type in assembly.GetTypes())
            {
                if (type.IsInterface || type.IsAbstract)
                {
                    continue;
                }

                var handlerInterface = type.GetFirstClosedGenericInterfaceBasedOnOpenGenericInterface(typeof(IEventHandler));
                if (handlerInterface == null)
                {
                    continue;
                }

                // the type is registered with a name, because only named registrations can be resolved by unity.ResolveAll method
                Container.RegisterType(handlerInterface, type, type.FullName, new TransientLifetimeManager());
            }

            return this;
        }
    }
}

Simple and powerful, isn’t it? :)

I demand configuration

In the previous posts I described the intentions behind Themis. It’s time do deepen into demand and the whole configuration. Imagine a domain of car drivers. It includes car drivers as well as driven cars. The simplest demand which one can come up with is a simple check, whether one can drive a specific car.

public class CanDrive : IDemand<bool>
{
    public CanDrive(Car car)
    {
        Car = car;
    }

    public Car Car { get; private set; }
}

Speaking about a car and car driver:

public class DriverRole
{
    public DriverRole(int skillLevel)
    {
        SkillLevel = skillLevel;
    }

    public int SkillLevel { get; private set; }
}

public class Car
{
    public Car(int requiredSkillLevel)
    {
        RequiredSkillLevel = requiredSkillLevel;
    }

    public int RequiredSkillLevel { get; private set; }
}

The last but one thing, which should be done is to create a defintion of the DriverRole. By creating a definition I mean, creating a set of rules applied to evaluated demands in scope of the role context:

public class DriverRoleDefinition : RoleDefinition<DriverRole>
{
    public DriverRoleDefinition( )
    {
        Add<CanDrive, bool>((d, r) => d.Car.RequiredSkillLevel <= r.SkillLevel);
    }
}

Having all of this, during application startup one can easily configure the demand service with fluent configuration:

var demandService = Fluently.Configure()
    .AddRoleDefinition(new DriverRoleDefinition())
    .BuildDemandService();

and use the created service to evaluate demand canDrive created on a retrieved car basis, passing the driver role as the argument

return _demandService.Evaluate<CanDrive, bool>(canDrive,  driverRole).Any(b=>b);

In the solution, I created a more meaningful example (Themis.Example project), showing how with simple extension methods get rid of plenty of generic parameters.

In the next posts I’ll write about Themis extension points and describe a way in which I want to integrate it with NHibernate.

I demand IDemand

The main interface of Themis is IDemand.

public interface IDemand
{
}

An example implementations:

public class EntityPermission : IDemand
{
    public EntityPermission(TEntity entity)
    {
        Entity = entity;
    }

    public TEntity Entity { get; private set; }
}

public class View : EntityPermission
{
    public View(TEntity entity)
        : base(entity)
    {
    }
}

A simple interface, with no members at all. The class implementing it defines the demand name as well as the result which should be provided by the system evaluating the demand. Implementations of IDemand should be contextful, passing all the needed context for the demand evaluation in their properties. The TResult can be any type, but in majority of authorization cases it’ll be nothing more then bool. We want a yes/no answer for ‘Can I?’ question, don’t we? :)

Having the defined roles and demands we can move to the UI and write a simple authorization check. Being given an instance of IDemandService (by constructor injection), I can write in my ASP MVC controller a simple check of a demand/authorization. Below you can find the signature of IDemandService, a simple extension method written to ease using the service in a specific domain (a wrapping service is also a good solution) and the usage example itself.

public interface IDemandService
{
    IEnumerable<TResult> Evaluate<TDemand, TResult>(TDemand permission, params object[] roles)
        where TDemand : IDemand<TResult>;
}

Useful extension methods written in the app using Themis. The names, I hope, are self-descriptive.

    public static class AuthorizationServiceExtensionMethods
    {
        public static bool Can<TDemand>(this IDemandService demandService, TDemand permission, 
            params object[] roles)
            where TDemand : IDemand<bool>
        {
            return demandService
                .Evaluate<TDemand, bool>(permission, roles)
                .Any(b => b);
        }

        public static bool CanEdit<TEntity>(this IDemandService demandService, TEntity entity, 
            params object[] roles)
        {
            return demandService
                .Evaluate<Edit<TEntity>, bool>(new Edit<TEntity>(entity), roles)
                .Any(b => b);
        }

        public static bool CanView<TEntity>(this IDemandService demandService, TEntity entity, 
            params object[] roles)
        {
            return demandService
                .Evaluate<View<TEntity>, bool>(new View<TEntity>(entity), roles)
                .Any(b => b);
        }
    }

And finally, the app code example in which you can see that role management was delegated to an enigmatic IRoleService. The ‘Should I use NH session to manage roles?’ question is out of scope of this entry.

public ActionResult View(Guid id)
{
    var car = _session.Get<Car>(id);
    var roles = _roleService.GetCurrentUserRoles();
    if (!_demandService.CanView(car, roles))
        throw new InvalidOperationException("You cannot view this car!");

    return View(car);
}

Isn’t is simple? What about the expressions and the configuration? I’ll cover it soon.