In the recent post Rinat Abdullin provides a retrospective for Lokad.CQRS framework which was/is a starting point for many CQRS journeys. It’s worth to mention that Rinat is the author of this library. The whole article may sound a bit harsh, it provides a great retrospection from the author’s and user’s point of view though.
I agree with the majority points of this post. The library provided abstractions allowing to change the storage engine, but the directions taken were very limiting. The tooling for messages, ddd console, was the thing at the beginning, but after spending a few days with it, I didn’t use it anyway. The library encouraged to use one-way messaging all the way down, to separate every piece. Today, when CQRS mailing lists are filled with messages like ‘you don’t have to use queues all the time’ and CQRS people are much more aware of the ability to handle the requests synchronously it’d be easier to give some directions.
The author finishes with
So, Lokad.CQRS was a big mistake of mine. I’m really sorry if you were affected by it in a bad way.
Hopefully, this recollection of my mistakes either provided you with some insights or simply entertained.
which I totally disagree with! Lokad.CQRS was the tool that shaped thinking of many people, when nothing like that was available on the market. Personally, it helped me to build a event-driven project (you can see the presentation about this here) based on somehow on Lokad.CQRS but with other abstractions and targeted at very good performance, not to mention living documentation built with Mono.Cecil.
Lokad.CQRS was a ground breaking library providing a bit too much tooling and abstracting too many things. I’m really glad if it helped you to learn about CQRS as it helped me. Without this, I wouldn’t ask all the questions and wouldn’t learn so much.
The provided retrospective is invaluable and brings a lot of insights. I’m wishing you all to make that kind of ground breaking mistakes someday.
It’s been a few days since the last Warsaw .NET User Group meeting. The main presentation was provided by me & Tomasz Frydrychewicz. The title was: “Event Driven Architecture in practice”. Being given a high number of answers to the pool and the overall was very positive response I may call it one of my best presentations ever. Anyway, I was being asked many questions during these days, the main one is what/who should I read/watch to immerse into this event-based approach. The list below tries to answer it somehow, grouped by author:
- Martin Fowler
- http://martinfowler.com/eaaDev/EventSourcing.html – the top 1 Google search result. Martin provides a good intro, mixing a bit a concept of storying commands and events. Anyway, this is a must read if you starts with this topic
- http://martinfowler.com/eaaDev/RetroactiveEvent.html – the article which one should become familiar with after spending some time with event modelling. Some domains are less prone to result in special cases for handling this kind of events, other may be very fragile and one should start with this
- Lokad, CQRS, Rinat Abdullin
- http://lokad.github.io/lokad-cqrs/ – a must-read if you want to choose the event way. Plenty of materials and tooling. To me some parts are a bit frameworkish, but still, it’s one of the best implementations I’ve seen. Understanding this might be your game changer.
Additionally, it provides an Azure storage implementation.
- Rinat Abdullin & Kerry Street
- Being the worst – how to become a master? Immerse yourself in a new field as the worst. That’s how winning is done! Am amazing journey through learning about DDD, Event Sourcing and many paradigms.
- Microsoft Patterns and Practices:
- CQRS Journey – a free book about a group of developers using event driven approach with DDD in mind, to build a new system. I love the personas they use to drive dialogues between different opinions/minds/approaches. It’s not a guide. I’d rather consider it a diary of all the different cases you can meet when implementing solutions using these approaches.
- Event Store
- The whole Event Store database is an actual event store for storying events from the event sourced systems. I encourage you to spend a week or more on reading its code. It’s a good codebase.
- Event sourcing documentation is a short introduction to the ES world. After all these years, it still uses the Word generated pictures :) but this doesn’t diminish its value.
- NEventStore is an open source library for storying and querying your events. It’s opinionated, for instance it stores all the events as one commit object. I’ve read it carefully, although I don’t like its approach still. One should read it though, it’s always worth to know what’s already provided.
It’s a bit long list but nobody said that you can learn a new paradigm over one weekend. So read, learn and apply it successfully :)
Currently I’m helping to model one domain with Event Sourcing. A few days ago we spent ~15 minutes on modelling some cases on the whiteboard. The result of the first phase was distilling a few aggregates with events. Later on, we described some processes as we needed to react to some events. At first to much behavior was put in a process manager, to finally be moved to a separate aggregate – a process itself. The process manager was left as a simple router. Because of the strong foundations (a library) providing us at-least-once semantics with a idempotent receivers and handling process managers, the whole discussion was on a much higher level. No imperative programming, just declarative pieces of the domain itself!
A few hours later an implementation was finished. The most important thing was that there was no mapping!. The code was a simple representation of the model being discussed and agreed on. No ORM magic, no enterprise onion with tons of layers and mappings. It was just a simple model of this piece of a domain. And it’s been working like a charm.
In the previous post I covered the process manager subscribing to and consuming events from multiple sources. Additionally, it was show that saving the position of read logs after performing action is sufficient to get at-least-once delivery (retry in case of errors).
Let me consider an aggregate which an action is invoked on. As the only transactional boundary that can be used is the aggregate itself, to each call from process manager we’ll add additional data:
- hash (unique, SHA1 probably) of the process manager identifier and the name of the origin module where the handled event was taken from
- the order number of the handled event
This two values combined in an event, will allow in one transaction to check, whether the action has been already applied and skip it if needed. Everything in one transaction.
As order numbers for the given hash can only increase, the state of this idempotent received can be modeled as a dictionary with Sha1 value as its key and the order number as its value.
The only disadvantage is additional event added to the aggregate for each action performed within a process manager. Fortunately, a scavenging process, a similar one to this from EventStore. When events are dumped to a file from a store of your choice, only the last value for the given Sha1 hash can be stored.
There is a pattern which can be used to orchestrate collaboration of different aggregates, even if they are located in different contexts/domains. This patters is called a process manager. What it does is handling events which may result in actions on different aggregates. How can one create a process manager handling events from different sources? How to design storage for a process manager?
In the latest take of event sourcing I used a very same direction taken by EventStore. My first condition was to have a single natural number describing the sequence number for each event committed in the given context/domain (represented as module). This, because of using an auto-incrementing identity in a relational database table, even when some event may be rolled back by transaction, has resulted in an monotonically increasing position number for any event appended in the given context/domain. This lets you to use the number of a last read event as a cursor for reading the events forward. Now you can imagine, that having multiple services results in having multiple logs with the property of monotonically increasing positions, for example:
- Orders: e1, e2, e3, e6
- Notifications: e1, e2, e3, e4
If a process manager reads from multiple contexts/domains, you can easily come to a conclusion that all you need to store is a last value of a cursor from the given domain. Once an event is dispatched, in terms of finishing handling by the process manager, the cursor value for the context event was created within is updated. This creates an easy but powerful tool for creating process managers with at-least-once process guarantee for all the events they have to process.
If a process provides guarantee of processing events at-least-once and can perform actions on aggregates, it may, as action on aggregate and saving the state of a PM isn’t transactional, perform the given action more than once. It’s easy: just imagine crushing the machine after the action but before marking the event as dispatched. How can we ensure that no event will result in a doubled action? This will be the topic of the next post.
If you’re into software development you’ve probably heard about Behavior-driven development. Recently I had a discussion whether or not business people think in this way. Fortunately, I was involved in a business workshop, so I could make some observations.
The way mentioned earlier is the only language business uses to define and discuss aspects of their actions. They are some abbreviations like:
Once we reach 1000 participants, we assign them rooms
Which can be easily translated into
- Given 999 participants registered
- When a participant registered
- Then the rooms are allocated
This can be easily read by business as by developers.
If you can model your solutions towards this kind of testing, which not necessarily must be performed with tools for BDD but can be easily done by introducing Event Sourcing and structuring your tests like in Lokad CQRS examples then you can finally start to discuss business ideas with business instead of describing how your db is updated. And this, for sure makes the difference.
A: Hello, have you CREATEd a new car?
B: No! I just UPDATEd its Owner field, setting it to my id. I needed to UPDATE the balance field of my Account row as well.
A: Oh, I see. Yesterday Tom DELETEd a few employees. They were stealing money. Unfortunately there is a transition period, so first he needed to UPDATE their IsActive to false, then after the period he could finally DELETE them.
B. Yes, that’s the way you do it.
No it’s not. People do not use only four verbs to describe their activities, and if they do, they have a real problem. The scope of vocabulary used by business as well as other people is much wider and their is a reason behind it. You can name everything a THING, you can use only four CRUD verbs to describe activities but instead of meaningful phrases you get a long sentences filled with clarifications. Using a vocabulary consisting of a few words only will not only increase the number of words to describe something but for sure will for sure loose some of the meaning. Can you afford? Can your company afford it as well?