I had an idea about querying and projecting over big streams of messages serialized with Google Protocol Buffers. If one needs only a few fields to his/her projection, why don’t make it implicit and prepare an optimal way of deserializing only these fields? That’s the way Protobuf-linq has been born. It’s simple, fast and eager to help you iterate over big streams of data.
Check it out!
Ripple on your NuGet
Recently I’ve been involved in a project which consists of many teams collaborating to deliver a big project. The application architecture gives this teams an opportunity to work with their code in a module scope. There’s no one big Visual Studio solution, all modules are scoped in their own slns. Living in a world like this means, that your CI, package management and local deployment must be fast, effective and easy to use.
Ripple is a project from the FubuMVC family. It’s an alternative NuGet client, so all the changes in behavior are made on the client side. It makes it a perfect fit for a scenario where one can mix up a local NuGet feeds for their internal packages and official ones. What about its advantages?
The very first difference is a versioning. Ripple extracts packages into non versioned folders. It means that your projects will no longer be changed, when a new version of a package arrives. It helps a lot, especially in environments with packages being frequently published.
Ripple makes a distinction between referenced packages. There are float and fixed packages. Float packages are these changing rapidly. It’s meant that all your packages should be floating during development. The fixed packages are rock solid versions of libraries used by your system. They will not be updated, unless the force option are used.
The Ripple is still being developed. The most important thing it’s that it is active and thriving. If you develop applications in .NET with a plenty of solutions which are meant to publish packages, then give Ripple a try. It’s worth it.
Backbone & underscore beauty
Recently I’ve joined a project which is based on the Single Page Application paradigm. This means no page refreshes after the first load, after login and a plenty of client side scripting. To enhance the project speed a stack of client side libraries is used including, RequireJS (for module management), Underscore, Backbone and Backbone Marionette. There are more of them, but the last three are very interesting because of their paradigm.
Underscore, Backbone and Marionette are not frameworks. They are libraries, great libraries providing a few tools to deal with a web app development. They do not make you to use all of they features components (as ‘features’ are overloaded, framework term). You can pick wisely and use only a few of the provided tools. Of course there is a risk of creating a monster, an unusable app, but it’s up to you and your paradigms, what to use and what to implement on your own. Strongly encourage you to get accustomed to them, especially all of them have a beatiful common denominator, they provide the annotated codebase (code as documentation) which can be found here: Underscore, Backbone, Marionette.
Use libraries, don’t let frameworks use you!
Enhancing log4net exception logging
Using a library like log4net is surely helpful in your day to day app development. One of the most important cases where an entry should be logged is an application exception occurring on the production environment. It’s quite common to use Windows’ event logs as a storage for log entries and log4net has a special appender which is used in this cases called simply EventLogAppender. All it does by default is appending an entry to the Application event log using a parametrized event source name. It does not set an event id or a category of the entry, disabling a simple querying of entries by this param. There are imperative ways of adding one, like by adding an element EventID or Category to a dictionary log4net.ThreadContext.Properties. This makes you either wrap each call to log4net with a proxy (it’s useful and may be used in your project) or pollute your code with plenty of a logging logic. Can it be done in another way?
Filter to the rescue
It’s not quite hard to provide a better, separated way of enhancing your log entries on the information based on the exception object being thrown. One of them is using a filter, which has an ability to access an exception by a LoggingEvent instance. A simple code of doing it can be found in the following gist.
As you can see, the consts of EventID and Category are stored in the filter’s implementation as the original EventLogAppender uses inline consts to check them. Once their added they will be used by the mentioned EventLogAppender to mark the given entries with EventId and Category. The logic of determining them is up to you. You can hash the exception stacktrace, use an exception’s source property or something else. At the very end remember to add this filter to your appender. Happy logging! 
Learning
How do you learn? How much time do spend on learning new things? What tools do you use? Below you can find a few my opinions, a few tools that I use.
The tools I use to learn are, in the order from the most used:
- An interesting projects
- A smartphone
- An e-book reader
- Books (yes, good, old-fashioned printed books)
- A computer or a laptop
The first and foremost is an interesting project. By a project I mean my day-to-day job (choose it wisely, it’s a lot of your time) as well as some minor things done as OSS. Under this category go also all the spikes which haven’t been finished and all the fun projects I do at home.
A smartphone with a good RSS reader allows me to deal with an enormous number of blog entries published by brothers in code. The phone enables me to drag down a Twitter bar in the Android app a few times a day when I have a few spare minutes(waiting for a coffee, etc.). I’m not an addict though. I do prefer to have internet access switched off most of the time. I use it offline with no notifications etc. distracting me all the time.
An e-book reader (I like to use a bigger format of one) is perfect for longer periods of time (from 20 minutes and longer). Just have a few Google whitepapers unread and stored there.
I read non-technical books as well. I do like the paper versions, but sometimes they’re even harder to read than e-books (a crowded bus for instance). The time limit is also important. I tend to take one with me if I know that I’ll be able to read for more than 15, 20 minutes.
A computer for watching all the Infoq & Skillsmatter presentations. When I have one hour, I take one from my list of ‘to watch’ and watch it.
As you can see I have one rule: I’ve got to have sth on my to do list all the time, with different time requirements. I use it every time I have a few spare minutes. Things which may take longer (a good post on Twitter) are pushed to a to do list. The future version of me will handle it. Having a smartphone and a reader is more than enough for a standard day. Presentations watched on a computer are good for weekends or an evening spare time. I do encourage you. Make your list, assign time frames preferred by you and take a few of your knowledge sources with you. It’s worth it!
Protobuf-net: inheritance of messages
The last post was an introduction to a simple project called Protopedia, located here. The project is destined to bring in a simple manner, probably one test per case, solutions for complex scenarios like versioning, derivation of messages, etc. As the versioning was described by the previous entry, it’s right time to deal with derivation.
Inheritance
It’s well known fact that one should favor composition over inheritance. Dealing with derivation trees with plenty of nodes can bring any programmer to his/her knees. How about messaging? Does this rule apply also in this area? It’s common for messages to provide a common denominator, containing fields common for all messages (headers, correlation identifiers and so on), especially if they’re meant to be sent/saved as a stream of messages of the base type (example: Event Sourcing with events of a given aggregate). Using a set of messages with a distilled root greatly simplifies concerns mentioned earlier. Consider the following scenario, serialization of a collection of A messages (or its derivatives) being given the following structure:
How would Protobuf-net serialize such collection? First, take a look at the folder from Protopedia. You can notice, that all the classes: A, B, C, have been mapped with different types. It’s worth to notice the ProtoInclude attributes with tag values of the types located one level deeper in the derivation tree. The second important thing is the values of the derived type tags, which do not collide with tags of the class fields. In the example, you can find a constant value of 10 used for sake of future versions of the root, the A class. As one can see in the test of the derivation, the child classed of the given class are serialized as fields with the tags equal to the tag passed in the ProtoInclude attribute. To see the fields composed in a way the Protobuf-net serializes inherited messages take a look into following message contracts. There’s no magic and the whole idea is rather straightforward: serialize derivatives as fields, turning the inheritance into the composition. This working proposal of Protobuf-net will be sufficient and effective in all of your efforts of serialization of inheritance. Nice serializing!
Protobuf-net: message versioning
Welcome again. Recently I’m involved in a project where Protobuf-net library is used for the message serialization concern. The Protobuf-net library was developed by Marc Gravell of Stackoverflow. Beside standard Google Protocol Buffers concepts there is a plenty of .NET based options, like handling derivation, using standard Data Contracts, etc. This is the first post of a few, which will deal with some aspects of Protobuf-net which might be nontrivial for a beginner. Beside the posts, a project Protopedia was created to show cases of Protobuf usages. All the examples below are stored in this repository. All the posts requires a reader to get accustomed to the official Google Protocol Buffers documentation.
Versioning
The versioning of interfaces is a standard computer science problem. How to deal with sth which was released as looking in one way and after internal changes of the system it must be changed publicly.
Consider following scenario of having two versions of one message located here. As you can see, there are a few changes like the change of the name, the addition and removal of some fields. Imagine that a service A runs on the old version of the message. A sevice B uses a new version. Is it possible to send this message from A to B, and then back to A and have all the data stored in the message? With no loosing anything appended by the B service? With Protobuf’s Extensible class used as a base class it’s possible. The only thing one should remember is to do not reuse ProtoMemberAttribute tags’ values of field removed in the past. New fields should always be added with new tags’ values.
How does it work?
When Protobufs deserialize a message, all the data with tags found in the message contract are deserialized into the specific fields, the rest of data is hold in a private ‘storage’ of the Extensible class. During serialization, these additional fields are added to the binary form allowing another message consumer to retrieve fields according o their version of the message.
