CQRS and Event Sourcing

I was recently at the Agile Architecture Open Space Conference, and Command/Query Responsibility Segregation (CQRS) as an architecture was a big topic on the conference. Mark Seeman introduced the terminology and the principles, and Jeppe Cramon described a practical application in a concrete case.

The idea is that you make an explicit separation of commands that change the state of the system, from queries that only read state and never changes it. Commands and queries are represented as objects (see the patterns Command, Unit of Work, and Context from DCI) that are sent to the system from a front-end or UI-layer, or external system. Each command or query represents an intention to do something, and may be synchronously or asynchronously rejected based on validation and other rules. When the system performs a command, it generates one or more events. Commands are often queued before they are executed, partly because this decouples the system from its clients, and partly because this allows multiple back-ends to run concurrently, in turn allowing deployment of new versions without any down-time at all. Queries, on the other hand, are often synchronous and executed against a separate set of nodes dedicated to the task. Commands, queries and events can have version numbers in their serialized formats, making Just-In-Time data-migration possible, which in turn makes it easier to have multiple versions of the system running concurrently.

An event is a piece of information about something that has happened. Events are immutable, because they represent the past, and you cannot change the past. Events can be persisted in a database, sent to a number of recipients through some publish-subscribe model, topic or queue, or be handled by generic event handlers. The architecture poses no limits to what can happen to the events. A typical scenario, however, is that the events are used to update a representation for the system state that is optimized for reading. Queries, because they only read state and do not change it, can then operate on the read-friendly dataset. They will have to be able to tolerate the slight lag there exist in between the command executions and the updates to the read-friendly dataset, but since most systems use an optimistic concurrency model they are unlikely to be able to observe this usually small lag anyway.

This way, writes and reads are separated and (can) happen on different databases. This way, writes and reads can scale independently, and the acknowledgement of eventual consistency makes the whole system more scalable in general. Another neat benefit is that audit logging and tracing become trivial to implement, simply by persisting the events. A persisted set of events, or a steady stream of events, also makes real-time Business Intelligence not only possible, but potentially easy, since running a report is really just a query against a specialized dataset – this fact alone can be reason enough to build systems on a CQRS architecture. And one more time for people who might not realize it: Real-time Business Intelligence is awesome!

Concrete CQRS Example

Jeppe Cramon of TigerTeam presented, in broad terms, a user administration system that was built on a CQRS architecture. The system was part of a larger installation, where multiple external systems were interested in information about users, and each of them had their own user representation in their own databases. A command in this system could be a user updating his password. In the absence of a single-sign on deployment, the system verified the password change and stored the hash of the new password in its database. Then it sent password-update events, containing the new password hash, to a number of event handlers that updated the user databases of the external systems. Adding a new system to the installation, or treating some of them slightly differently from the others, was easy to do because they each had their own event handler.

This way, the responsibility of maintaining user information was collected in a single part of a larger whole composed of multiple legacy systems. CQRS is useful for these types of loosely-coupled integrations, but the read-write segregation also presents advantages in enabling scalability, and the immutable nature of events make interesting things possible in terms of BI. On the other hand, there are cases where CQRS makes less sense. For instance, if you have a front-end that is very sensitive to latency, or if you do not have sufficient control over the database setup and schema to make immutable events and the read-write split possible.