EventSourcing with NodeJS
By: Diego D Ursel
Tags: Cqrs, Software Development, Nodejs, NoSQL, Event Sourcing

The “createGroup” command

We have a choice how to implement it:

either using a single event GroupWasCreated (which includes the original participants and subject) that would be enough to describe the participant list, subject, and group admin.

(which includes the original participants and subject) that would be enough to describe the participant list, subject, and group admin. or emit events ParticipantWasAdded and SubjectWasChanged along when emitting the GroupWasCreated.

The first solution reduce the number of events and has implicit behaviour; while the latter make it clear what happens at creation time and also allow more reusable logic for both the command side (createGroup calls addParticipant) and the query side (we don’t have to process GroupWasCreated as a special event and extract participant from it, because we know ParticipantWasAdded events are being emitted along with it).

For reusability (DRY) and clarity (no implicit behaviour), I’ll choose the second solution and emit each event.

I’m using a library that helps me store and fetch events : eventstore on npm.

npm install --save eventstore

npm install --save uid

There is an in-memory adapter (basically a fake database) but the lib also provide a MongoDB and Redis adapter. I’ve personally used Redis in a previous project, but MongoDB should be the default choice imo.

I’ll initialise the lib in the constructor of my service and add a small Promise wrapper around loading events to have a nice consistent Promise api.

But so far, I haven’t run the code yet.

So, because I’m lazy … and professional … (the lazy part is actually more accurate, but that’s another debate) I’m gonna write an automated test :)

Yes coders, if you don’t write automated tests … you’re only half professional! (according to Uncle Bob; and after 4 years of practicing BDD/TDD, I have to agree).

npm install --save-dev assert

npm install --save-dev mocha

As we currently have no way to get the “end-state” of our entity, we can only verify that the events are being emitted.

Let me run that and check if this works … Ok just after a few fixes and that actually runs.

Code at this step is on github on branch “step-1”.

You can pull it and try to run npm install and npm test .

Notice that the aggregate behaviour is defined by (at least for the creation):

When Command

Then Events

And that’s reflected in the test.

When we’ll be implementing “update” command (like “addParticipant”), we would have:

Given Events

When Command

Then Events

We’ve been building so far the “Command Side” of our application with currently a single command, but no feature to read the state of the application (except the stream of events which is not super user-friendly).

And that’s why EventSourcing is coupled with CQRS: Command / Query segregation.

The code of the application at this stage can be found on github branch step-1.

Whatsup so far ?

Let’s realise that with this, we have a persistence model that can accept unstructured data and persist a Has-Many relationship for the Participants.

This is powerful on it’s own: you don’t have to do any database migration or deep thinking if you want to add a new commands or persist another “belongs-to” or “has-many” association; as long as you have a clear AggregateRoot defined in your ubiquitous domain language and Event-Sourcing for it’s persistence, it’s is very easy to add behaviour in a clear and consistent manner. We’ll see that in the next step.

Let’s say we want to add notification settings particular to this group, it’s simply a new command and a new event. Really straightforward in terms of the Command Model: there is no database changes and the only changes required is limited to the domain needs: a command, an event; that’s it.

Let’s compare to CRUD:

We would have need a Group model and a GroupParticipant model linked with a has_many/belongs_to relationship. Both would have method like "create" and "update" and "destroy" from the underlying ORM.

The CreateGroup logic you would either add participant using GroupParticipant.create inside the controller (bad idea imo).

A bit better would be in the model, overriding the Group.create method to the custom domain logic which would use GroupParticipant.create .

But better: you could build a service module (outside of the model) around each AggregateRoot (exactly as we did here) that would define all commands and queries available around that aggregate. This will end up with a code structure quite similar to what we have. But not very common in MVC fameworks: Rails and others don't encourage this type of code structure.

Developing outside of ORM defaults can make your code cleaner using Clean Architecture concepts of aggregate, repository and value-object.

But even with a Clean-Code architecture, with classic MVC you would need two DB tables and any new feature would likely need a database migration.

State ?

Now, let’s look at how we can get some state.

Currently we can query events, usually by aggregate_id , but could be loaded by aggregate_type or maybe by event_name .

But that's not going to be efficient (for example, querying all Groups with more than 2 participants would require to process all ParticipantWasAdded events) nor human-friendly (we don't have an easy way to get the end-state of the group, only its events).

The common solution to this problem is to implement a Projection which is processing all events and updating a special database made for querying.

Building state in a Projection

We can see this as a standalone micro-service streaming events from a message bus, updating a database with incremental changes, combining the Projection state with the new event.

It wouldn’t have access to history of events, but rather the real time stream.

A Projection is:

Given State (Projection state)

When Event (new Aggregate event)

(new Aggregate event) Then State (new Projection state)

Reducing state in a value object

In Event-Sourcing literature, usually we talk about the Read Model being a Projection built out of an event stream. But there’s another option: to load an Aggregate’s history and reduce it to it’s end-state.

Motivation: A few years ago, at my previous work, after we were using Event-Sourcing for around 1.5 year, I was struggling with the complexity of Projection and replay: it wasn’t easy to be 100% confident with the end-state of an entity.

For this, I came up with the idea of building Reducers around Aggregate’s history. Back then I called them StateModel : as they represent a way of building an end-state of an Aggregate; there could be multiple StateModel (different interpretation) for a single Aggregate type.

Now I believe Reducer is the right term as it is well known in ReactJS … and the idea is very similar.

A Reducer is:

Given Events (an Aggregate’s history)

(an Aggregate’s history) Then State

and it can be seen as a ValueObject (functional).

This has the huge advantage of not being dependent on a Projection Database (which would usually imply eventual consistency).

With this idea, we can return the end-state of an entity (that would be returned by a REST endpoint like /group/:id ) without the need of an extra Projection.

Of course, this won't allow us to easily query Groups.

So, let’s write a minimalistic Read Model to easily get the state of our aggregate.

Let’s do it, but … let’s start with the tests :D

We should load the events for our test … but because I’m lazy, and because I prefer my tests to use the service api, I’m not gonna extract the events from the previous test to create my “Given” case, but instead, I’m gonna use the service api and simply use the CreateGroup command to populate the events in the database:

We can see that, through the test, I defined the api behaviour and the expected payload.

I also went ahead and added createdAt and updatedAt attributes that will be inferred from the events.

Let’s implement the Reducer:

And the module function to complete our aggregate api:

Which give us a green test and the following end-state payload:

We now have a Command that generates Events, which those Events can be reduced into a human friendly end-state.

Let’s note that the reducer is simplified by the fact we’re emitting multiple events when creating the group.

From this, it is easy to create a Query Database: simply save this end-state into a NoSQL database in an after-save hook of the AggregateRoot!

But … you might not need that for simple applications…

Code at this state is on github branch step-2

Let’s add the missing actions: AddParticipant & RemoveParticipant

AddParticipant is straightforward if we don’t verify that the user is not already present (we should, to avoid redundant events).

For RemoveParticipant we have to check if the userId is in the participant list. If not, we should raise an error as the command is invalid.

For that, I’m gonna use the aggregateState and add a helper function hasParticipant(userId) to the state object.

First writing the tests:

And the domain code:

I added a validation for not executing command on aggregate that doesn’t not exist (untested 🤦).

I extract specific attributes of the command payload with

const {groupId, userId, admin} = payload; to avoid unknown attributes in my events. More on data validation later.

And the domain state object:

Ok my tests passes, but there is one thing missing, if I load the state after removing a participant, the participant is still in the list, and that is because I don’t reduce the ParticipantWasRemoved event.

Let’s adapt the test and include the state assertion along inside the current test.

Let’s add the event to our reducer to return a valid state:

And voilà!

That’s it for the code today ..

What about updating groupSubject?

Easy! No ?

Basically, what is required is to implement the changeSubject method of our service so that it emits SubjectWasChanged on an aggregateId (ie. groupId) that exists, no more.

It doesn’t require reducing the aggregate’s state to validate the command, nor adding the event to the Reducer: it’s already there.

If you’re up for a little challenge, comment your code snippet of changeSubject(payload) in the comments! ;-)