Baking Peixinhos da Horta with ING’s Baker Library
By: João Esperancinha
Tags: Baker, Scala, Multithreading, Ing, Orchestration

It is important to understand at this point that Interactions and Events work a bit like interfaces. They essentially establish the layout of how the structure of the Recipe is made. The actual driving force behind the Baker orchestration are actually the InteractionInstances and the EventInstances.

The engine that drives Baker is called the akka toolkit. The original implementation of this library is made in scala and to use this library in Java can be a challenge, but not difficult. For this I will write future articles to illustrate how we can use Reflection to achieve this. I will also make a walkthrough on how we can use Baker in Clojure and Kotlin environments. But for now and to keep it simple, in this article, my main intention is to give you an introduction to this awesome library.

1.2. About events

The generation of events and how they run in baker can provide us with a myriad of solutions to our specific problems. One of the things we want to be concerned about when discussing orchestration, parallelism and multithreaded systems is control over how events are triggered. In Baker we have two major options. We can create a baker Future which will resolve once all processing has been completed. Further we can create a baker Future which will resolve once the first Event has been received.

2. Domain and Recipe overview.

Before we dive into the code, it is important to understand that we are going to cook “Peixinhos da Horta”. We need basic, if not, complete knowledge on how to make this recipe. This, like in FINTECH, is important to know. We don’t have to learn how to cook, but in order to implement this, we need at least to know how to cook this. So let’s get to it.

2.1. Ingredients

I’ve taken this recipe from the site from the from Vaqueiro. This is their recipe translated into English for 4 people:

500 g of green beans

Salt

100 g of flower

1 egg

pepper

Sunflower oil

This is our ingredients alignment. Let’s keep this in mind and let’s move on to establishing our events. Let’s keep in mind for the moment that in baker there is such a concept of creating ingredients during our recipe. As an example, we can have 500 g of green beans, but we can also have 500 g of washed green beans.

2.2. Events

In order to successfully build our recipe, it is very important to start out with Events. Interactions generate events but they can also depend on Events. This means that Events should be established before Interactions are created. But first, let’s read the recipe transcript translated to English:

“Wash the green beans and remove the bean threads. Cut the bean pods in half. Bring the water to moderate heat and cook the beans for 10 to 15 minutes according to taste. Drain the beans and prepare the batter. In a pan mix the flour with the egg. Season with salt and pepper to taste. If the batter becomes too thick, add a bit of cold water. Pass the pods of the green beans through the batter and fry them afterward in sunflower oil. Let them fry until crisp”

Reading this recipe, it may be a bit difficult to identify the Events and the Interactions. It is a fairly simple recipe, but to translate that to code and orchestration, it requires a bit of work. Let’s do that and start identifying the events:

Start bean handling — We start our recipe by making our beans ready to cook. We return all ingredients.

Start batter making — Our batter can be made independently of the means. We return all ingredients.

Cooking table setup for beans — All needed ingredients to get the beans ready. We return the green beans on the table only.

Cooking table setup for batter — All needed ingredients to get the batter ready. We return the egg and the flower.

Beans washed — Beans have been washed. We return the washed beans only.

Removed threads from green beans — Green beans have been de-threaded. We remove the unthreaded beans only.

Pods have been cut in half — All the green bean pods have been cut in half. We return the pods cut in half only.

Beans have been cooked — All green beans have been cooked. We return the cooked beans only.

Beans have been drained — All green beans have been drained. We return the drained beans only.

Flower with eggs mix has been done — We have created the base of our mix. We return our mix only.

Mix has been seasoned — Our base is now ready for the beans, but it needs more water. We return our seasoned mix only.

Coldwater added to mix — Our mix is now a bit softer now that water has been added. We return our softer mix only.

Pods have been passed through batter — We have created raw “Peixinhos da Horta”. We return our battered beans only.

Pods have been fried — We finally get our “Peixinhos da Horta”. We return our “Peixinhos da Horta Recipe”.

Ok, great. We have now identified and established the Events we need to implement. The final pieces to our Recipe layout are the Interactions.

2.3. Interactions

Interactions are actually interfaces to our business logic. What’s important here to understand is that with interactions, we are only going to establish the link between Events and Ingredients. Let’s start:

Setup cooking table for beans — We receive all ingredients and we will return an event which let us know that we have filtered out what we need in regards to making the beans ready.

Setup cooking table for batter — We receive all ingredients and we will return an event which let us know that we have filtered out what we need in regards to making the batter ready.

Wash bean — We receive the beans and we return an event that signals that the beans have been washed.

Remove thread from beans — We receive the washed beans and return an event that signals that the thread has been removed from the beans.

Pods are cut in half — We receive the washed beans and return an event that signals that the pods have been cut in half.

Beans are cooked — We receive our pods cut in half and we return an event that signals that the pods have been cooked.

Beans are drained — We receive the cooked beans and we return an event that signals that the beans have been drained.

Make batter — We receive the egg and the flower and return an event that signals that the batter base has been done.

Season batter — We receive our batter base and return an event that signals that the batter has been seasoned.

Make batter softer — We receive our complete batter and return an event that signals that the batter has been made softer with cold water.

Pass beans through batter — We receive our drained beans and the batter and return an event that signals that the pods have been battered.

Fry pods — We receive our battered beans and return an event that signals that the battered beans have been fried and that we finally have “Peixinhos da Horta”!!!

As you can see, just establishing the baker orchestration and its Ingredients, Events and Interactions, can be a lot of work at the beginning, but as we will see later this is a short and long term layout which makes it much more reliable to the development processes.

In regards to the Event implementation and the Interaction implementation, we will see later on in this article that Events can be implemented using their name as an Id. Ingredients also need to be implemented. All interactions can be implemented with any sort of business logic, but in this example, we are only going to see how everything behaves if the implementation is just a sleep mode of 100 milliseconds. It’s only important to know that It’s in these implementations, where we are free to choose if we want to call other systems, if we want to send files, if we want to start whatever process we want. The implementations are in other words open books to our needs and creativity.

3. Implementation

After running through the theory and the domain knowledge of getting the recipe right, it is now time to get to our code. I have prepared two modules for this article:

peixinhos-da-horta-scala — In this module, I’m making an example that blocks the event handler. I have made this in order to block the event generation. We will make a quick run over this.

— In this module, I’m making an example that blocks the event handler. I have made this in order to block the event generation. We will make a quick run over this. peixinhos-da-horta-parallel-scala — In this module, I’m making an example that will hold all interactions for 100 ms. Here we are going to have a look at how parallelism and orchestration works in baker and how can we use it to our benefit.

In order to explain Baker I think it’s better that we first take a look at one Recipe from one module project in detail and then go through the analysis of other implementations by enhancing potential differences between them.

Please bear in mind that for this implementation we are not going to resource the Reflection techniques described in the documentation. We are only going to use the Baker library interfaces and source code.

3.1. Ingredients

Let’s first establish the ingredients to our recipe. We can find these in the peixinhos-da-horta-scala module:

Pexinhos da Horta Ingredients

As you can see, not only have we created our necessary ingredients for our recipe, we have also created the ingredients that will be generated.

3.2. Events

In the same module, we can find the interfaces to our events. Take note that the input parameters are the actual generated Ingredients which will be passed on to the Interaction after the Event has been triggered.

Peixinhos da Horta Events

3.3. Interactions

Now that we have our Events and Ingredients let’s just finish our interfaces with our Interactions. This is the layout which we will have to consider in order to complete the implementation:

Peixinhos da Horta Interactions

At this moment maybe it’s important to compare what we have here and what we discussed during the Domain overview. This code doesn’t differ that much from our requirement document. It’s literally, in a Scala way, implementing what we have discussed. Let’s take an example. Let’s look at interaction removeBeanThread. This interaction is essentially describing that it has the name “Remove Bean Thread” and that it will receive the Ingredient greenBeansWashed and return the Event greenBeansThreadRemoved. We can look at the other examples, but since the description of all of them is just as simple as this one, it will be more beneficial to our analysis if we move on further taking into account what we have learned just now.

3.4. The recipe

In order to start the recipe we need an actual representation of the recipe. This is done by class Baker. Let’s have a look at how this is done:

Peixinhos da Horta Recipe

Let’s analyse how this works. We first declare that “Peixinhos da Horta Recipe”, is the name of our recipe. Then we declare all our interactions using the withInteraction method. Our recipe needs to start somewhere. In our case, we have a family which sits at a restaurant table and it’s dinner time! 🍽! These are events dinnerTime and familyIsHungry. We declare these using the withSensoryEvents method. If for some reason, our orchestration fails, we can declare another orchestration flow using the withDefaultFailureStrategy method. In this example, we are only saying that after a fail we are going directly to trigger an event called “ooops”. Here we can also see that naming is very important. Remember when we said that the name is also an Id in baker. This our first example of that. We will now see further examples of this in action.

3.5. Event and Interaction Implementation

We finally reach the endpoint of the complete implementation. In order to make the “Baking” of our recipe possible we need to implement it. Let’s first have a look at the code:

Peixinhos da Horta Interaction Implementation

Let’s just pick the last example to illustrate how we do this. Looking at fryPodsInstance we can easily see that this is an instance of the implementation of the fryPods interaction. We gave it the name of the fryPods. This is because this name is exactly what determines that our instance is the implementation of the fryPods interaction. There needs to be a perfect match between the Interaction and the InteractionInstance. In other words, the InteractionInstance, needs to have the same name, the same number of input Ingredients, and it’s run method needs to be implemented in order to return a Future with a return Option instance type of an EventInstance. Finally we only need to essentially create our EventInstance. To create it, we only need to insert two parameters. The first is the EventInstance and of course that is the Event name. In this case that is the name property of the event podsAreFried. The second is the providedIngredients parameter. The provided Ingredient is also an implementation, but in this case we only need to provide a Map. This map uses a key type of String and a value type of Type. Type is a baker type. To make this exercise simple we are just using PrimitiveType with the name of our implemented Ingredient.

This concludes our implementation. We will now have a look at the unit tests and interpret its digraphs results. The point here is to get us familiar with Baker and with Graphviz. We will leave the analysis to chapter 4 of this article.

4. Firing unit tests.

Let’s look at the unit tests one by one. These are also located in the peixinhos-da-horta-scala module. The unit tests are all using the same ActorSystem:

implicit val actorSystem: ActorSystem = ActorSystem(“peixinhosDaHorta”)

4.1. A peixinhos da horta recipe should compile and validate

Let’s generate our first digraph! Lets first run this unit test:

A peixinhos da horta recipe should compile and validate

We will get a digraph output in our console. If we copy-paste that into their online website at http://www.webgraphviz.com/, we will get the following diagram.

Note that these digraphs are quite big and at this point we are not interested in seeing all of it. This just gives us a sense of how it works. Let’s examine the colors we are seeing. In light grey, we can see Sensory Events. These are the events that need to occur if the process is to be started. In purple, we see the representation of the Interactions. In dark grey, we can see the normal Event representation. In orange we can see the representation of the Ingredients. Finally we can also see that all Rhombus shapes represent Events in general, regardless of their type. In the same way a rectangle represents Interactions and the circles represent Ingredients. A green color will represent a successful use or execution. We will see this in more depth later.

4.2. Baking a peixinhos da horta recipe fire event after interaction completion and don’t wait should complete baking it

Let’s now test our first Baking example. For this example only we’ll look in detail how the code is constructed in order to begin a baking process.

“Baking a peixinhos da horta recipe fire event after interaction completion and don’t wait” should “complete baking it” in {

Baking a peixinhos da horta recipe fire event after interaction completion and don’t wait should complete baking it

In this example, we say we don’t wait, because we will not explicitly wait for the main Thread to finish. Instead, we will only wait a maximum of 5 seconds and 10 seconds for our baking process, also named as “program” to finish. Though not strictly necessary we will also wait for the visual state to be ready until it’s ready to a maximum of 10 seconds. This is just to guarantee that we wait only the necessary time until we can actually check for state. In this case, we are waiting for our cook to finish. This is done by combining fireEventAndResolveWhenCompleted and the use of Await for the Future returned by the bake method. If we put the output through the http://www.webgraphviz.com/ website we will get:

4.3. Baking a peixinhos da horta recipe fire event only when received and don’t wait should start baking it but stop in the middle

In this next case we will have a look at how the fireEventAndResolveWhenReceived works and how we can use it to our advantage.

Baking a peixinhos da horta recipe fire event only when received and don’t wait should start baking it but stop in the middle

This is the example where we actually don’t stop the processing in the middle of it. Our unit tests do. Because we resolve only when the events are fired, this means that our Await methods will actually only wait until the first fired event is done processing its interaction and creates its eventual Ingredient. If we put the result of our unit test through http://www.webgraphviz.com/, this is what we will get:

Notice that our SensoryEvents have all been triggered, the Interaction also and so has its triggered Event with its created Ingredients. This means in practical terms, that after the family is hungry waiting for food at the dinner table, the cook has started to bake the dinner with all its ingredients. However we stop the process immediately after.

4.4. Waiting for baking

For the last 2 unit tests I have created another two where we will wait 3 seconds before we use our Await methods. In these scenarios we get the following same resulting digraph:

As we can see, all processing has been completed. This means that, in other words, that whether our cook is alone cooking or together with the help of others, it will always finish its baking process. In other words this will happen regardless of the difference between fireEventAndResolveWhenCompleted and fireEventAndResolveWhenReceived respectively.

4.5. Pre conclusion

We have yet to understand the fundamental purpose of Baker. Remember we want to make orchestration simple, seamless, fast and easy to implement. Let’s have a look at the following graph:

What this graphic tells us in both cases is how events are being generated. They are not telling us anything about the actual processing of the Interactions. This graph has been generated by simply logging to System out the number 1, the event name and the timestamp.

If we look back at our unit tests, this is happening here:

baker.registerEventListener((_, event) => TaskSimulator.waitMilliseconds(event.name, 10))

TaskSimulator is only an object class that I have created in order to make this log possible. It receives the name of the event and the number of times we want to have 10 milliseconds as the waiting time. This is all logged out to the standard output. In this example it would be 10 x 10 = 100 ms. We will use it in the next examples for the peixinhos-da-horta-parallel-scala module. What is important to notice for now is that the event triggering is happening sequentially and that it’s not really a good idea to perform heavy processing during the event handling. It’s important to know that it exists and that it may be useful in the future. I will discuss this further in future articles.

4.6. Parallel analysis

In this final section of our testing analysis we will have a look at the peixinhos-da-horta-parallel-scala module.

In this module I am performing exactly the same last two tests with a waiting period of 3 seconds. However in this case, I moved all the log into the Interactions. As an example let’s have a look at fryPods example:

Parallel analysis

The waitMilliseconds function is now located in the Future run declaration of the EventInstance return method.

4.7. Pre conclusion

Running the same tests, we will be able to generate a graphic like this:

In this graph we can see clearly that we have created parallelism. We have orchestrated a bunch of Interactions, which were processed in parallel when possible. This is why Baker is such an interesting library. We have made possible the orchestration of multiple processes for a very low cost. We can see very clearly that the green beans were created at the same time the batter was created. We can then see that we stop seeing parallelism, but that is only because the batter and the green beans have been mixed together and all we have to do is to fry the resulting mix. In other words, our cook finally had the help of a least another cook which made this parallel processing possible.

4.8. Bonus fail test

Because error handling is so important, Baker of course has a way to do this. This is the unit test that simulates that fail:

Bonus fail test

If we look closely, we see that instead of using fryPodsInstance, we are using fryPodsInstanceFail. This is the interactionInstant that throws the Exception:

fryPodsInstanceFail

We have already looked at the function withDefaultFailureStrategy and saw how this works. Let’s have a look at our resulting digrah:

As we can see it is very easy to visualize in our digraph when exceptions occur. It is also easy to establish its business logic.

5. Conclusion

In our study, we have seen how to create Futures of a baking process leading to completion or the next received Event. We have seen how to create and understand Events, Interactions, and Ingredients and how they work together. We have looked at code examples and with them have clear examples on how to create basic architectures.

Using Graphviz, we have seen how baker also allows the communication between developers and business to flow much more easily.

As we have seen, the Baker library is an option to consider if we want to implement orchestrations and multithreaded processes. I’m not saying that its the right option. In my opinion it’s just another one we can consider.

Here is our full recipe diagram:

Complete Peixinhos da Horta Recipe

I have placed all the source code of this application in GitHub

I hope that you have enjoyed this article as much as I enjoyed writing it.

I’d love to hear your thoughts on it, so please leave your comments below.

Thanks in advance for your help and thank you for reading!

6. References