A dig into elixir’s umbrella projects
By: Siddhant Singh
Tags: Phoenix, Backend Development, Elixir

A dig into elixir’s umbrella projects

Sometimes our project may get really really big and the elixir’s mix build tool allows you to breakdown your code into multiple apps. With the aim to make the project more manageable as it grows, elixir lets you define one root project with multiple sub-applications underneath which together make up the final solution to your project.

How Do I Create One?

To create an umbrella app you just pass an umbrella flag to the mixed call and that will create you a root project which looks something like this.

Now we need to define some subprojects. To do that, we navigate to the apps folder and then you can create elixir projects as you would normally. At this point, it’s worth noting that you can pass all the usual flags when you create your mix projects here. Let me show you how it’s done. So follow my command:

You can either run all the tests and all the subprojects from the root of your project or you can navigate to a particular subproject you’re interested in and run the tests just for that project.

Dependencies

Sometimes we may have dependencies between our sub apps. Suppose we want one sub app use code that’s in another supplication. Umbrella projects make it very easy to have child dependencies as another sub app. To define dependencies within the umbrella application we could just use the umbrella true flag and elixir knows

where to look for that dependency within that project. This is useful if you don’t want to host a library on git or hex or something like that.

Why an Umbrella?

We wanted to check if umbrella applications are a sleek solution for microservices. This is what I understand about learning an umbrella project which I’m going to show you now.

So the first lesson is around breaking up your project

At the beginning of the project, it’s really really hard to know how best to split up your architecture and it’s even more difficult to get it right for the first time and so how do you know the boundary of each sub-application?

Suppose you have a web application so probably the most naive way to split the architecture would have been to have a sub-app with just a phoenix layer and a core sub app with all our business logic.

This way our web layer would be very thin it would just have our controllers routing and maybe authentication manipulating headers and new requests and so on. On the other hand, you could decide to split your responsibility perhaps if your domain a bit more well known you can already identify different services.

2. Elixir Versions

So in your umbrella project, you can define a different version of elixir per sub-project if you want now when you define a version of dependency you’re actually stating a requirement. You can say this is the version that I’m willing to work against in this project and under the hood elixir will call a function called version which will check the requirement that can actually be satisfied.

3. Circular Dependencies

You’ve got to careful with circular dependencies where you have two sub apps that are dependent on each other. In other languages, I’ve used it’s not always been that obvious if I’ve had a cycle. Teams have often restored to using like code analysis tools or it’s only in investigating a weird bug that people have realized oh actually there’s a cycle here and as developers, we want fast feedback that makes us effective. Elixir will give you very fast feedback for this so under your umbrella if you end up with a cycle you basically can’t do anything because mix tells you straight away that you have to break the cycle before you can go on.

If you do end up in this situation it’s probably best to check if you’ve split your services to a two smaller level. Maybe the two things are dependent on each other should actually be combined and they may be related

4. Transient dependency clashes

So under our umbrella, we have a mix file per sub-application where you can define the dependencies that each project needs.

Suppose you’re using some dependency for your sub_app_a and that same dependency is listed in other sub apps so it will give you an exception where it will tell you that the version of a dependency is clashing between sub-application. But thankfully elixir has a way that you can handle this you have to explicitly override the clashing dependency and you do this by using override true flag and that tells the project that it should use the version stated here and ignore any other version. Isn’t it cool? okay, let’s go to the next part.

5. Namespace Clashes

Suppose you’re using sub_app_a and it has some search service but sub_app_b also has the same search service. When I try to run individually using mix it will run but when I try using root project it will give me an error. So to solve this problem we can use the namespace.

6. Test Data Cleanup

The next battle is around cleaning up test data in the database. So suppose we have a few services in our project which are interacting with each other either directly or through dependencies. We’ve got the database service which was directly accessed directly persist and fetch from our Postgres database. Now suppose you have a post_service which had the database service as a dependency and you have a test for post_service which ultimately would put data in the database. You have another service called search-service which has a dependency on post_service which also has a dependency on the database service so you have tests that were also bidding data in the database.

So in the end, you want your database to clean after each of your tests. So after a bit of research, you could use the sandbox the ecto sandbox to create transactional tests. So each test would start a transaction and then at the end of the test the transactions would roll back thus emptying the database.

That was unfairly trivial to set up to have to state in your test config that you want to use the sandbox pool

and you have to get an explicit connection in the tests that will interact with the database so you can do this in a setup block

So if you run into a problem where more then one process is not able to access your database. That’s where you have to use something called Shared Mode.

You can run the shared box in a shared mode so that more than one process can use the database. You just have to update your test helper and where you state that you want to use shared mode and then the test round cleaned up after themselves.

That’s it for this blog. Will continue more about Umbrella Project on to next blog.

Conclusion

If you made it this far, congratulations! 🎉. Now you can tell me is the Umbrella Project a good fit for MicroServices?

Also, give me feedback if you find something more interesting about umbrella projects.

Thank you!!