Scripting projects
Published: 2020-09-23 08:01:57

---
title: Scripting projects
published: true
date: 2020-09-22 22:11:00 UTC
tags: 
canonical_url: https://tomandrews.co.uk/blog/2020-09-19-scripting-projects/
---

Over my career the move from monolithic applications to micro-services has come with a number ofbenefits. In my experience some of the best features are that they can reduce overall complexity,enforce design discipline, scale much better to larger applications (and teams) and allow for moreappropriate technical choices. One of the problems they bring is that the developer’s environmentcan get a lot more complex. Each service you require to run the app is an additional bit ofcomplexity as each service needs certain dependencies installing, needs the correct environmentvariables, the correct branch etc. Then once you’ve got them running you need to be able to accesseach one so you can make the necessary change for your work - you might even need to be able makechanges to multiple services ones in one go.

As a new developer working on one of these applications can be a bit of a nightmare. You often needa decent amount of the services running every time you want to start work. You need the servicesstarted, you might need the tests running as well and then you’ll need a terminal in the rightdirectory as well. For years I’ve been doing this all manually - log on in the morning and spenda couple of minutes getting everything started or if its a specific laptop just for this projectthen hope everything survived the wake from sleep and hope I don’t need to restart it ever! As youcan see this was never sustainable especially when I started working on a mature project that pushedthis much further than I have seen before.

For this mature project to run the public facing website required a dozen services and a multi-dayset up process using internal tools to import the necessary data. This was also running on my laptopso when I wanted to stop working on the project and work on home projects in the evening I needed toable to stop all of these services and then start them all when I started work the next morning.

To solve this problem I chatted firstly with the existing developers and they were using acombination of [iterm](https://www.iterm2.com/) and [screen](https://www.gnu.org/software/screen/screen.html).All the solutions were very custom to their individual developer and contained many caveats so Icouldn’t take them wholesale and just get going. So I needed to build my own. From a technology the`iterm` solution was not ideal, the scripting seemed very complex so might work for the currentproject but would need a custom solution for each project. `screen` looked much better but I havebeen listening to a lot of [podcasts from people in the ruby community](https://www.bikeshed.fm/)who have heavily been advocating [tmux](https://github.com/tmux/tmux/wiki) which I have noodled within the past (for adding tabs to [alacritty](https://github.com/alacritty/alacritty)). So if I’mgoing to have to learn a tool I think its best I use the one I am starting from a better positionespecially as I didn’t have a lot of time to do this.

So I ended creating a pretty small but useful script for the project that would start all theservices and bring them up in their own tabs and two panes for each application - one running it theother just in the correct directory for `git` commands, running tests etc. There is also a killcommand to bring it all back down again so that there is nothing left running when I have finishedfor the day. This solution has worked well enough and I’ve really enjoyed living without any `iterm`tabs and just living in `tmux`. So much so that I’ve also started creating similar scripts for allmy projects. Even if they don’t require multiple services, just having a nice consistent way ofgoing into the project and running a single command its all up and running is really nice. Thisspace needs a lot of work there are a few things that don’t work well _yet_.

1. Starting a dozen or more services starts them ALL at the same time which means that your CPU isoverwhelmed for those first couple of minutes
2. A proper way of defining shutdown commands for each of the services rather than just killingeverything seems necessary
3. Some sort of priority order would also be nice but I would hope most services should be prettyresilient.
4. Integration of existing tooling like [docker compose](https://docs.docker.com/compose/) etcwould be nice so that you don’t have to define too much for each app.
5. Allowing a user to customise what additional things every app gets or defining some sort ofstandard layout would be nice as people work in different ways and this is defining the servicesand how to run the tools not forcing a workflow.

Here are my current versions

- The one that started it all [dp-start](https://github.com/tgandrews/dp-start)
- A simple version for an in progress dynamodb ORM called [dynamodb2](https://github.com/tgandrews/dynamodb2/blob/rewrite/start.sh)