Let's create microservices system from scratch - part 3
Tags: beginners, microservices, architecture
Published: 2021-02-16 16:11:19

In the [last article](https://dev.to/digitalcrafting/let-s-create-microservices-system-from-scratch-part-2-5eoh) we decided that the most important thing while developing microservices system is to have a good way of managing and running them. 

After that, the first feature I wanted to implement was to create proxy and service discovery mechanism. How would that work ?

1. The first to start should be *discovery-service* which will hold the list of available services and their location,
2. After *discovery* is started, all other services should start and register themselves with it using REST Api,
3. Requests to the services will go through *api-gateway* which will query *discovery* for the location of the service and then proxy the requests.

Sounds simple, and the actual implementation is, but there are a few things that needed to be done before I could finish it: 
1. Create Rest API standard to follow,
2. Control startup order,
3. Tell each service where it is,
4. Connect services to each other,
5. Be able to test it, which is the reason we need to #6,
6. Refactor the CLI

Let's walk through them.

# REST API standard
For our proxy to work, it needs to be able to somehow connect the request to a specific service. We should also consider the versioning of our services. What I came up with for *root* URI is this:
`GET /api/<service-name>/<version>`
In example:
`GET /api/user/v1`

Which is pretty straightforward, the *gateway* can just split request URI by `/` and use the second value to determine the target service. 

As for versioning there are actually few methods to do this:
1. through URI path (that's what I did)
2. through header of a request
3. through query parameter
4. through content negotiation

While #2 is probably *more* correct from a RESTful point of view, I feel that #1 is more natural and easier to implement. As for the others, I didn't even consider them as they seem weird to me. 

This is by no means the complete standard, just something to let me finish current feature.

# Controlling startup order
While this feature is available from `docker-compose.yml` configuration it doesn't work as expected because it waits for the *container* to start and **not** the service. 
Pretty neat solution is to use [bash](https://www.gnu.org/software/bash/) script and [curl](https://curl.se/) to determine whether the *discovery-service* is up or not. If yes, then we can start other services.

The *discovery-service* URL will be different on *localhost* and in *docker-compose* so, for now, I created 2 different scripts with different URLs (yeah, I know, I could use one and pass parameters, but the problems were piling up, so I just used what works for now, I will refactor it later) which will be copied to a correct place by CLI when building the project using:
`theonecli (local|docker) build`

The script itself is rather simple:
``` bash
serviceUp=false

for (( i=0; i<10; i++ ))
do
  if $(curl --output /dev/null --silent --head --fail http://the-one-discovery:8081/discovery/v1/)
  then
    serviceUp=true
    break
  else
    echo "Service not up - try $i"
  fi
  sleep 2
done

if [ $serviceUp == true ]
then
  echo "Service is up - starting command"
  exec $@
else
  echo "Service is unavailable"
fi
```

And the example usage can be look up, for example in the [user service Dockerfile](https://github.com/DigitalCrafting/microservices-the-one-project/blob/master/python/user-service/Dockerfile).

# Services location
This is actually something, that would be done by administrators. Each service should have it's own IP and PORT, which should be passed to them via *config* file. They should also know the location of the *discovery-service*, which will be one of the few that have not dynamically assigned IP, but a static one. I did that by extending the *config* files from [previous article](https://dev.to/digitalcrafting/let-s-create-microservices-system-from-scratch-part-2-5eoh). 

# Interservice communication
This part is actually the easiest one. On *localhost* the services are available, well, on `localhost` so the only think we need to do is to assign different PORT numbers.

When using `docker-compose.yml` it's almost as easy - we just just need  create internal network in the config file:
``` yaml
networks:
  the-one-network:
    driver: bridge
```

and specify this network, as the one, each service belongs to:
``` yaml
the-one-user-service:
    build: ./python/user-service
    container_name: the-one-user-service
    ports:
      - 8083:8083
    networks:
      - the-one-network
```

# Testing the proxy and discovery mechanism
Now, that's where all the hell broke loose when I started developing this feature. As it turns out, testing microservices is the second biggest obstacle right after managing them and there are some solutions to consider but I'll leave that topic for another whole article. Suffice to say, there is no easy way to do this, and the most complete approach would be to start everything you need to and test it then. In my case, I need to start 4 services:
- *discovery-service*
- *api-gateway*
- *user-service*
- *outpost-service*

And check if *user* and *outpost* are correctly registered withing *discovery*, and if the *api-gateway* correctly proxies the request. That's when I realized that my CLI Tool is not good enough to actually do this and I had to refactor it. After that, I simply run:
`curl localhost:8080/api/user/v1`
and checked if the response is what I expected. This is by no means sufficient testing strategy and I will work on that.

# Refactoring the CLI
First of all, I decided to move `docker-compose` functionality to `docker` command in the CLI, meaning, when typing:
`theonecli docker start`
we will actually start `docker-compose.yml` configuration. After killing it, we have to call
`theonecli docker stop`
and there is no option to run individual docker images right now. After that, I also had to:
- fix the build process, so that *wait-for-service.sh* script will be copied into correct directory of each serivice,
- change the Dockerfiles so that the services start in correct order
- fix starting multiple services on *localhost* by spawning new gnome-terminal for each service

After all that, we end up with 2 levels of configuration files:
- project level
- service level

On *project level* we have the *wait-for-service.sh* scripts for each environment and on the *service level* we have service specific configurations, again, for each of our environments. Right now we only have 2 environments and it will probably not change, since it's only personal project and not a commercial one, but we can already see that the amount of configuration needed for running this project is going to bubble up, especially when we add databases and UI application.

# Summary
All of that took me waaay to much time to finish, but, I'm learning about these challenges right now :wink:

This article was more of a log of what happened, [the next one](https://dev.to/digitalcrafting/let-s-create-microservices-system-from-scratch-part-4-4jl3) will be more theoretical, probably about testing options for microservices. 

I hope this article will be of some help, and will let you avoid some problems with your projects. Current version is available on [my github](https://github.com/DigitalCrafting/microservices-the-one-project), feel free to download it and try it out.