Workers pool
Tags: spacelift, aws, configuration, tutorial
Published: 2023-03-04 20:28:04

---
title: Workers pool
series: spacelift
published: true
description: Create workers to use with Spacelift
tags: #spacelift #aws #configuration #tutorial
cover_image: https://www.pawelpiwosz.net/assets/covers/spacelift-cover.png
---

In this episode we will create workers pool, to run workloads on "our" servers. This process requires a few steps, let's see how quickly and easily we can add new worker to the app.

During this episode I will heavily use Spacelift documentation.

# Create private key

First step is to create the key which wil be used to authenticate worker with application.

I'll use WSL2 for it.

```bash
openssl req -new -newkey rsa:4096 -nodes -keyout spacelift.key -out spacelift.csr
```

These files we need to setup our pool.

# Create workers pool

Navigate to `Worker pools` in the Spacelift app. You should see something similar like on the screen below.

![Create workers pool](https://www.pawelpiwosz.net/assets/spacelift-intro/spacelift-intro-23.jpg)

Click `New worker pool`.

![Create workers pool](https://www.pawelpiwosz.net/assets/spacelift-intro/spacelift-intro-24.jpg)

Add the `csr` file generated in previous step. Also, I changed space to `root`. Yes, you can have pools set by space.

Please remember, that one worker is able to run one execution. If this is good or bad, I leave it to you, however it doesn't sound very appealing if I will use virtual machines, am I right? But! We can use this VM as containers platform and it starts to look a little bit better.

Ok, we can create our pool.

My pool is created and the file with token is downloaded automatically. Well, I am not fully sure here. I almost missed this download, my browser didn't show anything. I knew there will be a token, so I was careful, but I think it will be nice if Spacelift improve the UX here.

# Create worker

I decided to go easy way and I run the EC2 on AWS using Spacelift image.

![Select Spacelift AMI on AWS](https://www.pawelpiwosz.net/assets/spacelift-intro/spacelift-intro-25.jpg)

I've changed the default volume only. The type I selected is gp3 and I increased the size to 20G.

The provided AMI is not completed. I mean, it contains everything what we need to *download and configure* the launcher. Let's do it then!

First, we need to encode the key we generated earlier

```bash
cat spacelift.key | base64 -w 0
```

Now we can login to the newly created machine and do

```bash
sudo -i
wget https://downloads.spacelift.io/spacelift-launcher
install spacelift-launcher /usr/bin/spacelift-launcher
chmod +x /usr/bin/spacelift-launcher
export SPACELIFT_TOKEN="<token_from_file>"
export SPACELIFT_POOL_PRIVATE_KEY="key_from_encode_operation>"
/usr/bin/spacelift-launcher
```

What I do not like - to make the activity quickly I had to use `root` account.

> What we've just done is very unstable and should be treated as "let me check it" way only. When we do it properly, all these should be automated and run without any access from human side. In fact... Spacelift [provides a repository with IaC to provision the infrastructure](https://github.com/spacelift-io/terraform-aws-spacelift-workerpool-on-ec2)!

Let's see our workers pool

![Copnfigured workers pool](https://www.pawelpiwosz.net/assets/spacelift-intro/spacelift-intro-26.jpg)

Yep, we are done!

# The 'hmmm' moment

I need to dig deeper into the workers pool. I saw couple of things which made me think. 

* What will happen if I stop my worker node? 
* Logs for spacelift-launcher are set to `debug` by default (at least, this is what I see after quick check).
* In logs I saw information about S3. And this is something what I have to explore. What S3? Why? Who owns it? I have some ideas, what is behind, but I have to confirm it.

# Takeways

Spacelift allows us to create private pools of workers which can be used for different workloads. Creation is simple and management is quite effective.