Build own Kubernetes - Node commands
Tags: kubernetes, devops, go, programming
Published: 2022-10-12 16:33:42

In the previous post, we built the node image and wrote the agent, all done by executing commands on the shell with docker build and run, but the whole process should be automated by our program, this is what this article will focus on, on creating node on command, listing them and deleting them.

---
Before we begin I had some minor fixes in the code: rename the command variables in cmd/pod.go because we are going to use the same name in node, change the agent pods routes to pods and not pod as BP in REST, and moved the GenerateNewID function to pkg/util.go. Some big change was in the Dockerfile, I am relying on the environment that is running the commands to have locally the node image built, so I made a Makefile to build the main program, the agent, and the node image.
Makefile:
```Makefile
default: build

build: build-agent build-node-image
	go build -o bin/main main.go

build-node-image: 
	sudo docker build -t own-kube-node .

build-agent: 
	go build -o bin/agent pkg/agent/agent.go
```
As a result, we can rely on the binary for the agent is already built when we build out the image so I removed the build from the Dockerfile:
```Dockerfile
# Dockerfile for node image
FROM ubuntu

WORKDIR /agent

RUN apt-get update && apt-get install -y wget containerd

COPY bin/agent .

EXPOSE 10250

ENTRYPOINT [ "./agent" ]
```

---
Let's start with our node functionality, Like the pod, we have related files to node functionality in `pkg/node`.
We start on `pkg/node/constraints.go` to define some constraints for our work:
```go
package node

import "github.com/jonatan5524/own-kubernetes/pkg/agent/api"

const (
	NODE_NAME          = "node"
	NODE_IMAGE         = "own-kube-node"
	NODE_PORT          = api.PORT + "/tcp"
	NODE_PORT_HOST_IP  = "0.0.0.0"
	MEMORY_LIMIT       = 2.9e+9 // 2900MB
	CPU_LIMIT          = 2
	NODE_HOST_MIN_PORT = 10250
	NODE_HOST_MAX_PORT = 10300
)
```
All the constraints will be used in the following files and all the values are taken from the last article when we ran the docker container node. 
In the `pkg/node.go` we define a Node struct:
```go
type Node struct {
	Id   string
	Port string
}
```
Id - generated id for the node
Port - generate unused port for the node
In this file we also define function for creating a new node using the [docker sdk](https://docs.docker.com/engine/api/sdk/):
```go 
func NewNode(cli *client.Client, ctx context.Context) (*Node, error) {
	exists, err := isNodeImageExists(cli, ctx)
	if err != nil {
		return nil, err
	} else if exists == false {
		return nil, fmt.Errorf("node image: %s not exists locally, need to build the image", NODE_IMAGE)
	}

	id := pkg.GenerateNewID(NODE_NAME)
	config := &container.Config{
		Image: NODE_IMAGE,
	}

	hostConfig := &container.HostConfig{
		PortBindings: nat.PortMap{
			nat.Port(fmt.Sprintf("%s/tcp", api.PORT)): []nat.PortBinding{
				{
					HostIP:   NODE_PORT_HOST_IP,
					HostPort: "0",
				},
			},
		},
		Resources: container.Resources{
			Memory:    MEMORY_LIMIT,
			CPUShares: CPU_LIMIT,
		},
		Privileged: true,
	}

	_, err = cli.ContainerCreate(ctx, config, hostConfig, nil, nil, id)
	if err != nil {
		return nil, err
	}

	return &Node{Id: id}, nil
}
```
We first check if the node image exists, then we generate an id, and create some config for the container:
- node image
- port forwarding from the port inside the container (the agent API) to outside of the container. The HostPort is 0 because Linux assigns a random open port when a service assigns a 0 port.
- memory limit
- CPU limit
- privileged
All the configurations are equivalent to the command we ran in the previous article:
```bash
sudo docker run -it --memory="2900MB" --cpus="2" -p 10250:10250 --privileged --name test --rm containerd_test
```
Then we create the container with the config and generated Id and return a new Node struct.
The `isNodeImageExists` function implementation:
```go
func isNodeImageExists(cli *client.Client, ctx context.Context) (bool, error) {
	images, err := cli.ImageList(ctx, types.ImageListOptions{})
	if err != nil {
		return false, err
	}

	for _, image := range images {
		if strings.Contains(image.RepoTags[0], NODE_IMAGE) {
			return true, nil
		}
	}

	return false, nil
}
```
The function List the existing images, go through each one, and check if its RepoTag contains an `own-kube-node`

Let's move to `pkg/node/service.go`, the first function we will go through will be the `initDockerConnection` which initialized the docker client and context:
```go
func initDockerConnection() (*client.Client, context.Context, error) {
	ctx := context.Background()
	cli, err := client.NewClientWithOpts(client.FromEnv, client.WithAPIVersionNegotiation())
	if err != nil {
		return nil, nil, err
	}

	return cli, ctx, nil
}
```
Next, is the `NewNodeAndRun` function which creates new node and starts the container:
```go
func NewNodeAndRun() (*Node, error) {
	cli, ctx, err := initDockerConnection()
	if err != nil {
		return nil, err
	}
	defer cli.Close()

	node, err := NewNode(cli, ctx)
	if err != nil {
		return nil, err
	}

	if err := cli.ContainerStart(ctx, node.Id, types.ContainerStartOptions{}); err != nil {
		return nil, err
	}

	log.Printf("node created: %s\n", node.Id)
	log.Printf("starting node\n")

	container, err := cli.ContainerInspect(ctx, node.Id)
	if err != nil {
		return nil, err
	}

	// get linux generated port
	node.Port = container.NetworkSettings.Ports[nat.Port(fmt.Sprintf("%s/tcp", api.PORT))][0].HostPort

	log.Printf("node assign port: %s\n", node.Port)

	return node, nil
}
```
As you can see in this function, we are initializing the connection to the docker daemon, creating a new node and container, starting the container, and then because we assign port 0 we need to retrieve the assigned port by the system.

The next function is `ListRunningNodes`:
```go
func ListRunningNodes() ([]string, error) {
	cli, ctx, err := initDockerConnection()
	if err != nil {
		return []string{}, err
	}
	defer cli.Close()

	runningNodes := []string{}

	filter := filters.NewArgs(filters.KeyValuePair{Key: "ancestor", Value: NODE_IMAGE})
	containers, err := cli.ContainerList(ctx, types.ContainerListOptions{Filters: filter})
	if err != nil {
		return runningNodes, err
	}

	for _, container := range containers {
		runningNodes = append(runningNodes, container.Names[0])
	}

	return runningNodes, nil
}
```
The function initializes the connection, calls the `ContainerList` method with the `ancestor` filter which means the image we want to filter, and then runs over each container to create a list of names.

The last function is `KillNode` function which simply stops the container and removes it:
```go
func KillNode(name string) (string, error) {
	cli, ctx, err := initDockerConnection()
	if err != nil {
		return "", err
	}
	defer cli.Close()

	if err := cli.ContainerStop(ctx, name, nil); err != nil {
		return "", err
	}

	removeOptions := types.ContainerRemoveOptions{
		RemoveVolumes: true,
		Force:         true,
	}

	if err := cli.ContainerRemove(ctx, name, removeOptions); err != nil {
		return "", err
	}

	return name, nil
}
```

The last part we need to complete is `cmd/node.go` its structure is the same as `cmd/pod.go`, I'm not going to add this file here as this article is too long, you can see it in the source code.

we can see that everything works on the terminal:
```bash
❯ make
go build -o bin/agent pkg/agent/agent.go
sudo docker build -t own-kube-node .
...
❯ sudo ./bin/main node create
2022/10/12 19:18:26 node created: node-7daf80ec-0a46-4977-952c-66deb81e32f7
2022/10/12 19:18:26 starting node
2022/10/12 19:18:26 node assign port: 49167
❯ sudo docker ps
CONTAINER ID   IMAGE           COMMAND     CREATED         STATUS         PORTS                      NAMES
2066e8e0b933   own-kube-node   "./agent"   5 seconds ago   Up 4 seconds   0.0.0.0:49167->10250/tcp   node-7daf80ec-0a46-4977-952c-66deb81e32f
❯ curl -X POST localhost:49167/pods -H 'Content-Type: application/json' -d '{"name": "redis", "image registry": "docker.io/library/redis:alpine"}'
{"image registry":"docker.io/library/redis:alpine","name":"redis-a1cfce57-5db9-4894-a8cf-c277444e1b0c"}
```
---
And that's it, we have a node created by command using an API call that can also create for us a pod inside the node!

In the next article, we will cover some Linux network and get working communication with our pods and nodes.

As always, the source code can be found [here](https://github.com/jonatan5524/own-kubernetes), the changes were in `pkg/node`, `Dockerfile`, `Makefile`, `cmd/node`.