Learning Go by examples: part 12 - Deploy Go apps in Go with Pulumi
Tags: go, beginners, pulumi, docker
Published: 2023-08-21 06:43:16

---
title: Learning Go by examples: part 12 - Deploy Go apps in Go with Pulumi
published: true
description: Serie of article in order to learn Golang language by concrete applications as example
series: Learning Go by examples
tags: Golang, beginners, pulumi, docker
cover_image: https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ithvox3k5u8qcxn53e79.png
---

In previous articles we created a [HTTP REST API server](https://dev.to/aurelievache/learning-go-by-examples-part-2-create-an-http-rest-api-server-in-go-1cdm), and another kind of applications and we deployed them manually locally.

Deploying applications manually is cool but today, we will try and use Pulumi to deploy, programmatically, in Go of course ^^, our awesome apps.

Ready?

## Pulumi

![Pulumi](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/wfgg4rdupa8v7nctqw5n.png)

[Pulumi](https://www.pulumi.com/) is an Infrastructure as code (IasC) tool that allows you to build your infrastructures with a programming language. It supports a variety of programming languages: Python, Node.js, Go, Java, .Net...

Like [Terraform](https://www.terraform.io/), Pulumi have an architecture based on providers/plugins. There are official providers (AWS, GCP, Kubernetes, Docker...) but it is possible to create our own providers too.

How Pulumi is working?

Concretely, users defined the desired state in Pulumi programs and Pulumi create the desired resources.

To provision, update or delete infrastructures, Pulumi have an intuitive Command Line Interface (CLI). If you are familiar with Docker Compose CLI and Terraform CLI, you will adopt [Pulumi CLI](https://www.pulumi.com/docs/cli/) too.

Let's install the Pulumi CLI.
In this guide we will install it with brew but you can install in many ways, [follow the installation guide](https://www.pulumi.com/docs/install/). 

```bash
$ brew install pulumi/tap/pulumi
```

Let's check the CLI is correctly installed locally:

```bash
$ pulumi version
v3.77.1
```

## Pre-requisites

In this article, we will use the feature to save locally the Pulumi state. If you want to save it in the Cloud, don't use the `--local` flag in `pulumi login` command, instead create an account on [Pulumi](https://www.pulumi.com/) and retrieve an [access token](https://app.pulumi.com/account/tokens).

On my side I am using a free Pulumi account and save locally the state.

## What do we want?

An Infrastructure as Code (IaC) tool is originally used to deploy infrastructures in Cloud providers but we can also handle (deploy, change and destroy) our Go apps and that's what we will do in this article.

We will deploy two applications:
* our cute [Gophers API](https://github.com/scraly/gophers-api) that list existing Gophers, display information, create, update and delete a Gopher
* a Node.js HMI [Gophers API Watcher](https://github.com/LostInBrittany/gophers-api-watcher) that displays our cute Gophers (who don't love UI? ^^)

As you maybe know, I like to run apps in containers so we will run our apps in containers.

## Pre-requisites - Create a Docker image from a Go app

We will use the [Docker provider](https://www.pulumi.com/registry/packages/docker/) in Pulumi to deploy and run our applications so before we need to have Docker images with our apps.

Let's do it!

Uh... come on Aurélie ... You want really to explain deployment with Pulumi and Docker images creation in only one article?

Yes, with `docker init` command we can generate necessary Docker files and create our images easily without headaches or "marabout tips".

I have already explained it in video:
{% youtube 9CDnTW_uIOM %}

So thanks to the `docker init` command I generated a `Dockerfile`, built (for different platform and architecture), tagged and pushed the image.

Our Gophers API is available on Docker Hub (with several tags, depending on your host platform): https://hub.docker.com/r/scraly/gophers-api

And the Gophers API watcher too: https://hub.docker.com/r/scraly/gophers-api-watcher

## Initialization

First of all, we can create our repository in GitHub (in order to share and open-source it).

For that, I logged in [GitHub website](https://github.com/), clicked on the repositories link, click on "New" green button and then I created a new repository called [“pulumi-gophers”](https://github.com/scraly/pulumi-gophers/).

Now, in your local computer, `git clone` this new repository where you want:

```bash
$ git clone https://github.com/scraly/pulumi-gophers.git
$ cd pulumi-gophers
```

Login (the state will be saved locally):
```bash
$ pulumi login --local

Logged in to scraly-pulumigophers-o1kkpsmgr3k as gitpod (file://~)
```

Initialize our project:

```bash
$ pulumi new go --force

This command will walk you through creating a new Pulumi project.

Enter a value or leave blank to accept the (default), and press <ENTER>.
Press ^C at any time to quit.

project name: (pulumi-gophers) 
project description: (A minimal Go Pulumi program) 
Created project 'pulumi-gophers'

Please enter your desired stack name.
To create a stack in an organization, use the format <org-name>/<stack-name> (e.g. `acmecorp/dev`).
stack name: (gophers) 
Created stack 'gophers'

Installing dependencies...

go: downloading github.com/pulumi/pulumi/sdk/v3 v3.60.1
go: downloading golang.org/x/net v0.7.0
...
go: downloading github.com/kr/text v0.2.0
Finished installing dependencies

Your new project is ready to go! 

To perform an initial deployment, run `pulumi up`
```

The command create a `gophers` stack and the code organization of your project:

```bash
$ tree
.
├── go.mod
├── go.sum
├── main.go
├── Pulumi.yaml
└── README.md
```

## Create our application (Pulumi Go program)

Our application will:
* retrieve `Gophers API` Docker image
* retrieve `Gophers API Watcher` Docker image
* create a Docker network (thanks to that our containers should communicate with each other)
* create a `gophers-api` container and run it
* create a `gophers-api-watcher` containr and run it

For that, we will use the Pulumi official Docker provider.

Let's install Pulumi SDK and Pulumi Docker provider in order to use it in our code:

```
$ go get github.com/pulumi/pulumi-docker/sdk/v3@v3.6.1
$ go get github.com/pulumi/pulumi/sdk/v3@v3.44.2
```

Good, now we can create a `main.go` file and copy/paste the following code into it.

Go code is organized into packages. So, first, we initialize the package, called `main`, and all dependencies/libraries we need to import and use in our main file:

```go
package main

import (
	"fmt"

	"github.com/pulumi/pulumi-docker/sdk/v3/go/docker"
	"github.com/pulumi/pulumi/sdk/v3/go/pulumi"
	"github.com/pulumi/pulumi/sdk/v3/go/pulumi/config"
)
```

With the imports added, you can start creating the `main()` function that contains the intelligence of our app:

```go
func main() {
	pulumi.Run(func(ctx *pulumi.Context) error {
```

When you will execute `pulumi up` command to deploy our applications, Pulumi will run all the code we will wrote in the `pulumi.Run(func(ctx *pulumi.Context) error {` block code.

Let's write our program.

First, we define the configuration: 

```go

		//Configuration
		protocol := "http://"
		//tag := "latest" //for linux/arm64
		tag := "linux-amd64" //if you run this program in a linux/amd64 arch, on GitPod for example ;-)

		cfg := config.New(ctx, "")
		gophersAPIPort := cfg.RequireFloat64("gophersAPIPort")
		gophersAPIWatcherPort := cfg.RequireFloat64("gophersAPIWatcherPort")
```

Then, we pull the `Gophers API` Docker image from the Docker Hub with the tag you defined in the configuration:

```go
		// Pull the Gophers API image
		gophersAPIImageName := "gophers-api"
		gophersAPIImage, err := docker.NewRemoteImage(ctx, fmt.Sprintf("%v-image", gophersAPIImageName), &docker.RemoteImageArgs{
			Name: pulumi.String("scraly/" + gophersAPIImageName + ":" + tag),
		})
		if err != nil {
			return err
		}
		ctx.Export("gophersAPIDockerImage", gophersAPIImage.Name)
```

Then, we pull the `Gophers API Watcher` Docker image from the Docker Hub with the tag you defined in the configuration:

```go
		// Pull the Gophers API Watcher (frontend/UI) image
		gophersAPIWatcherImageName := "gophers-api-watcher"
		gophersAPIWatcherImage, err := docker.NewRemoteImage(ctx, fmt.Sprintf("%v-image", gophersAPIWatcherImageName), &docker.RemoteImageArgs{
			Name: pulumi.String("scraly/" + gophersAPIWatcherImageName + ":" + tag),
		})
		if err != nil {
			return err
		}
		ctx.Export("gophersAPIWatcherDockerImage", gophersAPIWatcherImage.Name)
```

Our containers will need to connect to each other, so we will need to create a Docker Network:

```go
		// Create a Docker network
		network, err := docker.NewNetwork(ctx, "network", &docker.NetworkArgs{
			Name: pulumi.String(fmt.Sprintf("services-%v", ctx.Stack())),
		})
		if err != nil {
			return err
		}
		ctx.Export("containerNetwork", network.Name)
```

Create the Gophers API container:

```go

		// Create the Gophers API container
		_, err = docker.NewContainer(ctx, "gophers-api", &docker.ContainerArgs{
			Name:  pulumi.String(fmt.Sprintf("gophers-api-%v", ctx.Stack())),
			Image: gophersAPIImage.RepoDigest,
			Ports: &docker.ContainerPortArray{
				&docker.ContainerPortArgs{
					Internal: pulumi.Int(gophersAPIPort),
					External: pulumi.Int(gophersAPIPort),
				},
			},
			NetworksAdvanced: &docker.ContainerNetworksAdvancedArray{
				&docker.ContainerNetworksAdvancedArgs{
					Name: network.Name,
					Aliases: pulumi.StringArray{
						pulumi.String(fmt.Sprintf("gophers-api-%v", ctx.Stack())),
					},
				},
			},
		})
		if err != nil {
			return err
		}
```

Create the Gophers API Watcher container:

```go
		// Create the Gophers API Watcher container
		_, err = docker.NewContainer(ctx, "gophers-api-watcher", &docker.ContainerArgs{
			Name:  pulumi.String(fmt.Sprintf("gophers-api-watcher-%v", ctx.Stack())),
			Image: gophersAPIWatcherImage.RepoDigest,
			Ports: &docker.ContainerPortArray{
				&docker.ContainerPortArgs{
					Internal: pulumi.Int(gophersAPIWatcherPort),
					External: pulumi.Int(gophersAPIWatcherPort),
				},
			},
			Envs: pulumi.StringArray{
				pulumi.String(fmt.Sprintf("PORT=%v", gophersAPIWatcherPort)),
				pulumi.String(fmt.Sprintf("HTTP_PROXY=backend-%v:%v", ctx.Stack(), gophersAPIPort)),
				pulumi.String(fmt.Sprintf("PROXY_PROTOCOL=%v", protocol)),
			},
			NetworksAdvanced: &docker.ContainerNetworksAdvancedArray{
				&docker.ContainerNetworksAdvancedArgs{
					Name: network.Name,
					Aliases: pulumi.StringArray{
						pulumi.String(fmt.Sprintf("gophers-api-watcher-%v", ctx.Stack())),
					},
				},
			},
		})
		if err != nil {
			return err
		}

		return nil
	})
}
```

And that's it! We defined everything we want in our infrastructures: 2 applications running in containers thanks to Pulumi.

### The configuration

As you saw, we didn't hardcode the apps port number:

```go
        cfg := config.New(ctx, "")
		gophersAPIPort := cfg.RequireFloat64("gophersAPIPort")
		gophersAPIWatcherPort := cfg.RequireFloat64("gophersAPIWatcherPort")
```

Instead we define them as config parameters. The program will get them in a `Pulumi.<your-stack-name>.yaml` file.

To define them and generate the config file, execute the following commands:

```bash
$ pulumi config set gophersAPIPort 8080
$ pulumi config set gophersAPIWatcherPort 8000
```

After editing our `main.go` file and defining our configuration fields and values, it's time to ask to Go to download and install all the Go providers and dependencies:

```bash
$ go mod tidy
```

## Let's deploy our apps

Now we can deploy our apps, to do that just execute the `pulumi up` comand.
This will display the plan/the preview of the desireed state. A prompt will ask you to choose the stack (`dev` by default) and to confirm of you want to perform/apply the changes.

```
$ pulumi up
Please choose a stack, or create a new one: gophers
Previewing update (gophers)

View in Browser (Ctrl+O): https://app.pulumi.com/scraly/pulumi-gophers/gophers/previews/cb2a49a5-e17e-4e58-9525-a1931b214b23

     Type                         Name                       Plan       
 +   pulumi:pulumi:Stack          pulumi-gophers-gophers     create     
 +   ├─ docker:index:RemoteImage  gophers-api-watcher-image  create     
 +   ├─ docker:index:RemoteImage  gophers-api-image          create     
 +   ├─ docker:index:Network      network                    create     
 +   ├─ docker:index:Container    gophers-api-watcher        create     
 +   └─ docker:index:Container    gophers-api                create     


Outputs:
    containerNetwork            : "services-gophers"
    gophersAPIDockerImage       : "scraly/gophers-api:linux-amd64"
    gophersAPIWatcherDockerImage: "scraly/gophers-api-watcher:linux-amd64"

Resources:
    + 6 to create

Do you want to perform this update? yes
Updating (gophers)

View in Browser (Ctrl+O): https://app.pulumi.com/scraly/pulumi-gophers/gophers/updates/3

     Type                         Name                       Status              
 +   pulumi:pulumi:Stack          pulumi-gophers-gophers     created (9s)        
 +   ├─ docker:index:Network      network                    created (2s)        
 +   ├─ docker:index:RemoteImage  gophers-api-watcher-image  created (8s)        
 +   ├─ docker:index:RemoteImage  gophers-api-image          created (5s)        
 +   ├─ docker:index:Container    gophers-api                created (0.97s)     
 +   └─ docker:index:Container    gophers-api-watcher        created (1s)        


Outputs:
    containerNetwork            : "services-gophers"
    gophersAPIDockerImage       : "scraly/gophers-api:linux-amd64"
    gophersAPIWatcherDockerImage: "scraly/gophers-api-watcher:linux-amd64"

Resources:
    + 6 created

Duration: 13s
```

We can check if images have been successfully pulled from the registry:

```bash
$ docker image ls
REPOSITORY                   TAG           IMAGE ID       CREATED        SIZE
scraly/gophers-api-watcher   linux-amd64   ee8c626fdeab   3 hours ago    288MB
scraly/gophers-api           linux-amd64   83e5cf52694c   3 hours ago    22.6MB
scraly/gophers-api-watcher   latest        4de2009ea463   23 hours ago   286MB
scraly/gophers-api           latest        0e32fa8f8e18   4 months ago   22.3MB
```

And check if containers are running as well:

```bash
 $ docker container ls
CONTAINER ID   IMAGE                        COMMAND                  CREATED         STATUS         PORTS                    NAMES
ba0b23d0af11   scraly/gophers-api-watcher   "docker-entrypoint.s…"   9 minutes ago   Up 9 minutes   0.0.0.0:8000->8000/tcp   gophers-api-watcher-gophers
7b6d448f3d01   scraly/gophers-api           "/bin/server"            9 minutes ago   Up 9 minutes   0.0.0.0:8080->8080/tcp   gophers-api-gophers
```

## Let's test it locally

Ahh I like this moment where we will be able to test what we have created!

First, let's test our API. 

```bash
$ curl localhost:8080/gophers
[{"displayname":"5th Element","name":"5th-element","url":"https://raw.githubusercontent.com/scraly/gophers/main/5th-element.png"}]
```

Cool, and now let's display our cute HMI, for that go to `localhost:8000/demo` with your favorite browser:

![Gophers API Watcher](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/3rdzb0kk73fqbrawrsks.png)

Awesome, it's working (and our Gopher is so cute ^^)!

Now, if you want, you can now play with the Gophers API (add and edit existing gophers) and watch them appears in the cute HMI done by [Horacio Gonzalez](https://twitter.com/LostInBrittany/) 🙂.

## Cleanup

To easily destroy created resources, you can use `pulumi destroy` command.

```bash
$ pulumi destroy
Please choose a stack: gophers
Previewing destroy (gophers)

View in Browser (Ctrl+O): https://app.pulumi.com/scraly/pulumi-gophers/dev/previews/2344bad2-xxxx-xxxx-xxxx-846c828f7102

     Type                         Name                       Plan       
 -   pulumi:pulumi:Stack          pulumi-gophers-gophers     delete     
 -   ├─ docker:index:Container    gophers-api                delete     
 -   ├─ docker:index:Container    gophers-api-watcher        delete     
 -   ├─ docker:index:Network      network                    delete     
 -   ├─ docker:index:RemoteImage  gophers-api-image          delete     
 -   └─ docker:index:RemoteImage  gophers-api-watcher-image  delete     


Outputs:
  - containerNetwork            : "services-gophers"
  - gophersAPIDockerImage       : "scraly/gophers-api:linux-amd64"
  - gophersAPIWatcherDockerImage: "scraly/gophers-api-watcher:linux-amd64"

Resources:
    - 6 to delete

Do you want to perform this destroy? yes
Destroying (gophers)

View in Browser (Ctrl+O): https://app.pulumi.com/scraly/pulumi-gophers/gophers/updates/2

     Type                         Name                       Status              
 -   pulumi:pulumi:Stack          pulumi-gophers-gophers     deleted             
 -   ├─ docker:index:Container    gophers-api-watcher        deleted (0.26s)     
 -   ├─ docker:index:Container    gophers-api                deleted (0.51s)     
 -   ├─ docker:index:RemoteImage  gophers-api-watcher-image  deleted (0.64s)     
 -   ├─ docker:index:Network      network                    deleted (2s)        
 -   └─ docker:index:RemoteImage  gophers-api-image          deleted (0.86s)     


Outputs:
  - containerNetwork            : "services-gophers"
  - gophersAPIDockerImage       : "scraly/gophers-api:linux-amd64"
  - gophersAPIWatcherDockerImage: "scraly/gophers-api-watcher:linux-amd64"

Resources:
    - 6 deleted

Duration: 6s

The resources in the stack have been deleted, but the history and configuration associated with the stack are still maintained. 
If you want to remove the stack completely, run `pulumi stack rm gophers`.
```

## Known issues

### creating failed

The first time you will "play" with Pulumi, specially the `Docker provider`, you can face to the issue I had:

```
Do you want to perform this update? yes
Updating (gophers)

View in Browser (Ctrl+O): https://app.pulumi.com/scraly/pulumi-gophers/gophers/updates/1

     Type                         Name                       Status              
 +   pulumi:pulumi:Stack          pulumi-gophers-gophers     created (9s)        
 +   ├─ docker:index:Network      network                    created (2s)        
 +   ├─ docker:index:RemoteImage  gophers-api-watcher-image  created (8s)        
 +   ├─ docker:index:RemoteImage  gophers-api-image          created (5s)        
 +   ├─ docker:index:Container    gophers-api                ** creating failed**    1 error     
```

`creating failed`... OK... 

I admit that when you have this problem, it can be disappointing.

A way, to find what is happening is to add the `--debug` flag to the `pulumi up` command.

Another way is to try, simply, to run the container locally:

```bash
$ docker run scraly/gophers-api:latest 
WARNING: The requested image's platform (linux/arm64/v8) does not match the detected host platform (linux/amd64/v3) and no specific platform was requested
exec /usr/local/bin/docker-entrypoint.sh: exec format error
```

As you can see, I built and pushed my image into another platform/architecture (mac m1) than the server/machine I'm running it on (ubuntu).

![fire gopher](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/5y93l3aip8n41fqrxyh0.png)

So the solution was to use, on my mac, `docker buildx build` command with the `platform` flag, to build and push the image with the good platform, like this:

```bash
$ docker buildx build --platform linux/amd64 -t scraly/gophers-api:linux-amd64 . --push
```

## What am I thinking about Pulumi

Before to conclude this blog post I need to tell you my thought about Pulumi.

I am doing a lot of Terraform since 2017, I trained my ex colleagues, used it in many projects, for several Cloud providers (AWS, OVHcloud...), and even maintaning a Terraform provider daily. So I admit I thought I don't need Pulumi because I know and uses already an IaC tool.

As you know, I am curious, so I wanted, since several years to test it but without time to do it. I wanted to try it for a concrete need and I found one: deploying a Kubernetes cluster & a node pool (and other resources) on OVHcloud.

The journey was not easy, I had several troubles (mainly the [tf2pulumi](https://www.pulumi.com/tf2pulumi/) converter and then the existing Pulumi OVH community provider). I lost several hours and days, but didn't give up and thanks to Engin we finally found solutions.

I think the experience with Pulumi you can have, will depends on the provider(s) you will use.

If you already know Terraform, you will easily understand the Pulumi concepts. If you are a developer, I think it can be easier for you to write your infrastructure in your favorite language, instead of write it in `HCL` (Hashicorp Configuration Language).
But, not all the company who have a Terraform provider have a Pulumi provider yet, it's a fact.

On my side the journey was not easy, but I am stubborn and hopefully [Engin Diri](https://twitter.com/_ediri) helped me. Without his help, I would probably have given up or postponed my umpteenth attempt for several months.

## Conclusion

As you have seen in this article and previous articles, it's possible to create applications in Go and even now deploying them in Go too with Pulumi.

All the code of our app is available in: https://github.com/scraly/pulumi-gophers

So if you want, now, you can use Pulumi, or another tool, to deploy your apps and automatize this task.
For me, there is no magic wand, no magic tool and technology  better then other, depending on your team, the context, the need, use the technology, the tool or the language you want :-).

In the following articles we will create others kind/types of applications in Go.

Hope you'll like it.