Building a Fargate API Server with Go, Gin, Docker, and AWS Copilot
Tags: go, aws, tutorial, docker
Published: 2023-10-25 16:16:42


![GPT-Powered Translator with LangChain Hosted on Fargate](https://www.doctorew.com/shuttlebay/Screen-Shot-2023-10-25-11-48-13.80.png)

Hello fellow tech enthusiasts!

As a self-taught engineer with over two decades of experience, starting back in the days the browser wars, my journey has been driven by curiosity, innovation, and a passion for understanding the why and how technology does its "magic". 

I chose to use Golang over Python not just because it's fun, but because it presented an opportunity to demystify its "magic" and deepen my understanding (even though I've been using it since it was in beta). This became especially relevant during a conversation with a respected leader who posed a seemingly simple question about Go's struct vs. interface. As a self-taught engineer who's gone toe-to-toe with imposter syndrome, vocabulary questions like this give me sweaty palms. But I realized that this was a great opportunity to break down these concepts and really understand them at a deeper level. 

(PS A struct is a composite data type that groups together variables under a single name, while an interface is a collection of method signatures that a type must implement. Understanding the difference between these two concepts is crucial for writing clean and efficient Go code.) 

Langchain was another exciting venture, and I am thrilled that Harrison Chase figured this one out because chaining GPT-2 and GPT-3 calls in Python in 2021 was a pain the keister! 

And then there's AWS Fargate, a game-changer in serverless container platforms. Having used Fargate at NBA Digital and NBATV to serve content to 50 million concurrent users at scale nightly, its efficiency and scalability are undeniable. 

Through all these experiences, my hope is to inspire fellow self-taught engineers, innovators, and anyone with a thirst for knowledge. Let's explore, learn, and innovate together! 

In this blog post, we'll walk through the process of building an API server using Go and the Gin framework, containerizing it with Docker, and deploying it to AWS Fargate using AWS Copilot. We'll also delve into the importance of environment management and testing. Let's get started.

Please refer to the code in this git repo: [https://github.com/doctor-ew/go_skippy_lc](https://github.com/doctor-ew/go_skippy_lc)
## 1. Docker: Containerizing the Go Application

Docker allows us to package our application and its dependencies into a container, ensuring consistent behavior across different environments. Let's break down the Dockerfile from the [go_skippy_lc repository](https://github.com/doctor-ew/go_skippy_lc):

#### 1.1. DockerfileCopy code

`FROM golang:1.17-alpine`

**Explanation:** This line specifies the base image for our Docker container. In this case, you're using the official Go image (`golang`) with version `1.17` based on the lightweight Alpine Linux distribution (`alpine`). This image will have the Go runtime and tools pre-installed.

---

#### 1.2. DockerfileCopy code

`WORKDIR /app`

**Explanation:** This line sets the working directory inside the container to `/app`. All subsequent commands in the Dockerfile (like `COPY` or `RUN`) will be executed in this directory. Essentially, this directory will be the root for our application inside the container.

---

#### 1.3. DockerfileCopy code

`COPY go.mod . COPY go.sum .`

**Explanation:** These lines copy the `go.mod` and `go.sum` files from our local machine (outside the container) to the current directory inside the container (`/app`). These files are essential for Go's module system, ensuring that the correct dependencies are used when building our application.

---

#### 1.4. DockerfileCopy code

`RUN go mod download`

**Explanation:** This line runs the `go mod download` command inside the container. This command fetches all the dependencies listed in `go.mod` and `go.sum`, ensuring they're available in the container for the build process.

---

#### 1.5. DockerfileCopy code

`COPY . .`

**Explanation:** This line copies everything from our current directory on our local machine (i.e., the root of our Go project) to the current directory inside the container (`/app`). This ensures that all our application's source code and other necessary files are available inside the container.

---

#### 1.6. DockerfileCopy code

`RUN go build -o ./out/myapi .`

**Explanation:** This line runs the `go build` command inside the container to compile our Go application. The `-o ./out/myapi` flag specifies the output directory and name for the compiled binary. In this case, the binary will be named `myapi` and will be located in the `/app/out/` directory inside the container.

---

#### 1.7. DockerfileCopy code

`CMD ["./out/myapi"]`

**Explanation:** This line specifies the command that will be executed when the container starts. In this case, it's running the compiled Go application (`./out/myapi`). The `CMD` instruction allows the container to behave like an executable, meaning when you run the container, it will automatically start our Go application.

## 2. Setting Up the Gin Server, Environment Management, and .gitignore

Before diving into the code, it's essential to understand the importance of environment management and the role of `.gitignore` in safeguarding sensitive information.

### 2.1. Environment Files and `.gitignore`

Before we delve into the code, it's essential to mention the importance of environment files and `.gitignore`. Storing sensitive information, like API keys, in environment variables is a best practice. This ensures that these keys are not hard-coded into the application, reducing the risk of accidental exposure. The `.gitignore` file ensures that certain files, like the `.env` containing these keys, are not committed to version control, further safeguarding them.

---

### 2.2. Imports

Here's a brief overview of each import:

- **Standard Library Imports**:
    
    - `context`: Provides a way to carry deadlines, cancellations, and other request-scoped values across API boundaries.
    - `fmt`: Implements formatted I/O functions.
    - `log`: Provides logging capabilities.
    - `net/http`: Provides HTTP client and server implementations.
    - `os`: Provides a platform-independent interface to operating system functionality.
    - `os/signal`: Provides a way to intercept and act upon signals sent to the application.
    - `syscall`: Contains an interface to the low-level operating system primitives.
    - `time`: Provides functionality for measuring and displaying time.
- **Third-party Imports**:
    
    - `github.com/gin-gonic/gin`: Gin is a web framework for building APIs in Go. It's known for its performance and small memory footprint.
    - `github.com/joho/godotenv`: A package to load environment variables from a `.env` file.
    - `github.com/tmc/langchaingo/llms`: A library related to the language chain (from the context, it seems to be related to interfacing with OpenAI).
    - `github.com/tmc/langchaingo/llms/openai`: Specific OpenAI related functionalities for the language chain.
    - `github.com/tmc/langchaingo/schema`: Defines the schema or structure for the messages and responses with OpenAI.

---

### 2.3. Code Structure

#### a. Interface vs. Struct

- **Interface**:
    
    goCopy code
    
    `type Chat interface {     Call(ctx context.Context, messages []schema.ChatMessage, options ...llms.CallOption) (*schema.AIChatMessage, error) }`
    
    **Explanation**: An interface `Chat` is defined, which any type must satisfy if it has a `Call` method with the specified signature. This allows for flexibility and can be used to mock the OpenAI chat for testing or to use different implementations.
    
- **Struct**:
    
    goCopy code
    
    ``type RequestBody struct {     Message string `json:"message"` }``
    
    **Explanation**: This struct `RequestBody` defines the structure of the request body that the `/ask-skippy` endpoint expects. The `json:"message"` tag indicates that when this struct is unmarshaled from a JSON object, the `Message` field corresponds to the `message` key in the JSON.
    

---

### 2.4. Functions

#### a. `srv` Function

This function sets up and starts the Gin server. It defines two endpoints: `/ask-skippy` for POST requests and `/` for GET requests. It also sets up graceful shutdown for the server when it receives an interrupt or termination signal.

#### b. `askSkippy` Function

This function interfaces with the OpenAI chat (or any other implementation that satisfies the `Chat` interface). It sends a system message to set the context for the AI and then sends the user's message. It then waits for a response from the AI and returns it.

#### c. `main` Function

This is the entry point of the application. It loads the environment variables from the `.env` file, retrieves the OpenAI API key, initializes the OpenAI chat, and then starts the server.

---

This breakdown provides a high-level overview of the code's structure and functionality. Each section and function plays a crucial role in setting up the server, interfacing with OpenAI, and serving responses to the user.

## 3. Testing: Ensuring Our Application Works as Expected

Testing is a crucial aspect of software development. It ensures that our application behaves as expected and helps catch issues early in the development process. Let's dive into the testing code for the `askSkippy` function:

### 3.1. Imports

- **Standard Library Imports**:
    
    - `context`: As before, this provides a way to carry request-scoped values.
    - `errors`: Provides functions to manipulate errors.
    - `testing`: The standard Go testing package.
- **Third-party Imports**:
    
    - `github.com/stretchr/testify/mock`: The `testify` library's mocking package. It provides utilities to easily mock interfaces for testing.
    - `github.com/tmc/langchaingo/schema`: Defines the schema or structure for the messages and responses with OpenAI.

---

### 3.2. MockChat Struct

goCopy code

`type MockChat struct { 	mock.Mock }`

**Explanation**: The `MockChat` struct embeds the `mock.Mock` type from the testify library. This allows `MockChat` to have all the methods and functionalities provided by `mock.Mock`, enabling easy setup of expected method calls and their return values.

---

### 3.3. Test Functions

#### a. `TestAskSkippy`

This test function checks the "happy path" scenario where everything works as expected.

- **Mock Setup**:
    
    goCopy code
    
    `mockChat.On("Call", mock.Anything, mock.Anything, mock.Anything).Return(&schema.AIChatMessage{Content: "Mocked response"}, nil)`
    
    Here, we're setting up the expectation that the `Call` method of `mockChat` will be invoked with any arguments (`mock.Anything` is a placeholder that matches any value). When invoked, it should return a mocked AI chat message with the content "Mocked response" and no error.
    
- **Function Call**:
    
    goCopy code
    
    `response, err := askSkippy(context.Background(), mockChat, "English", "Test message for Skippy")`
    
    We then call the `askSkippy` function with our mock chat and check the response.
    
- **Assertions**: The test checks two things:
    
    1. That there's no error.
    2. That the response matches the expected "Mocked response".

#### b. `TestAskSkippy_Error`

This test function checks the scenario where the `Call` method returns an error.

- **Mock Setup**:
    
    goCopy code
    
    `mockChat.On("Call", mock.Anything, mock.Anything, mock.Anything).Return(nil, errors.New("Mocked error"))`
    
    Here, we're setting up the expectation that the `Call` method of `mockChat` will be invoked and will return an error "Mocked error".
    
- **Function Call**:
    
    goCopy code
    
    `_, err := askSkippy(context.Background(), mockChat, "English", "Test error message for Skippy")`
    
    We then call the `askSkippy` function with our mock chat and check for an error.
    
- **Assertion**: The test checks that an error is returned.
    

---

### Potential Issues and Solutions:

1. **MockChat Implementation**: The `MockChat` struct is defined, but its methods aren't. For the tests to work, the `MockChat` needs to have a `Call` method that uses the testify mock's functionalities. This method should look something like:
    
    goCopy code
    
    `func (m *MockChat) Call(ctx context.Context, messages []schema.ChatMessage, options ...llms.CallOption) (*schema.AIChatMessage, error) {     args := m.Called(ctx, messages, options)     return args.Get(0).(*schema.AIChatMessage), args.Error(1) }`
    
2. **Dependencies**: Ensure that the testify library is installed. If not, it can be added using:
    
    bashCopy code
    
    `go get github.com/stretchr/testify`
    
3. **Integration with Main Code**: Ensure that the `Chat` interface in the main code and the `MockChat` in the test code are in sync. If the interface changes, the mock and tests need to be updated accordingly.
    

---

This breakdown provides an overview of the testing code's structure and functionality. The tests are designed to validate the behavior of the `askSkippy` function in both normal and error scenarios.

---

## 4. AWS Fargate and Copilot: Deploying and Managing the Service

AWS Fargate and Copilot provide a seamless experience for deploying and managing containerized applications on AWS without the need to manage the underlying infrastructure. With AWS Copilot, you can define, release, and manage services using simple CLI commands. Let's dive into the Copilot configuration files from the [go_skippy_lc/copilot directory](https://github.com/doctor-ew/go_skippy_lc/tree/aws-copilot/copilot) to understand how the service and environment are set up:

## 4.1. Service Configuration: `copilot/go-skippy-lc/manifest.yml`

### Overview:
This file defines the configuration for the `go-skippy-lc` service. It's a Load Balanced Web Service, which means it's a public-facing web service that's behind a load balancer.

### Breakdown:

- **name & type**: 
  - `name: go-skippy-lc` specifies the name of the service.
  - `type: Load Balanced Web Service` indicates the type of service.

- **http**:
  - `path: '/'` specifies the path that the load balancer should forward requests to.
  - `alias`: This is a list of domain names that should route to this service. This is useful for custom domain routing.
  - `healthcheck`: (commented out) would specify a custom health check path for the service.

- **image**:
  - `build: Dockerfile` specifies the Dockerfile to use for building the container image.
  - `port: 80` is the port the container listens on.

- **cpu, memory, count**:
  - These fields specify the resources for the ECS task. It uses 256 CPU units, 512 MiB of memory, and there should always be 1 task running.

- **exec**: 
  - `exec: true` allows you to run commands in our container.

- **network**:
  - `connect: true` enables Service Connect for intra-environment traffic between services.

- **storage**:
  - `readonly_fs`: (commented out) would limit the mounted root filesystems to read-only access.

- **variables & secrets**: 
  - (Both commented out) would allow us to pass environment variables and secrets to our service.

- **environments**:
  - (Commented out) would allow us to override any of the above values for specific environments.

## 4.2. Environment Configuration: `copilot/environments/test/manifest.yml`

### Overview:
This file defines the configuration for the `test` environment.

### Breakdown:

- **name & type**: 
  - `name: test` specifies the name of the environment.
  - `type: Environment` indicates the type of configuration.

- **network**:
  - (Commented out) would allow us to specify a custom VPC or configure how the VPC should be created.

- **http**:
  - `public`: Specifies the configuration for the public load balancer in the environment.
  - `certificates`: Lists the ARN of the SSL certificate to use with the load balancer.

- **observability**:
  - `container_insights: false` specifies that container insights (for monitoring) should not be enabled for this environment.

### Manual Step:
I had to manually point the A Record to the ALB. This step is necessary because while AWS Copilot can automate the creation of resources like the ALB, the final step of updating DNS records to point to the ALB often requires manual intervention, especially if you're using a third-party DNS provider or have specific routing requirements.


--- 
## Conclusion

As an architect or engineer, we need to think through the atoms that compose the thing we're building. In this case, a robust and scalable API server encompasses everything from server setup and environment management to containerization and rigorous testing. I've shared this guide to provide a solid foundation for deploying a Go-based API server to AWS Fargate. 

My hope is that this not only serves as a practical guide but also inspires fellow self-taught engineers, innovators, and all curious minds. Let's continue to explore, learn, and push the boundaries of innovation together. 

Excelsior!