Run and test DynamoDB applications locally using Docker and Testcontainers
Tags: docker, testing, dynamodb, go
Published: 2024-02-07 11:05:34

***Use the [DynamoDB Local Testcontainers module](https://github.com/abhirockzz/dynamodb-local-testcontainers-go/) for your Go applications!***

[DynamoDB Local](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/DynamoDBLocal.html) is a version of Amazon DynamoDB that you can run locally as a Docker container (or [other forms](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/DynamoDBLocal.DownloadingAndRunning.html)). 

It's super easy to get started:

```shell
# start container
docker run --rm -p 8000:8000 amazon/dynamodb-local

# connect and create a table
aws dynamodb create-table --endpoint-url http://localhost:8000 --table-name Books --attribute-definitions AttributeName=ISBN,AttributeType=S --key-schema AttributeName=ISBN,KeyType=HASH --billing-mode PAY_PER_REQUEST

# list tables
aws dynamodb list-tables --endpoint-url http://localhost:8000
```

> More on the `--endpoint-url` soon

## Hello Testcontainers!

This is a good start. But DynamoDB Local is a great fit for [Testcontainers](https://testcontainers.com/) which *"is an open source framework for providing throwaway, lightweight instances of databases, message brokers, web browsers, or just about anything that can run in a Docker container."*

It supports multiple languages (including Go!) and databases (also messaging infrastructure etc.) - **All you need is Docker**. [Testcontainers for Go](https://golang.testcontainers.org/) makes it simple to programmatically create and clean up container-based dependencies for automated integration/smoke tests. You can define test dependencies as code, run tests and delete the containers once done.

Testcontainers has the concept of [modules](https://testcontainers.com/modules/) that are *"preconfigured implementations of various dependencies that make writing your tests even easier"*


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/qj6m5nom6xvhkyjp3rt5.png)

Having a piece of infrastructure supported as a Testcontainer module provides a seamless, plug-and-play experience. The same applies to DynamoDB Local, and that's where [Testcontainers module for DynamoDB Local](http://github.com/abhirockzz/dynamodb-local-testcontainers-go) comes in! It allows you to easily run/test your Go based DynamoDB applications locally using Docker.

## Getting started with the Testcontainers module for DynamoDB Local

Super easy!

```shell
go mod init demo
go get github.com/abhirockzz/dynamodb-local-testcontainers-go
```

Go ahead and use the sample code in the [project README](https://github.com/abhirockzz/dynamodb-local-testcontainers-go/blob/master/README.md). 

To summarize, it consists of three simple steps:

1. Start the DynamoDB Local Docker container - `dynamodblocal.RunContainer(ctx)`
2. Gets the client handle for the DynamoDB (local) instance - `dynamodbLocalContainer.GetDynamoDBClient(context.Background())`
3. Uses the client handle to execute operations. In this case - create a table, add an item, query that item.
4. Terminate it at the end of the program (typically register it using `defer`) - `dynamodbLocalContainer.Terminate(ctx)`

## Module options

The following configuration parameters are supported: 

- `WithTelemetryDisabled` - When specified, DynamoDB local will not send any telemetry.
- `WithSharedDB` - If you use this option, DynamoDB creates a shared database file in which data is stored. This is useful if you want to persist data for e.g. between successive test executions.

To use `WithSharedDB`, here is a common workflow:

1. Start container and get client handle
2. Create table, add data and query it
3. Re-start container
4. Query same data (again) - Should be there

And here is how you might go about it (error handling and logging omitted):

```go
func withSharedDB() {
	ctx := context.Background()

	//start container
	dynamodbLocalContainer, _ := dynamodblocal.RunContainer(ctx)
	defer dynamodbLocalContainer.Terminate(ctx)

	//get client
	client, _ := dynamodbLocalContainer.GetDynamoDBClient(context.Background())

	//create table, add data
	createTable(client)
	value := "test_value"
	addDataToTable(client, value)

	//query same data
	queryResult, _ := queryItem(client, value)
	log.Println("queried data from dynamodb table. result -", queryResult)

	//re-start container
	dynamodbLocalContainer.Stop(context.Background(), aws.Duration(5*time.Second))
	dynamodbLocalContainer.Start(context.Background())

	//query same data
	client, _ = dynamodbLocalContainer.GetDynamoDBClient(context.Background())
	queryResult, _ = queryItem(client, value)
	log.Println("queried data from dynamodb table. result -", queryResult)
}
```

To use these options together:

```go
container, err := dynamodblocal.RunContainer(ctx, WithSharedDB(), WithTelemetryDisabled())
```

The [Testcontainers documentation](https://golang.testcontainers.org/modules/) is pretty good in terms of detailing how to write an extension/module. But I had to deal with a specific nuance - related to DynamoDB Local.

## DynamoDB Endpoint Resolution

Contrary to the DynamoDB service, in order to access DynamoDB Local (with the SDK, AWS CLI etc.), you must specify a local endpoint - `http://<your_host>:<service_port>`. Most commonly, this is what you would use: `http://locahost:8000`


The endpoint resolution process has changed since AWS SDK for Go v2 - I had to do some digging to figure it out. You can read up in the [SDK documentation](https://aws.github.io/aws-sdk-go-v2/docs/configuring-sdk/endpoints/#customization), but the short version is that you to specify a custom endpoint resolver. In this case, all it take is to retrieve the docker container host and port.

Here is the implementation - [this is used in the module as well](https://github.com/abhirockzz/dynamodb-local-testcontainers-go/blob/master/dynamodb_local_endpoint_resolver.go).

```go
type DynamoDBLocalResolver struct {
	hostAndPort string
}

func (r *DynamoDBLocalResolver) ResolveEndpoint(ctx context.Context, params dynamodb.EndpointParameters) (endpoint smithyendpoints.Endpoint, err error) {

	return smithyendpoints.Endpoint{
		URI: url.URL{Host: r.hostAndPort, Scheme: "http"},
	}, nil
}
```

## This was fun! 

Like I mentioned, Testcontainers has excellent documentation which was helpful as I had to wrap my head around how to support [-sharedDb flag](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/DynamoDBLocal.UsageNotes.html#DynamoDBLocal.CommandLineOptions) (using [WithSharedDB](https://github.com/abhirockzz/dynamodb-local-testcontainers-go/blob/master/dynamodb.go#L90)). The solution was easy (ultimately), but the [Reusable container](https://golang.testcontainers.org/features/creating_container/#reusable-container) section was the one which turned on the lightbulb for me!

If you find [this project](https://github.com/abhirockzz/dynamodb-local-testcontainers-go/) interesting/helpful, don't hesitate to ⭐️ it and share it with your colleagues. Happy Building!