How to run disposable databases for your tests. Improve your integration tests accuracy with Testcontainers
Tags: webdev, dotnet, csharp, testing
Published: 2023-01-09 16:03:12

## Introduction
Integration tests are essential to ensure that the different components of our system work together as expected and continue to work after changes.

In this post, I'll explain how to spin up disposable database containers to use in integration tests.

## Integration tests and managed resources

As explained in [this article](https://blog.genezini.com//p/integration-tests-without-api-dependencies-with-asp.net-core-and-wiremock.net/), in integration tests, we should mock unmanaged dependencies (dependencies that are external to our system and not controlled by us, like APIs) but test against real managed dependencies (dependencies that are controlled by our system, like databases). This improves the reliability of the integration tests because the communication with these dependencies are a complex part of the system and can break with a package update, a database update or even a simple change in a SQL command.

## What is Testcontainers?

Testcontainers is a that provides lightweight, throwaway instances of databases, selenium web browsers, or anything that can run in a container. These instances can be especially useful for testing applications against real dependencies, like databases, that can be created and disposed of after the tests. 

## Why not run the containers manually?

One key benefit of using Testcontainers instead of running the containers manually is that we can make use of libraries such as AutoFixture to generate data to seed the database instead of running scripts to insert the data. This also helps in avoiding collision between data used in different tests because the data is random.

Also, there are other advantages in usings Testcontainers:

- To run the tests locally, you don't need any extra steps, like running a docker-compose command;
- You don't have to wait or implement a waiting strategy to check if the containers are running before accessing them. Testcontainers already implements this logic;
- Testcontainers have properties to access the port in which the container is running, so you don't need to specify a fixed port, avoiding port conflict when running in the CI/CD pipeline or other machines;
- Testcontainers stops and deletes the containers automatically after running, free resources on the machine.

## Running a disposable container with Testcontainers

To use Testcontainers, you will need to have a container runtime ([Docker](https://www.docker.com/products/docker-desktop/), [Podman](https://podman-desktop.io/), [Rancher](https://www.rancher.com/products/rancher-desktop), etc) installed on your machine. 

Then, you need to add the Testcontainers NuGet package to your test project:

```bash
dotnet add package Testcontainers
```

To run a container, we first need to use the `TestcontainersBuilder<T>` class to build a `TestcontainersContainer` or a derived class, for instance, `MySqlTestcontainer`:

```csharp
await using var mySqlTestcontainer = new TestcontainersBuilder<MySqlTestcontainer>()
    .WithDatabase(new MySqlTestcontainerConfiguration
    {
        Password = "Test1234",
        Database = "TestDB"
    })
    .Build();
```

Then, we start the container and use the `ConnectionString` property where needed:

```csharp
await using var mySqlTestcontainer = new TestcontainersBuilder<MySqlTestcontainer>()
    .WithDatabase(new MySqlTestcontainerConfiguration
    {
        Password = "Test1234",
        Database = "TestDB"
    })
    .Build();

await mySqlTestcontainer.StartAsync();

await using var todoContext = TodoContext
    .CreateFromConnectionString(mySqlTestcontainer.ConnectionString);
```

> ❗ `TestcontainersContainer` implements `IAsyncDisposable` and needs to be disposed of after use. We can use the `await using` syntax or call the `DisposeAsync` method.

Testcontainers have classes for many different databases (called modules), for example:

- Elasticsearch;
- MariaDB;
- Microsoft SQL Server;
- MySQL;
- Redis.

The full list can be seen [here](https://dotnet.testcontainers.org/modules/).

But we can also create containers from any image, as in the example below, where it creates a Memcached instance:

```csharp
await using var MemCachedTestcontainer = new TestcontainersBuilder<TestcontainersContainer>()
    .WithImage("memcached:1.6.17")
    .Build();

await mySqlTestcontainer.StartAsync();
```

More details in the [official documentation](https://dotnet.testcontainers.org/api/create_docker_container/).

## Creating integration tests with Testcontainers

In this example, I'm using the xUnit and the `WebApplicationFactory<T>` class from ASP.NET Core.

If you don't know how to use the `WebApplicationFactory<T>` class, I explained in [this post](https://blog.genezini.com/p/integration-tests-without-api-dependencies-with-asp.net-core-and-wiremock.net/).

In this example, I have a controller with a `GET` method that returns a ToDo item and a `POST` method that add a ToDo item:

```csharp
[ApiController]
[Route("[controller]")]
public class TodoController : ControllerBase
{
    private readonly TodoContext _todoContext;

    public TodoController(TodoContext todoContext)
    {
        _todoContext = todoContext;
    }

    [HttpGet("{itemId}", Name = "GetTodoItem")]
    public async Task<ActionResult<TodoItem?>> GetByIdAsync(int itemId)
    {
        var item = await _todoContext.TodoItems.SingleOrDefaultAsync(a => a.ItemId == itemId);

        if (item is null)
        {
            return NotFound();
        }

        return Ok(item);
    }

    [HttpPost]
    public async Task<ActionResult<int>> PostAsync(TodoItem todoItem)
    {
        _todoContext.Add(todoItem);

        await _todoContext.SaveChangesAsync();
        
        return CreatedAtRoute("GetTodoItem", new { itemId = todoItem.ItemId }, todoItem);
    }
}
```

> ⚠️ The business logic is in the controller just for the sake of simplicity. In a real-world application, the logic should be in another layer.

### Testing the GET method

The test does the following actions:

1. Create and start a MySql container;
1. Create a Entity Framework DbContext using the ConnectionString from the MySql in the container;
1. Create the database tables using Entity Framework; (Can also be done passing a script to the `ExecScriptAsync` method from the `mySqlTestcontainer` object);
1. Create a random object with AutoFixture and add it to the database table;
1. Override our application configuration with the connection string from the container;
1. Create an `HttpClient` pointing to our application; 
1. Make a request to the GET endpoint passing the Id of the random object we added to the database;
1. Validate that the Status Code is `200` and the object returned is the same we added to the database.

```csharp
public class TodoIntegrationTests : IClassFixture<WebApplicationFactory<Program>>
{
    private readonly WebApplicationFactory<Program> _factory;

    public TodoIntegrationTests(WebApplicationFactory<Program> factory)
    {
        _factory = factory;
    }

    [Fact]
    public async Task GetOneItem_Returns200WithItem()
    {
        //Arrange
        await using var mySqlTestcontainer = new TestcontainersBuilder<MySqlTestcontainer>()
            .WithDatabase(new MySqlTestcontainerConfiguration
            {
                Password = "Test1234",
                Database = "TestDB"
            })
            .WithImage("mysql:8.0.31-oracle")
            .Build();

        await mySqlTestcontainer.StartAsync();

        await using var todoContext = TodoContext
            .CreateFromConnectionString(mySqlTestcontainer.ConnectionString);

        // Creates the database if not exists
        await todoContext.Database.EnsureCreatedAsync();

        Fixture fixture = new Fixture();
        var todoItem = fixture.Create<TodoItem>();

        todoContext.TodoItems.Add(todoItem);

        await todoContext.SaveChangesAsync();

        var HttpClient = _factory
            .WithWebHostBuilder(builder =>
            {
                builder.UseSetting("MySqlConnectionString", mySqlTestcontainer.ConnectionString);
            })
            .CreateClient();

        //Act
        var HttpResponse = await HttpClient.GetAsync($"/todo/{todoItem.ItemId}");

        //Assert
        HttpResponse.StatusCode.Should().Be(HttpStatusCode.OK);

        var responseJson = await HttpResponse.Content.ReadAsStringAsync();
        var todoItemResult = JsonSerializer.Deserialize<TodoItem>(responseJson);

        todoItemResult.Should().BeEquivalentTo(todoItem);
    }
}
```

> ❗ Be aware that I pass the container tag version in the `WithImage` method even when using the typed `MySqlTestcontainer` container class. This is very important because when we don't pass the tag, the container runtime will use the `latest` tag and a database update may break the application and the tests.

![Tests running and container lifecycle](https://blog.genezini.com/p/how-to-run-disposable-databases-for-your-tests.-improve-your-integration-tests-accuracy-with-testcontainers/images/testsrunning.gif)

### Testing the POST method

First, let's migrate the `MySqlTestcontainer` and the `DbContext` creation to a Fixture and a Collection so they can be shared between all tests. This is recommended because unless we do this, all tests will spin up and dispose of the container, making our tests slower than needed.

```csharp
[CollectionDefinition("MySqlTestcontainer Collection")]
public class MySqlTestcontainerCollection: ICollectionFixture<MySqlTestcontainerFixture>
{
}

public class MySqlTestcontainerFixture : IAsyncLifetime
{
    public MySqlTestcontainer MySqlTestcontainer { get; private set; } = default!;
    public TodoContext TodoContext { get; private set; } = default!;

    public async Task InitializeAsync()
    {
        MySqlTestcontainer = new TestcontainersBuilder<MySqlTestcontainer>()
            .WithDatabase(new MySqlTestcontainerConfiguration
            {
                Password = "Test1234",
                Database = "TestDB"
            })
            .WithImage("mysql:8.0.31-oracle")
            .Build();

        await MySqlTestcontainer.StartAsync();

        TodoContext = TodoContext
            .CreateFromConnectionString(MySqlTestcontainer.ConnectionString);

        // Creates the database if it does not exists
        await TodoContext.Database.EnsureCreatedAsync();
    }

    public async Task DisposeAsync()
    {
        await MySqlTestcontainer.DisposeAsync();

        await TodoContext.DisposeAsync();
    }
}
```

The test does the following actions:

1. MySql container and DbContext are injected by the xUnit fixture;
1. Create a random object with AutoFixture;
1. Override our application configuration with the connection string from the container;
1. Create an `HttpClient` pointing to our application;
1. Make POST request to the endpoint passing the random object previously created;
1. Validate that the Status Code is `200` and that the `Location` header has the correct URL for the GET endpoint of the created object;
1. Query the database and validate that the object created is equal to the randomly created object.

```csharp
[Collection("MySqlTestcontainer Collection")]
public class TodoIntegrationTests : IClassFixture<WebApplicationFactory<Program>>
{
    private readonly WebApplicationFactory<Program> _factory;
    private readonly MySqlTestcontainer _mySqlTestcontainer;
    private readonly TodoContext _todoContext;

    public TodoIntegrationTests(WebApplicationFactory<Program> factory, 
        MySqlTestcontainerFixture mySqlTestcontainerFixture)
    {
        _factory = factory;
        _mySqlTestcontainer = mySqlTestcontainerFixture.MySqlTestcontainer;
        _todoContext = mySqlTestcontainerFixture.TodoContext;
    }

    //Other tests
    //...

    [Fact]
    public async Task PostOneItem_Returns201AndCreateItem()
    {
        //Arrange
        Fixture fixture = new Fixture();
        var todoItem = fixture.Build<TodoItem>()
                .With(x => x.ItemId, 0)
                .Create();

        var HttpClient = _factory
            .WithWebHostBuilder(builder =>
            {
                builder.UseSetting("MySqlConnectionString", _mySqlTestcontainer.ConnectionString);
            })
            .CreateClient();

        //Act
        var HttpResponse = await HttpClient.PostAsJsonAsync($"/todo", todoItem);

        //Assert
        HttpResponse.StatusCode.Should().Be(HttpStatusCode.Created);

        var responseJson = await HttpResponse.Content.ReadAsStringAsync();
        var todoItemResult = JsonSerializer.Deserialize<TodoItem>(responseJson);

        HttpResponse.Headers.Location.Should().Be($"{HttpClient.BaseAddress}Todo/{todoItemResult!.ItemId}");

        var dbItem = await _todoContext.TodoItems
            .SingleAsync(a => a.ItemId == todoItemResult!.ItemId);

        dbItem.Description.Should().Be(todoItem.Description);
    }
}
```

## Liked this post?

I post extra content in my personal blog. [Click here](https://blog.genezini.com/) to see.

## Follow me

- [On Feedly](https://feedly.com/i/subscription/feed/https://blog.genezini.com/index.xml)
- [RSS](https://blog.genezini.com/index.xml)
- [Linkedin](https://www.linkedin.com/in/danielgenezini/)
- [Twitter](https://twitter.com/GeneziniDaniel)

## Source code of this sample

https://github.com/dgenezini/TestcontainersMySqlSample