Testcontainers with MySQL and Redis with an Spring Boot Kotlin Application
Tags: kotlin, mysql, redis, testcontainer
Published: 2022-07-07 13:01:32

*Microservices* architecture gives us some facilities, such as independently deploy, services with a unique responsibility, working with different languages, and so on. However, it also brings us some new complexities, such as duplicated code, observability tools, distributed caches tools, and logs tools. When we think about all these resources working together, we can think "How can we test these resources?". For instance, if our service uses *Redis* for cache, and *MySQL* to save and retrieve data, how can we do integration tests? Doing these tests by yourself can be difficult, but we have some manners to create that without much effort. One option is using *TestContainers*. To show you the tool, I will test a Spring Boot Kotlin Application as example.  

## TestContainers 
	
*Testcontainers* is a Java library that supports *JUnit* tests, providing lightweight, throwaway instances of common databases, *Selenium* web browsers, or anything else that can run in a *Docker* container.

*Testcontainers* make the following kinds of tests easier:

* Data access layer integration tests: use a containerized instance of a *MySQL*, *PostgreSQL* or *Oracle* database to test your data access layer code for complete compatibility, but without requiring complex setup on developers' machines and safe in the knowledge that your tests will always start with a known DB state. Any other database type that can be containerized can also be used.

* Application integration tests: for running your application in a short-lived test mode with dependencies, such as databases, message queues or web servers.

* UI/Acceptance tests: use containerized web browsers, compatible with *Selenium*, for conducting automated UI tests. Each test can get a fresh instance of the browser, with no browser state, plugin variations or automated browser upgrades to worry about. And you get a video recording of each test session, or just each session where tests failed.

* Much more! Check out the various contributed modules or create your own custom container classes using `GenericContainer` as a base.

## Let's Code

The application that will be used has a Car domain. We can perform operations of inserting new cars, searching for available cars, and changing existing cars. The system has a controller, a service, and a repository. An overview of the architecture follows.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/4l5d935lxbkcvo5u645r.jpeg)

So, when we do a POST request to an application, the data is saved on *MySQL* DB. 

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/t6wrkjpgau8pdcdw9gxf.png)

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/dnqujrdgj406xi269dfr.png)

When we do a GET request, the application retrieves the data that is saved on DB and adds the information to *Redis* cache. 

*Redis is an open-source, in-memory data store used by millions of developers as a database, cache, streaming engine, and message broker. Redis' speed makes it ideal for caching database queries, complex computations, API calls, and session state.*

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/k2f13gpr2vwrux19quov.png)

## Data in Redis

We need to guarantee that the cache was saved in *Redis*. For this, we can use a tool called *RedisInsight* to see the data. *RedisInsight* provides an intuitive *Redis* admin GUI and helps optimize your use of *Redis* in your applications. It supports *Redis Open Source, Redis Stack, Redis Enterprise Software, Redis Enterprise Cloud, and Amazon ElastiCache*. *RedisInsight* now incorporates a completely new tech stack based on the popular *Electron* and *Elastic UI* frameworks. And it runs cross-platform, supported on *Linux*, *Windows*, and *MacOS*. 

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ncf1pq9d7m64b7aamkr0.png)

As we can see, the cache's data is saved like a hash. And if we don't do POST and PUT operations to the application, this data always will be retrieved. 

## Testing the application

Now that we know how the application works, a controller test will be performed, where we will use *MockMVC*. *MockMvc* simulates a real request to the application, doing exactly what happens when the client calls an endpoint. This implies that at the time of running the tests, we need to have all external resources working. It is common for projects to use a real test-specific database instance to perform integration tests, however, as we saw earlier, we have a more elegant and modern way to do them, using *TestContainers*. As the cached service method and the repository will be called in the controller flow, it will be necessary to upload two containers, one for *MySQL* and the other for *Redis*. Let's start developing our test.

The first thing we need to do is add the *TestContainers* dependencies to the application's pom.xml.

```xml
<dependencyManagement>
  <dependencies>
    <dependency>
      <groupId>org.testcontainers</groupId>
      <artifactId>testcontainers-bom</artifactId>
      <version>${testcontainers.version}</version>
        <type>pom</type>
        <scope>import</scope>
    </dependency>
  </dependencies>
</dependencyManagement>

<dependencies>
  <dependency>
    <groupId>org.testcontainers</groupId>
    <artifactId>junit-jupiter</artifactId>
    <scope>test</scope>
  </dependency>
  <dependency>
    <groupId>org.testcontainers</groupId>
    <artifactId>mysql</artifactId>
    <scope>test</scope>
</dependency>
```

We need the *TestContainers* `junit-jupiter` dependency because it provides some resources for more efficient testing. As we are using *MySQL* as a database, it's necessary a dependency that provides resources to create a *MySQL* container for tests in a simplified way.

We need to create the configuration class of the containers that will be created at test time. It will be called `DatabaseContainerConfiguration` and will have the following content

```java
@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
abstract class DatabaseContainerConfiguration {

    companion object {
        @Container
        private val mysqlContainer = MySQLContainer<Nothing>("mysql:latest").apply {
            withDatabaseName("testdb")
            withUsername("joao")
            withPassword("12345")
        }

        @Container
        private val redisContainer = GenericContainer<Nothing>("redis:latest").apply {
            withExposedPorts(6379)
        }

        @JvmStatic
        @DynamicPropertySource
        fun properties(registry: DynamicPropertyRegistry) {
            registry.add("spring.datasource.url", mysqlContainer::getJdbcUrl)
            registry.add("spring.datasource.password", mysqlContainer::getPassword)
            registry.add("spring.datasource.username", mysqlContainer::getUsername)

            registry.add("spring.redis.host", redisContainer::getContainerIpAddress)
            registry.add("spring.redis.port", redisContainer::getFirstMappedPort)
        }
    }
}
```

Now let's break down the meaning of each of the above features:

`@Container`: is used in conjunction with the `@Testcontainers` annotation to mark containers that should be managed by the *Testcontainers* extension.

*We'll see about the @TestContainers annotation yet*

`MySQLContainer<Nothing>(...)`: Class that supports the creation of a *MySQL* container for testing. We passed a Nothing in the generics of `MySQLContainer` because of the class signature - `public class MySQLContainer<SELF extends MySQLContainer<SELF>> extends JdbcDatabaseContainer<SELF>` - as we can see, there is a recursive generic type, which with *Java*, we can ignore and just instantiate the class without generics. Using *Kotlin*, we don't have this option, but we have two ways to handle it.
1. Define a subclass of `MySQLContainer` and pass the child class to *generics*
2. Use `Nothing`, which by *Kotlin* doc's is:

> Nothing has no instances. You can use Nothing to represent "a value that never exists": for example, if a function has the return type of Nothing, it means that it never returns.

In the `MySQLContainer` constructor, the version that we want to use from the *MySQL* image is informed.

`withDatabaseName`: Method of the `MySQLContainer` class that receives the name of the database as a parameter. This database will be created at runtime after the test container goes up.

`withUsername`: Method of the `MySQLContainer` class that receives the database username as a parameter.

`withPassword`: Method of the `MySQLContainer` class that receives the password from the database as a parameter.

`GenericContainer<Nothing>()`: As the name implies, it is a class that allows you to create a container in a generic way, which means that the methods contained in them can be reused for various resources, such as *Redis*, *NoSQL* Databases, among others. We are using it because *Redis* does not have a specific class like *MySQL*. As we are seeing in the code, to create a generic container, we just need to insert the image of the resource that we want to instantiate as a parameter.

`withExposedPorts`: Method of the `GenericContainer` class that receives the port that we will make available from the test container. This port will be used at runtime.

The properties function is used to dynamically add *Redis* and *MySQL* properties to the application. Before *TestContainers* uploads the two containers, the application will not be aware of them, so how would we carry out the integration tests? Using `@DynamicPropertySource` *Kotlin* understands that these properties will be dynamically set at runtime, so when uploading the *MySQL* and *Redis* container, the information that the application needs to know to reach them will be included.

Now we have everything we need for *TestContainers* to upload both containers. We just need to use the class where needed. As stated at the beginning of the section, we want to do an API test, where *MockMVC* will make a request to the controller and thus follow the flow to the database. So let's create this test.

```java
@Testcontainers
@AutoConfigureMockMvc
@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
class CarControllerTest : DatabaseContainerConfiguration() {

    @Autowired
    private lateinit var mockMvc: MockMvc

    companion object {
        private const val URI = "/cars"
    }

    @Test
    fun `should return 200 code when call cars`() {
        mockMvc.get(URI).andExpect { status { isOk() } }
    }
```

As mentioned before, we can see that we have an annotation in the class declaration, which is `@TestContainers`. This annotation makes that, at runtime, the library identifies it and uploads the available containers through the `@Container` annotation, which is in our `DatabaseContainerConfiguration` class, which in turn was inherited by the test. This is enough for when we run the tests, the containers are instantiated.

> The intention is not to talk about MockMVC, but according to the documentation, it is the main entry point for server-side Spring MVC testing support.

One last detail before running the tests is that it will be necessary to configure the *migration*. When the application goes up and reaches the containers, it is applied.

```sql
#src/main/resources/db/migration/V01__create_car_table_add_data.sql

create table car(
    id bigint not null auto_increment,
    name varchar(50) not null,
    model varchar(50) not null,
    primary key(id)
);

insert into car values('Golf', 'VW');

```

It's time to run the tests and remember, *docker* must be active. Running the `mvn -B test` command will run the tests. We can observe the containers going up at the time of the test execution, using the docker ps command in the terminal or the docker desktop.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/b52x3jx4o2tzcs8ztd46.png)

In the test execution logs, we can see the moment when the *MySQL* container rises

22:17:53.206 [main] DEBUG 🐳 [mysql:latest] - Trying to create JDBC connection using com.mysql.cj.jdbc.Driver to jdbc:mysql://localhost:52550/testdb?useSSL=false&allowPublicKeyRetrieval=true with properties: {password=12345, user=joao}
22:17:53.582 [main] INFO 🐳 [mysql:latest] - Container is started (JDBC URL: jdbc:mysql://localhost:52550/testdb)
22:17:53.583 [main] INFO 🐳 [mysql:latest] - Container mysql:latest started in PT23.428294S

Just like *Redis*

22:17:53.583 [main] DEBUG 🐳 [redis:latest] - Starting container: redis:latest
22:17:53.583 [main] DEBUG 🐳 [redis:latest] - Trying to start container: redis:latest (attempt 1/1)
22:17:53.583 [main] DEBUG 🐳 [redis:latest] - Starting container: redis:latest
22:17:53.583 [main] INFO 🐳 [redis:latest] - Creating container for image: redis:latest

And finally the expected result

`Tests Passed: 1 of 1 test - 631 ms`

## Conclusion

Integration tests are very important for the application, and one of the main tasks is to validate if the application's connection with external resources is being done successfully. There are N ways to do them. We can use database integration tests, in-memory databases, real instances and as presented in the post, TestContainers. *TestContainers*, with small settings, bring big advantages. Disposable containers, as they are only used at the time of test execution. External resource agnostic, with the `GenericContainer` class we can do integration testing with any resource we uploaded in docker. An excellent testing tool.

*This post is in collaboration with Redis.*

Application's repository: https://github.com/joao0212/car-service

References: 
https://www.testcontainers.org/
https://docs.spring.io/spring-framework/docs/current/javadoc-api/org/springframework/test/web/servlet/MockMvc.html