Spring Boot Testing — Testcontainers and Flyway
Tags: java, junit5, testcontainers, flyway
Published: 2021-10-19 17:41:25

This is the second part of the Spring Boot Testing article series. The code snippets are taken from this [repository](https://github.com/SimonHarmonicMinor/spring-boot-test-example). You can clone it and run tests to see how it works.

Previous Chapter. [Spring Boot Testing — Data and Services](https://dev.to/kirekov/spring-boot-testing-data-and-services-288f)

So, we have learned how to test the [service layer](https://martinfowler.com/eaaCatalog/serviceLayer.html) and the [repository layer](https://martinfowler.com/eaaCatalog/repository.html) with the H2 database. Such tests have some benefits. For example, they are pretty fast and require no complex configuration neither on the local machine nor in the CI/CD environment. However, H2 is not the database that usually runs in production. If we use some vendor-specific features of one database, H2 might not help us. So, today we are going to discuss how to run test cases against the true DB server.

![Cover Image](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/x3aq161w7y6sf05s5qav.png)

## Automatic Schema Creation

How can we run tests with the real database? Well, we could create a new instance locally and configure the test environment by editing `src/test/resources/application.yml`. It does work but it makes the build hard to reproduce. Every developer that is working on the project has to be sure that they have two separate databases. One for development and another one for tests running. Besides, it makes executing the build on CI/CD environment a real challenge.

So, [Testcontainers](https://www.testcontainers.org/) to the rescue! It’s a Java library that launches the service within the [Docker](https://www.docker.com/) container, runs tests, and eventually destroys the container. You don’t need to worry about anything, the framework does the job. Just make sure you have Docker installed and then you are ready to go. The library supports dozens of different databases and modules ([PostgreSQL](https://www.testcontainers.org/modules/databases/postgres/), [MySQL](https://www.testcontainers.org/modules/databases/mysql/), [MongoDB](https://www.testcontainers.org/modules/databases/mongodb/), [Kafka](https://www.testcontainers.org/modules/kafka/), [Elasticsearch](https://www.testcontainers.org/modules/elasticsearch/), Nginx, [Toxiproxy](https://www.testcontainers.org/modules/nginx/), and many others). Even if you didn’t find the one that you need, you could use [generic container creation](https://www.testcontainers.org/features/creating_container/).

The first step is to add the required dependencies.

```groovy
testImplementation 'org.testcontainers:junit-jupiter'
testImplementation 'org.testcontainers:postgresql'
runtimeOnly 'org.postgresql:postgresql'
```

Then we need to create a separate configuration file in `src/test/resources`. Spring Boot is able to distinguish different configuration files by the [profiles](https://www.baeldung.com/spring-profiles). The profile name should be placed as a suffix like `application-PROFILE_NAME.yml`. For example, the configuration file named `application-test-containers.yml` is applied only when `test-containers` profile is active.

```yml
spring:
  datasource:
    url: jdbc:tc:postgresql:9.6.8:///test_database
    username: user
    password: password
  jpa:
    hibernate:
      ddl-auto: create
```

Have you noticed the `tc` suffix in the JDBC-connection string? That’s the magic that comes with the union of JUnit 5 and Testcontainers. The thing is that you don’t need any programmatic configurations at all! When the framework sees that `url` contains the `tc` suffix it runs all necessary Docker commands internally. You can find more examples [here](https://www.testcontainers.org/modules/databases/jdbc/).

> We set `spring.jpa.hibernate.ddl-auto=create` property
> so the database schema shall be created automatically
> according to definition of entity classes.
> Flyway integration is described in the next section.

Now let’s take a look at `PersonCreateService.createFamily` method and its H2 test again.

```java
@Service
@RequiredArgsConstructor
public class PersonCreateServiceImpl implements PersonCreateService {

  private final PersonValidateService personValidateService;
  private final PersonRepository personRepository;

  @Override
  @Transactional
  public List<PersonDTO> createFamily(Iterable<String> firstNames, String lastName) {
    final var people = new ArrayList<PersonDTO>();
    firstNames.forEach(firstName -> people.add(createPerson(firstName, lastName)));
    return people;
  }
  
  @Override
  @Transactional
  public PersonDTO createPerson(String firstName, String lastName) {
    personValidateService.checkUserCreation(firstName, lastName);
    final var createdPerson = personRepository.saveAndFlush(
        new Person()
            .setFirstName(firstName)
            .setLastName(lastName)
    );
    return DTOConverters.toPersonDTO(createdPerson);
  }
}
```

```java
@SpringBootTest(webEnvironment = WebEnvironment.NONE)
@AutoConfigureTestDatabase
class PersonCreateServiceImplSpringBootTest {

  @Autowired
  private PersonRepository personRepository;
  @MockBean
  private PersonValidateService personValidateService;
  @Autowired
  private PersonCreateService personCreateService;

  @BeforeEach
  void init() {
    personRepository.deleteAll();
  }

  @Test
  void shouldCreateOnePerson() {
    final var people = personCreateService.createFamily(
        List.of("Simon"),
        "Kirekov"
    );
    assertEquals(1, people.size());
    final var person = people.get(0);
    assertEquals("Simon", person.getFirstName());
    assertEquals("Kirekov", person.getLastName());
    assertTrue(person.getDateCreated().isBefore(ZonedDateTime.now()));
  }

  @Test
  void shouldRollbackIfAnyUserIsNotValidated() {
    doThrow(new ValidationFailedException(""))
        .when(personValidateService)
        .checkUserCreation("John", "Brown");
    assertThrows(ValidationFailedException.class, () -> personCreateService.createFamily(
        List.of("Matilda", "Vasya", "John"),
        "Brown"
    ));
    assertEquals(0, personRepository.count());
  }
}
```

How do we run the test with Testcontainers PostgreSQL instance? All we have to do is to add two annotations. `@AutoConfigureTestDatabase` though should be removed.

1. `@ActiveProfiles("test-containers")` — activates the `test-containers` profile so the Spring could read the configuration file that was described earlier
1. `@Testcontainers` — tells to run PostgreSQL instance in Docker [automagically](https://www.testcontainers.org/modules/databases/jdbc/)

```java
@SpringBootTest(webEnvironment = WebEnvironment.NONE)
@Testcontainers
@ActiveProfiles("test-containers")
class PersonCreateServiceImplTestContainers {

  @Autowired
  private PersonRepository personRepository;
  @MockBean
  private PersonValidateService personValidateService;
  @Autowired
  private PersonCreateService personCreateService;

  @BeforeEach
  void init() {
    personRepository.deleteAll();
  }

  @Test
  void shouldCreateOnePerson() {
    final var people = personCreateService.createFamily(
        List.of("Simon"),
        "Kirekov"
    );
    assertEquals(1, people.size());
    final var person = people.get(0);
    assertEquals("Simon", person.getFirstName());
    assertEquals("Kirekov", person.getLastName());
    assertTrue(person.getDateCreated().isBefore(ZonedDateTime.now()));
  }

  @Test
  void shouldRollbackIfAnyUserIsNotValidated() {
    doThrow(new ValidationFailedException(""))
        .when(personValidateService)
        .checkUserCreation("John", "Brown");
    assertThrows(ValidationFailedException.class, () -> personCreateService.createFamily(
        List.of("Matilda", "Vasya", "John"),
        "Brown"
    ));
    assertEquals(0, personRepository.count());
  }
}
```

![Test results](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/e8dgoz3cijg0f6ptxhz1.png)

See? Piece of cake!

## Repository Tests

What about testing repository layers? Let’s see H2 example again.

```java
@DataJpaTest
class PersonRepositoryDataJpaTest {

  @Autowired
  private PersonRepository personRepository;

  @Test
  void shouldReturnAlLastNames() {
    personRepository.saveAndFlush(new Person().setFirstName("John").setLastName("Brown"));
    personRepository.saveAndFlush(new Person().setFirstName("Kyle").setLastName("Green"));
    personRepository.saveAndFlush(new Person().setFirstName("Paul").setLastName("Brown"));

    assertEquals(Set.of("Brown", "Green"), personRepository.findAllLastNames());
  }
}
```

The rules are the same but there is a slight difference. `@DataJpaTest` is annotated with `@AutoConfigureTestDatabase` itself. This annotation replaces any data source with the H2 instance by default. So, we need to override this behavior by adding `replace=Replace.NONE` property.

```java

@DataJpaTest
@Testcontainers
@ActiveProfiles("test-containers")
@AutoConfigureTestDatabase(replace = Replace.NONE)
class PersonRepositoryTestContainers {

  @Autowired
  private PersonRepository personRepository;

  @Test
  void shouldReturnAlLastNames() {
    personRepository.saveAndFlush(new Person().setFirstName("John").setLastName("Brown"));
    personRepository.saveAndFlush(new Person().setFirstName("Kyle").setLastName("Green"));
    personRepository.saveAndFlush(new Person().setFirstName("Paul").setLastName("Brown"));

    assertEquals(Set.of("Brown", "Green"), personRepository.findAllLastNames());
  }
}
```

![Test results](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/tn14rvwmg44mbmst595a.png)

Everything still works fine.

## Flyway Integration

[The Evolutionary Database Design](https://martinfowler.com/articles/evodb.html) principle has been described a long time ago. Nowadays it is standard routine to use tools that implement this pattern. [Flyway](https://flywaydb.org/) and [Liquibase](https://www.liquibase.org/) are the most popular ones in the Java world. We are going to integrate Testcontainers with Flyway.

Firstly, the Flyway dependency is required.

```groovy
implementation "org.flywaydb:flyway-core"
```

Secondly, it’s necessary to disable Flyway in `application-test-containers.yml` because there will be a separate configuration file.

```yml
spring:
  datasource:
    url: jdbc:tc:postgresql:9.6.8:///test_database
    username: user
    password: password
  jpa:
    hibernate:
      ddl-auto: create
  flyway:
    enabled: false
```

Then we are going to create `application-test-containers-flyway.yml`. The library provides lots of auto-configuration. So, actually, we don’t need to tune anything.

```yml
spring:
  datasource:
    url: jdbc:tc:postgresql:9.6.8:///test_database
    username: user
    password: password
```

Now it’s time to add SQL migrations. The default directory is `resources/db/migration`.

```sql
create table person
(
    id           serial primary key,
    first_name   text,
    last_name    text,
    date_created timestamp with time zone
);
```

Finally, we need to replace `test-containers` profile with `test-containers-flyway` one.

```java
@SpringBootTest(webEnvironment = WebEnvironment.NONE)
@Testcontainers
@ActiveProfiles("test-containers-flyway")
class PersonCreateServiceImplTestContainersFlyway {

  @Autowired
  private PersonRepository personRepository;
  @MockBean
  private PersonValidateService personValidateService;
  @Autowired
  private PersonCreateService personCreateService;

  @BeforeEach
  void init() {
    personRepository.deleteAll();
  }

  @Test
  void shouldCreateOnePerson() {
    final var people = personCreateService.createFamily(
        List.of("Simon"),
        "Kirekov"
    );
    assertEquals(1, people.size());
    final var person = people.get(0);
    assertEquals("Simon", person.getFirstName());
    assertEquals("Kirekov", person.getLastName());
    assertTrue(person.getDateCreated().isBefore(ZonedDateTime.now()));
  }

  @Test
  void shouldRollbackIfAnyUserIsNotValidated() {
    doThrow(new ValidationFailedException(""))
        .when(personValidateService)
        .checkUserCreation("John", "Brown");
    assertThrows(ValidationFailedException.class, () -> personCreateService.createFamily(
        List.of("Matilda", "Vasya", "John"),
        "Brown"
    ));
    assertEquals(0, personRepository.count());
  }
}
```

![Tests result with Flyway schema creation](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/83im8cawj73bur4uw1gk.png)

## CI/CD Tests Running

Although Testcontainers’ purpose is to make tests easy to run there are some caveats in CI/CD environment build. Some vendors integrate with Testcontainers transparently. For example, [Travis-CI](https://www.travis-ci.com/) spots the requirement to run a container automatically and does the job internally. But some other tools may require additional configuration — e.g., Jenkins.

### Docker Wormhole

It’s a common approach to package the whole application build in a Docker image during the CI run. Testcontainers can handle it and run the defined containers on the host Docker server. In this case, we need to bind `/var/run/docker.sock` as a volume. More than that, the directory inside the container should be the same as the one where the container was launched.

For instance, that’s a hypothetical command to run tests on a CI environment with `gradle`.

```shell
docker run -it   \
           --rm  \
           -v $PWD:$PWD \
           -w $PWD  \
           -v /var/run/docker.sock:/var/run/docker.sock gradle:7.1-jdk8 \
           gradle test
```

You can find more information on the official [Testcontainers guide page](https://www.testcontainers.org/supported_docker_environment/continuous_integration/dind_patterns/).

## Conclusion

Today we have discussed how to test the data and service layer with Testcontainers and Flyway. Next time we will test the API (controllers) level. If you have any questions or suggestions, please, leave your comments down below. Thanks for reading!