Testcontainers for Hashicorp Consul and Vault
Tags: vault, consul, microservices, springcloud
Published: 2022-02-16 16:17:35

---
title: Testcontainers for Hashicorp Consul and Vault
published: true
date: 2022-02-16 15:59:39 UTC
tags: vault,consul,microservices,springcloud
canonical_url: https://hmh.engineering/testcontainers-for-hashicorp-consul-and-vault-85f2bba14290
---

![3 cubes representing testcontainers for Vault, Consul and LocalStack in a Docker-In-Docker environment with host.testcontainers.internal pointing from Vault to the other testcontainer cubes.](https://cdn-images-1.medium.com/max/1024/1*92KE3Rr_GSdODI_zY5wvHw.png)
_Testcontainers for Vault, Consul and LocalStack with DIND._

In my earlier post, I touched on some interesting architectural patterns for [Configuration and Secret Management](https://dev.to/krisiye/spring-boot-configuration-and-secret-management-patterns-on-kubernetes-1179) for your microservices on k8s. I highly recommend reading if you haven’t already. This article builds on the same idea and highlights the importance of integration tests of your service and how we could leverage Testcontainers to ease out some of these challenges such as divergent test environments and configuration or even mock components used in your tests that may not be enough to catch issues early on in your continuous integration pipelines and provide adequeate code coverage.

### Testcontainers

[Testcontainers](https://www.testcontainers.org/) is a JVM library that allows users to run and manage Docker images with Java code and frameworks such as JUnit. The most common use-cases for Testcontainers include integration tests against microservices with external dependencies such as Vault, Consul, AWS, Databases, Cache Frameworks, and more. With an API-based approach, it is easy to manage the lifecycle of a container along with the configuration that may be required for your services.

### Integration tests with Testcontainers

Some obvious benefits for integration testing with Testcontainers:

- TestContainers can help your tests mirror managed environments and components as closely as possible.   
 - For sourcing secrets and configuration from Vault (includes the various secret backends)and consul.  
 - Integration with cloud providers such as AWS (most commonly used services if not all), GCloud (incubating), Azure (incubating).  
 - Databases such as PostgreSQL or MySQL etc.  
 - Kafka, RabbitMQ for distributed messaging  
 - The list goes on. Check out the full list of Testcontainer [modules](https://github.com/testcontainers/testcontainers-java/tree/master/modules).
- Testing compatibility and tech stack upgrades for client libraries and dependencies such as spring cloud (Vault, Consul, AWS), AWS SDK, etc.

### Why Vault and Consul?

**Secret Management** in microservices needs to be high on your priority list to build secure and scalable microservices. Secrets can be **sensitive** , **dynamic, and time-bound**. They require proper **access control** models with **audit logs** and **encryption**. We also need to support unique **life-cycle** policies and rotations for microservices. While there are a few options that may work for your needs, [**Hashicorp Vault**](https://www.hashicorp.com/products/vault) is certainly the most popular and comprehensive solution in this space.

**Configuration Management** also presents a set of challenges with microservices. Support for static and dynamic configuration, externalized configurations, watching for changes, and updating application configuration without any service disruption are some of the key features we would expect from microservices. In addition, the microservice ecosystem in mature organizations often leads to a web with many microservices deployed but also inter-connected where [**Service Discovery**](https://www.consul.io/use-cases/discover-services) or even a [**Service Mesh**](https://www.consul.io/use-cases/multi-platform-service-mesh) becomes important in a cloud infrastructure to manage endpoint configuration, load balance, security, etc. [**Hashicorp Consul**](https://www.consul.io/)is a great fit for meeting these requirements.

### Testcontainers with Vault

The [Hashicorp Vault Testcontainer](https://www.testcontainers.org/modules/vault/) module aims to solve your app’s integration testing with Vault. You can use it to source static and dynamic credentials for your application as well as test how your application may behave with Vault by writing different test scenarios in Junit such as corner cases like lease rotations, lease expiry, exception handling, etc.

```
<dependency>
    <groupId>org.testcontainers</groupId>
    <artifactId>vault</artifactId>
    <version>${testcontainers.version}</version>
    <scope>test</scope>
</dependency>
```

### Testcontainers for Consul

One of the challenges we hit was that there wasn’t a Testcontainer module for consul. Based on some discussions with the test container teams ([#4680](https://github.com/testcontainers/testcontainers-java/issues/4860)) we decided to fork off an existing [project](https://github.com/denverx/testcontainers-consul), polished it a bit, and published this artifact for the OSS community as part of [Houghton Mifflin Harcourt](https://www.hmhco.com/). This also turns out to be a tiny milestone in some ways as it is our first (out of many more to come) OSS artifact on Sonatype!!!

```
<dependency>
 <groupId>com.hmhco</groupId>
 <artifactId>testcontainers-consul</artifactId>
 <version>0.0.4</version>
 <scope>test</scope>
</dependency>
```

The module supports legacy and newer versions of Consul, ACL, Clustering, and more. The project can be found on [Github](https://github.com/hmhco/testcontainers-consul) and we welcome contributions via pull request and/or the discussion forum for any issues or improvements! Eventually, the hope would be to get this module added to [testcontainers-java](https://github.com/testcontainers/testcontainers-java) and be supported alongside the rest of the modules.

### Some useful pointers when working with Testcontainers

- There may be test scenarios that require a Testcontainer to be able to talk to another in the scope of a test. TestContainers [networking](https://www.testcontainers.org/features/networking/) support makes it easy for you to do that with a generic host address “ **host.testcontainers.internal:{port}** ” when looking up containers and ports that may be exposed.  
- An example would be to write a test when integrating Vault and Consul KV via ACL or even Vault integrating with other secret backends such as AWS (localstack) or Database backends (PostgreSQL or similar.)
- Supports Junit 4 and Jupiter/Junit 5 and Spock. Choose what’s best for your test framework.
- Manage [Testcontainer life-cycles](https://www.testcontainers.org/test_framework_integration/manual_lifecycle_control/)appropriately.   
- Often you may want to re-use a Testcontainer across your tests. This may also help speed up your test phase.
- Use containers for your tests and pipelines. Various patterns including [DIND](https://www.testcontainers.org/supported_docker_environment/continuous_integration/dind_patterns/) are available.  
- Running an image for every test method, image per class, or even running one image for all integration test executions. When sharing an image we need to pay close attention to test data and rollback to clean up the state after test execution.
- Configure testcontainers.properties when working with private docker registries.  
- **hub.image.name.prefix** ={your\_private\_registry}  
- Also see image [dependencies](https://www.testcontainers.org/supported_docker_environment/image_registry_rate_limiting/) for what you may need on your private registry for Testcontainers and getting around any Docker Rate Limiting.
- Configuring logback to see Testcontainer logs is useful at development time and troubleshooting any issues. You could also [stream container logs](https://www.testcontainers.org/features/container_logs/) if you choose to.
- Use container labels and image pull policy as appropriate to make sure you benefit from any caching for images that are not changing often. Please see [advanced](https://www.testcontainers.org/features/advanced_options/) options for more details on this.
- Configure [wait\_timeouts](https://www.testcontainers.org/features/startup_and_waits/) for containers. Also not a bad idea to Assert the container is up before you kick off your tests.

```
Assert.assertTrue(yourContainer.isRunning()));
```

- Follow the **FIRST** principle for your tests as defined in the book [**_Clean Code: A Handbook of Agile Software Craftsmanship_**](https://www.oreilly.com/library/view/clean-code-a/9780136083238/) written by Robert C. Martin. It’s a great read for coding best practices and I highly recommend it!

### Getting Started

- A [quick start](https://github.com/krisiye/springcloud_vault_consul) for a spring-boot service with Spring Cloud Vault and Consul has been provided.   
- This example includes integration tests for Consul KV, Vault Secret Backends (KV, Consul, AWS).   
- It also includes a [docker-compose](https://github.com/krisiye/springcloud_vault_consul/tree/main/spring_cloud_vault_consul_with_acl/docker-compose) recipe for running all of these integrations on your local development environment.

### Summary

The addition of consul as an independent Testcontainer module allows you to integrate Vault and Consul for your test pipelines and add a lot more coverage for your code! I also hope the quickstart examples provided above serve as a good starting point for adding integration tests to your microservice and standardizing your development and test pipelines with best practices for configuration and secret management.

Also, tune in to [HashiTalks 2022](https://events.hashicorp.com/hashitalks2022) on Feb 17/18, 2022 if you are interested in learning more about HashiCorp Vault, Consul, and many more services for your cloud infrastructure.

![Speaker card for Hashitalks 2022.](https://cdn-images-1.medium.com/max/1024/1*0mlYcsgPTjzjdmz95mBILw.png)
_Speaker card for Hashitalks 2022._

If you would like to learn more about Spring Cloud and integration with Vault and Consul for your microservice please join me at the HashiTalks on Feb 17, 12:05–12:35 GMT. We have a great lineup for speakers and topics this year and looking forward to speaking at the event as well as learning a lot more from the Hashicorp user group!

See you there!

* * *