Reliable Integration Tests with Testcontainers

-

Integration tests

We, developers, write tests for our code to fall asleep peacefully, knowing that the implemented features meet the requirements defined by the specification and use case scenarios. We cover the code with a layer of unit tests, ensuring that the created abstractions do not rust. However, as they say, unit testing doesn't make a summer.

Therefore, we write integration tests to confirm that the separate system components operate as planned and interact with each other as intended. Components that interact with our applications may include relational databases, noSql databases, message brokers, identification and access systems, full-text search engines, browsers and many, many more. How do we solve the problem of test environments? We want such an environment to be identical to the production environment, because, as the commune rumor is, it promotes progress in programming and good dreams.

Without going into too much detail, there are production environments that cannot be easily copied. They require additional resources (more servers) that often need to be shared with other services, projects. This is expensive to install and maintain, most often it means constant administration of such environments. Maintaining several versions of the production environent is very costly, not talking about licenses. Embeded and in-memory solutions can be used only as an emulator in a very restricted way thus they are often too simple. So what can we do to make our tests reliable?

We bite the bullet, prepare provisions and gather the team, ... then we move on to Mordor.

Well, no! At least, not this time.

If we are writing code in Java, Python, Javascript, .NET, Scala, Rust or Go, we take the Testcontainers. Thanks to this library, we are able to create environment components from the production, that we can use during tests.

 

Testcontainers

What is Testcontainers?

Testcontainers is a library that supports tests by providing lightweight, recyclable instances of the most commonly used components, e.g. databases, message brokers, servers, and everything that can be run in the docker.

What are the requirements?

Very low. Good operating system.

But most of all, one of the requirements we need to ensure in order to use Testcontainers is

  • working docker, and
  • a suitable library for testing (In the case of Java it will be JUnit4 or newer JUnit5. But also Spock, if for some reason you are more fond of Groovy).

 

How it works?

From the first use you can feel the wind in your hair, and the chilly mountain air fills your lungs. We can do anything. The future has no secrets for us and the universe has no limits ...

The production environment includes a MySQL database, Postgres, or maybe Mongo? Not a problem. Testcontainers includes 16 ready-to-use database modules, and all we need is to provide a database image with version. On top of that we can configure the basic settings. Besides, Testcontainers includes support for JDBC, R2DBC. Not convinced?

We need a message broker? The Kafka module, or RabbitMQ is ready for use.

We use the Nginx module, Mockserver, to be able to flood the HTTP server with requests with impunity.

We test Elasticsearch because we can, and if necessary, GCloud and a fully functional AWS cloud stack ... and these are only prepared modules in the Testcontainers library.

If we let our imagination run wild, or the production environment has specific, custom services, we know that we can use a generic container for this purpose, which will create a testcontainer just for us. We can even use the docker compose testcontainer module for that.

So at first, we define modules that reflect the components in the production environment, and then run those modules during the test execution. If the docker image is not in the system, the image is downloaded and the appropriate container is created before running all tests. Together with the containers used in a given test run, another container is launched - the Ryuk, which manages the life cycle of all active containers used for integration tests.

 

Necessary settings

Let's stick to Java. Testcontainers library is distributed as a JAR file, versioned.

It is important to add the main library to the dependency in the Mavena pom.xml file:

or Gradle build.gradle:

If we need a specific module creating the test environment, then we add another dependency, e.g. in Maven:

or Gradle:

 

Running instance test

Let's make it clear and simple.

First, create a test class, annnotate it with @Testcontainers.

Then, initialize a container, e.g.

and start the container executing ... start() method:

That's it. (A bare minimum).

The following code presents a very naive way to test that container is up and running. We create more sophisticated abstractions with a little complex structure (including custom settings) to reflect the production environment as accurately as possible.

 

Who is using Testcontainers?

We, AMB SOFTWARE,

but also, Zipkin, Apache James, Playtika, JetBrains, Instana, Spring Session, Lightbend, Plumbr, Zalando SE, Skyscanner, eBay Marketing, jOOQ, Marquez, and many, many others...

 

At the end of a day

Integration tests are not a cure for everything, but they are the way to prove that the application works with separate environment components in a predictable manner.

Testcontainers is a library. It is just a tool, nothing more. But it greatly helps to focus on delivering business value, implementing use case scenarios, which is an essential part of our profession.

 

More about unit and integration tests:

Testcontainers library official page: https://www.testcontainers.org/

Repository: https://github.com/testcontainers

Social: https://twitter.com/testcontainers?lang=en

 

 

AMB'S TECH INSIGHTS - AN ARTICLE BY LESZEK

Leszek joined the AMB team in 2018 as a java backend engineer with extensive experience in data analysis. Leszek quickly established himself as a good specialist and a great team member.

The main areas of Leszek's activities are the maintenance and development of data processing components based on Kafka, monitoring & health-check of system components using Grafana, maintenance & deployment of components on the Google Cloud Platform.

 

 

DO YOU NEED HELP WITH WEB DEVELOPMENT? CONTACT US AT

CONSULTING@AMBSOFT.DE