Easy Integration Testing with Venom!
Tags: testing, integration
Published: 2022-11-07 08:47:17

---
title: Easy Integration Testing with Venom!
published: true
description: Discover how to create and execute Integration Tests (IT) with YAML files thanks to Venom
tags: testing, integration
cover_image: https://dev-to-uploads.s3.amazonaws.com/uploads/articles/93b9pcvzh17dv4fhjhsc.png
---

When working in the development of applications / and websites, the diction “testing is life” is true and we will see in this article that it is not only unit tests and that creating and automating integration tests can be child's play ;-).

# Integration tests, what for?

You have participated in the development of a site or an application, you have, of course, coded the unit tests (UT) and now you are “bored” with the integration tests, but what is that again?

Integration tests make it possible to test the functionalities of a system on all the components built via user scenarios.

With unit tests, we check the proper functioning of a specific part of an application/software, of a unit, while integration tests, as its name suggests, make it possible to test all the components in functional scenarios:

* Does the website http://www.mysupersite.com respond (return code 200)?
* Does it respond in less than 1 second?
* Does the call to the URL http://www.mysupersite.com/mysuperjson.json contain an array composed of two elements composed of the key “id” and the key “description”
* …

In the testing phase, integration testing falls between unit testing and validation testing.

The objective of integration tests is to detect errors that could not be detected (because they were not tested) during the unit testing phase.

# Venom

To write and run our integration tests, we'll use **Venom**.
Venom is a tool created and made open-source by OVHcloud: https://github.com/ovh/venom

![Venom](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/hw7x45f69r2dxz3g26kg.png)

The tool, written in Go (and yes another one ;-) ), dates from 2017, it is open-sourced and I learned about it following a presentation made during an edition of [Devoxx France](https://www.devoxx.fr/) conference.
We found it very promising at the time and decided to use it for our integration tests. A few years later of use in one of my previous professional experiences, I am still happy to talk about this great little tool and this will be an opportunity to test the new features of version 1.O+.

Concretely, it is necessary to write test suites (`testsuite`) in a YAML file. Venom runs executors (scripts, HTTP request, web, IMAP, etc.) and enforces assertions. It can also generate xUnit result files.

The tool has a list of existing executors:

* amqp: https://github.com/ovh/venom/tree/master/executors/amqp
* dbfixtures: https://github.com/ovh/venom/tree/master/executors/dbfixtures
* exec: https://github.com/ovh/venom/tree/master/executors/exec (exec is the default type for a step)
* grpc: https://github.com/ovh/venom/tree/master/executors/grpc
* http: https://github.com/ovh/venom/tree/master/executors/http
* imap: https://github.com/ovh/venom/tree/master/executors/imap
* kafka https://github.com/ovh/venom/tree/master/executors/kafka
* mqtt https://github.com/ovh/venom/tree/master/executors/mqtt
* odbc: https://github.com/ovh/venom/tree/master/executors/plugins/odbc
* ovhapi: https://github.com/ovh/venom/tree/master/executors/ovhapi
* rabbitmq: https://github.com/ovh/venom/tree/master/executors/rabbitmq
* readfile: https://github.com/ovh/venom/tree/master/executors/readfile
redis: https://github.com/ovh/venom/tree/master/executors/redis
* smtp: https://github.com/ovh/venom/tree/master/executors/smtp
* sql: https://github.com/ovh/venom/tree/master/executors/sql
* ssh: https://github.com/ovh/venom/tree/master/executors/ssh
* web: https://github.com/ovh/venom/tree/master/executors/web

For my part, I mainly use two of them: `exec` and `http`, which already cover a large majority of the cases to be tested.

But if this list is not enough for you, you can create your own executors ;-).

The tool has a list of assertions/statements, here is a non-exhaustive list:

* ShouldEqual
* ShouldNotEqual
* ShouldAlmostEqual
* ShouldBeNil
* ShouldNotBeNil
* ShouldBeTrue
* ShouldBeFalse
* ShouldBeZeroValue
* ShouldBeGreaterThan
* ShouldBeGreaterThanOrEqualTo
* ShouldBeLessThan
* ShouldBeLessThanOrEqualTo
* ShouldBeBetween
* ShouldNotBeBetween
* ShouldContain
* ShouldNotContain
* ShouldContainKey
* ShouldNotContainKey
* ShouldBeIn
* ShouldNotBeIn
* ShouldBeEmpty
* ShouldNotBeEmpty
* ShouldHaveLength
* ShouldStartWith
* ShouldEndWith
* ShouldBeBlank
* ShouldContainSubstring
* ShouldNotContainSubstring
* ShouldBeChronological
* ShouldNotExist
* ...

Full list: https://github.com/ovh/venom

Venom runs integration tests which must be listed in a YAML format file.

In these test scenarios, Venom provides common variables:

* `{{.venom.testsuite}}`
* `{{.venom.testsuite.filename}}`
* `{{.venom.testcase}}`
* `{{.venom.teststep.number}}`
* `{{.venom.datetime}}`
* `{{.venom.timestamp}}`

One of the handy features of Venom is that we can use the output of a test case as a parameter or input to a next test case.

For example, we can query a web service that returns a JSON containing two elements: an ID and a user name for example, then reuse the user ID as a query parameter for a second web service.

We will see this case in more detail and as an example later in the article.

# Installation

It has been pointed out to me that I very often put a certain expression in (all?) my articles, so I will not break the rule and use it again:
"Let's start with the beginning", let's start by installing Venom on our machine.

## Go, go, go!

We can now install the Venom tool on our machine. Here are the commands to run:

* For linux:

```bash
$ curl https://github.com/ovh/venom/releases/download/v1.0.1/venom.linux-amd64 -L -o /usr/local/bin/venom && chmod +x /usr/local/bin/venom
```

* For MacOS:

```bash
$ curl https://github.com/ovh/venom/releases/download/v1.0.1/venom.darwin-amd64 -L -o bin/venom
```

And that's all … ;-). The tool is installed and ready to use! 

To test it, we will execute the help command of the tool:

```bash
$ venom -h
Venom - RUN Integration Tests

Usage:
  venom [command]

Available Commands:
  help        Help about any command
  run         Run Tests
  update      Update venom to the latest release version: venom update
  version     Display Version of venom: venom version

Flags:
  -h, --help   help for venom

Use "venom [command] --help" for more information about a command.
```

# Let's understand the tool with an example

To understand how Venom works, let's create and run a first test suite together. The first assertions we are going to make, in this test suite, consist in checking whether the site we want to test (a public REST API, OVHcloud API for example):
* is reachable (respond with a 200 status code)
* responds in less than 5 seconds
* returns a valid response (in JSON format)

Start by creating your test suite in a file named `testsuite.yml` for example.
Open it in your favorite editor or IDE and fill it with this content:

```yaml
name: APIIntegrationTest

vars:
  url: https://eu.api.ovh.com/

testcases:
- name: GET http testcase, with 5 seconds timeout
  steps:
  - type: http
    method: GET
    url: {{.url}}/1.0/
    timeout: 5
    assertions:
    - result.statuscode ShouldEqual 200
    - result.timeseconds ShouldBeLessThan 1
    - result.bodyjson ShouldContainKey apis
    - result.body ShouldContainSubstring /dedicated/server
    - result.body ShouldContainSubstring /ipLoadbalancing
```

And to run our test suite, just call the run command and pass it the path to your yml file as a parameter, if in your directory you have several yml files containing test suites. Or you just need to execute only the `venom run` command (without argument), if there is only one testsuite in your folder:

```bash
$ venom run

 • APIIntegrationTest (testsuite.yml)
     • GET-http-testcase-with-5-seconds-timeout SUCCESS
```

You have written and run your first test suite with the HTTP Runner, congratulations! :)

Now, let's dissect this test suite:

```yaml
name: APIIntegrationTest (1)

vars: (2)
  url: https://eu.api.ovh.com/

testcases: (3)
- name: GET http testcase, with 5 seconds timeout (4)
  steps:
  - type: http (5)
    method: GET
    url: {{.url}}/1.0/
    timeout: 5
    assertions: (6)
    - result.statuscode ShouldEqual 200
    - result.timeseconds ShouldBeLessThan 1
    - result.bodyjson ShouldContainKey apis
    - result.body ShouldContainSubstring /dedicated/server
    - result.body ShouldContainSubstring /ipLoadbalancing
```

(1) A test suite (testsuite) must have a name
(2) You can define variables that you can use in test cases
(3) It is composed of one or more test cases (testcase)
(4) A test case is composed of a name and a sequence of steps
(5) A step is composed of a type (exec by default), a script (if type = exec) as well as assertions (tests to be performed)
(6) The assertions (affirmations) are the tests that we will do to validate that our application/site/script works as expected

# Let's write our integration tests

We are now going to write a suite of integration tests that will allow us to get to the heart of the matter, to see several Venom executors and to realize that it is good to know some tips when doing Venom to avoid wasting several hours on a problem.

Let's start by creating a yaml file named “testsuite.yaml”:

```yaml
name: IntegrityTest

vars:
  url: https://dog.ceo/api

testcases:
```

The first test we are going to do, in this test suite, is to test whether the site we want to test (a public REST API):
* is accessible
* if its answer is not empty
* if the expected elements are in its response (which is in JSON format)

```yaml
name: IntegrityTest

vars:
  url: https://dog.ceo/api

testcases:
- name: GetDogs
  steps:
  - type: http
    method: GET
    headers:
       Accept: application/json
       Content-Type: application/json
    url: "{{.url}}/breeds/list/all"
    retry: 1
    delay: 2
    assertions:
    - result.statuscode ShouldEqual 200
    - result.bodyjson ShouldContainKey message
```

In this small example we use the notion of variable which is to be defined in the `vars` block at the top of the yaml file. We define a url variable.
This variable can be called and reused throughout the file thanks to the formalism `{{.myvarname}}`.

Once we've made the API call, we're testing two things:
* that the result of the HTTP code is 200 (everything is fine)
* that the JSON we retrieve contains the message key (it does because the JSON starts like this): `{"message":{"affenpinscher":[],"african":[],"airedale" :[],"akita":[],"appenzeller":[],"basenji":[],"beagle":[],"bluetick":[],"borzoi":[],"bouvier": [],"boxer":[],"brabancon":[],"briard":[],"buhund":["norwegian"],"bulldog":["boston","english","french" ...]...}}`

We do not change a winning team, we will test our test case:

```bash
$ venom run testsuite.yaml
 • IntegrityTest (testsuite.yaml)
     • GetDogs SUCCESS
```

Great, the test works ;-).

Concretely, our JSON (once formatted in a readable way) is presented in this form:

```json
{
    "message": {
        "affenpinscher": [],
        "african": [],
        "airedale": [],
        "akita": [],
        "appenzeller": [],
        "basenji": [],
        "beagle": [],
        "bluetick": [],
        "borzoi": [],
        "bouvier": [],
        "boxer": [],
        "brabancon": [],
        "briard": [],
        "buhund": [
            "norwegian"
        ],
        "bulldog": [
            "boston",
            "english",
            "french",
          ...
        ],
        ...
      }
}
```

To go even further, we are now going to explore our JSON in order to test if in our JSON we have a **bulldog** attribute that contains an array and that its first element is equal to “bulldog”.

We therefore add an assertion that tests the existence of a “**boston**” element as the first element of the **bulldog** array:

```yaml
    assertions:
    - result.statuscode ShouldEqual 200
    - result.bodyjson ShouldContainKey message
    - result.bodyjson.message.bulldog.bulldog0 ShouldEqual "boston"
```

Let's run the test suite again:

```bash
$ venom run testsuite.yaml
 • IntegrityTest (testsuite.yaml)
     • GetDogs SUCCESS
```

Cool, our test case still works.

Note that a nice feature of Venom is that you can reuse a value retrieved from a result of a test case, in another test case.
So if we want to retrieve the first sub-breed of bulldog retrieved above, it will suffice to call it like this:

```yaml
{{.GetDogs.result.bodyjson.message.bulldog.bulldog0}}
```

We are going to use it to make an API call that lists the images available for **boston** type **bulldogs**:

```yaml
- name: GetBostonBulldog
  steps:
  - type: http
    method: GET
    url: "https://dog.ceo/api/breed/bulldog/{{.GetDogs.result.bodyjson.message.bulldog.bulldog0}}/images"
    retry: 1
    delay: 2
    assertions:
    - result.statuscode ShouldEqual 200
```

Another cool feature is writing test cases that run a CLI or a script, for example:

```yaml
- name: GetDogName
  steps:
  - script: echo 'bulldog'
    assertions:
    - result.code ShouldEqual 0
```

This test case executes `echo ‘bulldog’` and tests if the result of the script is successful or not (exit code equal to 0).

If we run a `venom run` on our three test cases, we can see that the test suite still works:

```bash
$ venom run testsuite.yaml
 • IntegrityTest (testsuite.yaml)
     • GetDogs SUCCESS
     • GetBostonBulldog SUCCESS
     • GetDogName SUCCESS
```

One thing to know is that if you need to run a script on several lines, there is a trick to know for it to work. The “multi line script” works with this formalism for the script executor:

```yaml
- name: Says hi to readers
  steps:
  - script: |
           #!/bin/bash
           echo “salut les lecteurs” | base64
    assertions:
       - result.code ShouldEqual 0
       - result.systemout ShouldNotEqual ""
       - result.systemout ShouldEndWith "=="
       - result.systemerr ShouldEqual ""
```

You must imperatively put a pipe “|”.

# Bonus tracks

## Running integration tests automatically

When you write your integration tests, very quickly comes the need for automatic execution of these, on several different environments.

Is it possible ? I have good news for you, the answer is yes.

You can, create a Docker image containing Venom, use it in your CI/CD chain, and depending on the environment on which you want to run your tests, you just have to pass your parameters from your CI tool: Jenkins/Gitlab /CircleCI…, until running the tests through Venom.

How is it possible ? Thanks to the ability to pass variables/parameters to our testsuite when running the latter, thanks to setting a variable in the testsuite and using the `--var` parameter during the `venom run`:

`mytestsuite.yml`:

```yaml
name: IntegrityTest

vars:
  url: https://myurl.dev/myendpoint

testcases:
- name: CreateUser
  steps:
  - type: http
    method: POST
    headers:
       Accept: application/json
       Content-Type: application/json
    url: "{{url}}/v1/user"
    retry: 1
    delay: 2
    assertions:
    - result.statuscode ShouldEqual 201
    - result.bodyjson ShouldContainKey id
    extracts:
      result.bodyjson.id: '{{license=.+}}'
```

```bash
$ venom run mytestsuite.yml --var url=http://myurl.dev/myendpoint
```

The example above takes a URL as a parameter, with a default value, and tests an HTTP call with the POST method to the url http://myurl.dev/myendpoint/v1/user, passing in the headers an Accept and a Content-Type and validating if the HTTP response code is a 201 code AND that in the response, in JSON format, there is an element named “id”.

## Export of the report in xUnit format

Venom allows an export of the result as a report in xUnit format:

```bash
$ venom run testsuite.yaml  --format=xml --output-dir="test-results"
 • IntegrityTest (testsuite.yaml)
     • GetDogs SUCCESS
     • GetBostonBulldog SUCCESS
     • GetDogName SUCCESS
Writing file test-results/test_results.xml
```

Let's look at the contents of this output XML file:

```bash
$ cat test-results/test_results.xml
<?xml version="1.0" encoding="utf-8"?><testsuites>
  <testsuite name="IntegrityTest" package="testsuite.yaml" tests="3">
    <testcase classname="testsuite.yaml" name="GetDogs" time="0.415826625">
      <system-out></system-out>
      <system-err></system-err>
    </testcase>
    <testcase classname="testsuite.yaml" name="GetBostonBulldog" time="0.1746975">
      <system-out></system-out>
      <system-err></system-err>
    </testcase>
    <testcase classname="testsuite.yaml" name="GetDogName" time="0.047147083">
      <system-out><![CDATA[bulldog]]></system-out>
      <system-err></system-err>
    </testcase>
  </testsuite>
</testsuites>
```

I find this export super practical as a step to add to its pipeline of its CI/CD chain.

## Running multiple test suites

If you want to run several test suites, without having to launch several `venom run` it is possible:

```bash
$ venom run *
 • APIIntegrationTest (tests/testsuite.yml)
     • GET-http-testcase-with-5-seconds-timeout SUCCESS
 • IntegrityTest (testsuite.yaml)
     • GetDogs SUCCESS
     • GetBostonBulldog SUCCESS
     • GetDogName SUCCESS
```

Or if your YAML are in a “tests” directory:

```bash
$ venom run tests/*
 • Title of TestSuite (tests/test.yaml)
     • TestCase-with-default-value-exec-cmd-Check-if-exit-code-1 SUCCESS
     • Title-of-First-TestCase SUCCESS
 • APIIntegrationTest (tests/testsuite.yml)
     • GET-http-testcase-with-5-seconds-timeout FAILURE
```

# Conclusion

Venom is a small tool very practical for writing and running integration tests. The only downside is that it's not well known. A few years ago I regretted the fact that its documentation was not thorough enough and that there was slowness concerning the processing of issues and PRs and I am happy to see that this problem is corrected and that the tool is actively maintained :).