How to create 1040 tests targeting Gitlab in 5 minutes with Humlix
Tags: testing, startup, webdev, productivity
Published: 2020-10-04 13:10:21

[Humlix](https://www.humlix.se) is a brand new web API request builder and quality assurance tool that automatically generates tests. All you need to do is provide examples of how to call your API methods, and Humlix will use it to start doing its magic.

To put Humlix to a real test, we decided to try it on a real industry giant, and GitLab seemed to be a suitable candidate because of its size and that we can run it locally. Therefore, in this post, we will let Humlix create tests targeting GitLab Community Edition (gitlab-ce) and specifically its GitLab Groups API. We will use *gitlab/gitlab-ce:11.2.0-ce.0* docker image and later try to repeat the same scenario on the currently latest version *gitlab/gitlab-ce:13.1.0-ce.0*

#Let’s start with the end
By showing Humlix how to call the */api/v4/groups* *Gitlab Group* API method, we were able to discover a bug that would have been hard to detect using regular unit tests. You can see that Humlix generated 1040 tests towards *gitlab-ce* in the screenshot below and that calling:

*/api/v4/groups?page=461168601842738792*

resulted in a status **500 Internal Server Error**.

![Humlix test result](https://blog.humlix.com/content/images/size/w2400/2020/10/screenshot.png)

We can see the cause of the problem by going into the production.log file for GitLab and see that during the execution of some SQL statement the error was:
>2020-10-02_19:43:52.85686 ERROR:  bigint out of range
>2020-10-02_19:43:52.85689 STATEMENT:  SELECT "namespaces".* FROM "namespaces" WHERE "namespaces"."type" = 'Group' ORDER BY "namespaces"."name" ASC, "namespaces"."id" ASC LIMIT 20 OFFSET 9223372036854775820

![GitLab production log](https://blog.humlix.com/content/images/size/w2400/2020/10/production-log.png)

So in just a couple of minutes, we were able to detect bugs in production code.

A user has already reported this issue in the GitLab issue tracker system. To check if GitLab has fixed the bug, we can repeat the same scenario on the currently latest version *gitlab/gitlab-ce:13.1.0-ce.0*. Humlix once again located the same problem.

![Repeated issue](https://blog.humlix.com/content/images/size/w2400/2020/10/1073-errors-repeat.png)

We can also verify it using this cURL command:

```curl -I localhost/api/v4/groups?page=461168601842738792
HTTP/1.1 500 Internal Server Error```

As you can see, this problem has existed for a long time, even between different versions, but Humlix was able to find the bug within minutes.

#How to generate tests with Humlix
You need to [download Humlix](https://www.humlix.com). Remember to [sign up for the Humlix mailing list](https://www.humlix.com) to stay up to date with the latest news. You can follow the Humlix getting started guide if you don’t know how to run it.

Select the *HOME* menu and click on *ADD SUITE* and create a test suite called *GitLab Suite*.

![Add suite](https://blog.humlix.com/content/images/size/w2400/2020/10/gitlab-suite.png)

Press the *Add request* link and create a new test request called *Get a group list by page*.

![Add request](https://blog.humlix.com/content/images/size/w2400/2020/10/gitlab-request.png)

Go to the *ENVIRONMENT VARIABLES* tab and change the environment name to *GitLab Environment*. Add two variables *base_url* and *gitlab_token*. Set the values according to your setup.

![Environment variables](https://blog.humlix.com/content/images/size/w2400/2020/10/gitlab-environment.png)

Go to the *REQUEST* tab so that you can see the Humlix test request builder. Select *GET* verb and *Http* protocol and add *base_url/api/v4/groups*. Add a new query parameter page of type *integer* according to the picture below.

![Request builder](https://blog.humlix.com/content/images/size/w2400/2020/10/gitlab-request-builder.png)

Now we are ready to perform some tests. To verify that your setup is working, you can run a test with precisely the data you entered in the request builder by pressing the *Run single test* button.

![Run single test](https://blog.humlix.com/content/images/size/w2400/2020/10/single-test-button.png)

If everything works as expected, you should receive something like this in the log area.

![Test result](https://blog.humlix.com/content/images/size/w2400/2020/10/one-off-run.png)

The result above shows that our setup works and that your Gitlab API token is correct.

>Disclaimer: Don’t perform the next step unless you are sure you are targeting a test environment running GitLab that you can re-create without losing anything important. Humlix will generate many requests and could cause a heavy load on production servers.

Now you can press the *Generate tests* button to see some real magic.

![Generate tests](https://blog.humlix.com/content/images/size/w2400/2020/10/1048-errors-1.png)

If you don’t get the same result, you can experiment with the test request settings. E.g., you can change the Max number of generated test calls to 1000 and rerun the test generation.

![Test settings](https://blog.humlix.com/content/images/size/w2400/2020/10/settings.png)

Now you have seen what Humlix is capable of in just a few minutes and how easy it is to let Humlix create tests for you. [Download](https://www.humlix.com) and try it out if you haven’t already!