Use Python to run parallel (and parametrized) Robot Framework tests
Tags: python, testing, concurrency, robotframework
Published: 2019-10-04 09:32:06

---
title: Use Python to run parallel (and parametrized) Robot Framework tests
published: true
description: Parametrize Robot Framework tests and run in parallel
tags: Python, Testing, Concurrency, Robot Framework
---

*Disclaimer:* I am neither an expert regarding the Robot Framework nor parallelization with Python. I simply found this possibility very use- (and learn-)ful and would like to share it whilst trying to explain how it works. 
If you are looking for more sophistiacted ways to execute concurrent Robot tests you should take a look at the [pabot library](https://github.com/mkorpela/pabot) or [Taurus](https://gettaurus.org/docs/Robot/).

## Requirements:
*  Knowledge of (and/or interest in) the Robot Framework
*  Basic Python knowledge


## What?

* ... how to use concurrent subprocesses in Python
* ... to run our Robot test multiple times in parallel
* ... with different parameters without writing multiple tests (for example to allow testing with different test users)

## Why?

&nbsp;&nbsp;&nbsp;👾 simple way performing stress tests against a system
&nbsp;&nbsp;&nbsp;🐢 speed up your testing
&nbsp;&nbsp;&nbsp;🐿️ fun

### Excursion: Robot Framework

From [robotframework.org](https://robotframework.org):

<blockquote cite="https://robotframework.org">
Robot Framework is a generic open source automation framework for acceptance testing, acceptance test driven development (ATDD), and robotic process automation (RPA).
</blockquote>

Its Keyword based syntax is easy to learn and makes it possible to write human readable tests very fast.

I like to use it together with the [Selenium Library](http://robotframework.org/Selenium2Library/Selenium2Library.html) which provides a rich set of keywords for testing web pages and applications.
In cases where those are not sufficient Robot Framework allows you to extend its functionality by:

* Using additional [libraries](https://robotframework.org/#libraries)
* Implementing your [own keywords](http://robotframework.org/robotframework/3.0/RobotFrameworkUserGuide.html#creating-static-keywords) in Java or Python
* Executing Javascript via [Selenium Keywords](http://robotframework.org/Selenium2Library/Selenium2Library.html#Execute%20Javascript) 🎉
* ...

See the [official documentation](https://robotframework.org/#documentation) for more information and posibilites.

#Let's start ⌨️

### Parameterizing Robot tests

We start with a single Robot test file called `suite.robot` containing our test(s). This file contains a global variable for the login name of our test user:

```
${USER}           test-user
...

Login Test
    Go To    ${HOMEPAGE}
    Wait Until Element Is Visible    username
    Input Text    username    ${USER}
    Input Password    password    ${PASSWORD}
    ...

```

Via [command line options](https://robotframework.org/robotframework/latest/RobotFrameworkUserGuide.html#using-command-line-options) you are able to override the default value for this variable when running the test.

For example
```
robot --variable USER:another-user suite.robot
```
runs the test for the user `another-user` (instead of `test-user`).

### Subprocesses with Python

To make use of the above described options via Python we can use `subprocess.Popen` ([doc](https://docs.python.org/2.7/library/subprocess.html#subprocess.Popen)). Above example would be translated into: 

```python
import subprocess
args = ['robot', '--variable', 'USER:another@user.org', 'suite.robot']
subprocess.Popen(args)
``` 

If you try this, you will see that the output of our Robot test gets logged to the console. This is not very helpful if you want to run multiple tests at the same time. 
We can disable this behavior by redirecting the output to `/dev/null` like this:

```python
import os
FNULL = open(os.devnull, 'w')
subprocess.Popen(args, stdout=FNULL)
```

Another argument for `Popen` we will use is `cwd`. It allows us to specify the working directory for the created process.
This is useful since `robot` will create the log files of our test in the current working directory. If we would run all our tests within the same directory those log files would collide. 

In our final script we make sure to create a separate folder per user, containing only the logs of the test for this user:

```python
os.mkdir(userPath)
subprocess.Popen(cmdParts, cwd=userPath, stdout=FNULL)
```

### Put it together

Now we are able to put together a script expecting following files as input:

* `suite.robot` containing our Robot test(s)
* `user_list.txt` containing list of usernames (one per line)

and doing the following:

1. Create a base folder containing our user folders
2. Load `user_list.txt`
3. For each user:
  * a subfolder is created
  * a subprocess is created running our test with the users parameter(s) within its subfolder
4. Wait until all subprocesses finished


Full script:

{% gist https://gist.github.com/mbarde/77ba76f32cb62126a39a11df9fc9fb27 %} 

Possible improvements:

* Make it more dynamic: Create input file containing arbitrary variable parameters per test (not only the username)
* Live view of each test in the console by redirecting `stdout` more sophisticated
* Thread pool management (for example to restrict maximum number of concurrent subprocesses)
* Aggregated report

Thank you for reading :)