Chapter 5: Testing Multiple Environments with TOX.
Tags: python, pytest, tox, testing
Published: 2022-10-21 12:24:23

- This chapter is part of the series: {% embed https://dev.to/zim95/understanding-the-python-tox-ecosystem-3noi %}
- This tutorial builds upon the project structure from Chapter 3 and Chapter 4. I suggest understanding Chapter 3 and Chapter 4 before moving forward in order to understand the project structure.
    - Chapter 3: {% embed https://dev.to/zim95/chapter-3-using-pytest-mypy-and-flake8-for-testing-30cl %}
    - Chapter 4: {% embed https://dev.to/zim95/chapter-4-a-quick-tour-of-pytest-264d %}
- So far, this is the structure of our code.
    
    ```markdown
    - src/
    	- something/
    		- __init__.py
    		- py.typed
    		- app.py
    - tests/
    	- __init__.py
    	- conftest.py
    	- test_app.py
    - .gitignore
    - LICENSE
    - pyproject.toml
    - README.md
    - requirements.txt
    - setup.cfg
    - setup.py
    
    ```
- ### Table of contents:
   - [5.1 - The Need for Tox and the Tox configuration file](#part-5_1)
   - [5.2 - Understanding the Tox environments and variables](#part-5_2)
   - [5.3 - Understanding the flow of Tox environments](#part-5_3)
   - [5.4 - Executing TOX](#part-5_4)

### 5.1 The Need for Tox <a name="part-5_1"></a>
- The code so far has run on our local environment.
- How do we know that the code runs on multiple environments? Since we are building packages, we should know which versions of python can install our package and run it.
- This is where `Tox` comes in. To work with `Tox` we need a `tox.ini` file which is `Tox`'s own configuration file.
- Let’s consider the following `tox.ini` file in the root directory of the project.
    
    ```
    [tox]
    minversion = 3.8.0
    envlist = py36, py37, py38, py39, flake8, mypy
    isolated_build = true
    
    [gh-actions]
    python =
        3.6: py36, mypy, flake8
        3.7: py37
        3.8: py38
        3.9: py39
    
    [testenv]
    setenv =
        PYTHONPATH = {toxinidir}
    deps =
        -r{toxinidir}/requirements_dev.txt
    commands =
        pytest --basetemp={envtmpdir}
    
    [testenv:flake8]
    basepython = python3.6
    deps = flake8
    commands = flake8 src tests
    
    [testenv:mypy]
    basepython = python3.6
    deps =
        -r{toxinidir}/requirements_dev.txt
    commands = mypy src
    ```
    
- To read more about `tox` file structure, we can read: [https://tox.wiki/en/latest/config.html](https://tox.wiki/en/latest/config.html).


### 5.2 Understanding the Tox environments and variables <a name="part-5_2"></a>

- We can read about all of this from `[tox.wiki](http://tox.wiki)` ,  [https://tox.wiki/en/latest/config.html](https://tox.wiki/en/latest/config.html).
- All global settings are defined in the `[tox]` section. In our case,
    
    ```
    [tox]
    minversion = 3.8.0
    envlist = py36, py37, py38, py39, flake8, mypy
    isolated_build = true
    ```
    
    - `minversion` is the minimum version of `tox-library` required to parse this `tox` file.
        - `3.8.0` is the minimum version of `tox-library` required to parse this `tox` file.
    - `envlist` is for determining the environment list that `tox`is to operate on and so on.
    - There are many other global setting variables that are available in the `[tox.wiki](http://tox.wiki)` website. We do not need to understand everything as of now.
- Test environments are defined under the `testenv`section and individual `testenv:NAME`sections, where `NAME`is the name of a specific environment. This section also has a lot of options to set. For our case, we will go through:
    - `basepython`: Name or path to a Python interpreter which will be used for creating the virtual environment.
    - `deps`: Environment dependencies. Installed usually from requirements file or manually written.
    - `commands`: The commands to be called for testing. Only execute if **`[commands_pre](https://tox.wiki/en/latest/config.html#conf-commands_pre)`** succeeds. `commands_pre` is another option we have not talked about.
    - `setenv`: Each line contains a NAME=VALUE environment variable setting which will be used for all test command invocations.
- We might also notice `{toxinidir}` which is a variable inbuilt in `tox`. We can learning more about changing its working directory: [https://stackoverflow.com/questions/52503796/change-tox-workdir](https://stackoverflow.com/questions/52503796/change-tox-workdir).
    - For now, we just need to understand that: from `tox global settings`section from `tox` documentation, the `.tox` directory which is working dir, is created in directory where `tox.ini`is located.
    - Turns out `tox` also creates a working directory just like `git` does.
- We also have `{envtmpdir}` which is another variable inbuilt in `tox`.
    - `{envdir}` basically points to the `virtualenv` directory that `tox` creates.
    - If we do `{envdir}/tmp` it points to the `tmp` directory inside the `virtualenv`.
    - This is what `{envtmpdir}` points to. Basically `{envtmpdir} = {envdir}/tmp`.
    - It will be cleared each time before the group of test commands is invoked. In our case, we have three group of test commands: a `[test]` which is inbuilt in `tox` and two `test:NAME` which are our custom test groups.
    - There are other similar variables: `{envlogdir}={envdir/log}`. It defines a directory for logging where `tox` will put logs of `tool invocation`.
- **`tox-gh-actions`**is a `tox` plugin which helps running `tox` on `GitHub Actions` with multiple different `Python versions` on multiple workers in parallel.

### 5.3 Understanding the flow of Tox environments <a name="part-5_3"></a>

- `Tox` allows us to create a bunch of different `virtual environments`, `install` our `package` into those `environments` and then run the `test groups` on each of those `environments`.
- The first four `py36, py37, py38, py39,` are `built-in` versions of `python` that `tox` already knows about.
    
    ```
    [tox]
    minversion = 3.8.0
    envlist = py36, py37, py38, py39, flake8, mypy
    isolated_build = true
    ```
    
- Their configuration goes in the `testenv` block and the tests are run in those environments with the following commands.
    
    ```
    [testenv]
    setenv =
        PYTHONPATH = {toxinidir}
    deps =
        -r{toxinidir}/requirements_dev.txt
    commands =
        pytest --basetemp={envtmpdir}
    ```
    
    - We set the `PYTHONPATH` to `{toxinidir}`. From the previous section we know it equals to `PYTHONPATH=<top level of project directory>`. Since, we know `toxinidir` points to wherever the `tox.ini` file is located. In our case, it is located at the top level of the project directory.
    - As you can see, in `testenv` , we have set `deps=-r{toxinidir}/requirements_dev.txt`. From the previous section we know it equals to `deps=-r <project_directory>/requirements_dev.txt`. Since, we know `toxinidir` points to wherever the `tox.ini` file is located. In our case, it is located at the top level of the project directory.
    - We install the requirements and then run the `pytest --basetemp=<virtualenv>/tmp` command in all of the environments passed along. Since, we know that `{envtmpdir}` points to `<virtualenv>/tmp` directory.
- Then we have the last two environments: `flake8`, `mypy`.
    
    ```
    [tox]
    minversion = 3.8.0
    envlist = py36, py37, py38, py39, flake8, mypy
    isolated_build = true
    ```
    
- `flake8` and `mypy` are not `built-in` environments in `tox`. Therefore, we have to create separate `testenv:NAME` block for them. Which is why we added:
    
    ```
    [testenv:flake8]
    basepython = python3.6
    deps = flake8
    commands = flake8 src tests
    
    [testenv:mypy]
    basepython = python3.6
    deps =
        -r{toxinidir}/requirements_dev.txt
    commands = mypy src
    ```
    
- Here, `testenv:flake8` is the `flake8` virtual environment and `testenv:mypy` is the `mypy` virtual environment.
- Since `flake8` and `mypy` are not versions of python and only commands that we want to run, we need to specify which version of python we want to run on. We do this on `basepython`.
- `deps` again are all the dependencies. We can specify individual dependencies or we can specify the requirements file.
- `commands` is the actual command that runs in these blocks.
- We also have, `gh-actions` block which we will discuss in the next chapter.

### 5.4 Executing TOX <a name="part-5_4"></a>

- Once we have the `tox.ini` file and the `tox` command.
- To get the `tox` command, `pip install tox`.
- We can now just run the `tox` command and everything will be automated.
    
    ```bash
    $ tox
    ```
    
- When we run it, we notice that it takes very long. Why is that?
    - When we run the tests locally, it runs on our virtual environment.
    - But like we discussed, `tox` creates a new virtual environment, install everything into that and run tests in all of those environments.
- It is advised to run `tox` only before commit and push. Otherwise, just use `pytest` , to save time during development.