Chapter 1: The Need For Test Automation
Tags: python, tox, testing, pytest
Published: 2022-10-21 10:42:07

- This chapter is part of the series: {% embed https://dev.to/zim95/understanding-the-python-tox-ecosystem-3noi %}
- At first, we need to understand, how we can test python code without a testing library.
- We will then look into how to test with an existing library.
- Then we will look at how `tox` fits in the ecosystem.

### 1.1 Testing Without A Library

- Consider the following code structure of a Python application:
    
    ```markdown
    - README.md
    - LICENSE
    - .gitignore
    - app.py
    ```
    
- We may have various `assert statements` written inside `app.py`. However, these `tests` need to be executed manually by calling the function.
- Consider the following code for `app.py`.
        
  ```python
  def simple_calculator_function(evaluation_string: str) -> typing.Any:
    '''
    Reads an evaluation string.
    Evaluates it and returns the result.
    For example, eval("5*(4+5)") = 45.
    
    params:
        evaluation_string: str: The string that will be evaluated.
    '''
    try:
        return eval(evaluation_string)
    except Exception as e:
        raise Exception(e)
        
   def test_simple_calculator_function() -> None:
        assert simple_calculator_function("5*(4+5)") == 45 # test addition and multiplication
        assert simple_calculator_function("10 - (100/2)") == -40 # test subtraction and division. 
        assert simple_calculator_function("'a' + 'b'") == 'ab' # test string concatenation
   ```
- As you can see, `test_simple_calculator_function` will not run automatically unless called. We want to automate this.
- The first step is to create a separate `test_file` which uses libraries to write tests. The new project code would be:
    
    ```markdown
    - README.md
    - LICENSE
    - .gitignore
    - app.py
    - test_app.py
    ```
    
- We have added a `test_app.py` which will work on libraries.

### 1.2 USING AN EXISTING TESTING LIBRARY

- We have libraries such as the `unittest` library using which we can write test subclasses that are auto-detected and run automatically.
- Example `unittest` subclass:
    
    ```python
    import unittest
    from app import simple_calculator_function
    
    class TestSomething(unittest.TestCase):
    	def test_simple_calculator_function(self) -> None:
    		self.assertEqual(simple_calculator_function("5*(4+5)"), 45)
    		self.assertEqual(simple_calculator_function("10-(100/2)"), -40)
    		self.assertEqual(simple_calculator_function("'a' + 'b'"), 'ab')
    ```
    
- We can run this code with `unittest` tool in the shell as:
    
    ```bash
    $ python -m unittest discover -s './' -p 'test_*.py'
    ```
    
    - Here, `unittest discover` discovers all the subclasses of `unittest.TestCase`, i.e. those classes that have inherited `unittest.TestCase`.
    - `-s` flag denotes the start directory of where the tests are located. In this case, we can stay in the current working directory.
    - `-p` flag denotes the pattern of the names of test files. In this case, the test file naming pattern is `test_*.py` , where `*` can be replaced with any name.
- We also have the `pytest` library which is not inbuilt and needs to be installed separately.
  ```bash
  $ pip install pytest
  ```
- For example, `pytest` test file.
    
    ```python
    import pytest
    from app import simple_calculator_function
    
    def test_simple_calculator_function() -> None:
        assert simple_calculator_function("5*(4+5)") == 45 # test addition and multiplication
        assert simple_calculator_function("10 - (100/2)") == -40 # test subtraction and division. 
        assert simple_calculator_function("'a' + 'b'") == 'ab' # test string concatenation
    ```
    
- `pytest` can autodetect tests. We can just run `pytest` in the shell and all tests will run.
   ```bash
   $ pytest
   ```
- The test detection algorithm also known as the test discovery standard is is used to detect tests by default unless external modifications are made.
- There are other testing frameworks such as `Nose`, `DocTest`, etc. which will not be covered in this article.

### 1.3 TOX USE CASE IN THE ECOSYSTEM

- `Tox` is just another automation tool for testing.
- What exactly does it automate?
    - Since we are building a package, we need to make sure that our package runs on different versions of Python.
    - The tests that we have in `pytest` will run on the local version of the `virtualenv` that we are currently on. So, it will only check one version of python.
    - We need to make sure these tests run passes on all versions of Python.
    - This is what `Tox` automates. Multi-version testing.
- How does `Tox` do this?
    - `Tox` creates a virtual environment for each version of python and/or the operating system that we specify.
    - It runs the tests in all those environments and makes sure that our package works on those versions as well.
- When we run things on `github actions` :
    - We need to map `Tox` commands to `github actions` , essentially because `github actions` is completely capable of doing what `Tox` is doing.
    - We are only installing `Tox` to automate testing locally on our machines while developing.
    - Since we have a working `Tox` file, why create another `github action` section for the thing already automated in `Tox`.
    - We just map `Tox` environments to `github action` environments.