Testing in Python
Tags: testing, python, opensource, programming
Published: 2023-11-09 22:16:47

[Automated Testing](https://www.atlassian.com/continuous-delivery/software-testing/automated-testing) is the backbone of **quality software**. Without having tests setup in your project, there is no way to ensure if your code works as expected, and becomes next to impossible to keep it going in the **right direction** as more people start contributing to it. It is crucial to setup automated tests for complex projects, as manual testing or [exploratory testing](https://www.globalapptesting.com/blog/what-is-exploratory-testing) after each iteration of your product is simply not feasible without a dedicated team of QA Analysts.

{% embed https://www.youtube.com/embed/u6QfIXgjwGQ?si=jEaM8qvML1Epy6iC %}


There are various types of tests like [end-to-end tests](https://circleci.com/blog/what-is-end-to-end-testing/?utm_source=google&utm_medium=sem&utm_campaign=sem-google-dg--uscan-en-dsa-tCPA-auth-nb&utm_term=g_-_c__dsa_&utm_content=&gclid=Cj0KCQiAo7KqBhDhARIsAKhZ4uj7r0VfXZdZeR-F_0Zre8PfU1ZWczcyAbdDVUBBWqnhJ9UaeFYpk1waArdgEALw_wcB), [unit tests](https://en.wikipedia.org/wiki/Unit_testing), [integration tests](https://katalon.com/resources-center/blog/integration-testing), [performace tests](https://en.wikipedia.org/wiki/Software_performance_testing) etc., each differing with its approach of testing.

In this post, I'll be sharing how I set up both **unit** and **integration** testing in my python project [til-page-builder](https://github.com/Amnish04/til-page-builder), and how it helped me detect some hidden issues.

## Table of Contents
&nbsp;1. [Unit Testing 👨‍🔬](#unit-testing)
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 1.1. [Pytest 🐍](#pytest)
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 1.2. [pytest-watch 🔎](#pytestwatch)
&nbsp;2. [Integration Tests ⚙️](#integration-tests)
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 2.3. [Writing my first integration test](#writing-my-first-integration-test)
&nbsp;3. [Coverage 📈](#coverage)
&nbsp;4. [Makefile](#makefile)
&nbsp;5. [Conclusion 🎇](#conclusion)

## Unit Testing 👨‍🔬

Unit testing is an approach of automated testing in which the **smallest testable parts** of an application, called units, are individually run against certain scripts to verify for proper operation. These **units** could be indiviual **functions**, **classes** or **methods**that you write to implement functionalities for your application.

### Pytest 🐍

After looking through the various unit testing tools available for Python like [pytest](https://docs.pytest.org/en/7.4.x/), [unittest](https://docs.python.org/3/library/unittest.html) (built-in), and [nose](https://nose.readthedocs.io/en/latest/testing.html), I went with **pytest** for its simlpicity and ease of use.

![Pytest image](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/qyi6g162ab6yfpnjtxpg.png)

I took the following steps to setup **pytest** in my project.

**1. Installing Pytest**

Execute the following command from your terminal.

```
pip install pytest
```

**2. Setting up pytest configuration**

Create a `pytest.ini` file at the root of your project.

[**pytest.ini**](https://github.com/Amnish04/til-page-builder/blob/master/pytest.ini)
```ini
[pytest]
minversion = 7.0

# Directories to look for the test files
testpaths = tests

# Directories to look for the source code
pythonpath = src
```

**3. Writing my first unit test**

I started out by testing the [HeadingItem](https://github.com/Amnish04/til-page-builder/blob/master/src/builder/toc_generator/heading_item.py) class I was using for generating a TOC after parsing html.

[**tests/builder/toc_generator/test_heading_item.py**](https://github.com/Amnish04/til-page-builder/blob/master/tests/builder/toc_generator/test_heading_item.py)
```python
from builder.toc_generator.heading_item import HeadingItem


class TestHeadingItem:
    """
    Test suite for 'HeadingItem' class
    """

    class TestHeadingItemConstructor:
        """
        Tests for 'HeadingItem' class __init__ method
        """

        def test_default_arguments(self):
            """
            Test to verify the default values for HeadingItem constructor
            """
            heading = HeadingItem()

            assert heading.value == HeadingItem.DEFAULT_HEADING_VALUE
            assert heading.id == HeadingItem.generate_heading_id(
                HeadingItem.DEFAULT_HEADING_VALUE
            )
            assert heading.children == HeadingItem.DEFAULT_CHILDREN_VALUE

        def test_value_argument(self):
            """
            Test to verify the supplied value property is correctly set
            """
            sample_heading_value = "This is a sample heading"
            heading = HeadingItem(sample_heading_value)

            assert heading.value == sample_heading_value
            assert heading.id == HeadingItem.generate_heading_id(sample_heading_value)
            assert heading.children == HeadingItem.DEFAULT_CHILDREN_VALUE

        def test_value_and_children(self):
            """
            Test to verify the supplied value and children properties are correctly set
            """
            deep_nested_heading_1 = HeadingItem("1.1.1")
            deep_nested_heading_2 = HeadingItem("1.1.2")

            nested_heading_1 = HeadingItem(
                "1.1", [deep_nested_heading_1, deep_nested_heading_2]
            )
            nested_heading_2 = HeadingItem("1.2")

            top_heading = HeadingItem("1", [nested_heading_1, nested_heading_2])

            # Check for values
            assert top_heading.value == "1"
            assert nested_heading_1.value == "1.1"
            assert nested_heading_2.value == "1.2"

            # Check nested values
            assert top_heading.children[0].value == "1.1"
            assert top_heading.children[1].value == "1.2"

            # Check deep nested values
            assert top_heading.children[0].children[0].value == "1.1.1"
            assert top_heading.children[0].children[1].value == "1.1.2"

            # Check if children are correctly set
            assert nested_heading_1 in top_heading.children
            assert nested_heading_2 in top_heading.children

            # Check if deep nested children are correctly set
            assert deep_nested_heading_1 in top_heading.children[0].children
            assert deep_nested_heading_2 in top_heading.children[0].children

        def test_bad_values(self):
            """
            Check if default values are assigned when 'None' is passed as arguments
            """
            heading = HeadingItem(None, None)

            assert heading.value == HeadingItem.DEFAULT_HEADING_VALUE
            assert heading.id == HeadingItem.generate_heading_id(
                HeadingItem.DEFAULT_HEADING_VALUE
            )
            assert heading.children == HeadingItem.DEFAULT_CHILDREN_VALUE
```

I created **TestHeadingItem** class to group all the **tests** for **HeadingItem** class, and further grouped the tests related to **__init__** method in **TestHeadingItemConstructor** class.

**4. Running the tests**

Now that the tool was configured and the very first tests in place, it was time to run them with the following command.

```
pytest
```

It wasn't a smooth run for the first time, as there were some issues with my constructor's default values setup.

I was trying to generate the item's id before setting defaulting the `value` property to an empty string. This is what the fixed function looked like.

![constructor](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/dulb2p65pqzuq23c00hv.png)

I finally got the **green check** from all tests.

![Running unit tests](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/glnhr5351as3ng9352tr.png)

I would have never known about this **edge case** for a long time if not for the unit tests, and this problem might have turned into something much harder to **debug** by that time.

This is how
> Automated and quality tests can save companies millions of dollars and hundreds of wasted hours on fixing problems that could have been prevented in the first place.

### pytest-watch 🔎

After I was satisfied with the basic setup of unit tests, it was time to look for something that could **execute** my tests **automatically** everytime a file changed. This makes it really convenient to debug a problem as you don't have to manually run your tests after every little change.

I installed [pytest-watch](https://pypi.org/project/pytest-watch/) for this purpose and had to make following additions to the `pytest.ini` file we discussed above.

```ini
[pytest-watch]
# Re-run after a delay (in milliseconds), allowing for
# more file system events to queue up (default: 200 ms).
spool = 200

# Waits for all tests to complete before re-running.
# Otherwise, tests are interrupted on filesystem events.
wait = true
```

Once it was configured, all I had to do was execute the following command for the utility to start listening for changes to filesystem.

```
ptw
```

![ptw execution](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/nz20ayf7bqvnxv4f11dy.png)

## Integration Tests ⚙️

Unit tests do a good job in testing if individual units of program behave as expected. But no one knows if two or more units work together as expected, until we have some [integration tests](https://en.wikipedia.org/wiki/Integration_testing) in place.

### Writing my first integration test

To begin with, I added a very **general test** that called my program against a [markdown file](https://github.com/Amnish04/til-page-builder/blob/master/integration/test_samples/til-yattag.md), and comparted the generated html with the expected snapshot.

I created a dictionary to store the **expected snapshots** in a separate file.

[**snapshots.py**](https://github.com/Amnish04/til-page-builder/blob/master/integration/snapshots.py)
```python
"""Html snapshots to be used in integration testing"""

snapshots = {
    "yattag_html": """<!DOCTYPE html>
<html lang="en-CA">
  <head>
    <meta charset="utf-8" />
    <title># TIL *Yattag*</title>
    <meta name="viewport" content="width=device-width, initial-scale=1" />
  </head>
  <body>
    <h2>Table of Contents</h2>
    ...
    ...
    ...
    <p>Here's a simple code <em>snippet</em> that makes use of this library:</p>
    <p>with tag('p'):</p>
    <p>text("some random text")</p>
  </body>
</html>"""
}
```

And this is what my first **integration test** looked like.

[**test_integration.py**](https://github.com/Amnish04/til-page-builder/blob/master/integration/test_integration.py)
```python
""""This module is responsible for integration testing of this application"""

import os
from snapshots import snapshots
from til_builder_main import App


class TestIntegration:
    """Integration testing suites"""

    DEFAULT_OUTPUT = "til"

    def test_general(self):
        """Compare the generated output with a sample file to verify general expectations"""

        # Load the exected html from snapshot
        expected_html = snapshots["yattag_html"]

        # Run the application with default settings
        app = App()
        app.run("integration/test_samples/til-yattag.md", test_context=True)

        # Verify if the file was generated in the correct location
        assert os.path.isfile(f"{TestIntegration.DEFAULT_OUTPUT}/til-yattag.html")

        # Verify the contents of generated file match with the expected snapshot
        with open(
            f"{TestIntegration.DEFAULT_OUTPUT}/til-yattag.html", "r"
        ) as generated_file:
            generated_html = generated_file.read()

            assert generated_html == expected_html
```

I feel like this can be done better by replacing the keys in `snapshots.py` with the sample file paths, and a single function can iterate over all key-value pairs and run corresponding comparisons.

This can prevent lots of code repetition allowing the developer to add more test files and snapshots without adding more code.

But for now, all that matters is I was able to get a green check in the integration test as well.

**pytest.ini**
```
# Directories to look for the test files
testpaths = tests integration
```

After adding **integration tests** directory in testpath,

```
pytest
```

![Run integraion tests](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/gvfrwyv5ixbjclds2ti5.png)

## Coverage 📈

We also need a way to know how much of our code has been covered by the existing tests. For this purpose, I added another pytest plugin called [pytest-cov](https://pypi.org/project/pytest-cov/).

```
pip install pytest-cov
```

To check the code coverage, I ran

```
pytest --cov=src
```

![Code Coverage](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/brkg1e5v54w6braffm2j.png)

## Makefile

Even though I was almost done with the basic test setup, I still felt like something was missing. I am mostly used to working with **node** and **javascript**, and there we have an idea of configuring **custom scripts** required by the project in a **package.json** file.

I wanted to do something similar and hence, found a way by making use of the [GNU Make](https://www.gnu.org/software/make/) utility.

From the official documentation,
> GNU Make is a tool which controls the generation of executables and other non-source files of a program from the program's source files.

> Make gets its knowledge of how to build your program from a file called the makefile, which lists each of the non-source files and how to compute it from other files. When you write a program, you should write a makefile for it, so that it is possible to use Make to build and install the program.

I added a [Makefile](https://github.com/Amnish04/til-page-builder/blob/master/Makefile) to define all the custom scripts that I needed,

**Makefile**
```Makefile
install:
	pip install -r requirements.txt

format:
	black .

lint:
	pylint src/

test:
	pytest

run-failed-tests:
	pytest --last-failed 

test-watch:
	ptw

coverage:
	pytest --cov=src
```

Each of them can be executed with

```
make <script-name>
```

If you don't have the utility installed, refer to the installation instructions here,
https://www.gnu.org/software/make/#download

**Sweet!**

## Conclusion 🎇

In this post, we talked about the **importance of software testing** and its various **types**, setting up **pytest** and it **plugins**, writing both **unit** and **integration** tests, and the Makefile utility to setup custom scripts just like we do with npm.

Hope this helped!
Make sure to check out the other [posts](https://dev.to/amnish04).

### Image Attribution
<a href="https://www.freepik.com/free-vector/tester-developer-work-with-laptop-tablet-cross-platform-bug-founding-bug-identification-testing-team-concept-pinkish-coral-bluevector-isolated-illustration_11669402.htm#query=unit%20testing%20software&position=7&from_view=search&track=ais" style="font-size: 6px;">Cover Image by vectorjuice</a> on Freepik