Mock It Till You Make It: Mocking in Python's unittest
Tags: python, tutorial, beginners, testing
Published: 2024-04-29 01:09:47

---
title: "Mock It Till You Make It: Mocking in Python's unittest"
published: true
series: Testing Strategies for Python
---
# Something a little different

Remember when I was job searching and I would have given anything to be employed? Well, now I am employed – which is great – and a side effect of full-time employment is well, it’s _full-time_. Some days I miss the time I had to pursue my personal projects. I don’t, however, miss worrying about paying rent. I also don’t want to give up on my personal projects and my personal learning, which means I just need to get creative (and a little more disciplined) with the time I _do_ have for projects like [_The DevRel Digest_](https://dev.to/lizzzzz/series/25904).

So this issue of The DevRel Digest is actually going to be the tutorial I have been meaning to write about mocking in Python as part of a larger series on software testing strategies. Lest you argue that _isn’t_ a DevRel Digest, don’t worry, I have my justification ready: If you are in Developer Relations, it’s that much more critical to exercise your technical muscles and occasionally refresh your understanding of some of the more basic concepts in code. It’s also helpful to read other instructionals to inspire the tutorials you have to write for work. So, you see? It makes perfect sense to occasionally switch things up with the DevRel Digest. And anyway, this is _my_ personal project, which means I can proceed however I wish.

And with that, let’s talk about mocking in Python testing.

# Eat your vegetables: The benefits of writing tests

I don’t eat enough vegetables. I know I _should_ eat my vegetables, but in a world of pizza and cupcakes, they are just so unappealing. _And_ I feel so much better when I _do_ eat the recommended serving of vegetables consistently. There are lots of different ways and opportunities to eat vegetables and I am most successful when I have found ways to make vegetables part of every meal.

It’s the same thing for writing tests. And notice I said the benefit is in _actually writing_ them. AI is definitely something you want in your toolbox, but not every test is a nail, and as you’ve probably already discovered, when it comes to writing code, the machines often get it wrong. So sure, let AI handle the boilerplate code, but don’t trust it to be able to perform the kind of discernment relegated to humans. We are already bad at writing tests, so imagine just how bad the robots trained on _our_ tests are at writing tests.

Besides the quantitative benefits of software testing such as bug prevention, reduction in development costs, and improved performance, the most compelling benefit of writing tests is it makes us better engineers. Writing tests forces us to ask ourselves, “What exactly is the expected behavior of this method or application?” When software becomes “difficult to test,” it is usually a good indicator of code smells and an opportunity to refactor a method or reconsider the entire design of a system.

# Documentation: The less obvious benefit of writing tests

I don’t know about you, but when I pull down new code, the first thing I do is try to get the tests to run (and pass!). This confirms that I’ve at least got the foundation correctly configured. Then, after a tour through the main script, I begin to dissect the separate modules by checking out the unit tests for them.

Best practices for unit tests call for long, descriptive function names. These function names not only make verbose test output more readable and quick to assess, they also provide documentation for the function under test. Test input and test output indicate what the function expects and returns; assertions indicate the function’s intended behavior. Assertions on errors and exceptions indicate how the function handles stuff when things go wrong; assertions on side effects and calls indicate how the function fits into the rest of the code. If you’re not going to write your documentation (you never do anyway), at least write your tests.

# What is mocking?

In Plato’s [allegory of the cave](https://utm.guru/ugSht), the Greek philosopher describes a group of people whose reality is exclusively the shadow representations of the real world as projected onto a cave wall. These projections can be really, really close to the actual thing they represent, but they’re never quite the same as the real deal – they are only shadows and they can be easily distorted or destroyed as soon as all the lights go out.

<figure>
![An illustration of Plato's allegory of the cave](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/0zb698d4awl9wcpekfxd.jpeg)
<figcaption>An illustration of Plato's allegory of the cave by <a href="https://commons.wikimedia.org/w/index.php?curid=73850232">4edges - Own work, CC BY-SA 4.0</a>
</figcaption>
</figure>

Mocking is the shadow in the cave. Mocks are representations of methods, API calls, behaviors, classes, or infrastructure that stand in for the real things for the duration of your tests. Mocking goes by other names, such as “stubbing” or “dummies,” but whatever you call it, at its core, mocking enables us to make our tests more reliable, repeatable, and resilient by temporarily replacing components that are outside of our control. We trust the makers of those components have also written thorough tests as well.

Mocking is helpful in testing because it enables us to:

* Isolate and control our code and systems under test because we don’t have to rely on outside dependencies
* Save money and performance because we’re not making real calls to APIs or databases
* Ensure security best practices by not signing into real services and not passing along real secrets
* Contain and limit our tests to just our input and output so we can make more precise assertions and trust our tests

Like all things in code, mocking has its limitations and faults. Mocks are additional code we have to maintain or an additional dependency we have to rely on, so they are always more work. Mocks can fall out of sync with reality, rendering our tests irrelevant and meaningless, which is in conflict with unit testing best practices.

And there’s no way to avoid needing mocks in modern software. So what are some best practices we can keep in mind?

# Mocking best practices

* **Make one precise assertion at a time:** A mock will only make a bad test worse. A good test should have just one precise assertion.
* **Use a mocking library:** While these are usually only available for more established or well-supported APIs and libraries, a mocking library with official maintenance can be worth the tradeoff of an additional dependency.
* **Mock only what you need:** Wherever possible, use the real thing, and mock only what you control. Too many mocks can be a sign the test is no longer meaningful.
* **Verify your mocks for accuracy:** Write tests that occasionally compare mocks against the real components they represent; use “recorded” responses whenever possible. 
* **Keep mocks simple:** When you start wondering if your test actually means anything anymore because so much is mocked, you’re probably mocking the wrong thing.
* **Refactor your code:** When writing tests starts to feel like solving a LeetCode problem, it’s time to revisit your code. This is the best possible outcome of writing tests.

# Meet unittest.mock

From the [Python documentation](https://utm.guru/ugShw): “unittest.mock is a library for testing in Python. It allows you to replace parts of your system under test with mock objects and make assertions about how they have been used.” The most useful features of the mock library are the classes `Mock` and `MagicMock`, and the `patch()` decorator.

## The classes: Mock and MagicMock

`Mock` is a base class that enables you to instantiate an object that you can create attributes and methods for as you access them. A `Mock` object is like a ball of clay and we can shape it into different things as we need to. `MagicMock` is a subclass of `Mock` “with default implementations of most of the magic methods.” (Read the documentation [here](https://utm.guru/ugShv).) “Magic methods” are the ones that come baked in – methods like mathematical operations, iteration, and dictionary keys.

<figure>
![Did you really think I wouldn't make a Gary tarot card? Here is Gary poorly Photoshopped as The Magician from the Rider-Waite deck](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/5cp60hl1qss680kjfk2b.png)
<figcaption>Did you really think I wouldn't make a Gary tarot card for this post? Here is Gary poorly Photoshopped as <a href="https://en.wikipedia.org/wiki/The_Magician_(tarot_card)">The Magician</a> from the Rider-Waite deck</figcaption>
</figure>
Want to see it in action? Fire up Python in your terminal and execute the following:

```
# Import the mock library

from unittest.mock import Mock

# Create a Mock integer object 

mock_number = Mock(1)

# Try to add mock_number with another integer

mock_number + 2

```

You should get an error like this: `TypeError: unsupported operand type(s) for +: 'Mock' and 'int'` – Python doesn’t even know what to do with the plus sign.

Now try the same thing with a `MagicMock`:

```
# Import the mock library

from unittest.mock import MagicMock

# Create a Mock integer object 

mock_number = MagicMock(1)

# Try to add mock_number with another integer

mock_number + 2

```

You shouldn’t get an error this time. You should get a `MagicMock` object returned to you: `<MagicMock name='mock.__add__()' id='4392271440'>`. The `add` method was implicitly created with the mock object because it is _magical_.

If a `Mock` object is a ball of clay we can shape, a `MagicMock` object is a ball of clay that comes with a set of moulds we can use to shape it.

## The decorator: patch()

From the [Python documentation](https://utm.guru/ugShM):

“`patch()` acts as a function decorator, class decorator or a context manager. Inside the body of the function or with statement, the _target_ is patched with a _new_ object. When the function/with statement exits the patch is undone.”

When used, `patch()` creates a `MagicMock` object to replace the target it is called with. So if a `MagicMock` object is a ball of clay with a set of moulds, then `patch()` tells us where the ball of clay goes in relation to the rest of the code.

# Get your hands dirty with mocking in Python

## A little bit of context and setup

You can find the code for this tutorial in [this repo](https://utm.guru/ugShL) on the `mocking-tutorial` branch.

While the repo contains a Flask application, the full web application is not necessary for this tutorial. This tutorial is concerned with the Pug class in `pug.py` and in particular, the methods for `get_pug_facts`, `build_pug`, and `check_for_puppy_dinner`. The accompanying tests are in `tests/unit/test_pug.py`.

`get_pug_facts() `makes a call to an external static API called [Dog API](https://utm.guru/ugShu). Dog API returns a JSON object with dog breed facts. In the case of our application, it returns facts about pugs.

On the other hand, the method `build_pug` uses the [OpenAI client](https://utm.guru/ugShz) to generate an image of a pug based on the prompt we send it. This method is also making a call to an external service, however, unlike the API we are using for `get_pug_facts()`, OpenAPI returns a dynamic response.

The `check_for_puppy_dinner` method (inspired by [Puppy Songs](https://utm.guru/ugShB)) gets the current time and compares it with the puppy dinner time supplied to an instance of Pug class. If the current time equals puppy dinner time, the function returns a happy message; if the times do _not_ match, the function returns a sad message.

In the accompanying test, there are two versions of the tests for each of these methods. Each version is meant to illustrate a point. In both cases, there is a commented-out set of test data containing an intentional test failure. (I have done my best to indicate all of this in the comments.)

For this tutorial, I am using Visual Studio Code’s Testing interface, however, you are encouraged to use whatever tools you are already familiar with. For help configuring the Python plugin to run your tests in VS Code, check out the [documentation](https://utm.guru/ugShy).

## Run the tests

1. In terminal in the same directory as the tutorial code, run `pip install pipenv --user` to install [pipenv](https://utm.guru/ugShA)
2. Install the dependencies from `Pipfile.lock` with `pipenv install`
3. Now switch over to Visual Studio Code to run the tests.
    (For even more human readability, it might be easier to configure tests to `View as Tree`, which you can select by clicking the three dots in the upper right-hand corner of the Testing interface.)
    ![Screenshot of the VS Code Testing configuration menu](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/kjbrvv53ipg40pzznq7e.png)
4. Run the test by clicking the `Run Test` arrow icon for `test_get_pug_facts()`. The test should pass.
5. Now run the test for `test_build_pug()`. It should also pass.
6. Lastly, run the tests for `test_check_for_puppy_dinner()`. They should also pass.

Now that you’ve successfully run the tests, let’s take a look at how mocking is used in them.

## Mocking in the test for get_pug_facts(): Autospeccing

In the test for `get_pug_facts()` we are using the `patch()` decorator. What this does is tell Python that for the code block below, replace the the `Requests` object in `Pug.py` with a `MagicMock`. We set `autospec` to `True` and this ensures you can only “[access attributes on the mock that exist on the real class](https://utm.guru/ugShw).”

Because the `Mock` class has the power to create attributes and methods on the fly, it’s easy to accidentally access an attribute that doesn’t actually exist. Moreover, as we refactor and update our code, it is easy for mocks to fall out of sync. Autospeccing ensures mocks remain faithful to the real classes they represent.

Notice also that one of the tests does not assert on a comparison, but on whether or not the mocked request was made with the correct URL with `assert_called_with()`. I’m sure you’ve seen similar tests that create a mocked response for the mocked request to return and then compares the test results against a set of expected results. This doesn’t actually test anything because the two sets of results we are comparing were created for the test. What this is actually a test of is `Mock` itself – it is not a test of our code.

```
mock_requests.get.assert_called_with(PUG_FACTS_URL)
```

We don’t “own” the Dog API endpoint, so we shouldn’t be mocking it. What we _do_ own is the specific instance of `Request` that we create in our code, so that’s what we mock, and then we assert that the mock is called with the correct URL. The method under test (`get_pug_facts()`) does return a dictionary object, so a more accurate test is to assert on a comparison between the keys in the test results and the keys in the expected results.

```
self.assertEqual(expected_results, list(test_results.keys()))
```

1. Uncomment the code for `test_get_pug_facts_without_autospec` and run the test. It should pass.
2. Uncomment the code for `test_get_pug_facts_with_autospec_error` and run the test. It should raise the error `Mock object has no attribute 'get_response'`.

This error occurred because of the following line:

```
mock_requests.get_response.return_value = mocked_pug_facts_response \
```

`get_response` is not a method available on the `Request` class. Autospeccing prevented this test from passing silently. The test for `test_get_pug_facts_without_autospec` contains the same line and passed – it is a meaningless test and yet its passing creates a false sense of confidence.

## Mocking in the test for build_pug(): Using the real thing

In `test_build_pug()` we mock the `images` attribute of the `OpenAI` object we created using the `patch()` decorator. We could have mocked the response the client receives, but I was pleased to discover that I could import the `ImageResponse` object out of the OpenAI SDK, so in the test, we use a real instance of `ImageResponse` with the values we give it.

```
mock_openai_response = ImagesResponse(created=1234, data[Image(b64_json=None, revised_prompt=None, url="https://ai-generated-pug")])
```

## Mocking in the test for check_for_puppy_dinner(): The peculiars of patching

In order to correctly use the `patch()` decorator, we must use the fully qualified name of the function we are targeting. This [blog post ](https://utm.guru/ugShx)explains it best: “If a class is imported using a from `module import ClassA` statement, `ClassA` becomes part of the namespace of the module into which it is imported.”

```
@patch('pug.datetime', autospec=True)
```

This can be especially confusing when it comes to libraries like `datetime`.

1. Run the test for `test_check_for_puppy_dinner`. It should pass.
2. Run the test for `test_check_for_puppy_dinner_wrong_target`. It should fail.

Notice that the target class we specified is `datetime.datetime`. This is not actually patching the instance of `datetime` we are using the `Pug` module, therefore `mock_datetime.now.return_value = data['time']` does not replace the `datetime.now()` in the method under test. `now()` is actually called and correctly returns whatever time it is at the moment `check_for_puppy_dinner() ` is called while running the test.

```
@patch('datetime.datetime', autospec=True)
```   

# In summary

Mocking is an important and powerful tool in our testing toolbox and this tutorial covers just a small fraction of what the mock library that comes with unittest can do. We looked at how to use autospeccing to ensure the accuracy of our mocks, as well as what to consider when writing our assertions on mocks and how to properly patch a target function or class. Though this is just an introduction to mocking, you can probably see how the power of mocks can also be a liability, which is why it’s important to keep best practices in mind.  

What are your thoughts on mocking? How will you use mocks the next time you write your tests?

## More resources on mocking

* [TalkPython: Testing without dependencies, mocking in Python](https://utm.guru/ugShD)
* [Understanding the Python Mock Object Library](https://utm.guru/ugShE)
* [The Art of Mocking in Software Testing](https://utm.guru/ugSia)
* [“Don’t Mock What You Don’t Own” in 5 Minutes](https://utm.guru/ugShK)
* [Python Mocking 101: Fake It Before You Make It](https://utm.guru/ugShG)

# Events and resources and other notable things

* I am obligated to invite you to [Dev Day](https://utm.guru/ugKfK), but even if I didn’t work for Streamlit, I would still invite you because it sounds cool.
* It’s been interesting to see digital video evolve from firewires (Remember those?) to cameras built into our laptops. So how should we be thinking about video in 2024? Check out the [State of Video Report](https://utm.guru/ugShC) to find out.
* _Hacker News_ can be a great place to engage developers, but how do you do it with authenticity? Dan Moore offers his thoughts on this over at [The MonkCast](https://utm.guru/ugShJ).
* Know how we’re always talking about how DevRel needs to get comfortable with sales and marketing? Here’s a [DevRel guide to all the jargon you need to know](https://utm.guru/ugShF).
* Need a developer journey map template for Figma? [DevRel.Agency has got you covered](https://utm.guru/ugShN).
* Pydantic is a a popular data validation library for Python and here’s a [tutorial](https://utm.guru/ugShO) on how to use it.
* Take a journey down the rabbit hole of [AI hallucinations](https://utm.guru/ugShH).