A comprehensive guide to pytest.
By: Richard Shi
Tags: Development, Python, Pytest, Unit Testing, Mocking

Testing. A facet of software engineering evoking groans from developers everywhere since epoch. While I do not have a panacea, if you’re working with Python and you’re in need of exhaustive literature on unit testing (with pytest), look no further.

To procure such knowledge, I stumbled through the mystifying forests of Documentation, dove deep into the hostile waters of Stack Overflow, and scaled across the sheer precipice of Page 37 within the unforgiving mountains of Google. May this be the last article you will ever need on the topic.

Getting Started

Install pytest and related dependencies:

pipenv install --dev pytest pytest-cov pytest-mock

There are many more plugins, but coverage and mocks should go a long way before you need anything else.

I also use a package manager called pipenv, which I highly recommend for easy and convenient package management. Opensource.com has an excellent writeup on why you should be using it over vanilla pip.

pytest Setup

Put a conftest.py file in the root of the project. This file:

contains “global” settings for pytest, including mark registration (more on that later) and globally shared fixtures (also more on this later)

is responsible for giving pytest the scope; by putting the file in root, pytest will recognize the src module without needing to manually add it to PYTHONPATH

pytest does a great job of discovering tests, so the actual tests can be placed anywhere. I recommend the following folder structure:

/

|- <project name>/

|- tests/

|- acceptance/

|- unit/

|- # tests go here

Out-of-the-box, all pytest test files must be prefixed or postfixed with “test” (singular). For example, if you were testing a module called handler.py , your test file should be called test_handler.py or handler_test.py . The test file doesn’t actually need to share the base name of the module to be tested — it could be called test_zzyzx_the_end_of_the_world.py and it would still work the same — but for the sake of sanity you probably shouldn’t do that.

The unit tests themselves can be organized however makes sense, but I recommend reflecting the structure of the actual code. This allows you to quickly find relevant tests to a given piece of code. Unit tests should only test the logic contained within each file (in other words: mock any imports), with a notable exception to this being custom Exception/Error classes, which may inadvertently be tested in a given module.

Writing Tests

For barebones tests, you don’t need to import the pytest module. The simplest pytest test you can write for a method is as follows:

def hello_world():

return 'hello world' def test_function():

assert hello_world() == 'hello world'

pytest uses standard Python assert s for most assertions. pytest can even check the context of the asserted objects, e.g.:

def hello_worlds():

return {

'english': 'hello world',

'chinese': '你好世界'

} def test_function():

assert hello_world() == {'english': 'hello wurld',

'japanese': 'ハロー・ワールド'} E AssertionError: assert {'english': 'hello world', 'chinese': '你好世界'} == {'english': 'hello wurld', 'japanese': 'ハロー・ワールド'}

E Differing items:

E {'english': 'hello world'} != {'english': 'hello wurld'}

E Left contains 1 more item:

E {'chinese': '你好世界'}

E Right contains 1 more item:

E {'japanese': 'ハロー・ワールド'}

That is the extent of what can be done without importing pytest. The rest of this document covers features that require import pytest .

pytest can also check for the existence and context of exceptions.

import pytest def hello_error():

raise NotImplementedError def test_function():

with pytest.raises(NotImplementedError):

hello_error() def test_context():

with pytest.raises(NotImplementedError) as e:

hello_error()

assert e.xyz == abc

# note that the exception context object is referenced *outside* the `with` block

It’s possible to have multiple asserts under a single test function; whether or not this should be done is entirely a semantic decision.

Fixtures

Fixtures are separate pieces of reusable code that can be called from tests and other fixtures. They’re often used for setup and teardown, but can also be used to abstract repeated actions.

Fixture Basics

Fixtures are decorated with @pytest.fixture and are requested by passing in the name of a fixture. Multiple fixtures can be requested by a single method. Fixtures can also request other fixtures.

import pytest



@pytest.fixture

def hello():

return 'hello'



@pytest.fixture

def world():

return 'world'



@pytest.fixture

def hello_world(hello, world):

return hello + ' ' + world



def test_function(hello_world, hello, world):

assert hello_world = hello + ' ' + world @pytest.fixture

def hello():

return 'hello' @pytest.fixture

def world():

return 'world' @pytest.fixture

def hello_world(hello, world):

return hello + ' ' + world def test_function(hello_world, hello, world):

assert hello_world = hello + ' ' + world

Fixtures have independent execution contexts except when called multiple times in a single test.

The following example has the same fixture requested in different tests, and provides distinct objects to each test.

import pytest



@pytest.fixture

def a_list():

return ['a']



def test_str(a_list):

a_list.append('b')

assert a_list == ['a', 'b']



def test_int(a_list):

a_list.append(2)

assert a_list == ['a', 2]

On the other hand, this example has the same fixture requested twice in the same test (once transitively and the second directly), and returns the same object.

import pytest



@pytest.fixture

def a_list():

return ['a']



@pytest.fixture

def append_b(a_list):

return a_list.append('b')



def test_int(b_list, a_list):

assert a_list == ['a', 'b']

Fancy Fixtures (officially “factory as a fixture”)

To be able to pass parameters to a fixture (e.g. to dynamically generate input data), you need to use the factory as a fixture paradigm. Essentially, the fixture itself returns a function that does what you need.

import pytest



@pytest.fixture

def gen_input():

def _gen_input(a: bool):

return {'a': a}



return _gen_input



def test_input(gen_input):

assert gen_input(True) == {'a': True}

Unfortunately, docstrings and code-completion features don’t play well with this paradigm, but that doesn’t mean you shouldn’t include documentation!

Autouse Fixtures

You can specify that a fixture be used automatically on every test. This is useful if all tests have a common setup component.

import pytest



@pytest.fixture(autouse=True)

def some_func():

# do stuff before every test

Note, however, that autouse fixtures will not yield any objects unless they are explicitly requested by a testing function.

Sharing Fixtures & Fixture Lifecycle

To share fixtures across different modules, stick them in the conftest.py file.

By default, fixtures are invoked and torn down once per function. You can change this to have a fixture’s execution context be shared across the following scopes (shamelessly stolen from the docs):

function (default) — the fixture is destroyed at the end of the test

(default) — the fixture is destroyed at the end of the test class — the fixture is destroyed after the last test in the class

— the fixture is destroyed after the last test in the class module — the fixture is destroyed after the last test in the module

— the fixture is destroyed after the last test in the module package — the fixture is destroyed after the last test in the package

— the fixture is destroyed after the last test in the package session — the fixture is destroyed at the end of the test session.

The scopes are passed in as an argument to the fixture decorator.

import pytest



@pytest.fixture(scope="session")

def test_something():

# something

Yielding Fixtures

Sometimes you may have complicated objects involved in setup and teardown. Enter yield fixtures.

import pytest



@pytest.fixture

def read_file():

f = open('hello_world.txt', 'r')

yield f.readlines()

f.close() # teardown after yield



# alternatively, using `with`:

@pytest.fixture

def read_file():

with open('hello_world.txt', 'r') as f:

yield f.readlines()



def test_file(read_file):

assert read_file = ['hello world']

yield provides an object to the requesting function and "pauses" execution of the fixture. When the requesting function completes, the fixtures are torn down in reverse order ("resuming" execution of the fixture).

Prior to pytest v3, @pytest.yield_fixture was the decorator to be used. Though we’re way past that now, some old docs and Stack Overflow questions/answers may be dated to that time.

Mocking

Unit tests should be atomic; mocking enables that. You want to mock calls to most (if not all) external functions, especially if said function is slow.

Mocking in pytest requires pytest-mock , which you'll have installed following the Getting Started section above. It provides the mocker fixture and can mock objects, methods, and variables.

Mocking Variables

Variable mocking is mainly used for mocking globals (outside of function scope). Say you have the following file that contains a Lambda handler:

# lambda_handler.py is_cold_start = True

def handler():

if is_cold_start:

# do stuff

is_cold_start = False

# do more stuff

return True # arbitrary return value for illustration

You want to be able to test handler() when is_cold_start = False ; to do so, you must mock is_cold_start .

# handler_test.py import pytest

import lambda_handler



def test_handler(mocker):

mocker.patch.object(lambda_handler, 'is_cold_start', False)

assert handler()

The signature is as follows:

mocker.patch.object(

module, # this is NOT a string

'variable', # this IS a string

value # this is whatever

)

The module name follows the import name. For example, if lambda_handler.py were in a folder src/ , the module would then be src.lambda_handler .

Mocking Functions

There are some nuances with function mocking. Here are a series of examples.

Basics

# hello_world.py import os



def say_passphrase():

passphrase = os.environ.get('PASSPHRASE')

return passphrase or 'I need somebody (Help!)'

To mock the call to os.environ.get() :

# hello_world_test.py import pytest

from hello_world import say_passphrase



def test_passphrase(mocker):

mocker.patch('hello_world.os.environ.get', return_value='hello world')

assert say_passphrase() == 'hello world'

Note that the first argument is the fully-qualified name of the method to be mocked as a string.

Because hello_world.py imported the os module, the target of the mock is the instance of os.environ.get() that resides in the hello_world module. Alternatively, if in hello_world.py you did from os import environ , then the fully-qualified name would be hello_world.environ.get .

Different Return Values

If the same mock is required in multiple tests, you can put the mocker in a fixture and return the mocked object. Given the same hello_world.py as above, here's what the tests would look like:

# hello_world_test.py import pytest

from hello_world import say_passphrase



@pytest.fixture

def mock_getenv(mocker):

return mocker.patch('hello_world.os.environ.get')



def test_passphrase_exists(mock_getenv):

mock_getenv.return_value = 'hello world'

assert say_passphrase() == 'hello world'



def test_passphrase_not_exists(mock_getenv):

mock_getenv.return_value = None

assert say_passphrase() == 'I need somebody (Help!)'

A few things to note here:

the call to mocker.patch() doesn't include the return_value argument — this creates a generic mock object that we can manipulate later, which the fixture then returns

doesn't include the argument — this creates a generic mock object that we can manipulate later, which the fixture then returns the tests don’t request mocker directly; they request the mock_getenv fixture instead — this provides the tests with the mocked object, and because the tests don’t require any additional mocks, mocker doesn’t need to be separately passed in

directly; they request the fixture instead — this provides the tests with the mocked object, and because the tests don’t require any additional mocks, doesn’t need to be separately passed in the return_value s can be unique per test; this is due to fixture behavior (see above)

Dynamic Return Values (via side_effect)

Let’s say the same method was called multiple times in the same function, with different arguments. How do you mock that? Let’s find out!

# hello_world.py import os



def say_passphrase():

greeting = os.environ.get('GREETING')

subject = os.environ.get('SUBJECT')

return greeting + ' ' + subject

In this example, os.environ.get() is called twice, each with different arguments. We’ll need a way to differentiate the two calls.

# hello_world_test.py import pytest

from hello_world import say_passphrase



def getenv_side_effect(key, default = None):

if key == 'GREETING':

return 'hello'

elif key == 'SUBJECT':

return 'world'

else:

return default



@pytest.fixture

def mock_getenv(mocker):

return mocker.patch('hello_world.os.environ.get', side_effect=getenv_side_effect)



def test_passphrase(mock_getenv):

assert say_passphrase() == 'hello world'

The side_effect argument allows you to pass a function that will be called in place of the actual function. This allows you to specify distinct return values depending on the arguments passed in. Also note that the side effect function is not a fixture and is not requested.

Even-More-Dynamic Return Values

If you had multiple calls to the same method, and you wanted to succinctly test different return scenarios of that method, how would that work?

The idea is to combine factory fixtures with the side effect functionality, like so:

# hello_world.py import os



def say_passphrase():

greeting = os.environ.get('GREETING', 'hola')

subject = os.environ.get('SUBJECT', 'mundo')

return greeting + ' ' + subject

This is nearly identical to the previous example, except default values are now passed in to os.environ.get() .

# hello_world_test.py import pytest

from hello_world import say_passphrase



def getenv_side_effect(key_exists: bool):

def _getenv_side_effect(key, default = None):

if key == 'GREETING':

return 'hello' if key_exists else default

elif key == 'SUBJECT':

return 'world' if key_exists else default



return _getenv_side_effect



@pytest.fixture

def mock_getenv(mocker):

def _mock_getenv(key_exists):

return mocker.patch('hello_world.os.environ.get',

side_effect=getenv_side_effect(key_exists))



return _mock_getenv



def test_passphrase_exists(mock_getenv):

mock_getenv(True)

assert say_passphrase() == 'hello world'



def test_passphrase_not_exists(mock_getenv):

mock_getenv(False)

assert say_passphrase() == 'hola mundo'

Alternatively, to get the same effect you could have multiple side effect functions for different scenarios, and in each test load a different side effect.

Mocking Objects/Classes

Mocking objects is similar to mocking functions. Here are two examples.

Implicit Class Mocking

In this example, you’re using requests to make some API call. requests.get() returns a Response object, which has some number of attributes, one of which is status_code .

# hello_world.py import requests



def check_status():

response = requests.get('hello.world') # this returns a Response class

return response.status_code

In the test, you mock the requests.get() call and store it as a mock object ( mock_response ). You can then chain attributes of the mock object so that the access of status_code attribute is mocked without explicitly mocking the Response class.

# hello_world_test.py import pytest

from hello_world import check_status



def test_check_status(mocker):

mock_response = mocker.patch('hello_world.requests.get')

mock_response.return_value.status_code = 200

assert check_status() == 200

Explicit Class Mocking

Here you’re passing in an object that’s been previously instantiated. Let’s say the name attribute is of type str , and get_employer() returns a Company object that has a name attribute as well.

# hello_world.py from people import Person



def greet(person: Person):

name = person.name

company = person.get_employer().name

return f'hello, {name} from {company}'

In the test, you’ll have to explicitly mock the Person class, but Company can be implicitly mocked.

# hello_world_test.py import pytest

from hello_world import greet



def test_greet(mocker):

mock_person = mocker.patch('hello_world.Person') # explicitly mocked Person

mock_person.name = 'Rich Fairbank'

mock_person.get_employer.return_value.name = 'Capital One' # implicitly mocked Company

assert greet() == 'hello, Rich Fairbank from Capital One'

Internal Class Mocking

In this scenario, you are trying to mock an object that is internal to the method being tested. Let’s say for example greet() is a part of some class (left out for simplicity) and it creates its own instance of a Person to greet you. For the sake of example, assume that the values passed in to initialize the Person are static (though in reality would be retrieved from somewhere else).

# hello_world.py from people import Person



def greet():

person = Person("Dumbledore", "Hogwarts")

name = person.name

company = person.get_employer().name

return f'{name} from {company} says "Greetings."'

In the test, you’ll have to explicitly mock the internal Person method a little differently from when you pass it in as an argument.

# hello_world_test.py import pytest

from hello_world import greet



def test_greet(mocker):

mock_person = mocker.patch('hello_world.Person') # the overall class is mocked the same way

mock_person().name = 'Dumbledore' # however, note the parentheses here...

mock_person().get_employer.return_value.name = 'Hogwarts' # and here



assert greet() == 'Dumbledore from Hogwarts says "Greetings."'

Marking

That’s marking with an a. Marks are a way to tag/label your tests, so that you can selectively run a subset of tests (for example, you want to run only the test you’re working on).

To mark something, just decorate it with @pytest.mark.<your mark> , e.g.:

import pytest



@pytest.mark.wip

def test_something():

# blah

You’ll also have to register your mark with pytest in conftest.py :

# conftest.py def pytest_configure(config):

config.addinivalue_line("markers", "wip: mark test that is in progress")

You can add additional marks by adding another config.addinivalue_line() .

To run marks, use pytest -m <mark1> <mark2> ...

Parameterization

Sometimes you want to run the same test with different sets of inputs. Enter parameterization!

# hello_world.py def greet(name: str, is_casual: bool, use_exclamation: bool):

greeting = 'sup' if is_casual else 'salutations,'

ending = '!' if use_exclamation else ''

return f'{greeting} {name}{ending}'

You want to be able to easily test the different boolean combinations without rewriting basically the same test multiple times.

# hello_world_test.py from hello_world import greet



@pytest.mark.parameterize('is_casual', 'use_exclamation',

[(True, True), (True, False), (False, True), (False, False)])

def test_greet(is_casual, use_exclamation):

greeting = 'sup' if is_casual else 'salutations,'

ending = '!' if use_exclamation else ''

assert greet('Rich', is_casual, use_exclamation) == f'{greeting} Rich{ending}'

With parameterization, you provide the arguments to be parameterized and a list of the values to use. pytest will run the test once for each set of arguments provided.

That’s All Folks!

If you’ve made it this far, you’re a trooper and I hope I was able to help you with your testing needs. If not, reach out or comment on what you’d like to accomplish and I’ll take a look! Also, if you notice any issues with the info I’ve presented, please do let me know. The last thing I want to do is spread wrong information.