Test Your Python Program the Right Way
Tags: Programming, Python Programming, Python, Software Development, Tdd

Test Your Python Program the Right Way

How to use Python Unittest to find bugs in your Python projects easily and faster Matteo Follow Jul 15 · 7 min read

Photo by Chris Ried on Unsplash

Why a testing framework?

For a long time, I have assumed that finding bugs in my programs was just a matter of running them a few times manually. This may be okay for small projects, but it rapidly becomes impractical as the number of methods and classes increases.

That’s the main reason why I started using a testing framework. It doesn’t mean, however, that you should only use these libraries if you are starting a big project: on the contrary, I am writing tests for every program I create and I have found that this makes the whole bug-finding process much faster.

But what is exactly a testing framework?

To answer concisely, it is a library that allows you to check if methods and classes behave as expected, and often can also give a clear explanation of what has gone wrong if a test fails. In the following tutorial, I will use the unittest library as a testing framework for Python. I chose it because it is really easy to use, and has many useful features.

The program we are going to test

In this guide, we will use as a running example a (very simple) class for a calculator, which is able to do some arithmetic operations. Here is the code to implement it (I created it in a file named Calculator.py):

import math class Calculator: def add(self, a, b):

return a+b def subtract(self, a, b):

return a-b def squareRoot(self, a):

return math.sqrt(a) def multiply(a, b):

return a*b

Note: At the end of this article there is, for reference, the full program with all the tests we will create.

Let’s start testing!

Now that we have our program, we only need to write the tests for it! First of all, we need to create a new file which will contain the tests. In it we have to import the Calculator class and obviously unittest :

from Calculator import Calculator

import unittest

Now we can create the class that will implement the tests. It has to be a subclass of unittest.TestCase , and inside it each test will be a new method.

class CalculatorTests(unittest.TestCase):

The first function we want to test is add. We want to check that, for example, add(7, 3) gives the right result ( 10 in this case). To check if the return value of a function is the one we are expecting, we can use assertEqual(expected, result) from the TestCase class. Here is the code to put inside CalculatorTests :

def test_add(self):

calc = Calculator()

self.assertEqual(10, calc.add(7, 3))

Now is time to run the tests!

To do this, simply add these lines of code at the end of the file:

if __name__ == '__main__':

unittest.main()

And run the program. The result should be something like:

.

--------------------------------------------------------------------

Ran 1 test in 0.001s OK

This is a quick report of our tests: we have executed one test, and all the tests passed (this is why it wrote OK). But what would have happened if the test failed? Let’s check this by modifying the assertEqual and run the test again.

self.assertEqual(13, calc.add(7, 3)) #This should NOT pass

Now the output is a bit longer:

F

====================================================================

FAIL: test_add (__main__.CalculatorTest)

--------------------------------------------------------------------

Traceback (most recent call last):

File "d:\Desktop\PythonTestingTutorial\CalculatorTest.py", line 10, in test_add

self.assertEqual(13, self.calc.add(3, 7))

AssertionError: 13 != 10 --------------------------------------------------------------------

Ran 1 test in 0.001s FAILED (failures=1)

In the first line, the report prints an “F” for each test that failed and a “.” for each one that passed (in this case we only have one failed test, so it shows one “F” ). Then for each failed test there is a description, starting with the test name ( test_add in this case) followed by a description of the error. In the last line of the description, we can see that the test failed because 13 (the expected result) is different from 10 (the result we obtained).

But wouldn’t it be better to have a more clear description of the error? Luckily unittest provide an easy way to achieve this: we only need to add a string as the third parameter for assertEqual , and it will be shown whenever the test fails:

self.assertEqual(13, calc.add(3, 7), "The adddition is wrong")

And the output now becomes:

F

====================================================================

FAIL: test_add (__main__.CalculatorTest)

--------------------------------------------------------------------

Traceback (most recent call last):

File "d:\Desktop\PythonTestingTutorial\CalculatorTest.py", line 10, in test_add

self.assertEqual(13, calc.add(3, 7), "The adddition is wrong")

AssertionError: 13 != 10 : The adddition is wrong --------------------------------------------------------------------

Ran 1 test in 0.000s FAILED (failures=1)

Now it is more clear what went wrong in our test. You may even make a more detailed message, such as:

def test_add(self):

calc = Calculator()

a = 3

b = 7

expected = a+b

res = calc.add(a, b)

message = "The sum of "+ str(a) + " and " + str(b) + " should

be " + str(expected) + " but the result is " + str(res)

self.assertEqual(expected, res, message)

The setUp and tearDown methods

Suppose we have created a few tests for addition, subtraction and so on. All these methods will need to create a new Calculator object, so the same line will be repeated across all our tests:

class CalculatorTest(unittest.TestCase):

def test_add(self):

calc = Calculator()

self.assertEqual(10, calc.add(3, 7), "The addition is wrong") def test_subtract(self):

calc = Calculator()

self.assertEqual(12, calc.subtract(15, 3), ”Subtraction is wrong”) def test_multiply(self):

calc = Calculator()

self.assertEqual(30, calc.multiply(5, 6), ”Multiplication is wrong”)

Luckily unittest has a feature that helps to avoid the repetition of the same code over and over: the setUp and tearDown methods.

1. setUp() contains the code that should be run before each test. In this case it would be:

setUp(self):

self.calc = Calculator()

2. tearDown() , on the other hand, contains the code that should be run after each test (for example the code for closing a file).

Now the code of the tests will be much cleaner:

def setUp(self):

self.calc = Calculator() def test_add(self):

self.assertEqual(10, self.calc.add(3, 7), "The addition is wrong") def test_subtract(self):

self.assertEqual(12, self.calc.subtract(15, 3), ”Subtraction is wrong”) def test_multiply(self):

self.assertEqual(30, self.calc.multiply(5, 6), ”Multiplication is wrong”)

Tests with multiple parameters

Until now we have only tested a single case (for example, 7+3 in the test for addition). However, it is often more useful to test multiple cases for the same function, because not all of them may cause bugs. To do so we may use the subTest functionality. It allows us to run a loop inside a test function, and at each iteration a different sub-test is executed. The following code does the work:

def test_multiple_addition(self):

for a in range(10):

for b in range(10):

msg = "add " + str(a) + " and " + str(b)

with self.subTest(msg):

self.assertEqual(a+b, self.calc.add(a, b))

Now we run 100 tests, each one with different parameters, to check if the addition is correct. The msg variable contains the name of the subtest, to be shown (together with the general name of the test) in case it fails. In this way you know exactly which one of the multiple inputs caused an error.

Checking for an exception

Sometimes we want to check that a function throws an exception as a result of a certain input. For example in our Calculator class we expect squareRoot(-1) to return a Value Error .

As before, unittest can help us in such cases: there is a method assertRaises that can be used to achieve this.

def test_negative_root(self):

self.assertRaises(ValueError, self.calc.squareRoot, -1)

assertRaise takes as input the exception that should be thrown, the function to check, and then the arguments for the function (in this case -1 ). The test will pass only if the desired exception is raised.

Other Assertions

Until now we have only used AssertEqual to check for equality and AssertRaises to check that an assertion was thrown. However, there are many more methods that can be used inside the tests, depending on our needs. Here is a list of the most useful (you can add a message as an optional parameter to each of them, as we did for assertEqual ):

assertNotEqual(expected, result) checks that the result is not equal to the expected value

checks that the result is equal to the expected value assertTrue(result) checks that the result is true

checks that the result is assertFalse(result) checks that the result is false

checks that the result is assertIn(element, container) checks that the element is one of the values in the container (e.g. a list)

checks that the element is one of the values in the container (e.g. a list) assertAlmostEqual(expected, result, places=7) checks that the two numbers are the same up to a certain number of decimal digits (7 by default, can be changed using the places optional parameter)

checks that the two numbers are the same up to a certain number of decimal digits (7 by default, can be changed using the optional parameter) assertGreater(expected, result) and assertLess(expected, result) check that the expected value is greater (or less) than the result

and check that the expected value is greater (or less) than the result assertGreaterEqual(expected, result) and assertLessEqual(expected, result) check that the expected valuer is greater/less or equal to the result

To see the full list of assertions, including assertDictEqual , assertListEqual and many more, read the documentation here.

How to test efficiently

Now that you have learned the basis of testing with unittest, here are a few suggestions to test your projects efficiently.

Test EVERYTHING The only way to really avoid bugs is to test every line of code. In fact, you may be sure that a simple function is working when you first write it, but future changes may break it, and having tests to immediately check that will save you hours of work (and headache). Run the tests after each change Every time you change something in your project, you should rerun all the tests, because you never know what are the effects of your new code on the rest of the program. Write tests before code This is a principle called Test Driven Development (TDD) and it helps you make sure that you are testing extensively and your tests are not influenced by the code you just wrote. Keep tests simple It’s easy to get excited and write over-complicated tests, but beware! You do not want to have to write tests to check your tests. As a rule of thumb, each test should only be a few lines of code and should check only one specific functionality.

The mock test class

Here is, for reference, the complete test class we have created in this guide:

import unittest

from Calculator import Calculator class CalculatorTest(unittest.TestCase): def setUp(self):

self.calc = Calculator() def test_add(self):

self.assertEqual(10, self.calc.add(3, 7), "The addition is

wrong") def test_subtract(self):

self.assertEqual(12, self.calc.subtract(15, 3), "Subtraction is

wrong") def test_multiply(self):

self.assertEqual(30, self.calc.multiply(5, 6), "Multiplication

is wrong") def test_multiple_addition(self):

for a in range(10):

for b in range(10):

with self.subTest("add " + str(a) + " and " + str(b)):

self.assertEqual(a+b, self.calc.add(a, b))



def test_negative_root(self):

self.assertRaises(ValueError, self.calc.squareRoot, -1) if __name__ == '__main__':

unittest.main()

Conclusion

And there we have it! I hope you have found this to be a useful introduction to testing Python code. Be sure to let us know your thoughts in the comments.

More content at plainenglish.io