Debugging
Tags: python, beginners, ebook, exercises
Published: 2021-01-18 11:38:02

---
title: "Debugging"
published: true
description: "Debugging in Python and general tips"
canonical_url: https://learnbyexample.github.io/100_page_python_intro/debugging.html
series: 100 Page Python Intro
tags: Python,beginner,ebook,exercises
cover_image: https://dev-to-uploads.s3.amazonaws.com/i/x0gbxspe60srzezgp6gd.png
---

>Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it. — Brian W. Kernighan

>There are 2 hard problems in computer science: cache invalidation, naming things, and off-by-1 errors. — Leon Bambrick

>Debuggers don't remove bugs. They only show them in slow motion. — Unknown

Above quotes chosen from [this collection at softwareengineering.stackexchange](https://softwareengineering.stackexchange.com/questions/39/whats-your-favourite-quote-about-programming).

## General tips

Knowing how to debug your programs is crucial and should be ideally taught right from the start instead of a chapter at the end of a beginner's learning resource. [Think Python](https://greenteapress.com/wp/think-python-2e/) is an awesome example for such a resource material.

Debugging is often a frustrating experience. Taking a break helps. It is common to find or solve issues in your dreams too (I've had my share of these, especially during college and intense work days).

If you are stuck with a problem, reduce the code as much as possible so that you are left with minimal code necessary to reproduce the issue. Talking about the problem to a friend/colleague/inanimate-objects/etc can help too — famously termed as [Rubber duck debugging](https://rubberduckdebugging.com/). I have often found the issue while formulating a question to be asked on forums like stackoverflow/reddit because writing down your problem is another way to bring clarity than just having a vague idea in your mind.

Here's some awesome articles on this challenging topic:

* [What does debugging a program look like?](https://jvns.ca/blog/2019/06/23/a-few-debugging-resources/)
* [How to debug small programs](https://ericlippert.com/2014/03/05/how-to-debug-small-programs/)
* [Debugging guide](https://uchicago-cs.github.io/debugging-guide/)
* [Problem solving skills](https://ryanstutorials.net/problem-solving-skills/)

Here's an interesting snippet (modified to keep it small) from a collection of [interesting bug stories](https://stackoverflow.com/questions/169713/whats-the-toughest-bug-you-ever-found-and-fixed).

>A jpeg parser choked whenever the CEO came into the room, because he always had a shirt with a square pattern on it, which triggered some special case of contrast and block boundary algorithms.

See also [curated list of absurd software bug stories](https://500mile.email/).

## Common beginner mistakes

The previous chapter already covered syntax errors. This section will discuss more Python gotchas.

Python allows you to redefine built-in functions, modules, classes etc (see [stackoverflow: metaprogramming](https://stackoverflow.com/questions/514644/what-exactly-is-metaprogramming)). Unless that's your intention, do not use [keywords](https://docs.python.org/3/reference/lexical_analysis.html#keywords), [built-in functions](https://docs.python.org/3/library/functions.html) and modules as your variable name, function name, program filename, etc. Here's an example:

```python
# normal behavior
>>> str(2)
'2'

# unintentional use of 'str' as variable name
>>> str = input("what is your name? ")
what is your name? learnbyexample
# 'str' is no longer usable as built-in function
>>> str(2)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'str' object is not callable
```

Here's another example:

```python
>>> len = 5
>>> len('hi')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'int' object is not callable
```

>As an exercise, create an empty file named as `math.py`. In the same directory, create another program file that imports the `math` module and then uses some feature, say `print(math.pi)`. What happens if you execute this program?

See also:

* [Think Python: Debugging chapter](https://greenteapress.com/thinkpython2/html/thinkpython2021.html)
* [inventwithpython: common runtime errors](https://inventwithpython.com/blog/2012/07/09/16-common-python-runtime-errors-beginners-find/) and [common python gotchas](https://inventwithpython.com/beyond/chapter8.html)
* [pythonforbiologists: common beginner errors](https://pythonforbiologists.com/29-common-beginner-errors-on-one-page/)
* [stackoverflow: common pitfalls in Python](https://stackoverflow.com/questions/1011431/common-pitfalls-in-python)

## pdb

Python comes with a handy built-in library `pdb` that you can use from the CLI to debug small programs. See [docs.python: pdb](https://docs.python.org/3/library/pdb.html) for documentation. Here's some of the frequently used commands (only short form is shown, see documentation for long form and more details).

* `l` prints code around the current statement the debugger is at, useful to visualize the progress of debug effort
* `ll` prints entire code for the current function or frame
* `s` execute current line, steps inside function calls
* `n` execute current line, treats function as a single execution step
* `c` continue execution until the next breakpoint
* `p expression` print value of an expression in the current context, usually used to see the current value of a variable
* `h` list of available commands
    * `h c` help on `c` command
* `q` quit the debugger

Here's an example invocation of the debugger for the `num_funcs.py` program seen earlier in the [Importing your own module](https://learnbyexample.github.io/100_page_python_intro/importing-and-creating-modules.html#importing-your-own-module) section. Only the `n` command is used below. The line with `>` prefix tells you about the program file being debugged, current line number and function name. The line with `->` prefix is the code present at the current line. `(Pdb)` is the prompt for this interactive session. You can also see the output of `print()` function for the last `n` command in the illustration below.

```bash
$ python3.9 -m pdb num_funcs.py
> /home/learnbyexample/Python/programs/num_funcs.py(1)<module>()
-> def sqr(n):
(Pdb) n
> /home/learnbyexample/Python/programs/num_funcs.py(4)<module>()
-> def fact(n):
(Pdb) n
> /home/learnbyexample/Python/programs/num_funcs.py(10)<module>()
-> num = 5
(Pdb) n
> /home/learnbyexample/Python/programs/num_funcs.py(11)<module>()
-> print(f'square of {num} is {sqr(num)}')
(Pdb) n
square of 5 is 25
> /home/learnbyexample/Python/programs/num_funcs.py(12)<module>()
-> print(f'factorial of {num} is {fact(num)}')
```

Continuation of the above debugging session is shown below, this time with `s` command to step into the function. Use `r` while you are still inside the function to skip until the function encounters a `return` statement. Examples for `p` and `ll` commands are also shown below.

```bash
(Pdb) s
--Call--
> /home/learnbyexample/Python/programs/num_funcs.py(4)fact()
-> def fact(n):
(Pdb) ll
  4  -> def fact(n):
  5         total = 1
  6         for i in range(2, n+1):
  7             total *= i
  8         return total
(Pdb) n
> /home/learnbyexample/Python/programs/num_funcs.py(5)fact()
-> total = 1
(Pdb) p n
5
(Pdb) r
--Return--
> /home/learnbyexample/Python/programs/num_funcs.py(8)fact()->120
-> return total
(Pdb) n
factorial of 5 is 120
--Return--
> /home/learnbyexample/Python/programs/num_funcs.py(12)<module>()->None
-> print(f'factorial of {num} is {fact(num)}')
```

If you continue beyond the last instruction, you can restart from the beginning if you wish. Use `q` to end the session.

```bash
(Pdb) n
--Return--
> <string>(1)<module>()->None
(Pdb) n
The program finished and will be restarted
> /home/learnbyexample/Python/programs/num_funcs.py(1)<module>()
-> def sqr(n):
(Pdb) q
```

>You can call [breakpoint()](https://docs.python.org/3/library/functions.html?#breakpoint) or [pdb.set_trace()](https://docs.python.org/3/library/pdb.html#pdb.set_trace) to set breakpoints in the code and use it in combination with `c` command.

See also:

* [awesome-python: Debugging tools](https://github.com/vinta/awesome-python#debugging-tools)
* [pdb tutorial](https://github.com/spiside/pdb-tutorial)
* [docs.python: Basic Logging Tutorial](https://docs.python.org/3/howto/logging.html)

## IDLE debugging

Sites like [Pythontutor](http://www.pythontutor.com/visualize.html#mode=edit) allow you to visually debug a program — you can execute a program step by step and see the current value of variables. Similar feature is typically provided by IDEs. Under the hood, these visualizations would likely be using the `pdb` module discussed in the previous section.

This section will show an example with `IDLE`. Before you can run the program, first select **Debugger** option under **Debug** menu. You can also use `idle3.9 -d` to launch IDLE in debug mode directly. You'll see a new window pop up as shown below:

![Debug window.png](https://learnbyexample.github.io/100_page_python_intro/images/debug_window.png)

Then, with debug mode active, run the program. Use the buttons and options to go over the code. Variable values will be automatically available, as shown below.

![IDLE debug in action](https://learnbyexample.github.io/100_page_python_intro/images/debug_in_action.png)

>You can right-click on a line from the text editor to set/clear breakpoints.

>See [realpython: Debug With IDLE](https://realpython.com/python-debug-idle/) for a more detailed tutorial.