Creating Beautiful Tracebacks with Python's Exception Hooks
Tags: python, tutorial, programming, webdev
Published: 2022-02-02 08:08:41


We all spend a good chuck of our time debugging, sifting through logs or reading tracebacks. Each of these can be difficult and time-consuming and in this article we will focus on making the last one - dealing with tracebacks and exceptions - as easy and efficient as possible.

To achieve this we will learn how to implement and use custom _Exception Hooks_ that will remove all the noise from tracebacks, make them more readable and display just the information we need to troubleshoot our code and exceptions in Python. On top of that, we will also take a look at awesome Python libraries that provide ready to use exception hooks with beautiful tracebacks, which can be installed and used without any additional coding. 

## Exception Hooks

Whenever exception is raised and isn't handled by `try`/`except` block, a function assigned to `sys.excepthook` is called. This function - called _Exception Hook_ - is then used to output any relevant information to standard output using the 3 arguments it receives - `type`, `value` and `traceback`.

Let's now look at a minimal example to see how this works:

```python
import sys

def exception_hook(exc_type, exc_value, tb):
    print('Traceback:')
    filename = tb.tb_frame.f_code.co_filename
    name = tb.tb_frame.f_code.co_name
    line_no = tb.tb_lineno
    print(f"File {filename} line {line_no}, in {name}")

    # Exception type and value
    print(f"{exc_type.__name__}, Message: {exc_value}")

sys.excepthook = exception_hook
```

In the above example we leverage each of the arguments to provide basic traceback data in output. We use traceback (`tb`) object to access the traceback frame which contains data describing where the exception occurred - that is - filename (`f_code.co_filename`), function/module name (`f_code.co_name`) and line number (`tb_lineno`).

Apart from that, we also print information about exception itself using the `exc_type` and `exc_value` variables.

With this hook in place, we can invoke a function that raises some exception and we will receive the following output:

```python
def do_stuff():
    # ... do something that raises exception
    raise ValueError("Some error message")

do_stuff()

# Traceback:
# File /home/some/path/exception_hooks.py line 22, in <module>
# ValueError, Message: Some error message
```

The above example provides some information about exception, but to get all the information needed for debugging, as well as a full picture of where and why the exception happened, we need to dig a bit deeper into the traceback object: 

```python
def exception_hook(exc_type, exc_value, tb):

    local_vars = {}
    while tb:
        filename = tb.tb_frame.f_code.co_filename
        name = tb.tb_frame.f_code.co_name
        line_no = tb.tb_lineno
        print(f"File {filename} line {line_no}, in {name}")

        local_vars = tb.tb_frame.f_locals
        tb = tb.tb_next
    print(f"Local variables in top frame: {local_vars}")

...

# File /home/some/path/exception_hooks.py line 41, in <module>
# File /home/some/path/exception_hooks.py line 7, in do_stuff
# Local variables in top frame: {'some_var': 'data'}
```

As you can see here, the traceback object (`tb`) is actually a linked list of all the exceptions that occurred - a stacktrace. This allows us to loop through it using `tb_next` and print information for each frame. On top of that, we can also use `tb_frame.f_locals` attribute to dump local variables to console, which can also aid in debugging.

Digging through the traceback object like we saw above works, but it's cumbersome and becomes quite unreadable very quickly. Better solution is to use `traceback` module instead, which provides lots of helper functions for extracting information about exceptions.

So, now that we know the basics, let's see how we can build our own exception hooks with some real, useful features...

## Make Your Own

There are more things that we can do then just dump data on `stdout`. One of them would be logging the output to a file automatically:

```python
LOG_FILE_PATH = "./some.log"
FILE = open(LOG_FILE_PATH, mode="w")

def exception_hook(exc_type, exc_value, tb):
    FILE.write("*** Exception: ***\n")
    traceback.print_exc(file=FILE)

    FILE.write("\n*** Traceback: ***\n")
    traceback.print_tb(tb, file=FILE)
    
# *** Exception: ***
# NoneType: None
# 
# *** Traceback: ***
#   File "/home/some/path/exception_hooks.py", line 82, in <module>
#     do_stuff()
#   File "/home/some/path/exception_hooks.py", line 7, in do_stuff
#     raise ValueError("Some error message")
```

This can be useful if you want to preserve information about uncaught exception for later debugging.

By default, uncaught exception will go to `stderr`, which might be undesirable if you have a logging setup in place and want the logger to take care of the error output. You could use following hook to allow logger to take care of these exceptions:

```python
import logging
logging.basicConfig(
    level=logging.CRITICAL,
    format='[%(asctime)s] {%(pathname)s:%(lineno)d} %(levelname)s - %(message)s',
    datefmt='%H:%M:%S',
    stream=sys.stdout
)

def exception_hook(exc_type, exc_value, exc_traceback):
    logging.critical("Uncaught exception:", exc_info=(exc_type, exc_value, exc_traceback))


# [17:28:33] {/home/some/path/exception_hooks.py:117} CRITICAL - Uncaught exception:
# Traceback (most recent call last):
#   File "/home/some/path/exception_hooks.py", line 122, in <module>
#     do_stuff()
#   File "/home/some/path/exception_hooks.py", line 7, in do_stuff
#     raise ValueError("Some error message")
# ValueError: Some error message
```

The first thing that comes to mind when trying to improve the console output is making it pretty by giving it some colours highlighting the important bits:

```python
# pip install colorama
from colorama import init, Fore
init(autoreset=True)  # Reset the color after every print

def exception_hook(exc_type, exc_value, tb):

    local_vars = {}
    while tb:
        filename = tb.tb_frame.f_code.co_filename
        name = tb.tb_frame.f_code.co_name
        line_no = tb.tb_lineno
        # Prepend desired color (e.g. RED) to line
        print(f"{Fore.RED}File {filename} line {line_no}, in {name}")

        local_vars = tb.tb_frame.f_locals
        tb = tb.tb_next
    print(f"{Fore.GREEN}Local variables in top frame: {local_vars}")
```

There's obviously much more you could do, for example printing local variables in each frame, or even lookup variables that were referenced on the line on which the exception occurred. Unsurprisingly, exception hooks for these use-cases already exist, so instead of dumping the code on you, I'd suggest you take a look at their source code from which you can draw some inspiration. 

- [printing local variables from each frame](https://github.com/Infinidat/infi.traceback/blob/develop/src/infi/traceback/__init__.py)
- [printing relevant info for each frame (referenced variables)](https://github.com/albertz/py_better_exchook/blob/master/better_exchook.py)

Finally, I want to include a word of caution, whenever you decide to install an exception hook, be aware that libraries can install their own hooks, so make sure you don't override those. In those cases you can instead catch the exception and use `except` block to output information you want to see, for example by using `sys.exc_info()`.

## Awesome Hooks in The Wild

Building your own exception hook can be a fun little exercise, but there are already quite a few cool ones out there. So, instead of reinventing a wheel, let's rather look at what we can just grab and start using immediately.

First and my favourite being _Rich_:

```python
# https://rich.readthedocs.io/en/latest/traceback.html
# pip install rich
# python -m rich.traceback

from rich.traceback import install
install(show_locals=True)

do_stuff()  # Raises ValueError
```

![Rich traceback](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/e279yfdokjevl803w7ih.png)

The installation is super easy, all you need to do is install the library, import it and run `install` function which puts exception hook in place. If you want to just check out a sample output without writing Python code, then you can also use `python -m rich.traceback`.

Another popular option is `better_exceptions`. It also produces nice output, but requires a little more setup:

```python
# https://github.com/Qix-/better-exceptions
# pip install better_exceptions
# export BETTER_EXCEPTIONS=1

import better_exceptions
better_exceptions.MAX_LENGTH = None
# Check if you TERM variable is set to `xterm`, if not set below variable - https://github.com/Qix-/better-exceptions/issues/8
better_exceptions.SUPPORTS_COLOR = True
better_exceptions.hook()

do_stuff()  # Raises ValueError
```

In addition to installing the library with `pip` we also need to set `BETTER_EXCEPTIONS=1` environment variable to enable it. Next, we need the above Python code for setup. The most important part being the call to `hook` function which installs the exception hook. Additionally, we also set `SUPPORTS_COLOR` to `True` which might be necessary depending on the terminal you're using - more specifically - you will need this if your `TERM` variable is set to anything other than `xterm`. 

![better_exceptions traceback](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/2pj82xjdvuulyszzj9e6.png)

Next up is `pretty_errors` library. This one is definitely the easiest one to configure, requiring just an import:

```python
# https://github.com/onelivesleft/PrettyErrors/
# pip install pretty_errors

import pretty_errors
# `configure` can be omitted if you're satisfied with default settings
pretty_errors.configure(
    filename_display    = pretty_errors.FILENAME_EXTENDED,
    line_number_first   = True,
    display_link        = True,
    line_color          = pretty_errors.RED + '> ' + pretty_errors.default_config.line_color,
    code_color          = '  ' + pretty_errors.default_config.line_color,
    truncate_code       = True,
    display_locals      = True
)

do_stuff()
```

Apart from the mandatory `import`, the above snippet also shows optional configuration for the library. This is just a small sample of what you can configure to produce below output. The full list of config options can be found [here](https://github.com/onelivesleft/PrettyErrors/#configuration-settings). 

![pretty_errors traceback](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/1l50dm93ezqmt0kqs89f.png)

Next one is a library whose output style will be familiar to everyone who uses Jupyter notebook. It's IPython's `ultratb` module which provides a couple of options for very pretty and readable exception and traceback error outputs:

```python
# https://ipython.readthedocs.io/en/stable/api/generated/IPython.core.ultratb.html
# pip install ipython
import IPython.core.ultratb

# Also ColorTB, FormattedTB, ListTB, SyntaxTB
sys.excepthook = IPython.core.ultratb.VerboseTB(color_scheme='Linux')  # Other colors: NoColor, LightBG, Neutral

do_stuff()
```

![IPython traceback](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/08j46jt6l6pymnqnbbry.png)

Last but not least is `stackprinter` library which produces concise output with all the debugging information you might need. Again, all you need to do to set it up is install the exception hook:

```python
# https://github.com/cknd/stackprinter
# pip install stackprinter
import stackprinter
stackprinter.set_excepthook(style='darkbg2')

do_stuff()
```

![stackprinter traceback](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/3rtlyvbclhoi8yfjgbjs.png)

## Conclusion

In this article we learned how to write an exception hook, but I don't actually recommend writing and using your own hooks. Implementing one such hook could be a fun exercise, but probably not a worthwhile effort. You're better off using one of the awesome hooks presented above and calling it a day.

I do however, strongly recommend choosing one of them and installing it across all the projects you're working on, both for improved debugging, but also for consistency. The more you use one of these exception hooks, the more used you will become to its output and consequently, the more benefit you will get from using it.

With that said though, you should consider excluding custom exception hooks from your production build, as the prettified outputs might obscure some information which might be critical in certain scenarios. One such example would be missing file paths in some of the outputs above, which aids readability when debugging locally, but might make it harder to debug code running on remote system.