How to debug Django inside a Docker container with VSCode
Tags: python, docker, vscode, debug
Published: 2024-01-26 16:46:23

_“If you want a thing done well, do it yourself.” ― Napoleon Bonaparte_

This tutorial is valid for these platforms:

- Linux :white_check_mark:
- Windows :x:
- OSX :x:
 
Changes will be indicated along with a changelog.

## Table Of Contents
- [Table Of Contents](#table-of-contents)
- [1. Requirements](#1-requirements)
- [2. Motivation](#2-motivation)
- [3. How to set up for this example](#3-how-to-set-up-for-this-example)
  - [3.1. Clone the project and have some requirements ready](#31-clone-the-project-and-have-some-requirements-ready)
  - [3.2. Enable virtual environment](#32-enable-virtual-environment)
  - [3.3. Add debugpy](#33-add-debugpy)
  - [3.4. Install the packages for definitions](#34-install-the-packages-for-definitions)
  - [3.5. Build the project for Docker](#35-build-the-project-for-docker)
  - [3.6. Create the debug file](#36-create-the-debug-file)
  - [3.7. Create the launch.json file to allow VSCode to attach to our application](#37-create-the-launchjson-file-to-allow-vscode-to-attach-to-our-application)
  - [3.8. Create a test view](#38-create-a-test-view)
- [4. Debugging your code](#4-debugging-your-code)
  - [4.1. Execution of docker compose up with debug](#41-execution-of-docker-compose-up-with-debug)
  - [4.2. Attach debugger to application](#42-attach-debugger-to-application)
  - [4.3. Request the URL and debug your code](#43-request-the-url-and-debug-your-code)
- [5. Debugging library code](#5-debugging-library-code)
  - [5.1. Changing `launch.json` to work with library code](#51-changing-launchjson-to-work-with-library-code)
  - [5.2. Configuring variables for library code debug](#52-configuring-variables-for-library-code-debug)
  - [5.3. Debugging the library code](#53-debugging-the-library-code)
- [6. Conclusion](#6-conclusion)
- [7. References](#7-references)
  - [7.1 Docker Engine and Docker Compose](#71-docker-engine-and-docker-compose)
  - [7.2 Pyenv](#72-pyenv)
  - [7.3 Poetry](#73-poetry)
  - [7.4 Git](#74-git)
  - [7.5 Project](#75-project)
  - [7.6 Debugpy](#76-debugpy)
  - [7.7 VSCode](#77-vscode)

## 1. Requirements

- [Docker Engine](https://docs.docker.com/engine/install/)
- [Docker Compose v2.23.3](https://docs.docker.com/compose/install/linux/#install-the-plugin-manually) (v2.24.1 has a bug when using two docker-compose.yml files that override)
- Python version manager [pyenv](https://github.com/pyenv/pyenv)
- Python >= 3.8 (for this example, 3.11.x is used) through `pyenv`
- Package manager [Poetry](https://python-poetry.org/docs/#installation) v1.4.2
- [VSCode](https://code.visualstudio.com/) editor at least v1.85
- Python package [debugpy](https://github.com/microsoft/debugpy) >= 1.8
- For this demonstration, this project will be used: https://github.com/wiamsuri/django-gunicorn-nginx-docker

## 2. Motivation

Many tutorials available online don't acknowledge the problems of debbuging Django applicants with Docker, usually they explain basic functioning and sometimes are just as vague on how it works. My intention is explaining how to set up a debugger for Django applications that use Docker, Poetry and VSCode, correctly.

One main problem that I constantly see is the **debugpy** module always being installed when executing the container for debugging, which consumes bandwidth and time. It should be **installed as a development dependency** for the project, so you can always use it when needed.

## 3. How to set up for this example

If you already have an project configured with `pyenv`, Poetry and Docker. **Skip steps 1, 2 and 8**.

### 3.1. Clone the project and have some requirements ready

Install the `pyenv` Python version manager, there are various ways to do it [here](https://github.com/pyenv/pyenv#installation). My personal choice is the automatic installer. Also check the [shell set up](https://github.com/pyenv/pyenv#set-up-your-shell-environment-for-pyenv), it will enable your environment to always have `pyenv` easily from your terminal.

After that, install the latest Python version available of `3.11`, do it using the following:

```sh
pyenv install 3.11
```

As soon as you installed the specific Python version, install the Poetry package manager. I recommend using the [official installer](https://python-poetry.org/docs/#installing-with-the-official-installer), once done, proceed to clone the repository.

Clone the project using [Git](https://git-scm.com/):

```sh
git clone https://github.com/wiamsuri/django-gunicorn-nginx-docker
```

You will need to specify the Python version to `3.11.x` at the root of the project:

```sh
pyenv local 3.11.<hotfix_version_installed>
```

At this point, I'm assuming you also have the **Docker Engine** and **Docker Compose V2** installed.

### 3.2. Enable virtual environment

Enable the virtual environment. If you don't have one, it will create for you with the command below:

```sh
poetry shell
```

### 3.3. Add debugpy

`debugpy` description from it's [repository](https://github.com/microsoft/debugpy):

>An implementation of the Debug Adapter Protocol for Python 3.

It is a module maintained by Microsoft in order to debug Python programs, allowing connection to a debugger.

If the project does not have the `debugpy` module, add it with the command below:

```sh
poetry add --group=dev debugpy
```

Done this way, `debugpy` will always be available at development and you won't consume bandwidth and lose your time waiting for the debugger module to install everytime.

### 3.4. Install the packages for definitions

The command below will install all the packages needed for the project so VSCode can identify the packages in the source code. This step is needed to allow definitions for library's to be available:

```sh
poetry install
```

### 3.5. Build the project for Docker

Our project example has a `README.md` file which explains how to build the project, each project has it's own configuration and explanations on how to build them. You can read it [here](https://github.com/wiamsuri/django-gunicorn-nginx-docker#for-development) or use the following commands sequentially:

```sh
cp django.env.example django.env
```

```sh
docker compose build
```

### 3.6. Create the debug file

This file is essential to allow debugging, as it will reuse all configurations defined in other files and only changes what is needed to accomplish our task.

1. At the same level of `docker-compose.yml`, create a new file called `docker-compose.debug.yml`.
2. It should have the service you need to debug, as also the `ports` and `command`.
3. It must have the same port used for the application and debug port defined.

File example:

```yml
version: "3.7"

services:
  app:
    ports:
      - "8000:8000"
      - "5678:5678"
    command: ["sh", "-c", "python3 -m debugpy --listen 0.0.0.0:5678 manage.py runserver 0.0.0.0:8000"]
```

As shown in the file, it's calling the `debugpy` module and listening for connections at the port `5678`, but it isn't waiting the connection of the debugger, it will run the application just as normal. If you need to wait for it to connect, you might need the flag `--wait-for-client`, then the application will only start when the debugger is connected. Check more information in the `debugpy` module documentation.

### 3.7. Create the launch.json file to allow VSCode to attach to our application

In case you don't have a `launch.json` file, create one. VSCode offers a template as default.

![Run and debug tab showing how to configure it](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/64r976oaybfog7kgoyq8.png)

Click on **create a launch.json file**.

1. You might need to select the debugger, choose `Python`.
2. Then choose the debug configuration, go for `Remote Attach`.
3. It will ask configurations for the remote debugging, stick with the default, or change to your needs.

The result will be something like this:

```json
{
  // Use IntelliSense to learn about possible attributes.
  // Hover to view descriptions of existing attributes.
  // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
  "version": "0.2.0",
  "configurations": [
    {
      "name": "Python: Remote Attach",
      "type": "python",
      "request": "attach",
      "connect": {
        "host": "localhost",
        "port": 5678
      },
      "pathMappings": [
        {
          "localRoot": "${workspaceFolder}",
          "remoteRoot": "."
        }
      ],
      "justMyCode": true
    }
  ]
}
```

**Change the property** `justMyCode` to **false**, so that the debugger can have access to code from the library that your code might call to see it's behavior.

### 3.8. Create a test view

As the project is just an example, it has a basic template for running Django with Docker. We have to create a view to test our debugger and check if it's running correctly.

At the `urls.py` file, in the `config/` folder, create a `hello_view` function and add it to the `urlpatterns` list.

```python
from django.contrib import admin
from django.http import HttpResponse
from django.urls import path

def hello_view(request):
    if request.method == "GET":
        return HttpResponse("Hello, world!")

urlpatterns = [
    path('admin/', admin.site.urls),
    path('hello/', hello_view)
]
```

With this function, the debugger will be able to stop the execution of our code and check what's happening inside the program.

## 4. Debugging your code

### 4.1. Execution of docker compose up with debug

If everything was done correctly, we can start our application with the debugging command. In order for this event to happen, specify the debug file in the command. Do it as follows:

```sh
docker compose -f docker-compose.yml -f docker-compose.debug.yml up
```

The **Docker Compose** will overlap the files and use the debug file specified. As it's using a debug file, a base file must be determined for the services. As you may have other `docker-compose` files, just ensure the `docker-compose.debug.yml` is the last one, so the program can overlap the files and get their configurations correctly.

### 4.2. Attach debugger to application

With the application running, select the "Run and Debug" tab at VSCode, it will look like this if the `launch.json` file is present:

![Image showing Run and Debug tab when it has a launch.json file](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/lgp7jvr31sa70iyv84eg.png)

Click the play button indicated as **Python: Remote Attach**. In normal scenarios, it will connect the debugger to your application and a toolbar will appear on screen.

![VSCode with debugger toolbar](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/kjrc0p6h3pn72s7c9719.png)

This means the debugger is working now, let's test it.

### 4.3. Request the URL and debug your code

Open your browser of choice and type `localhost:8000/hello/`, a call to the test view will be made. The result should be a "Hello, world!":

![Browser page with "Hello, World!" text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/hm8dfqtk0bz9651pgjk9.png)

Notice that the debugger hasn't triggered yet, that's because we didn't specified any breakpoints in the code.

Set up a breakpoint on line 22 (L22) by clicking on the left side of the line number:

![Line with breakpoint](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/haz61faou2rhyqvhvvy0.png)

Now we refresh the page and you will see the *magic* happening.

![Code execution stopped at line 22](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/7fl51opk727497abszt5.png)

Our code execution stopped at the breakpoint we've just specified, you will see variables, call stack and even watch for specific values or expressions. You can learn more about the debugger screen in the [VSCode documentation](https://code.visualstudio.com/docs/editor/debugging).

That's the basic to debug your code and see how it's behaving. However, **this configuration does not allow to make breakpoints inside libraries** that you might have in your project.

The next section will explain what's needed to do in order to debug libraries directly from inside your project.

## 5. Debugging library code

Sometimes you're not interested in debugging your code. Instead, you want to understand the library logic or behavior in order to fix your code, or the library itself.

In this scenario, we need breakpoints inside the library code. However, by default, the configuration created above (which solves most problems) isn't enough to allow debugging library code.

Under these circumstances, you will get this grey empty dot:

![Grey empty dot for breakpoint that does not work](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/iym4zp4ynuozppf492ac.png)

In order to solve this, changes are needed at the `launch.json`, environment variables in `.bashrc` and `settings.json` to make it work.

### 5.1. Changing `launch.json` to work with library code

To make the debugger aware that the library code is available, some modifications are needed.

To accomplish this, changes to `launch.json` file must be done, so that the debugger can seek for the path were the library code is. By adding another object at `pathMappings`, we make the code available to the debugger.

```json
{
  // Use IntelliSense to learn about possible attributes.
  // Hover to view descriptions of existing attributes.
  // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
  "version": "0.2.0",
  "configurations": [
    {
      "name": "Python: Remote Attach",
      "type": "python",
      "request": "attach",
      "connect": {
        "host": "localhost",
        "port": 5678
      },
      "pathMappings": [
        {
          "localRoot": "${workspaceFolder}",
          "remoteRoot": "."
        },
        {
          "localRoot": "${env:PACKAGES_ENV_PATH}/${config:python.poetry.project.venv}/lib/python3.11/site-packages/",
          "remoteRoot": "/usr/local/lib/python3.11/site-packages/"
        }
      ],
      "justMyCode": false
    }
  ]
}
```

With this configuration provided for the debugger, which contains the library code path, it will seek for the files and allow to set a breakpoint.

Although, I have to explain what is that object and how you can configure it.

### 5.2. Configuring variables for library code debug

`${env:PACKAGES_ENV_PATH}` is an environment variable defined in `.bashrc` (or similar) that tells us where to look for the libraries source code. This path can be obtained by using:

```sh
poetry env info --path
```

The result will be similiar to:

```sh
/home/youruser/.cache/pypoetry/virtualenvs/django-gunicorn-nginx-docker-ucLVPu3k-py3.11
```

Until `virtualenvs/`, the path leads to all virtual environments that your Poetry creates. After it, is the specific _virtualenv_ for the project.

The environment variable VSCode needs to read, must be defined at `.bashrc` (or similars) as:

```sh
#VSCode debug
export PACKAGES_ENV_PATH="/home/youruser/.cache/pypoetry/virtualenvs"
```

After adding this piece of code to `.bashrc`, VSCode must be restarted to allow the process to get the new defined _env_.
This process has to be done due to how VSCode is opened and how processes work, for more info, see [here](https://github.com/microsoft/vscode/issues/29679).

`${config:python.poetry.project.venv}` is a user defined variable inside `settings.json` which tells us which _virtualenv_ the debugger must look for.

The last part of the command mentioned above can be used to fulfill this part. If the project doesn't have a `settings.json` file, just create one under `.vscode/` folder, in case you have it, define the new variable to use:

```json
{
  "python.poetry.project.venv": "django-gunicorn-nginx-docker-ucLVPu3k-py3.11"
}
```

I tried to came up with a key that won't affect the VSCode ecosystem or any extensions (that I've heard of).

This was the best way I've found that allowed to easily define the path in order to share the `launch.json` file with other coworkers through source control software.

**Observation:** you can also hardcode the path to make your life easier, .

The Python version won't change frequently, so I fixed it in the configuration.

In this context, I'm assuming that `settings.json` **will never be commited** to the source control, and the user can change as he desires. If that's not the case, I would recommend to hardcode the path in the `localRoot`, but every developer path will be different, **be aware**. **DO NOT FORGET** to never include this change to your commits.

_If you know about a better way to do this configuration for library source code debugging, let me know in the comments or reach out to me._

### 5.3. Debugging the library code

Commands are the same from the [Debugging your code](#4-debugging-your-code) section, the only difference is where the breakpoint is indicated.

For the example, I'd choosen the `WSGIHandler` class as it's purpose is receiving the requests and serving them.

I set the breakpoint at line 123 (L123). If the dot is red, it means the debugger can use the breakpoint properly. Then I refreshed the page for our test view, and this is the result:

![Debugger with execution stopped at line 123 where breakpoint was set](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/s4oz27fj921nl3xv08tf.png)

From there, you can happily (or not) debug the code.

## 6. Conclusion

Debugging is not a easy task, but with this tutorial I expect that somehow I made your life easier. So you can achieve your objective and/or catch that bug that is annoying you.

Feel free to comment and warn me about any mistakes that this article may have.

Thanks for reading until here, Ferka out :v:. 

## 7. References

### 7.1 Docker Engine and Docker Compose

- https://docs.docker.com/engine/install/
- https://docs.docker.com/compose/install/linux/#install-the-plugin-manually (be aware of the v2.24.1 bug with overlap compose files)

### 7.2 Pyenv

- https://github.com/pyenv/pyenv
- https://github.com/pyenv/pyenv#installation
- https://github.com/pyenv/pyenv#set-up-your-shell-environment-for-pyenv

### 7.3 Poetry
- https://python-poetry.org/docs/#installation
- https://python-poetry.org/docs/#installing-with-the-official-installer
- https://python-poetry.org/docs/managing-environments/

### 7.4 Git

- https://git-scm.com/

### 7.5 Project

- https://github.com/wiamsuri/django-gunicorn-nginx-docker#for-development

### 7.6 Debugpy
- https://stackoverflow.com/questions/76876915/pydev-debugger-unable-to-find-translation-for-please-revise-your-path-mapping
- https://github.com/microsoft/vscode-python-debugger/issues/158

### 7.7 VSCode
- https://code.visualstudio.com/docs/editor/debugging#_launch-configurations
- https://code.visualstudio.com/docs/editor/variables-reference#_configuration-variables
- https://code.visualstudio.com/docs/editor/debugging
- https://code.visualstudio.com/docs/python/environments
- https://code.visualstudio.com/api/extension-guides/command
- https://stackoverflow.com/questions/66971452/how-to-add-virtual-environment-to-vscode-launch-json
- https://github.com/microsoft/vscode/issues/29679
- https://www.google.com/search?channel=fs&client=ubuntu-sn&q=how+to+populate+vscode+env
- https://stackoverflow.com/questions/75767757/how-can-i-specify-environment-variables-for-the-vs-code-process-in-my-vs-code-se
- https://unix.stackexchange.com/questions/38205/change-environment-of-a-running-process
- https://stackoverflow.com/questions/48595446/is-there-any-way-to-set-environment-variables-in-visual-studio-code
- https://stackoverflow.com/questions/44303316/vscode-defining-own-variables-in-tasks-json