Debug Python Scripts Like a Pro: The Ugly, the Lovely, and the Smart Ways of Debugging. | Towards Data Science

It took me some time to grasp the idea of debugging.
I’m sure that’s common with most code newbies. To me, as a self-taught Python programmer, locating issues and fixing them in the quickest possible way was challenging.
But over the years, I learned several techniques to spot bugs in my scripts. Some had changed, and some I discontinued. I’ll take you through what I consider the bad, the lovely, and the smart ways of debugging a Python code in this post.
towardsdatascience.com
Let’s begin with the bad.
Perhaps ‘print’ is the most used Python function of all.
Naturally, anyone who wants to know what’s happening in their code uses print statements. Because that’s the easiest and the widely available choice. And it seems to work fine for smaller scripts too.
Go ahead and use it if you intend to test whether your code reaches a point or to check a variable’s value at a specific point. And also if it’s only for one time.
But as your code grow bigger, you’re in trouble. Print statements are going to be on all over the project. You’ll quickly lose track of which printed line corresponds with which print statement in your code.
It gets more complicated if your project executes codes parallel or asynchronous. And it’s not uncommon these days to use parallel execution either.
An immediate solution for this problem is to use a description in every print statement.
But this is going to suck the soul out of a developer. There’s a better way of doing this. That’s our following technique on the list.
towardsdatascience.com
Print statements are a tedious, time-consuming process. Icecreams make debugging sweet.
Icecream is a Python package that lets you inspect variables at different points in your script. In a nutshell, it prints the value of variables, the function executed, and the line number and file names in some instances.
Here’s how to use Icecream to debug your Python code.
Install Icecream using pip.
Install Icecream using pip.
pip install icecream
2. Import Icecream to your script
from icecream import ic
3. Wrap your function calls and statements with ic
Icecream makes the life of a developer easy by automatically adding the context to every log it creates. That’s a lovely way to debug Python code.
But this still is additional work.
Modern IDE’s like PyCharm and VSCode allow you to stop the code at a specific line and inspect all the local and global variables at that point.
Here’s the smart way of debugging.
https://towardsdatascience.com/plotly-dashboards-in-python-28a3bb83702c
Live debugging is a way to speak with your code when running.
Some of us started coding in notepad apps. Later we switched to Notepad++. Then came the revolutionary Sublime Text and Atom editors. They are great tools of their times.
Some of us started with PyCharm. I was one of them who fell in love with this excellent IDE. Yet, I switched to VSCode, because it’s free, and its community was (and still is) skyrocketing.
Not to my surprise, debugging with VSCode is super easy for even beginners. That’s what the remainder of this post is about. Let’s break this section into three: debugging with breakpoints, configuration for popular frameworks, and debugging data frames.
towardsdatascience.com
thuwarakesh.medium.com
You can start a debugger in just three steps. Here’s how:
Click on the debugger on the sidebar. It’s this play button with a bug on it.Create breakpoints in your code. You can do it by clicking before the line number. A red indicator will appear for every breakpoint you create.Now, start the debugger by clicking the “Run and Debug” button and selecting “Python file” in the dropdown.
Click on the debugger on the sidebar. It’s this play button with a bug on it.
Create breakpoints in your code. You can do it by clicking before the line number. A red indicator will appear for every breakpoint you create.
Now, start the debugger by clicking the “Run and Debug” button and selecting “Python file” in the dropdown.
As you do this, VSCode will spin up a thread to run your script and stop where you have your first breakpoint. The screen will appear as follows:
As it stopped in the breakpoint you created, you can now see the line is highlighted, the sidebar has changed, and a new toolbar appears on the top right (this location may differ in your window) corner.
The toolbar is the controller for your debugger. The play button will ignore the current breakpoint and move on to the next. The stop button will stop the debugger, and the refresh button will restart the debugger all over again.
I find the step-in button extremely useful. It’s the down arrow key on the debugger toolbar. Please see the below recording to understand how it works and how useful it is.
My first breakpoint was on line number 14. This line calls a function called ‘sum_odd_numbers.’ As I click the step in key, the interpreter moves to the next line but stops there. It is within the function I call (line number 4.)
Pay attention to the variables section in the sidebar. As we are now within the function, the scope changes. Under the ‘local’ area, we see the value of n is 100. The value of v is available because the interpreter hasn’t executed line number 4 yet.
But as you keep clicking on the step-in key. You see, the value of ‘v’ is changing as well. As we step inside the loop, you can also see the value of ‘i’.
Notice that we get to line number 8 only when the value of ‘i’ is odd. As we execute line number 8, the value of ‘v’ increases by the value of ‘i’ at that time.
Great, we could narrate everything line by line while our script was running. That’s why live debugging is the more brilliant way to find and fix issues on your code.
VSCode supports popular frameworks such as Django, Flask, FastAPI, and Pyramid out of the box. In most cases, you don’t have to configure anything special for these frameworks. You only have to select the framework in the dropdown when starting the debugger.
If this is the first time you’re working on a Django project, you can install django with pip install django and create a django project using the command Django startproject <project_name>
But in certain conditions, you will have to override the defaults. For instance, if another process blocks the 8000 port, Django needs a different one to start. You can give additional information to the debugger using a launch.json file.
Go to the debugger window and select the “create a launch.json file” option. In the dropdown, choose the framework you’re using. I’ve chosen Django for this illustration.
You’ll see a new JSON file open up with some familiar options.
The above file runs the ‘manage.py’ file in the ${workspacefolder} (that’s the root of the project you’ve opened in VSCode). In addition, it takes a command-line argument ‘runserver.’
What it means is it’s about to execute python manage.py runserver when the debugger starts.
I’ve added another argument by extending the ‘args’ list. In line number 11, I configured the Django dev server’s port to be 5000 instead of its default 8000. This modification will let the debugger run python manage.py runserver 5000 .
Go to a new line and press the space holding your ctrl (or command) key to see what other options you have. As shown in the figure above, a list of options will pop up.
Let’s see the debugger in action.
Replace the content of <proj>/urls.py with the following content.
And create a file called views.py with the following content. We’re running the same odd number counter as a web service.
Go ahead and start the debugger by clicking the play button in the sidebar. Now, in your web browser, if you go to http://127.0.0.1:5000/sum_odd_numbers/10, the browser will show a result of 25. Try a different number instead of 10 in the URL to see how the webserver works.
Now, Let’s put a breakpoint at line 7 and revisit the URL on your browser again. This time it the browser keeps waiting for the response. But VSCode stopped processing the request at our breakpoint.
Look at what you’ve got in the variables panel. You can access your entire request object, including information regarding the path, type of request, and browser cookie information.
It’s effortless to debug web servers in live debugging mode. We’ll discuss one of the most recent features that let me debug data frames in the next part.
VSCode has already been very useful in data science with an in-built Jupyter notebook in it.
Yet, debugging an analysis project had been difficult if it was on a script. A very common encounter in my day-to-day work is processing datasets inside web requests.
Lists, dictionaries, and even objects are relatively straightforward to show in the variables section. But a pandas data frame is a complex object that needs to be in a tabular format. Only then it makes sense for the developer to find issues in it.
VSCode has solved this issue in the best possible way. You can simply right-click the data frame object in the variables section and choose ‘View Value in Data Viewer.’
In the above screenshot, I’ve added a breakpoint at line number 5. At this time, the interpreter has read the CSV and the data frame is in the memory. When we open it in the Dat Viewer, we get the following.
It’s a regex-supported filterable tabular view of our data frame. This feature is a handy technique to debug projects when you aren’t on a Jupyter notebook. That’s the case when finding and fixing bugs in most production systems.
If you don’t see the data viewer option on the right-click menu, you have to install or update the Python extension in VSCode. Only the recent versions have this feature.
Debugging can be very challenging, regardless of how experienced you are.
For most beginners, print statements are the easiest to do. But having print statements to find bugs in large production-grade systems is not practical. Only the right tools and techniques can help ease the work.
A lovely tool to rescue is Icecream. Icecream automatically includes a description of the context in which it logs the statement. Yet, every time you want to test something different, you’ll have to rerun the entire script.
Live debugging is the smartest way to find and fix issues in software projects. Most modern IDE’s support live debugging. In this post, we covered how the VSCode debugger works.
I hope this guide helps make your code bug-free.
Thanks for reading, friend! Say Hi to me on LinkedIn, Twitter, and Medium.
Not a Medium member yet? Please use this link to become a member because, at no extra cost for you, I earn a small commission for referring you.