5 Different Meanings of Underscore in Python | by Xiaoxu Gao | Towards Data Science

If you are a Python programmer, you are probably familiar with underscores. Python works with two types of underscore: single underscore _ and double underscores __. Don’t underestimate the underscore in Python, it’s a very powerful syntax. In this article, I will talk about 5 different underscore patterns.
Single standalone underscore _ is a valid character for a Python identifier, so it can be used as a variable name.
Represent the last expression in the interpreter
Represent the last expression in the interpreter
According to Python doc, the special identifier _ is used in the interactive interpreter to store the result of the last evaluation. It is stored in the builtin module.
Here is an example. At first, we check that _ is not stored in the builtin module, then we write a single expression without a variable name. If we check the builtin module again, we will find _ in the module and the value is the last evaluation.
>>> '_' in dir(__builtins__)False>>> 1+12>>> '_' in dir(__builtins__)True>>> _2
2. Represent the values that we don’t care
Another use case of single underscore _ is to represent the value that you don’t care or will not be used later in the program. If you apply a linter like Flake8 to your program, you will get an error from the linter (F841) if you have a variable name assigned but never used. Assigning variables that you don’t care to _ can solve this problem.
Let’s see some code. Example1 uses _ to represent the index of each element in a list. In Example2, we only care about year, month and day from the tuple, so we assign _ to the rest (hour, minute, second). But if we print out _, we will only get the last expression which is 59.
From Python 3.*, it supports extended iterable unpacking which means we can use *_ to represent multiple values. In Example3, _ actually represents a list of values that we want to ignore.
Lambda function also supports _. In Example4, lambda function is used to monkeypatch function random.randint, and will always generate the same output. In this case, the input argument is not important anymore.
3. Visual separator for digit grouping purposes
From Python 3.6, underscore _ can also be used as a visual separator for digit grouping purposes. As stated in PEP515, it works for integers, floating-point, and complex number literals.
integer = 1_000amount = 1_000_000.1binary = 0b_0100_1110hex = 0xCAFE_F00D>>> print(amount)1000000.1>>> print(binary)78>>> print(hex)3405705229
According to PEP8, single leading underscore _var is intended for internal use. from M import * doesn’t import objects whose names start with an underscore.
_ in front of a variable or method name is a weak internal use indicator. It warns the developer that this variable, method, or function is not supposed to be imported and used publicly. Nevertheless, Python doesn’t completely prevent them from being imported and used.
In the following example, we have a public variable external and a private variable _internal. If we do wildcard import from m import *, it doesn’t include _internal variable (main1.py). But it works if we import _internal explicitly (main2.py) or use regular import (main3.py).
Single leading underscore is used a lot in classes. Programmers can create private variables and methods, but like the previous examples, these variables and methods can still be used from the outside.
In general, the single leading underscore is only a naming convention to indicate the variable or function is for internal use. Programmers are still able to import the name if they really want.
There is only one reason to use single trailing underscore which is to avoid conflicts with Python keywords. This is also mentioned in PEP8.
We can use keyword module to check the list of keywords in Python. If you want to have a variable name called global, then you should name it gloabl_.
The patterns we’ve discussed so far are basically different naming conventions, but with a double leading underscore, Python will behave somewhat differently.
Python interpreter will do name mangling to identifiers with leading underscores. Name mangling is the process to overwrite such identifiers in a class to avoid conflicts of names between the current class and its subclasses.
In summary, __var will have a different name in the class. Let’s look at the following example. The example uses built-in function dir to return the list of names in the current local scope. At the end of the list, we see _hour, day, month and year, which means we can directly retrieve these attributes by time._hour or time.year.
However, for __minute, it has another story. Its name is overwritten to _Time__minute.
This will have 2 consequences:
Attribute __day is not accessible outside class Time. As you can see, __day is not recognized by the class Time, and it is replaced with _Time__day.
Attribute __day is not accessible outside class Time. As you can see, __day is not recognized by the class Time, and it is replaced with _Time__day.
print(time.__day)# AttributeError: 'Time' object has no attribute '__day'print(time._Time__day)# 10
2. Attribute __day can’t be overwritten in the subclasses of Time. In the example below, we create a subclass TimeSubClass which is extended from class Time, and use dir to check the local names.
In the output, you can find _TimeSubclass__day, _Time__day, _month and year. It means that _month and year have been overwritten with new values, but not for __day.
Let’s check the values. _TimeSubclass__day is assigned with the new value, and _Time__day still has the original value.
print(time_subclass._TimeSubclass__day)# 30print(time_subclass._Time__day)# 1
Although you can’t get the value via direct access, you can still get the value via a method, and in the method, you return self.__day. It also works for @property.
Unlike single trailing underscore, there is no special meaning for double trailing underscores. You can probably think of var__ as an extension of var_.
However, double leading and trailing underscore __var__ is completely different and it’s a very important pattern in Python. Python doesn’t apply name mangling to such attributes, but names with double leading and trailing underscores are reserved for special use in Python. They are called Magic Names. You can check Python documentation to get a list of magic names. Names like __init__, __call__, __slots__ are all magic methods.
These magic attributes and magic methods are allowed to be overwritten, but you need to know what you are doing. Let’s look at an example. __str__ is a method that returns the string representation of the object. This method is called when you do print() or str(). In the example, we overwrite the string presentation, then we will get a new format in the output.
You are allowed to have a customized name with double leading and trailing underscore like __day__. Python will just take it as a regular attribute name and will not apply name mangling on it. However, it’s not a good practice to have a name like this without a strong reason.
In this article, we’ve discussed 5 different patterns of underscore in Python. Let’s summarize it.
Single standalone underscore _ & Single leading underscore _var & Single trailing underscore var_.
Single standalone underscore _ & Single leading underscore _var & Single trailing underscore var_.
Patterns with a single underscore are basically naming conventions. Python interpreter doesn’t prevent them from being imported and used outside the module/class.
2. Double leading underscores __var & Double leading and trailing underscores __var__.
Patterns with double underscores are more strict. They are either not allowed to be overwritten or need a good reason to do so. Please notice that there is no special meaning for double trailing underscores var__.
I hope you enjoyed this article! Leave your comments below if you have any thoughts.