Common Python security pitfalls, and how to avoid them
Tags: python, pythonsecurity
Published: 2021-02-10 17:43:05

---
title: Common Python security pitfalls, and how to avoid them
published: true
description: Python's popularity attracts black and white-hat hackers alike. Here's how SCA solutions like Bandit can help us find and fix security flaws in our Python apps.
tags: Python, pythonsecurity
canonical_url: https://deepsource.io/blog/python-security-pitfalls/
---

Python is undoubtedly a popular language. It consistently ranks among the most popular and most loved languages [year](https://insights.stackoverflow.com/survey/2019) after [year](https://insights.stackoverflow.com/survey/2018). That’s not hard to explain, considering how fluent and expressive it is. Its pseudocode-like syntax makes it extremely easy for beginners to pick it up as their first language, while its vast library of packages (including the likes of giants like Django and TensorFlow) ensure that it scales up for any task required of it.

Being such a widely-used language makes Python a very attractive target for malicious hackers. Let’s see a few simple ways to secure your Python apps and keep the black-hats at bay.

## Problems and solutions

Python places a lot of importance on zen, or developer happiness. The clearest evidence of that lies in the fact that the guiding principles of Python are summarized in a poem. Try `import this` in a Python shell to read it. Here are some security concerns that might disturb your zen, along with solutions to restore it to a state of calm.

### Unsafe deserialization

[OWASP Top Ten](https://owasp.org/www-project-top-ten/), a basic checklist for web security, mentions unsafe deserialization as one of the ten most common security flaws. While it’s common knowledge that executing anything coming from the user is a terrible idea, serializing and deserializing user input does not seem equally serious. After all, no code is being run, right? Wrong,

PyYAML is the _de-facto_ standard for YAML serialization and deserialization in Python. The library supports serializing custom data types to YAML and deserializing them back to Python objects. See this serialization code here and the YAML produced by it.

```
# serialize.py
import yaml class Person: def __init__ (self, name, age): self.name = name self.age = age def __str__ (self): return f'<Person: {self.name} - {self.age}>' def __repr__ (self): return str(self) person = Person('Dhruv', 24)
with open('person.yml', 'w') as output_file: yaml.dump(person, output_file)
```

```
!!python/object: __main__.Person
age: 24
name: Dhruv
```

Deserializing this YAML gives back the original data type.

```
# deserialize.py
import yaml # class Person needs to be present in the scope
with open('person.yml', 'r') as input_file: person = yaml.load(input_file, Loader = yaml.Loader) print(person)
```

```
$ python deserialize.py ↵
<Person: Dhruv - 24>
```

As you can see, the line `!!python/object: __main__.Person` in the YAML describes how to re-instantiate objects from their text representations. But this opens up a [slew of attack vectors, that can escalate to RCE](https://github.com/yaml/pyyaml/wiki/PyYAML-yaml.load%28input%29-Deprecation) when this instantiation can execute code.

**Solution**

The solution, as trivial as it may seem, is to use safe-loading by swapping out the loader `yaml.Loader` in favor of the `yaml.SafeLoader` loader. This loader is safer because it completely blocks loading of custom classes.

```
# deserialize.py
import yaml # class Person needs to be present in the scope
with open('person.yml', 'r') as input_file: person = yaml.load(input_file, Loader = yaml.SafeLoader) print(person)
```

```
$ python deserialize.py ↵
ConstructorError: could not determine a constructor for the tag 'tag:yaml.org,2002:python/object: __main__.Person'
  in "person.yml", line 1, column 1
```

Standard types like hashes and arrays can still be serialized to and deserialized from YAML documents just like before. Most people, probably including you, won’t even realize the difference.

```
age: 24
name: Dhruv
```

```
$ python deserialize.py ↵
{'age': 24, 'name': 'Dhruv'}
```

### Dynamic execution

Python has a pair of very dangerous functions: `exec` and `eval`. Both are very similar in terms of what they do: process the strings passed to them as Python code. `exec` expects the string to be a statement, which it will execute and not return a value. `eval` expects the string to be an expression and will return the computed value of the expression.

Here is an example of what both these functions in action.

```
eval('2 + 5') # returns 7
exec('print("Hello")') # prints "Hello", no return
```

You could in theory, pass a statement to `eval` and get a similar effect as `exec`, because in Python returning `None` is virtually the same as not returning anything at all.

```
eval('print("Hello")') # prints "Hello", returns None
```

The danger of these functions lies in the ability of these functions to execute virtually any code in the same Python process. Passing any input to the function that you cannot be 100% certain about, is akin to handing over your server keys to malicious hackers on a plate. This is the very definition of RCE.

**Solution**

There are ways to mitigate the access that eval has. You can restrict access to globals and locals by passing dictionaries as the second and third arguments to `eval` respectively. Remember that locals take priority over globals in case of conflict.

```
x = 3
eval('x + 5') # returns 8
eval('x + 5', { 'x': 2 }) # returns 7
eval('x + 5', { 'x': 2 }, { 'x': 1 }) # returns 6
```

That makes the code safer yes. At least it somewhat prevents the data in the variables from being leaked.

![](https://i.kym-cdn.com/photos/images/original/001/422/274/627.png)

But it still doesn’t prevent the string from accessing any built-ins like `pow`, or more dangerously, ` __import__ `. To counter that, you need to override ` __builtins__ `.

```
eval(" __import__ ('math').sqrt(5)", {}, {}) # returns 2.2360679774997898
eval( " __import__ ('math').sqrt(5)", { " __builtins__": None }, {} # restricts access to built-ins
) # error
```

Safe to expose now? Not quite. Because regardless of how secure you make `eval`, it’s job is to evaluate an expression and nothing can stop the expression from taking too long and freezing the server for a long time.

```
eval("2 ** 2147483647", { " __builtins__": None }, {}) # goodbye!
```

As we Python developers like to say, “`eval` **is evil**.”

### Dependency management

Python’s popularity draws the attention of white-hat security researchers just as much as it does that of hackers with malicious intent. As a result, new security vulnerabilities that are constantly discovered, disclosed, and patched. To keep the malicious hackers at bay, your software needs to keep all its dependencies up to date.

A common technique for pinning packages in Python is the ubiquitous `requirements.txt` file, a simple file that lists all the dependencies and exact versions needed by your project.

Let’s say you install Django. As of writing, Django depends on three more packages. If you freeze your dependencies, you end up with the following requirements. Note that only one of these dependencies was installed by you, the other 3 are sub-dependencies.

```
$ pip freeze ↵
asgiref==3.3.1
Django==3.1.5
pytz==2020.5
sqlparse==0.4.1
```

`pip freeze` does not place dependencies in levels and that’s a problem. For smaller projects with a few dependencies that you can keep a track of mentally, this is not a big deal but as your projects grow, so will your top level dependencies. Conflicts arise when sub-dependencies overlap. Updating individual dependencies is a mess too, because the graph relationship of these dependencies is not clear from the plain text file.

![](https://i.imgflip.com/r6k4l.jpg)

**Solution**

[Pipenv](https://pipenv.pypa.io/en/latest/) and [Poetry](https://python-poetry.org) are two tools that help you manage dependencies better. I prefer Pipenv but Poetry is equally good. Both package managers build on top of `pip`. **Fun fact:** DeepSource is compatible with both Pipenv and Poetry as package managers.

Pipenv, for example, tracks your top-level dependencies in a `Pipfile` and then does the hard work of locking down dependencies in a lockfile name `Pipfile.lock`, similar to how `npm` manages Node.js packages. Here’s an example `Pipfile`.

```
[[source]]
name = "pypi"
url = "https://pypi.org/simple"
verify_ssl = true

[dev-packages]

[packages]
django = "*"

[requires]
python_version = "3.9"
```

With this, you can get a clear picture of the top-level dependencies of your app. Updating dependencies is also much easier because you just need to update the top level packages and the locking algorithm will figure out the most compatible and up-to-date versions of all the sub-dependencies.

Here is the same example with Django. Notice how Pipenv can identify your top-level dependencies and its dependencies and so on.

```
$ pipenv graph ↵
Django==3.1.5
  - asgiref [required: >=3.2.10,<4, installed: 3.3.1]
  - pytz [required: Any, installed: 2020.5]
  - sqlparse [required: >=0.2.2, installed: 0.4.1]
```

If your code is hosted on GitHub, make sure you turn on and configure [Dependabot](https://dependabot.com) as well. It’s a nifty little bot that alerts you if any of your dependencies has gone out of date or if a vulnerability has been identified in the pinned version of a dependency. Dependabot will also make PRs to your repo, automatically updating your packages. Very handy indeed!

### Runtime assertions

Python has a special `assert` keyword for guarding against unexpected situations. The purpose of `assert` is simple, verify a condition and raise an error if the condition is not fulfilled. In essence, `assert` evaluates the given expression and

- if it evaluates to a truthy value, moves along
- if it evaluates to a falsey value, raises an `AssertionError` with the given message

Consider this example.

```
def do_something_dangerous(user, command): assert user.has_permissions(command), f'{user} is not authorized' user.execute(command)
```

This is a very simple example where we check if the user has the permissions to perform an action and subsequently perform the given action. In this case, if `user.has_permissions()` returns `False`, our assertion would cause an `AssertionError` and the execution would be halted. Seems pretty safe, right?

![](https://media.giphy.com/media/njYrp176NQsHS/giphy.gif)

No. Assertions are tools for developers during development and debugging phases. Asserts should not be used to guard critical functionality. The Python constant, ` __debug__ ` is set to `False` during compilation, which removes assert statements from the compiled code to optimise the code for performance. Removing assert statements from the compiled code leaves the function unguarded.

Another reason to avoid `assert`s is that assertion errors are not helpful to debuggers as they provide no information other than the fact that an assertion did not hold up. Defining apt exception classes and then raising their instances is a much more solid solution.

**Solution**

For an alternative approach, go back to the basics. Here’s the same program, this time using `if/else` and a raising a `PermissionError` (you’ll need to define it somewhere) when the required assertion is unfulfilled.

```
def do_something_dangerous(user, command): if user.has_permissions(command): # [safer as it will not be removed by compiler user.execute(command) else: raise PermissionError(f'{user} is not authorized') # suitable error class
```

This code uses straightforward Python constructs, works the same with ` __debug__ ` set to `True` or `False` and raises clear exceptions that can be handled with much more clarity.

## Achieving Zen

The key takeaway from all of these examples is to never trust your users. Input provided by the users should not be serialized-deserialized, evaluated, executed or rendered. To be safe, you must be careful of what you write and thoroughly audit the code after its been written.

Do you know what’s better than scanning for vulnerabilities in code after it’s been written? Getting vulnerabilities highlighted as soon as you write code with them.

![](https://i.kym-cdn.com/entries/icons/original/000/028/987/lightningspeed.jpg)


### Bandit

[Static Code Analysis tools](https://dzone.com/articles/top-7-static-code-analysis-tools) such as linters and vulnerability scanners can help you find a lot of issues before they get exploited in the wild. An excellent tool for finding security vulnerabilities in Python is [Bandit](https://bandit.readthedocs.io/en/latest/). Bandit goes through each file, generates an abstract syntax tree (AST) for it and then runs a whole slew of tests on this AST. Bandit can detect a whole bunch of vulnerabilities out-of-the-box and can also be extended for specific scenarios and compatibility with frameworks via plugins. As a matter of fact, Bandit is capable of detecting all of the aforementioned security shortcomings.

If you’re a Python developer, I cannot recommend Bandit enough. I could write a whole article extolling its virtues. I might even do that.

### DeepSource
You should also consider automating this entire audit and review process using code review automation tools like DeepSource that scans your code, on every commit and for every PR, through it’s linters and security analyzers and can automatically fix a multitude of issues. [DeepSource](https://deepsource.io/) also has its own custom-built analyzers for most languages that are constantly improved and kept up-to-date. And it’s [incredibly easy to set up](https://deepsource.io/docs/analyzer/python.html)!

```
version = 1

[[analyzers]]
name = "python"
enabled = true

  [analyzers.meta]
  runtime_version = "3.x.x"
  max_line_length = 80

```

Who knew it could be so simple?

Experience the zen of Python, and be careful not to let black-hats disturb your peace!