Optimizing Memory Usage of Python Applications
Tags: python, performance, tutorial, programming
Published: 2022-02-28 19:51:52

When it comes to performance optimization, people usually focus only on speed and CPU usage. Rarely is anyone concerned with memory consumption, well, until they run out of RAM. There are many reasons to try to limit memory usage, not just avoiding having your application crash because of out-of-memory errors.

In this article we will explore techniques for finding which parts of your Python applications are consuming too much memory, analyze the reasons for it and finally reduce the memory consumption and footprint using simple tricks and memory efficient data structures.

## Why Bother, Anyway?

But first, why should you bother saving RAM anyway? Are there really any reason to save memory other than avoiding the aforementioned out-of-memory errors/crashes?

One simple reason is money. Resources - both CPU and RAM - cost money, why waste memory by running inefficient applications, if there are ways to reduce the memory footprint?

Another reason is the notion that _"data has mass"_, if there's a lot of it, then it will move around slowly. If data has to be stored on disk rather than in RAM or fast caches, then it will take a while to load and get processed, impacting overall performance. Therefore, optimizing for memory usage might have a nice side effect of speeding-up the application runtime.

Lastly, in some cases performance can be improved by adding more memory (if application performance is memory-bound), but you can't do that if you don't have any memory left on the machine.

## Find Bottlenecks

It's clear that there are good reasons to reduce memory usage of our Python applications, before we do that though, we first need to find the bottlenecks or parts of code that are hogging all the memory.

First tool we will introduce is `memory_profiler`. This tool measures memory usage of specific function on line-by-line basis:

```shell
# https://github.com/pythonprofilers/memory_profiler
pip install memory_profiler psutil
# psutil is needed for better memory_profiler performance

python -m memory_profiler some-code.py
Filename: some-code.py

Line #    Mem usage    Increment  Occurrences   Line Contents
============================================================
    15   39.113 MiB   39.113 MiB            1   @profile
    16                                          def memory_intensive():
    17   46.539 MiB    7.426 MiB            1       small_list = [None] * 1000000
    18  122.852 MiB   76.312 MiB            1       big_list = [None] * 10000000
    19   46.766 MiB  -76.086 MiB            1       del big_list
    20   46.766 MiB    0.000 MiB            1       return small_list
```

To start using it, we install it with `pip` along with `psutil` package which significantly improves profiler's performance. In addition to that, we also need to mark the function we want to benchmark with `@profile` decorator. Finally, we run the profiler against our code using `python -m memory_profiler`. This shows memory usage/allocation on line-by-line basis for the decorated function - in this case `memory_intensive` - which intentionally creates and deletes large lists.

Now that we know how to narrow down our focus and find specific lines that increase memory consumption, we might want to dig a little deeper and see how much each variable is using. You might have seen `sys.getsizeof` used to measure this before. This function however will give you _questionable_ information for some types of data structures. For integers or bytearrays you will get the real size in bytes, for containers such as list though, you will only get size of the container itself and not its contents:

```python
import sys
print(sys.getsizeof(1))
# 28
print(sys.getsizeof(2**30))
# 32
print(sys.getsizeof(2**60))
# 36

print(sys.getsizeof("a"))
# 50
print(sys.getsizeof("aa"))
# 51
print(sys.getsizeof("aaa"))
# 52

print(sys.getsizeof([]))
# 56
print(sys.getsizeof([1]))
# 64
print(sys.getsizeof([1, 2, 3, 4, 5]))
# 96, yet empty list is 56 and each value inside is 28.
```

We can see that with plain integers, everytime we cross a threshold, 4 bytes are added to the size. Similarly, with plain strings, everytime we add another character one extra byte is added. With lists however, this doesn't hold up - `sys.getsizeof` doesn't _"walk"_ the data structure and only returns size of the parent object, in this case `list`.

Better approach is to use specific tool designed for analyzing memory behaviour. One such is tool is [_Pympler_](https://pympler.readthedocs.io/en/latest/), which can help you get more realistic idea about Python object sizes:

```python
# pip install pympler
from pympler import asizeof
print(asizeof.asizeof([1, 2, 3, 4, 5]))
# 256

print(asizeof.asized([1, 2, 3, 4, 5], detail=1).format())
# [1, 2, 3, 4, 5] size=256 flat=96
#     1 size=32 flat=32
#     2 size=32 flat=32
#     3 size=32 flat=32
#     4 size=32 flat=32
#     5 size=32 flat=32

print(asizeof.asized([1, 2, [3, 4], "string"], detail=1).format())
# [1, 2, [3, 4], 'string'] size=344 flat=88
#     [3, 4] size=136 flat=72
#     'string' size=56 flat=56
#     1 size=32 flat=32
#     2 size=32 flat=32
```

Pympler provides `asizeof` module with function of same name which correctly reports size of the list as well all values it contains. Additionally, this module also has `asized` function, that can give us further size breakdown of individual components of the object.  

Pympler has many more features though, including [tracking class instances](https://pympler.readthedocs.io/en/latest/classtracker.html#classtracker) or [identifying memory leaks](https://pympler.readthedocs.io/en/latest/muppy.html#muppy). In case these are something that might be needed for your application, then I recommend checking out tutorials available in [docs](https://pympler.readthedocs.io/en/latest/tutorials/tutorials.html).
    
## Saving Some RAM

Now that we know how to look for all kinds of potential memory issues, we need to find a way to fix them. Potentially, quickest and easiest solution can be switching to more memory-efficient data structures.

Python `lists` are one of the more memory-hungry options when it comes to storing arrays of values:

```python
from memory_profiler import memory_usage

def allocate(size):
    some_var = [n for n in range(size)]

usage = memory_usage((allocate, (int(1e7),)))  # `1e7` is 10 to the power of 7
peak = max(usage)
print(f"Usage over time: {usage}")
# Usage over time: [38.85546875, 39.05859375, 204.33984375, 357.81640625, 39.71484375]
print(f"Peak usage: {peak}")
# Peak usage: 357.81640625
```

The simple function above (`allocate`) creates a Python `list` of numbers using the specified `size`. To measure how much memory it takes up we can use `memory_profiler` shown earlier which gives us amount of memory used in 0.2 second intervals during function execution. We can see that generating `list` of 10 million numbers requires more than 350MiB of memory. Well, that seems like a lot for a bunch of numbers. Can we do any better?

```python
import array

def allocate(size):
    some_var = array.array('l', range(size))

usage = memory_usage((allocate, (int(1e7),)))
peak = max(usage)
print(f"Usage over time: {usage}")
# Usage over time: [39.71484375, 39.71484375, 55.34765625, 71.14453125, 86.54296875, 101.49609375, 39.73046875]
print(f"Peak usage: {peak}")
# Peak usage: 101.49609375
```

In this example we used Python's `array` module, which can store primitives, such as integers or characters. We can see that in this case memory usage peaked at just over 100MiB. That's a huge difference in comparison to `list`. You can further reduce memory usage by choosing appropriate precision:

```python
import array
help(array)

#  ...
#  |  Arrays represent basic values and behave very much like lists, except
#  |  the type of objects stored in them is constrained. The type is specified
#  |  at object creation time by using a type code, which is a single character.
#  |  The following type codes are defined:
#  |
#  |      Type code   C Type             Minimum size in bytes
#  |      'b'         signed integer     1
#  |      'B'         unsigned integer   1
#  |      'u'         Unicode character  2 (see note)
#  |      'h'         signed integer     2
#  |      'H'         unsigned integer   2
#  |      'i'         signed integer     2
#  |      'I'         unsigned integer   2
#  |      'l'         signed integer     4
#  |      'L'         unsigned integer   4
#  |      'q'         signed integer     8 (see note)
#  |      'Q'         unsigned integer   8 (see note)
#  |      'f'         floating point     4
#  |      'd'         floating point     8
```

One major downside of using `array` as data container is that it doesn't support that many types.

If you plan to perform a lot of mathematical operations on the data, then you're probably better off using NumPy arrays instead:

```python
import numpy as np

def allocate(size):
    some_var = np.arange(size)

usage = memory_usage((allocate, (int(1e7),)))
peak = max(usage)
print(f"Usage over time: {usage}")
# Usage over time: [52.0625, 52.25390625, ..., 97.28515625, 107.28515625, 115.28515625, 123.28515625, 52.0625]
print(f"Peak usage: {peak}")
# Peak usage: 123.28515625

# More type options with NumPy:
data = np.ones(int(1e7), np.complex128)
# Useful helper functions:
print(f"Size in bytes: {data.nbytes:,}, Size of array (value count): {data.size:,}")
# Size in bytes: 160,000,000, Size of array (value count): 10,000,000
```

We can see that NumPy arrays also perform pretty well when it comes to memory usage with peak array size of ~123MiB. That's a bit more than `array` but with NumPy you can take advantage of fast mathematical functions as well as types that are not supported by `array` such as complex numbers. 

The above optimizations help with overall size of arrays of values, but we can make some improvements also to the size of the individual objects defined by Python classes. This can be done using `__slots__` class attribute which is used to explicitly declare class properties. Declaring `__slots__` on a class also has a nice side effect of denying creation of `__dict__` and `__weakref__` attributes:

```python
from pympler import asizeof

class Normal:
    pass

class Smaller:
    __slots__ = ()

print(asizeof.asized(Normal(), detail=1).format())
# <__main__.Normal object at 0x7f3c46c9ce50> size=152 flat=48
#     __dict__ size=104 flat=104
#     __class__ size=0 flat=0

print(asizeof.asized(Smaller(), detail=1).format())
# <__main__.Smaller object at 0x7f3c4266f780> size=32 flat=32
#     __class__ size=0 flat=0
```

Here we can see how much smaller the `Smaller` class instance actually is. The absence of `__dict__` removes whole 104 bytes from each instance which can save huge amount of memory when instantiating millions of values.

The above tips and tricks should be helpful in dealing with numeric values as well as `class` objects. What about strings, though? How you should store those generally depends on what you intend to do with them. If you're going to search through huge number of string values, then - as we've seen - using `list` is very bad idea. `set` might be a bit more appropriate if execution speed is important, but will probably consume even more RAM. Best option might be to use optimized data structure such as _trie_, especially for static data sets which you use for example for querying. As is common with Python, there's already a library for that, as well as for many other tree-like data structures, some of which you will find at <https://github.com/pytries>.

## Not Using RAM At All

Easiest way to save RAM is to not use it in a first place. You obviously can't avoid using RAM completely, but you can avoid loading full data set at once and instead work with the data incrementally where possible. The simplest way to achieve this is by using generators which return a [lazy](https://en.wikipedia.org/wiki/Lazy_evaluation#Python) iterator, which computes elements on demand rather than all at once.

Stronger tool that you can leverage is _memory-mapped_ files, which allows us to load only parts of data from a file. Python's standard library provides `mmap` module for this, which can be used to create memory-mapped files which behave both like files and bytearrays. You can use them both with file operations like `read`, `seek` or `write` as well as string operations:

```python
import mmap

with open("some-data.txt" "r") as file:
    with mmap.mmap(file.fileno(), 0, access=mmap.ACCESS_READ) as m:
        print(f"Read using 'read' method: {m.read(15)}")
        # Read using 'read' method: b'Lorem ipsum dol'
        m.seek(0)  # Rewind to start
        print(f"Read using slice method: {m[:15]}")
        # Read using slice method: b'Lorem ipsum dol'
```

Loading/reading memory-mapped file is very simple. We first open the file for reading as we usually do. We then use file's file descriptor (`file.fileno()`) to create memory-mapped file from it. From there we can access its data both with file operations such as `read` or string operations like _slicing_.

Most of the time, you will be probably interested more reading the file as shown above, but it's also possible to write to the memory-mapped file:

```python
import mmap
import re

with open("some-data.txt", "r+") as file:
    with mmap.mmap(file.fileno(), 0) as m:
        # Words starting with capital letter
        pattern = re.compile(rb'\b[A-Z].*?\b')

        for match in pattern.findall(m):
            print(match)
            # b'Lorem'
            # b'Morbi'
            # b'Nullam'
            # ...

        # Delete first 10 characters
        start = 0
        end = 10
        length = end - start
        size = len(m)
        new_size = size - length
        m.move(start, end, size - end)
        m.flush()
    file.truncate(new_size)
```

First difference in the code that you will notice is the change in access mode to `r+`, which denotes both reading and writing. To show that we can indeed perform both reading and writing operations, we first read from the file and then use RegEx to search for all the words that start with capital letter. After that we demonstrate deletion of data from the file. This is not as straightforward as reading and searching, because we need to adjust size of the file when we delete some of its contents. To do so, we use `move(dest, src, count)` method of `mmap` module which copies `size - end` bytes of the data from index `end` to index `start`, which in this case translates to deletion of first 10 bytes.

If you're doing computations in NumPy, then you might prefer its `memmap` features [(docs)](https://numpy.org/doc/stable/reference/generated/numpy.memmap.html) which is suitable for NumPy arrays stored in binary files.

## Closing Thoughts

Optimizing applications is difficult problem in general. It also heavily depends on the task at hand as well as the type of data itself. In this article we looked at common ways to find memory usage issues and some options for fixing them. There are however many other approaches to reducing memory footprint of an application. This includes trading accuracy for storage space by using probabilistic data structures such as [bloom filters](https://en.wikipedia.org/wiki/Bloom_filter) or [HyperLogLog](https://en.wikipedia.org/wiki/HyperLogLog). Another option is using tree-like data structures like [DAWG](https://github.com/pytries/DAWG) or [Marissa trie](https://github.com/pytries/marisa-trie) which are very efficient at storing string data.