Customize your own dictionary - Python Tips
Tags: python
Published: 2020-02-20 19:03:02

## TL;DR

Subclass `collections.UserDict`, not `dict`.

## Longer Description

I need a dictionary-like variable, and pass it to other function, when the function returned, I need to know if this dictionary has been changed (key or value), if it is, I'll do a database saving (if you are seeing this as the process of saving sessions in the server side, it exactly is). Since this is a frequent operation, I want to make this as fast as possible.

The first idea came to my mind is like this:

```python
session_dict = {}
old_dict = {**session_dict}
# now call function
func(session_dict)
if session_dict != old_dict:
    # change happend, do database saving
```

But this method require a dictionary copy, which cost more space, and compare two dictionary objects, I am sure there are some recursive comparing happened inside, which may cost more CPU time.


The second idea is using `pickle` to compare the pickled string:

```python
import pickle
session_dict = {}
finger_print = pickle.dumps(session_dict)
func(session_dict)
if pickle.dumps(session_dict) == finger_print:
    # change happend, do database saving
```

This one uses `pickle.dumps` to convert dictionary to string, then after the function call, it compares the string. Based on how long the string is, this may cost more CPU time.


The third idea is, to subclass the `dict`, and when `__setitem__` and `__delitem__` are being called, mark the change flag (you may also want to see my [previous post here on subclassing dict](https://dev.to/0xbf/use-dot-syntax-to-access-dictionary-key-python-tips-10ec) in Python to support dot syntax accessing):


```python
class ChangeDetectableDict(dict):
    def __init__(self, val=None):
        if val is None:
            val = {}
        super().__init__(val)
        self.changed = False

    def __setitem__(self, item, value):
        super().__setitem__(item, value)
        self.changed = True

    def __delitem__(self, item):
        super().__delitem__(item)
        self.changed = True
```

This works fine if we have code to set key:

```python
session_dict = ChangeDetectableDict()
session_dict["name"] = "bo"
print(session_dict.changed) # True
```

Or delete a key:
```python
session_dict = ChangeDetectableDict({"name": "bo"})
del session_dict["name"]
print(session_dict.changed) # True
```

This will make the change-detection to time `O(1)`, which is great. But You may see the problem that if I set the key's value to be the same value, the `changed` will still be marked as True:

```python
session_dict = ChangeDetectableDict({"name": "bo"})
session_dict["name"] = "bo"]
print(session_dict.changed) # True
```

In my case I am fine with it, as long as if something changed, the `changed` will be marked as True, I don't mind having some false-positive case. But the real problem is, if I don't use `del` to delete a key, use `pop` instead, the `__delitem__` method won't be called, thus `changed` is still False:

```python
session_dict = ChangeDetectableDict({"name": "bo"})
session_dict.pop("name", None)
print(session_dict.changed) # False
```

This is unacceptable for my use case, and the reason why this happens is because Python's built-in `dict` has some inline optimizations which leads `pop` not calling `__delitem__`.

So the fourth method I found is not subclassing `dict`, instead, subclass `collections.UserDict`:


```python
from collections import UserDict

class ChangeDetectableDict(UserDict):
    def __init__(self, val=None):
        if val is None:
            val = {}
        super().__init__(val)
        self.changed = False

    def __setitem__(self, item, value):
        super().__setitem__(item, value)
        self.changed = True

    def __delitem__(self, item):
        super().__delitem__(item)
        self.changed = True
```

Now no matter we use `del` or `pop`, the `__delitem__` method will always get called, thus will mark `changed` to be True.

Back to the beginning, now I can write code like this:

```python
session_dict = ChangeDetectableDict()
func(session_dict)
if session_dict.changed:
    # change happend, do database saving
```