Soft Deletes in Django
Tags: python, django
Published: 2021-07-02 19:13:16

## What is Soft Deletion?

Sometimes, you want to delete something from your database without _actually_ deleting it. You want to preserve it, in case you ever decide to restore it in the future, or to use it for some other purpose, such as analytics … but it should still appear as if it were actually deleted, not show up in unexpected places in your application. This is called the 'soft delete' pattern.

Instead of actually deleting a record from your database, which is irreversible, you simply 'mark' it as deleted, usually in another column in your table. Now the challenge remains, how to prevent soft-deleted records from 'leaking' into your application.

In this article, we will learn how to implement soft deletion in Django by taking the example of a simple note-taking app backend.

## Soft Delete Model

We will start by creating a base `SoftDeleteModel` that can be [inherited](https://pynative.com/python-inheritance/) by the rest of our models.

```python
class SoftDeleteModel(models.Model):

    is_deleted = models.BooleanField(default=False)

    def soft_delete(self):
        self.is_deleted = True
        self.save()

    def restore(self):
        self.is_deleted = False
        self.save()

    class Meta:
        abstract = True
```

Note that we have marked this model as `abstract = True`. This means that Django will not create a database table for it.

Now, we can create our models as subclasses of `SoftDeleteModel` to grant them the ability to be soft-deleted. Let's take the example of a `Note` model.

```python
class Note(SoftDeleteModel):

    user = models.ForeignKey(User, on_delete=models.CASCADE, related_name="notes")
    title = models.CharField(max_length=32)
    content = models.CharField(max_length=255)
```

We can query our database filtering by `is_deleted` in order to exclude records that have been soft-deleted.

```python
Note.objects.filter(is_deleted=False)
```

## Trying It Out

Let's try playing around a bit with the code we've written so far. First, open up the Django shell by typing `python manage.py shell` in your terminal.

Import the models required:

```python
from django.contrib.auth.models import User
from tutorialapp.models import Note
```

Since each note is foreign-keyed to a user, our first step is to [create a `User` object](https://docs.djangoproject.com/en/3.2/topics/auth/default/#creating-users):

```python
john = User.objects.create_user('john', 'lennon@thebeatles.com', 'johnpassword')
```

Now we can create a couple of notes:

```python
my_note = Note.objects.create(user=john, title="Strawberry Fields", content="Strawberry Fields Forever")
another_note = Note.objects.create(user=john, title="Here Comes The Sun", content="It's All Right")
```

You are now ready to soft delete and restore notes:

```python
my_note.soft_delete()
my_note.restore()
```

You can query for all notes, whether they have been soft deleted or not:

```python
Note.objects.all()
```

You can also filter only for notes that have not been soft deleted:

```python
Note.objects.filter(is_deleted=False)
```

## Soft Delete Manager

While our code is functionally correct, the disadvantage is that we will have to remember to filter by `is_deleted=False` each time we write a query.

We can improve upon this behaviour by creating a custom [model manager](https://docs.djangoproject.com/en/3.2/topics/db/managers/) to apply the filter automatically, behind the scenes. If you've used Django in the past, you might be familiar with statements that look like this: `MyModel.objects.all()`. The `.objects` part in the statement is the manager. Managers act as the 'bridge' between your Django code and the database. They control the database operations performed on the tables that they 'manage'.

Our new custom manager can be defined as:

```python
class SoftDeleteManager(models.Manager):

    def get_queryset(self):
        return super().get_queryset().filter(is_deleted=False)
```

We will need to add the new manager to our `SoftDeleteModel` base class:

```python
class SoftDeleteModel(models.Model):

    is_deleted = models.BooleanField(default=False)
    objects = models.Manager()
    undeleted_objects = SoftDeleteManager()

    def soft_delete(self):
        self.is_deleted = True
        self.save()

    def restore(self):
        self.is_deleted = False
        self.save()

    class Meta:
        abstract = True
```

Note that, since we have added a custom manager to our class, we are required to explicitly add the default `objects` manager as well.

Then, we can simply rewrite our query as, 

```python
Note.undeleted_objects.all()
```

to get a `QuerySet` of undeleted notes.

We can still use

```python
Note.objects.all()
```

to get the full list of notes, including those that have been soft-deleted.

## Handling Foreign Key Relationships

Now, what if you have multiple users, and you want to fetch all the notes belonging to a specific user? The naive approach is to simply write a query filtering against the user:

```python
Note.objects.filter(user=john, is_deleted=False)
```

However, a more elegant and readable solution is to make use of the [reverse relationships](https://stackoverflow.com/questions/17328910/django-what-is-reverse-relationship)  Django provides for this purpose.

```python
john.notes.all()
```

Try soft deleting some of your notes and running this query. Do you notice something unusual about the results?

We find that the resultant `QuerySet` contains records that we had soft deleted. This is because Django is using the default `objects` manager to perform the reverse lookup, which, as you may recall, does not filter out soft deleted records.

How can we force Django to use our custom `SoftDeleteManager` to perform reverse lookups? We can simply replace the default `objects` manager in our `SoftDeleteModel`:

```python
class SoftDeleteModel(models.Model):

    is_deleted = models.BooleanField(default=False)
    objects = SoftDeleteManager()
    all_objects = models.Manager()

    def soft_delete(self):
        self.is_deleted = True
        self.save()

    def restore(self):
        self.is_deleted = False
        self.save()

    class Meta:
        abstract = True
```

Now, the `objects` manager will automatically filter out soft-deleted objects when querying our database, ensuring they never leak into our application under any circumstances! If we want to, we can still include soft deleted objects in our queries by making use of the `all_objects` manager.

```python
Note.all_objects.all()
```

## Storing More Information

We've already got a pretty solid soft deletion framework in our Django app, but we can make one final improvement. Knowing whether a record is soft deleted or not is useful, but another piece of information that would be nice to know is _when_ the record was soft deleted. For this, we can add a new a attribute `deleted_at` to our `SoftDeleteModel`:

```python
deleted_at = models.DateTimeField(null=True, default=None)
```

We can also update our `soft_delete` and `restore` methods as follows:

```python
def soft_delete(self):
    self.deleted_at = timezone.now()
    self.save()

def restore(self):
    self.deleted_at = None
    self.save()
```

For undeleted records, the value of `deleted_at` will be null, while for soft deleted records, it will contain the date and time at which it was deleted.

The addition of the new `deleted_at` attribute makes our previously created `is_deleted` attribute redundant, because we can simply perform a null-check on `deleted_at` to find out whether the record is soft deleted or not.

Our rewritten `SoftDeleteModel` now looks like this:

```python
class SoftDeleteModel(models.Model):

    deleted_at = models.DateTimeField(null=True, default=None)
    objects = SoftDeleteManager()
    all_objects = models.Manager()

    def soft_delete(self):
        self.deleted_at = timezone.now()
        self.save()

    def restore(self):
        self.deleted_at = None
        self.save()

    class Meta:
        abstract = True
```

And our rewritten `SoftDeleteManager` looks like this:

```python
class SoftDeleteManager(models.Manager):

    def get_queryset(self):
        return super().get_queryset().filter(deleted_at__isnull=True)
```

In addition to our previous capabilities, we can now also see the exact date and time at which our record was soft deleted:

```python
my_note.deleted_at
```

## In Conclusion

Soft deletion is a powerful software pattern that comes with several advantages, including better data preservation and restoration, history tracking, and faster recovery from failures.

At the same time, it should be used with care. Sensitive and personal data including payment-related information should always be hard-deleted. Users should always have the option to have their data permanently deleted, if they wish. Several jurisdictions around the world have information privacy and data protection laws that include the 'right to be forgotten', such as the [European Union's GDPR](https://gdpr-info.eu/art-17-gdpr/). It might also make sense to periodically delete or archive data that is very old, to avoid eating up excess database storage space.

If you would like to view the complete source code for the example used in this tutorial, it is available [on GitHub](https://github.com/BikramjeetSingh/Django-Soft-Delete-Tutorial-Example).

Image by <a href="https://pixabay.com/users/makalu-680451/?utm_source=link-attribution&amp;utm_medium=referral&amp;utm_campaign=image&amp;utm_content=4661907">Makalu</a> from <a href="https://pixabay.com/?utm_source=link-attribution&amp;utm_medium=referral&amp;utm_campaign=image&amp;utm_content=4661907">Pixabay</a>

## References and Further Reading

[How to use managers in Django](https://www.bedjango.com/blog/Django-model-managers/)
[What Are Soft Deletes, and How Are They Implemented?](https://www.brentozar.com/archive/2020/02/what-are-soft-deletes-and-how-are-they-implemented/)
[Soft Deletion in Django](https://adriennedomingus.medium.com/soft-deletion-in-django-e4882581c340)