Testing against unmanaged models in Django
Tags: python, testing, database, django
Published: 2017-08-15 19:58:12

---
title: Testing against unmanaged models in Django
published: true
description: 
tags: python, testing, databases, django
---

# The problem
My Django application is supposed to read the tables of an already existing database. So the models of the database are generated using `inspectdb` command. They look like this:
```python
class User(models.Model):
    first_name = models.CharField(max_length=255)
    second_name = models.CharField(max_length=255)
    # other properties

    class Meta:
        managed = False
        db_table = 'user'
```
The `managed = False` property tells Django not to create the table as part of `migrate` command and not to delete it after `flush`. The catch here is that Django [won't create the table](https://docs.djangoproject.com/en/1.11/ref/models/options/#managed) during testing either. So when I wanted to test my code against non-production database, I ran into the following issues:
1. The need to create all of the tables for unmanaged models by hand just to run the tests. Otherwise I was getting `django.db.utils.OperationalError: no such table: user`.
2. It was not obvious to me how to generate fixtures for the unmanaged models. If the model is managed, you can run the migrations on the test database, populate the tables with the help of `shell` command and save it with `dumpdata`. But the tables of unmanaged models are not created during migrations, there's nothing to populate. Again, I had to create everything by hand.

My goal was to find a way to automate the process, making both testing and collaboration easier. Relevant bits and pieces of are scattered across the Internet. This is an attempt to collect them in one place.

# The simple solution
Before the test runner launches the tests, it runs the migrations creating the tables for managed models. The migrations themselves are Python code that creates the models and sets the `managed` property. It is possible to modify a migration so that it sets `managed` to `True` while we are testing and `False` otherwise for unmanaged models.
To do that, we will create `IS_TESTING` variable in `settings.py` and set it to `True` when we are testing. We will also modify the migration code:
```python
from django.conf import settings

# ...
operations = [
    migrations.CreateModel(
        name='User',
        fields=[
            # ...
        ],
        options={
            'db_table': 'user',
            # 'managed': False,
            'managed': settings.IS_TESTING,
        },
    ),
]
```
Now the table will be created whenever the migration is run with `IS_TESTING = True`.
The idea belongs to Kyle Valade, who described it [in his blog](https://kdazzle.svbtle.com/testing-with-unmanaged-databases-in-django).
To generate fixtures, the method with `shell` command described earlier will work.
The downside here is that you have to remember to modify the migration of every unmanaged model.

# Creating a custom test runner
A more complex solution is to create a custom test runner that will convert all unmanaged models to managed before running a test and revert the effect afterwards.
We'll put the runner in `appname/utils.py`:
```python
from django.test.runner import DiscoverRunner


class UnManagedModelTestRunner(DiscoverRunner):

    def setup_test_environment(self, *args, **kwargs):
        from django.apps import apps
        get_models = apps.get_models
        self.unmanaged_models = [m for m in get_models() if not m._meta.managed]
        for m in self.unmanaged_models:
            m._meta.managed = True
        super(UnManagedModelTestRunner, self).setup_test_environment(*args, **kwargs)

    def teardown_test_environment(self, *args, **kwargs):
        super(UnManagedModelTestRunner, self).teardown_test_environment(*args, **kwargs)
        for m in self.unmanaged_models:
            m._meta.managed = False
```
Now we will tell Django to use this runner by adding `TEST_RUNNER = 'app_name.utils.UnManagedModelTestRunner'` to our settings.
We are not yet ready because the `User` model [has custom table name](https://stackoverflow.com/questions/18085245/running-tests-with-unmanaged-tables-in-django) `user`. This is why we need to create the test tables without running migrations. There's [a small app](https://pypi.python.org/pypi/django-test-without-migrations/) for that. It's installed by running `pip install` and adding it to `INSTALLED_APPS`. Our tests will work if we run them with `-n` switch: `python manage.py test -n`. As a consequence, we will lose the ability to see if any of our migrations are broken (which is probably fine if all of them are generated by Django).

The idea of creating a custom test runner belongs to Tobias McNulty, who posted it in [Caktus Group blog](http://blog.birdhouse.org/2015/03/25/django-unit-tests-against-unmanaged-databases). The code from his post had to be [updated](https://stackoverflow.com/a/36234846/3694363).

When it comes to maintaining the code, there are complications. First, if we wanted to use some other test runner, we'd have to inherit from it:
```python
from django.test.runner import DiscoverRunner
from django_nose import NoseTestSuiteRunner


class UnManagedModelTestRunner(NoseTestSuiteRunner, DiscoverRunner):
    # ...
```
Second, even if the `django-test-without-migrations` app is simple, it [doesn't mean](https://github.com/henriquebastos/django-test-without-migrations/issues/13) it can't be broken by a new version of Django, so we need to be prepared to troubleshoot it.
Third, we have to generate fixtures in an unusual way. The tables of our unmanaged models are only available in `setUp()` method, so to generate the fixtures we would have to add and dump data [in the source code](https://stackoverflow.com/a/16076115/3694363) of the test:
```python
import sys

from django.core.management import call_command

# ...

    def setUp(self):
        models.User.objects.create(
            # ...
        )
        sysout = sys.stdout
        sys.stdout = open('fixtures/users.json', 'w')
        call_command('dumpdata', app_label='app_name')
        sys.stdout = sysout
```
After we ran the code, we can remove it and load the fixture in our tests [the usual way](https://stackoverflow.com/questions/2470634/loading-fixtures-in-django-unit-tests).

# The common weakness
When the tests run, they will treat the unmanaged models as managed. They won't fail if someone accidentally adds a field to an unmanaged model. The only way I know to get around this is to create the tables by hand.

# Final word
This is all I've got on the topic. If you happen to know another solution, I'd be happy to learn about it.