How Django keeps track of your model classes?
Tags: python, django, opensource, programming
Published: 2023-12-05 21:14:51


I have ran the `python manage.py makemigrations` command several hundreds of times. Recently I had a surge of curiousity to find out how does django detect changes and generate migration files for an app.

I needed many answers but one of the first question I had was "How does Django keep track of my model classes?"

In Django, the creation of model classes is fundamental to mapping your application's data model to a database. These model classes inherit from Django's `models.Model` base class, which provides them with essential database interaction capabilities.

Under the hood, Django employs metaclasses to achieve its model detection magic. 

### But what exactly is a metaclass in Python? 

Simply put, a metaclass allows you to customize class creation in Python. In case of Django's model system, the metaclass responsible for this process is called `ModelBase`.

Within the `ModelBase` metaclass, a crucial method known as `__new__` plays a central role. This method invokes the `register_model` function, which, in turn, ensures that a given model registered within the app.

```python
class ModelBase(type):
    """Metaclass for all models."""

    def __new__(cls, name, bases, attrs, **kwargs):
        super_new = super().__new__

        ...
        new_class._prepare()
        # this is where a model class gets registered.
        new_class._meta.apps.register_model(new_class._meta.app_label, new_class)
        return new_class
```

### But when does this __new__ method on ModelBase get triggered? 

To my surprise I found out that it gets triggered at import time. When you import a model class, Django's model detection process initiates. For example:

```python
from myapp.models import MyModel
```

Behind the scenes, Django's machinery begins to work its magic during this import.

### So, where does Django import all these models? 

The answer lies in `django.setup()`, the entry point for various Django commands such as `shell`, `makemigrations`, `migrate`, `runserver`, and more. 


Within `django.setup()`, the `apps.populate(settings.INSTALLED_APPS)` function registers all of your `INSTALLED_APPS`.

```python
# django/django/__init__.py
def setup(set_prefix=True):
    """
    Configure the settings (this happens as a side effect of accessing the
    first setting), configure logging and populate the app registry.
    Set the thread-local urlresolvers script prefix if `set_prefix` is True.
    """
    ...
    apps.populate(settings.INSTALLED_APPS)
```

When I followed the code to the `populate` method, I could see it is defined under the `Apps` class. Within this method there is a method call as `app_config.import_models()`. The `app_config` derives from the config class we define for every new django app under `apps.py`.

```python
# django/django/apps/registry.py
class Apps:
    """
    A registry that stores the configuration of installed applications.

    It also keeps track of models, e.g. to provide reverse relations.
    """

    def __init__(self, installed_apps=()):
        ...
        ...
        if installed_apps is not None:
            self.populate(installed_apps)

    def populate(self, installed_apps=None):
        """
        Load application configurations and models.

        Import each application module and then each model module.
        """
        ...
        ...
            # Phase 2: import models modules.
                for app_config in self.app_configs.values():
                    app_config.import_models()

```

And finally to where the place where "magic" happens. On the `AppConfig` class we have a method `import_models` which reads the `MODELS_MODULE_NAME` which by default is set to `models.py` and imports the whole file as a module. Thereby importing all the model classes in it.

```python
# django/django/apps/config.py
class AppConfig:
    """Class representing a Django application and its configuration."""
    def import_models(self):
        # Dictionary of models for this app, primarily maintained in the
        # 'all_models' attribute of the Apps this AppConfig is attached to.
        self.models = self.apps.all_models[self.label]

        if module_has_submodule(self.module, MODELS_MODULE_NAME):
            models_module_name = "%s.%s" % (self.name, MODELS_MODULE_NAME)
            self.models_module = import_module(models_module_name)

```


In summary, Django's model detection process leverages metaclasses, specifically the `__new__` method, to register model classes during import time. 

When you execute a Django command, such as `makemigrations` or `migrate`, Django uses `django.setup()` to import all the models from the `models.py` files of your installed apps. This is further utilised in various checks and actions performed by django for generating or executing migration files. You can see an example from the `makemigrations` command below.

```python
# django/django/core/management/commands/makemigrations.py
class Command(BaseCommand):
    help = "Creates new migration(s) for apps."
    ...
    @no_translations
    def handle(self, *app_labels, **options):
        ...
        ...
        for alias in sorted(aliases_to_check):
            connection = connections[alias]
            if connection.settings_dict["ENGINE"] != "django.db.backends.dummy" and any(
                # At least one model must be migrated to the database.
                router.allow_migrate(
                    connection.alias, app_label, model_name=model._meta.object_name
                )
                for app_label in consistency_check_labels
                for model in apps.get_app_config(app_label).get_models()
            ):
            ...
```

Cover image by [John Fowler](https://unsplash.com/@wildhoney?utm_content=creditCopyText&utm_medium=referral&utm_source=unsplash)