Some insights into the low-level workflow of the Django framework
Published: 2021-12-10 03:42:55

## Running flow
You can use multiple methods to run a Django app, one of which is `runserver` which uses the web server of Django itself, and it is mostly used in the development and testing phases. Another is using protocols like `fastcgi` and `uWSGI`. 

### 1. runserver
**Usage**: `python3 manage.py runserver`, this command is run under port 8080 by default. If you investigate into the source codes of `manage.py`, you will find that this line is actually executed through the internal `runserver` command in `execute_from_command_line`, and this command mostly does two things:

- Parse the parameters, and get the `wsgi handler` by `django.core.servers.basehttp.get_internal_wsgi_application`;
- Generate a `WSGIServer` object according to your IP address and port, and then accept the user's requests. The source codes of `get_internal_wsgi_application` is as followed:

```python
def get_internal_wsgi_application():
    from django.conf import settings
    app_path = getattr(settings, 'WSGI_APPLICATION')
    if app_path is None:
        return get_wsgi_application()
    return import_by_path(
        app_path,
        error_prefix="WSGI application '%s' could not be loaded; " % app_path
    )
```
By the above we know that, `Django` will first obtain the `handler` via `WSGI_APPLICATION` in `settings`. While building the `project`, `Django` will create a `wsgi.py` by default.

### 2. uWSGI
Another approach is the most prevalent one while running Django in production, it uses Ngnix combined with uWSGI. To dive more deeply into this method, we would need to learn more about two protocols: `WSGI` and `uWSGI`.

- **WSGI** is short for **Web Server Gateway Interface**. It is a universally applicable interface that bridges python-based web servers and web apps/framworks, and is designed based on the current CGI standard. `WSGI` is actually a gateway, which helps conversion between multiple protocols.
- **uWSGI** is a web server which supports protocols like WSGI protocol, UWSGI protocol and HTTP Protocol, etc. It is known for rapid transmission, low memory usage and its excellence in multi-app management.

## HTTP Request Process
Django's RESTful workflow is similar to other web frameworks: Receiving requests and returning responses. Let's see the details in the following:

When Django is creating the project via `django-admin`, it will automatically create some default files including `settings.py` and `manage.py`, and execute the following command before building the WSGIServer:
```python
from django.conf import settings
```
During the execution of the above command, Django will read the `DjANGO_SETTINGS_MODULE` config info in `os.environ` and load the config files before creating the `setting` object. Therefore, inside the `manage.py`, it will first add the path of project settings to the os path before obtaining the WSGIServer.

### 1. Creating the WSGIServer
No matter you are running your Django project with runserver or uWSGI, Django will always call `run()` in `django.core.servers.basehttp` during the start-up, creating an instance of `WSGIServer`, and later call its `serve_forever()` to start the HTTP services. Below is the source codes of `run()`:

```python
 def run(addr, port, wsgi_handler, ipv6=False, threading=False):
     server_address = (addr, port)
     if threading:
        httpd_cls = type(str('WSGIServer'), (socketserver.ThreadingMixIn, WSGIServer), {})
    else:
        httpd_cls = WSGIServer
        httpd = httpd_cls(server_address, WSGIRequestHandler, ipv6=ipv6)
        # Sets the callable application as the WSGI application that will receive requests
        httpd.set_app(wsgi_handler)
        httpd.serve_forever()
```
From the above, we see that: While creating the `WSGIServer` instance, `WSGIRequestHandler` is used to designate the HTTP request handler. When the user's request arrives at the server, `WSGIServer`will create a `WSGIRequestHandler` instance, using its `handler` function to process the http request, and actually in the end it calls `run()` in `wsgiref.handlers.BaseHandler`. With `set_app()`, `WSGIServer` set up a callable object as an application. The above mentioned `handler` function will call this application to process the request and return the response. Besides, `WSGIServer` inherits from `wsgiref.simple_server.WSGIServer`, but `WSGIRequestHandler` inherits from `wsgiref.simple_server.WSGIRequestHandler`, `wsgiref` is the reference implementation of `WSGI` from the Python standard library.

### 2. Process the request
In Django, the aforementioned `application` is in general an object of `wsgi.WSGIHandler` in `django.core.handlers`. `WSGIHandler` inherits from `base.BaseHandler` and it creates a `WSGIRequest` instance which further inherits from `http.HttpRequest`. These are the essenial logics on how Django processes the request.

### 3. Return the response
There is a `get_response` method in `BaseHandler` and it initially loads the `ROOT_URLCONF` in the Django project and then find out corresponding `view` methods or classes according to the url rules. The logics in the `view` functions/classes will create and return specific responses through the request instance.

After Django returns the results, the `run()` in `wsgiref.handlers.BaseHandler` will call `finish_response` to end the request, and return the contents to the user.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/5m528j1i3u1dexslgs4h.png)

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/m1xycdlvuq6rlhr5xop4.png)

Seeing from the above image. 1-9 represent 9 steps, and the first steo is to send the request from a page, which reaches the `Request` middleware in the 2nd step. In the 3rd step, `URLConf` is querying for the corresponding `View` in `url.py`, and later the `View`s are called to answer the request with their functions invoked. During the 6th step, the `view` will access the model information and any interactions between model and databases are done by a built-in manager. Here `View`s can have contexts if needed, and if there is any context, it will be passed to `Template` for further rendering.

Processes a-e are done mostly in `Template`s. First filters and tags are used to render the output, which is later returned to `View`s. At the same time, `Response` will receive instances of `HTTPResponse`, and any response could add up new information or simply return a brand new body. In the end, user could receive the response from the browser.

Two most important components in these procedures are: Middleware and URLConf, and below we will briefly walk through them.

# Middleware
Middleware does not exclusively belong to Django, but also exist in other frameworks. In Django, middleware could participate in 4 phases in the RESTful process: request, view, response, exception. Accordingly, every class of middleware has the following 4 methods in the pattern of: `process_x`, where x are `request`, `view`, `response` and `exception` You can define one or more of these methods depending on which stage you want the middleware to put effect on, and every method returns a response object.

# URLConf
URLconf is the mapping of the websites supported by Django. Basically, it is a mapping table between the URL pattern and the `view` functions.

# References
[1] https://hitesh.in/2009/django-flow/
[2] https://stackoverflow.com/questions/1836627/diagrams-explanations-of-django-request-processing
[3] https://docs.djangoproject.com/en/3.2/topics/http/urls/
[4] https://docs.djangoproject.com/en/3.2/topics/http/middleware/
[5] https://docs.djangoproject.com/en/3.2/topics/templates/
[6] https://medium.com/@jvdali966/understanding-djangos-work-flow-1ee521422092