Python: Django Development on Kubernetes with DevSpace
Tags: kubernetes, vcluster, devspace, loft
Published: 2022-06-16 16:05:10

By Levent Ogut

The possibilities that Kubernetes has brought to the container orchestration space are vast. Kubernetes simplifies the deployment and operation of such systems. However, from a developer's point of view, it may not be as simple as their previous workflows; there are some nuances of the system to be learned. DevSpace is a tool that closes that gap and helps to provide a better developer experience. This article will use DevSpace to develop and deploy an application based on the popular and feature-rich Python framework [Django](https://www.djangoproject.com/).

## Requirements and Setting Up Development Environment

You will need three tools to continue, namely Kubectl, Helm, and DevSpace. And access to a local or remote Kubernetes cluster, any flavors of Kubernetes cluster would work.

You can find the installation instructions of the three tools in the following links:

* [Kubectl](https://kubernetes.io/docs/tasks/tools/).
* [Helm](https://helm.sh/docs/intro/install/).
* [DevSpace](https://devspace.sh/cli/docs/getting-started/installation).

## Developing with DevSpace

The first step for setting up DevSpace is to run the `devspace init` command to initialize the project. This command will ask you some questions so that it can generate the skeleton `devspace.yaml` configuration according to your project's needs. If you do not have a Dockerfile file, it will be created for you as well. It is recommended to review the `devspace.yaml` configuration to make sure the settings work for your project. There are many configuration options that you can set within the `devspace.yaml`. Please review the [documentation](https://devspace.sh/cli/docs/introduction) for further info.

```
$ devspace init

     ____              ____                       
    |  _ \  _____   __/ ___| _ __   __ _  ___ ___ 
    | | | |/ _ \ \ / /\___ \| '_ \ / _` |/ __/ _ \
    | |_| |  __/\ V /  ___) | |_) | (_| | (_|  __/
    |____/ \___| \_/  |____/| .__/ \__,_|\___\___|
                            |_|


? How do you want to deploy this project? helm: Use Component Helm Chart [QUICKSTART] (https://devspace.sh/component-chart/docs)

? How should DevSpace build the container image for this project? Create a new Dockerfile for this project
                                                  
? Select the programming language of this project python


[info]   DevSpace does *not* require pushing your images to a registry but let's assume you wanted to do that (optional)

? Which registry would you want to use to push images to? (optional, choose any) Use hub.docker.com => you are logged in as leventogut
[done] √ Great! You are authenticated with hub.docker.com              

[info]   Configuration saved in devspace.yaml - you can make adjustments as needed
[done] √ Project successfully initialized
         
You can now run:
- `devspace use namespace` to pick which Kubernetes namespace to work in
- `devspace dev` to start developing your project in Kubernetes
- `devspace deploy -p production` to deploy your project to Kubernetes
- `devspace -h` to get a list of available commands
$
```

Set the correct context to deploy the application.

```Bash
$ kubectl config use-context docker-desktop 
Switched to context "docker-desktop".
```

Additionally, it is a good practice to select a namespace to develop in.

```Bash
$ devspace use namespace myproject-namespace
```

Let's set the variables we need, run the following command. It will ask values of missing variables:

```Bash
$ devspace list vars
```

```
? Please enter a value for DB_PASSWORD ***********

 Variable      Value           
 DB_DATABASE   django          
 DB_PASSWORD   db.password     
 DB_PORT       5432            
 DB_USERNAME   postgres        
 DB_VERSION    11.11.0         
 IMAGE         leventogut/app 
```

Now we are ready to run `devspace dev` to start our development environment. This command will deploy all components that are defined in the `devspace.yaml` file. Also, by default, it will open an interactive console session to the application container. DevSpace will create a port forwarding so that you can reach the application using the local port. Lastly, it will set up file sync between your project directory and the application container.

```
$ devspace dev

[warn]   Deploying into the 'default' namespace is usually not a good idea as this namespace cannot be deleted

[info]   Using namespace 'default'               
[info]   Using kube context 'docker-desktop'
[info]   Execute 'helm upgrade myproject /Users/logut/.devspace/component-chart/component-chart-0.8.0.tgz --namespace default --values /var/folders/3h/tq577p717mdccgjpcgtcvqv80000gn/T/478617250 --install --kube-context docker-desktop'
[info]   Execute 'helm list --namespace default --output json --kube-context docker-desktop'                              
[done] √ Deployed helm chart (Release revision: 1)                                                                        
[done] √ Successfully deployed myproject with helm                                                                        
[done] √ Scaled down Deployment default/myproject                      
[done] √ Successfully replaced pod default/myproject-86cf556785-txfj5              
[done] √ Port forwarding started on 8080:8080 (default/myproject-86cf556785-txfj5-devspace)
                                             
#########################################################
[info]   DevSpace UI available at: http://localhost:8090
#########################################################

[0:sync] Waiting for pods...
[0:sync] Starting sync...
[0:sync] Sync started on /Users/logut/dev/loft/devspace-django/myproject <-> . (Pod: default/myproject-86cf556785-txfj5-devspace)
[0:sync] Waiting for initial sync to complete
[info]   Opening 'http://localhost:8080' as soon as application will be started (timeout: 4m0s)
[info]   Opening shell to pod:container myproject-86cf556785-txfj5-devspace:container-0
Installing Python Dependencies
Collecting asgiref==3.4.1
  Downloading asgiref-3.4.1-py3-none-any.whl (25 kB)
Collecting Django==3.2.7
  Downloading Django-3.2.7-py3-none-any.whl (7.9 MB)
     |████████████████████████████████| 7.9 MB 7.8 MB/s 
Collecting pytz==2021.1
  Downloading pytz-2021.1-py2.py3-none-any.whl (510 kB)
     |████████████████████████████████| 510 kB 9.7 MB/s 
Collecting sqlparse==0.4.1
  Downloading sqlparse-0.4.1-py3-none-any.whl (42 kB)
     |████████████████████████████████| 42 kB 62 kB/s 
Collecting psycopg2-binary==2.9.1
  Using cached psycopg2_binary-2.9.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.4 MB)
Installing collected packages: sqlparse, pytz, asgiref, Django, psycopg2-binary
Successfully installed Django-3.2.7 asgiref-3.4.1 pytz-2021.1 sqlparse-0.4.1 psycopg2-binary-2.9.1
WARNING: Running pip as root will break packages and permissions. You should install packages reliably by using venv: https://pip.pypa.io/warnings/venv
WARNING: You are using pip version 21.1.2; however, version 21.2.4 is available.
You should consider upgrading via the '/usr/local/bin/python -m pip install --upgrade pip' command.

   ____              ____
  |  _ \  _____   __/ ___| _ __   __ _  ___ ___
  | | | |/ _ \ \ / /\___ \| '_ \ / _` |/ __/ _ \
  | |_| |  __/\ V /  ___) | |_) | (_| | (_|  __/
  |____/ \___| \_/  |____/| .__/ \__,_|\___\___|
                          |_|

Welcome to your development container!

This is how you can work with it:
- Run `python main.py` to build the application
- Files will be synchronized between your local machine and this container
- Some ports will be forwarded, so you can access this container on your local machine via localhost:


 Image   ImageSelector    LabelSelector   Ports (Local:Remote)  
         leventogut/app                   8080:8080     

root@myproject-86cf556785-txfj5-devspace:/app#
```

In the output above, DevSpace executed several actions:

* Deploy our project into the Kubernetes cluster via the Helm [component chart](https://devspace.sh/component-chart/docs/introduction).
* Start file [sync](https://devspace.sh/cli/docs/configuration/development/file-synchronization)
* Install Python dependencies as per the requirement.txt
* Replaced the actual container image with a development version.

Lastly, we are dropped into a shell of our application container. From here, we can run commands within the container.

An important point here is that by default, DevSpace uses replacedPods feature for development. The ability to replace the image allows the developer to use a different image with the same code base. A developer can create an image with debugging tools installed to be used in the development process without affecting production deployment. If this is not desired, you can remove the `replacedPods` stanza in the `devspace.yaml`.

## Adding a Database (PostgreSQL)

To add a PostgreSQL deployment, please add the following snippet under the deployments section.

```
- name: postgresql
  helm:
    componentChart: false
    chart:
      name: postgresql
      version: 10.4.3
      repo: https://charts.bitnami.com/bitnami
    values:
      image:
        tag: $!{DB_VERSION}
      postgresqlDatabase: $!{DB_DATABASE}
      postgresqlUsername: $!{DB_USERNAME}
      postgresqlPassword: $!{DB_PASSWORD}
      service:
        port: ${DB_PORT}
```

And modify the `vars` section, as follows:

```
vars:
- name: IMAGE
  value: leventogut/app
- name: DB_VERSION
  value: 11.11.0
- name: DB_DATABASE
  value: django
- name: DB_USERNAME
  value: postgres
- name: DB_PASSWORD
  password: true
- name: DB_PORT
  value: 5432
- name: DB_HOST
```

## Adding Environment Values to a Deployment

To add the environment value to our pod, we need to define them in the deployments section; you only need the env section to be added to the existing `devspace.yaml`.

```
- name: myproject
  # This deployment uses `helm` but you can also define `kubectl` deployments or kustomizations
  helm:
    # We are deploying the so-called Component Chart: https://devspace.sh/component-chart/docs
    componentChart: true
    # Under `values` we can define the values for this Helm chart used during `helm install/upgrade`
    # You may also use `valuesFiles` to load values from files, e.g. valuesFiles: ["values.yaml"]
    values:
      containers:
        - image: ${IMAGE} # Use the value of our `${IMAGE}` variable here (see vars above)
          env:
            - name: DB_DATABASE
              value: $!{DB_DATABASE}
            - name: DB_HOST
              value: $!{DB_HOST}
            - name: DB_PORT
              value: $!{DB_PORT}
            - name: DB_USERNAME
              value: $!{DB_USERNAME}
            - name: DB_PASSWORD
              value: $!{DB_PASSWORD}
      service:
        ports:
        - port: 8080
```

Now re-run `devspace dev` for changes to take effect.

## Modifying Django Database Settings

The database configuration can be found under myproject>settings.py. As we do not want to hardcore our sensitive data, we will use environment values.

In settings.py, first import os module so that we can use os.environ to get the values from the environment:

```
import os
```

Then go to the `DATABASES` section to add the database environment values as follows.

```
DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.postgresql',
        'NAME': os.environ['DB_DATABASE'],
        'USER': os.environ['DB_USERNAME'],
        'PASSWORD': os.environ['DB_PASSWORD'],
        'HOST': os.environ['DB_HOST'],
        'PORT': os.environ['DB_PORT'],

    }
}
```

## Running Database Migrations

As we defined our PostgreSQL database, we can now run the migrations from our shell.

```
root@myproject-97954557-bzvp9-devspace:/app# ./manage.py migrate
```

```
Operations to perform:
  Apply all migrations: admin, auth, contenttypes, sessions
Running migrations:
  Applying contenttypes.0001_initial... OK
  Applying auth.0001_initial... OK
  Applying admin.0001_initial... OK
  Applying admin.0002_logentry_remove_auto_add... OK
  Applying admin.0003_logentry_add_action_flag_choices... OK
  Applying contenttypes.0002_remove_content_type_name... OK
  Applying auth.0002_alter_permission_name_max_length... OK
  Applying auth.0003_alter_user_email_max_length... OK
  Applying auth.0004_alter_user_username_opts... OK
  Applying auth.0005_alter_user_last_login_null... OK
  Applying auth.0006_require_contenttypes_0002... OK
  Applying auth.0007_alter_validators_add_error_messages... OK
  Applying auth.0008_alter_user_username_max_length... OK
  Applying auth.0009_alter_user_last_name_max_length... OK
  Applying auth.0010_alter_group_name_max_length... OK
  Applying auth.0011_update_proxy_permissions... OK
  Applying auth.0012_alter_user_first_name_max_length... OK
  Applying sessions.0001_initial... OK
root@myproject-97954557-bzvp9-devspace:/app#
```

All migrations are successfully executed. These are Django's built-in admin and auth modules migrations, but the same process also applies to our migrations.

Lastly, we can run `./manage.py runserver 0.0.0.0:8080` to start the development server in our container.

```
root@myproject-97954557-bzvp9-devspace:/app# ./manage.py runserver 0.0.0.0:8080
```

```
Watching for file changes with StatReloader
Performing system checks...

System check identified no issues (0 silenced).
September 10, 2021 - 18:51:00
Django version 3.2.7, using settings 'myproject.settings'
Starting development server at http://0.0.0.0:8080/
Quit the server with CONTROL-C.
```

Let's visit the app via the URL `http://localhost:8080` and see the requests in the logs.

```
[11/Sep/2021 18:51:08] "GET / HTTP/1.1" 200 10697
[11/Sep/2021 18:51:08] "GET /static/admin/css/fonts.css HTTP/1.1" 200 423
[11/Sep/2021 18:51:08] "GET /static/admin/fonts/Roboto-Regular-webfont.woff HTTP/1.1" 200 85876
[11/Sep/2021 18:51:08] "GET /static/admin/fonts/Roboto-Bold-webfont.woff HTTP/1.1" 200 86184
[11/Sep/2021 18:51:08] "GET /static/admin/fonts/Roboto-Light-webfont.woff HTTP/1.1" 200 85692
Not Found: /favicon.ico
[11/Sep/2021 18:51:08] "GET /favicon.ico HTTP/1.1" 404 2113
[11/Sep/2021 18:52:07] "GET /admin HTTP/1.1" 301 0
[11/Sep/2021 18:52:07] "GET /admin/ HTTP/1.1" 302 0
[11/Sep/2021 18:52:07] "GET /admin/login/?next=/admin/ HTTP/1.1" 200 2214
[11/Sep/2021 18:52:07] "GET /static/admin/css/base.css HTTP/1.1" 200 19513
[11/Sep/2021 18:52:07] "GET /static/admin/css/nav_sidebar.css HTTP/1.1" 200 2271
[11/Sep/2021 18:52:07] "GET /static/admin/css/login.css HTTP/1.1" 200 939
[11/Sep/2021 18:52:07] "GET /static/admin/js/nav_sidebar.js HTTP/1.1" 200 1360
[11/Sep/2021 18:52:07] "GET /static/admin/css/responsive.css HTTP/1.1" 200 18545
[11/Sep/2021 18:52:07] "GET /static/admin/css/fonts.css HTTP/1.1" 304 0
[11/Sep/2021 18:52:07] "GET /static/admin/fonts/Roboto-Regular-webfont.woff HTTP/1.1" 304 0
[11/Sep/2021 18:52:07] "GET /static/admin/fonts/Roboto-Light-webfont.woff HTTP/1.1" 304 0
[11/Sep/2021 18:52:12] "POST /admin/login/?next=/admin/ HTTP/1.1" 200 2375
[11/Sep/2021 18:52:12] "GET /static/admin/fonts/Roboto-Bold-webfont.woff HTTP/1.1" 304 0
```

Our application is deployed in development mode into our Kubernetes cluster. Now we can start developing the application locally, and DevSpace will sync the changes to the container.

## Create a Sample Application and a View

We will use the django-admin tool to create our first app for the project. If you are unfamiliar with projects and applications, please refer [here](https://docs.djangoproject.com/en/3.2/ref/applications/).

```Bash
$ ../venv/bin/django-admin startapp myapp
```

Edit `myproject/myapp/views.py` file and add the following code:

```Python
from django.http import HttpResponse
from django.views import View

class MyView(View):

    def get(self, request, *args, **kwargs):
        return HttpResponse('pong')
```

This code creates a view and configures the view to return `pong` to all requests.

Edit `myproject/urls.py` and add the following code so that we glue `ping/` path to MyView:

```Python
from myapp.views import MyView

urlpatterns += [
    path('ping/', MyView.as_view(), name='myview'),
]
```

This code adds a path to the urlpatterns; ping/ is linked to the previously created view.

Now please visit the http://localhost:8080/ping/, and you should see `pong` as a text response.

Please note that we didn't re-deploy the application into the Kubernetes cluster. The files are synced from your local folder into the container after the change so that we are able to develop our application as we do locally. This is one of the powerful features DevSpace brings.

## Deploying to Production

DevSpace has a helpful feature called [profiles](https://devspace.sh/cli/docs/configuration/profiles/basics). This feature allows you to change the configuration itself based on a given profile. The changes are configured within the profiles stanza and applied with the `-p` flag, for example, `devspace dev -p staging`.

```
devspace deploy -p production
```

## Troubleshooting with DevSpace

### Logs

In the default configuration, DevSpace opens a shell into the container; however, you can change this behavior and get a stream of logs of one or all components. To get the logs, you can also use the following command.

```Bash
$ devspace logs
```

When you don't specify the container, DevSpace will ask you which container you would like to get the logs for.

```
? Select a container  [Use arrows to move, type to filter]
> myproject-97954557-bzvp9-devspace:container-0
  postgresql-postgresql-0:postgresql
[info]   Printing logs of pod:container postgresql-postgresql-0:postgresql
postgresql 18:44:31.47 
postgresql 18:44:31.47 Welcome to the Bitnami postgresql container
postgresql 18:44:31.47 Subscribe to project updates by watching https://github.com/bitnami/bitnami-docker-postgresql
postgresql 18:44:31.47 Submit issues and feature requests at https://github.com/bitnami/bitnami-docker-postgresql/issues
postgresql 18:44:31.48 
postgresql 18:44:31.49 INFO  ==> ** Starting PostgreSQL setup **
postgresql 18:44:31.51 INFO  ==> Validating settings in POSTGRESQL_* env vars..
postgresql 18:44:31.51 INFO  ==> Loading custom pre-init scripts...
postgresql 18:44:31.52 INFO  ==> Initializing PostgreSQL database...
```

### Entering into Containers

DevSpace allows you to enter containers with the container name instead of the usual way, using the long pod name and then the container name.

Without any parameters, it will ask you which pod/container you want to enter.

```Bash
$ devspace enter
```

```
? Which pod do you want to open the terminal for?  [Use arrows to move, type to filter]
> myproject-97954557-bzvp9-devspace:container-0
  postgresql-postgresql-0:postgresql
```

Here we specifically select the `container-0` to enter.

```
$ devspace enter -c container-0
```

```
[info]   Opening shell to pod:container myproject-97954557-bzvp9-devspace:container-0
root@myproject-97954557-bzvp9-devspace:/app# 
```

### Running Commands within a Container

DevSpace allows you to configure predefined commands in the `devspace.yaml` under `commands` stanza. Also, you have the option to use the following syntax to run a command in the container without a configuration.

```
$ devspace enter -c container-0 ./manage.py
```

We can see that `manage.py` command's help text here, as we didn't dive a subcommand or parameter.

```
[info]   Opening shell to pod:container myproject-97954557-bzvp9-devspace:container-0

Type 'manage.py help <subcommand>' for help on a specific subcommand.

Available subcommands:

[auth]
    changepassword
    createsuperuser
...
...
```

### Running Tests

You have several options to start testing. Here is a simple example:

```Bash
$ devspace enter -c container-0 ./manage.py test
```

```
[info]   Opening shell to pod:container myproject-97954557-bzvp9-devspace:container-0
System check identified no issues (0 silenced).

----------------------------------------------------------------------
Ran 0 tests in 0.000s

OK
```

Running tests is very similar to how you would run in the local environment. Also, note that this can be turned into a DevSpace command so that you can use it as `devspace run test`, and all specified tests will be run with a single command.

## Clean Up

DevSpace provides a subcommand that deletes all the deployments and the resources (except PV/PVC) easily so you can remove the development environment with ease.

```Bash
$ devspace purge
```

## Conclusion

We have seen how to set up DevSpace in a basic Django environment. We have also seen that DevSpace seamlessly synced changed files to the running container so that we didn't need to re-deploy the application to Kubernetes cluster every time there was a change. This syncing also works for compiled languages.

DevSpace is highly configurable and is very feature-rich to make your life easier. Have a look at the [documentation](https://devspace.sh/cli/docs/introduction) to learn how to create commands, attach event-based hooks, verify dependencies, and create profiles for different deployment environments such as staging, production, and CI/CD pipelines.

## Further Reading

* [DevSpace Documentation](https://devspace.sh/cli/docs/introduction)
* [Django Documentation](https://docs.djangoproject.com/)

*Photo by [Chris Ried](https://unsplash.com/@cdr6934?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText) on [Unsplash](https://unsplash.com/s/photos/python?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText)*