Using Python to Dump Data into Django Models for Testing Frontend or API Endpoints:A Complete Django REST API Tutorial
Tags: python, django, restapi, sqlite
Published: 2024-07-12 13:26:27

As a developer working on APIs or front-end applications, you need data in your database to test API endpoints and front-end components effectively. This article will show you how to import JSON data into Django models, a skill that's useful for other formats like CSV or text files in various backend frameworks.

Recently, I worked on an API development project where my data was stored in a JSON file. I built a backend using Django and Django Rest Framework to serve this data through API endpoints to a React-based front-end application. While Django offers tools like the Admin app, dumpdata, and loaddata for data management, these tools can be complex for beginners and don’t handle file processing by default.

To simplify the process, I used a straightforward Python script to load my JSON data into Django models.

This article assumes you have already set up a Django project and configured it to serve data from your models via an API endpoint. However, for completeness and clarity, I have included the necessary steps and provided examples of the `models.py` and `views.py` files. For this, article is divided into two parts (Part I** & Part II), if you have a Django app with REST setup, you can skip directly to Part II: Dumping data (books.json) to Book model (Book()).

## Part I: Steps to Set Up Your Django Environment**

1.Create a virtual environment using either _conda_ or _venv_ with Python 3.10 or later:

**Example my env name is djangorestapi**

Using venv:
```
$python -m venv myenv

$python -m venu djangorestapi
```
Using conda:

```
$conda create --name myenv python=3.10

$conda create --name djangorestapi python=3.10
```
2.Activate your virtual environment (Ubuntu/Linux):

```
$source djangorestapi/bin/activate

$conda activate djangorestapi

```
3.Install Django and Django Rest Framework
Offical Website:
 [Django](https://www.djangoproject.com/)

 [Django Rest Framework](https://www.django-rest-framework.org/)

Note: Notice the activated virtual environment **djangorestapi** and project folder name **devrestapi**.

```
**(djangorestapi)**ajit/**devrestapi**$pip install django djangorestframework
```
4.Create a Project using Django admin command (we will use popular bookstore example so will have project name as bookstore and later there will be a app name as book and Model name will Book())


Note: For simplicity, I have removed the virtual environment name and project folder, but throughout this tutorial, ensure you are working within your virtual environment.

```
$django-admin startproject bookstore

```
5.Create a new app and call it book.

Note: You need to move inside the project folder (bookstore) created after running the command in step 4. 

```
$cd bookstore
$python manage.py startapp book

```
6.Add the book app to the project’s settings by editing the INSTALLED_APPS list in the bookstore/settings.py file.

```
INSTALLED_APPS = [   
    ...     
    'book',
    ...      
]
```

7.Similar to book app you also need to add restframework app to use REST features in Django application.

```python
INSTALLED_APPS = [   
    ... 
    'rest_framework',
    'book',
    ...     
]
```
### Make and run migration


It is always good practice to create and run migration after adding a new app to the Django project or creating or making changes in the models.py. So, you should do that as follows:

```bash
$python manage.py makemigrations
$python manage.py migrate
```

```
Operations to perform:
  Apply all migrations: admin, auth, contenttypes, sessions
Running migrations:
  Applying contenttypes.0001_initial... OK
  Applying auth.0001_initial... OK
  Applying admin.0001_initial... OK
  Applying admin.0002_logentry_remove_auto_add... OK
  Applying admin.0003_logentry_add_action_flag_choices... OK
  Applying contenttypes.0002_remove_content_type_name... OK
  Applying auth.0002_alter_permission_name_max_length... OK
  Applying auth.0003_alter_user_email_max_length... OK
  Applying auth.0004_alter_user_username_opts... OK
  Applying auth.0005_alter_user_last_login_null... OK
....
```

The step 1-7 will provide all the boilerplate code to get started with your API application.

Now, you can create a model Book in your book/models.py as below:

```python
from django.db import models

class Book(models.Model):
    title = models.CharField(max_length=200)
    author = models.CharField(max_length=100)
    price = models.DecimalField(max_digits=6, decimal_places=2)

    def __str__(self):
        return self.title
```
### Serving the Book Model via API

To serve the Book model via an API, you need to accomplish two key tasks. First, create a serializer to convert model data to and from JSON. Second, create a view to map the serializer to your API endpoints.

Let's walk through these steps:

Create a Serializer:create a serializers.py file within your book app as follow:

```python
#book/serializers.py
from rest_framework import serializers
from .models import Book

class BookSerializer(serializers.ModelSerializer):
    class Meta:
        model = Book
        fields = ['id', 'title', 'author', 'price']
```

The views will handle the API requests and responses using the serializer.

We have two views for two API endpoints: 

BookListAPIView(): To create a view that serves all Book model information via an API endpoint (/api/books/)

BookDetail(): To create a view that show details of one book based on the id. (books/<int:pk>/)
```python
#book/views.py
from rest_framework import generics
from .models import Book
from .serializers import BookSerializer

class BookListCreate(generics.ListCreateAPIView):
    queryset = Book.objects.all()
    serializer_class = BookSerializer

class BookDetail(generics.RetrieveUpdateDestroyAPIView):
    queryset = Book.objects.all()
    serializer_class = BookSerializer
```


Once your model and view are ready, you can link your app route through urls.py (within the book app and the project urls.py).

```python
#book/urls.py
from django.urls import path
from .views import BookListCreate, BookDetail

urlpatterns = [
    path('books/', BookListCreate.as_view(), name='book-list-create'),
    path('books/<int:pk>/', BookDetail.as_view(), name='book-detail'),
]
```
Now, map your app routes to the project route, using urls.py from project.

```python
#bookstore/urls.py
from django.contrib import admin
from django.urls import path, include

urlpatterns = [
    path('admin/', admin.site.urls),
    path('api/', include('book.urls')),
]
```
Note: It is good time to create and run your migrations again.

```bash
$python manage.py makemigrations
```

Output:

```python
Migrations for 'book':
  book/migrations/0001_initial.py
    - Create model Book

```
Output: migration

```bash
$python manage.py migrate
```

```python
Operations to perform:
  Apply all migrations: admin, auth, book, contenttypes, sessions
Running migrations:
  Applying book.0001_initial... OK

```

By following these steps, you will have set up API endpoint /api/books/ and /api/book/<int:pk>/ that retrieves all Book model data or a single book data in JSON format when accessed via a GET request. 

Let check that every thing is working fine by running Django webserver and going to the api endpoints.

```
$python manage.py runserver
```

Go to the browser and open localhost url as :

http://127.0.0.1:8000/api/books/


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/rn2hwa4g179evjchpclq.png)

Now, if you try other API endpoint that is for getting the detail of a specific book id. such as:

http://127.0.0.1:8000/api/books/1

But, you will have a error because there is no data about any book and now in the next part we will dump data to Book Model.


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/068ji24r3nz6t21kjh5s.png)




## Part II: Dumping data (books.json) to Book model (Book())

By default, Django utilizes SQLite for data storage. If you haven't set up another database, the following steps will work seamlessly. Any adjustments needed for specific databases will be minimal.

### install and import required python modules[default modules in Python 3.10] (if these two are not available in your environment due to any reasons)

```
$pip install sqlite3
$pip install json
```

```python
import sqlite3
import json
```

In Django, the default database name is db.sqlite3, and you can find this file in the main project directory after performing migrations.

You can use any SQLite database viewer tool, or alternatively, the following online tool is recommended for visualizing tables.

[https://sqliteonline.com/](https://sqliteonline.com/)


[https://inloop.github.io/sqlite-viewer/](https://inloop.github.io/sqlite-viewer/)

It will surprise you to see there are so many tables along with a table for the Book() model. Don't worry, Django create these tables for Admin app that we were talking at the beginning of the article. 

Note: Django create a table for each model and the naming is done with app name as namespace and the model name. So, the table for Book model will be named as 'book_books'. 

You can connect with db.sqlite3 database using following python lines:

```python
conn = sqlite3.connect('db.sqlite3')
```

Now, you can load your json data using following line:

```python
with open('data.json', 'r') as json_file:
    data = json.load(json_file)
```

Now dumping data into the model is similar to a SQL insert operation, so you can loop through your data object and make insert into the book_books table as follows:

```python
for item in data:
    title = item["title"]
    author = item["author"]
    price = item["price"]

    conn.execute("INSERT INTO 'book_books' (title, author, price) VALUES (?, ?, ?)", 
                 (title, author,price ))  

```
It is important to perform commit and close the connection after a database operation.

```python
conn.commit()
conn.close()
```

You can see the data in sqlite table as below:
**
(Note: if you are wondering how the id field came, then you must know , Django add the id field with each model if the default is not overwritten.) **

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/adwwuvymqmm8qkoiynx1.png)

Now, if you try the following endpoints, you will details:

http://127.0.0.1:8000/api/books/1


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/20ouckwrsrdonscfwit0.png)



This will insert all the data into the table and now those data will be available via the endpoints. It is good to run a migration to make things good after a database operation.

You can run the Django server and test the api endpoints via browser or using any tools like postman or curl etc.

You can find the complete code via my [github repo] (https://github.com/ajitmp/restapi-blog-code-dev-to)

All questions, and queries are welcome in the comment.