How to Export & Import Data | Django Package Series #3
Tags: python, django, webdev, productivity
Published: 2019-12-08 17:23:23

This post cross-published with [OnePublish](https://www.onepubli.sh)

<p>Welcome to 3rd post of Django Package Series. In this post we are going to learn one of the best Django package which allows to import and export data easily.</p>
<p>Youtube Channel with video tutorials - <a href="https://www.youtube.com/channel/UCEwYRHE5ZAOMfUaIpCQvjEQ">Reverse Python Youtube</a></p>
<p>Before we go on, please visit <a href="https://reversepython.net/">Reverse Python</a> to find more cool stuff with Python.</p>
{% link https://dev.to/coderasha/let-me-introduce-you-reverse-python-1eef %}

<h3>django-import-export</h3>
<p>The <b>django-import-export</b> library supports multiple formats, including xls, csv, json, yaml, and all other formats supported by <b>tablib</b>. It also have a Django admin integration, which is really convenient to use.</p>

<h3>Installation & Configuration</h3>
<p>Let's build a project to understand it better. Create a new Django project named <b>dashboard</b> and inside it create a new app named <b>reports</b>.</p>

<p>To install the package, run the following command:</p>

```
pip install django-import-export
```
<p>Then, in your <b>settings.py</b> add your app name (reports) and import-export library into INSTALLED_APPS:</p>

```python
INSTALLED_APPS = (
    ...
    'import_export',
    'reports'
)
```
<p>I also recommend to add optional configuration to end of <b>settings.py</b>:</p>

```python
IMPORT_EXPORT_USE_TRANSACTIONS = True
```
<p>The default value is False. It determines if the library will use database transactions on data import.</p>

<blockquote>After this section I will show you main parts of the project. If you don't know how to configure urls.py or templates, I will put GitHub repo link end of this post, so you can clone and run the project.</blockquote>

<h3>Create Model and Resources</h3>
<p>In your <b>models.py</b> create a model named Employee:</p>

```python
from django.db import models

class Employee(models.Model):
    first_name = models.CharField(max_length=30)
    last_name = models.CharField(max_length=60)
    email = models.EmailField(blank=True)
    day_started = models.DateField()
    location = models.CharField(max_length=100, blank=True)

    def __str__(self):
        return self.first_name
```

<p>The django-import-export library works with the concept of Resource, which is class definition very similar to how Django handles model forms and admin classes.</p>

<p>If you want to handle data directly from Django admin then you should put the code inside the <b>admin.py</b> file.</p>

<p>However, our implementation for this project is not related to the Django admin, so let's create <b>resources.py</b> file inside app folder.</p>

```python
from import_export import resources
from .models import Employee

class EmployeeResource(resources.ModelResource):
    class Meta:
        model = Employee
```
<h3>Export Data</h3>
<p>Well, we are going to create form and let the user to select format of file. Update your <b>views.py</b>:</p>

```python
from django.shortcuts import render
from django.http import HttpResponse
from tablib import Dataset

from .resources import EmployeeResource
from .models import Employee

def export_data(request):
    if request.method == 'POST':
        # Get selected option from form
        file_format = request.POST['file-format']
        employee_resource = EmployeeResource()
        dataset = employee_resource.export()
        if file_format == 'CSV':
            response = HttpResponse(dataset.csv, content_type='text/csv')
            response['Content-Disposition'] = 'attachment; filename="exported_data.csv"'
            return response        
        elif file_format == 'JSON':
            response = HttpResponse(dataset.json, content_type='application/json')
            response['Content-Disposition'] = 'attachment; filename="exported_data.json"'
            return response
        elif file_format == 'XLS (Excel)':
            response = HttpResponse(dataset.xls, content_type='application/vnd.ms-excel')
            response['Content-Disposition'] = 'attachment; filename="exported_data.xls"'
            return response   

    return render(request, 'export.html')
```

<p>It is very simple, when user selects JSON, file will exported as JSON or if user selects CSV then file will exported as CSV.</p>

<b>export.html</b>

```html
{% extends 'base.html' %}

{% block content %}
<h3>Export Data</h3>
<p>exporting from database</p>
<form method="post" enctype="multipart/form-data">
    {% csrf_token %}
    <p>Please select format of file.</p>
    <select name="file-format" class="form-control my-3">
        <option selected>Choose format...</option>
        <option>CSV</option>
        <option>JSON</option>
        <option>XLS (Excel)</option>
      </select>
    <button class="btn btn-primary" type="submit">Export</button>
  </form>
  <a href="{% url 'home' %}">Return Home View</a>
  {% endblock %} 
```
<h3>Import Data</h3>
<p>Assume that we have file named <b>employees.csv</b>:</p>

```
first_name,last_name,email,day_started,location,id
Peter,Parker,peter@parker.com,2015-05-18,New York,
James,Bond,james007@bond.com,2014-08-11,London,
```
<p>The <b>id</b> must be present because it is the primary key. It will be generated automatically.</p>

<p>So, we need to import CSV or JSON file to database. Add following function to your <b>views.py</b>:</p>

```python
def import_data(request):
    if request.method == 'POST':
        file_format = request.POST['file-format']
        employee_resource = EmployeeResource()
        dataset = Dataset()
        new_employees = request.FILES['importData']

        if file_format == 'CSV':
            imported_data = dataset.load(new_employees.read().decode('utf-8'),format='csv')
            result = employee_resource.import_data(dataset, dry_run=True)                                                                 
        elif file_format == 'JSON':
            imported_data = dataset.load(new_employees.read().decode('utf-8'),format='json')
            # Testing data import
            result = employee_resource.import_data(dataset, dry_run=True) 
            
        if not result.has_errors():
            # Import now
            employee_resource.import_data(dataset, dry_run=False)

    return render(request, 'import.html')    

```
<p>Let the user to select format of file which will imported.</p>

<b>import.html</b>

```html
{% extends 'base.html' %}

{% block content %}
<h3>Import Data</h3>
<p>importing to database</p>
<form method="post" enctype="multipart/form-data">
    {% csrf_token %}
    <input type="file" name="importData">
    <p>Please select format of file.</p>
    <select name="file-format" class="form-control my-3">
        <option selected>Choose format...</option>
        <option>CSV</option>
        <option>JSON</option>
      </select>
    <button class="btn btn-primary" type="submit">Import</button>
  </form>
<a href="{% url 'home' %}">Return Home View</a>
  {% endblock %} 
```
<p>Once you imported your file go and check admin and you will see imported data in your model.</p>

<h3>Django Admin</h3>
<p>If you want to use import export in admin then simply add following code in your <b>admin.py</b>:</p>

```python
from import_export.admin import ImportExportModelAdmin
from django.contrib import admin
from .models import Employee

@admin.register(Employee)
class EmployeeAdmin(ImportExportModelAdmin):
    pass
```

<p>Here's how it looks like:</p>

<img src="https://thepracticaldev.s3.amazonaws.com/i/i1hl0wc9kph887v52qoh.png" alt="Reverse Python" />

<h3>GitHub</h3>
<p>Some of you can be confused so it is better to clone the project from my GitHub and do stuff with it. Here is my GitHub link for this project:</p>

{% github raszidzie/Django-Import-Export %}

<h3>Mission Accomplished!</h3>
<p>Great job! You learned really useful information today. For more information about django-import-export take look <a href="https://django-import-export.readthedocs.io/en/latest/getting_started.html#creating-import-export-resource">Documentation</a> </p>
<p>If you liked the tutorial please please visit <a href="https://reversepython.net/">Reverse Python</a> to find more cool stuff with Python.</p>
{% link https://dev.to/coderasha/let-me-introduce-you-reverse-python-1eef %}</p>