Prepopulate Firestore (Firebase) with a Python script
Tags: firebase, python, 100daysofcode, firstpost
Published: 2021-02-03 22:02:40

This is a quick post to show you how to import CSV or JSON data into Firestore. If you data is in CSV format, you will need to convert it to JSON format using an online tool called CSVJSON. This guide uses Python and assumes that you have a Firebase project already set up.

## Data

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/2niylcu5vggu72l4tq5i.png)

First thing, you need to do is prepare the data you are going to import. For this guide, I prepared the data using Google Sheets then exported it in csv format. I then used the CSVJSON online too mentioned above to export the data to json format.

We will use the `id` field as the Firestore document ID. We will also structure the code and files to allow for multiple collection imports. One JSON file will contain data for one collection. The name of the JSON file will be used as the name of the collection. For instance, when we import from `categories.json`, it will use (or create, if none exists) a collection called categories and import data into it. We will place all our JSON files in a `data/` folder in the root of the project.

## Credentials


The first step is to download the service key from your Firebase. Do this by navigating to the Users and Permissions settings. See image below:

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/03m8d3ev6jk16fpj9oy8.png)

In the Users and Permissions page, navigate to the Service Accounts tab. There you will see your service account email and a code snippet showing how to configure the service account in your code.

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/kr1sit4bhan5xotkynwi.png)

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/xu0i5ps1xprtdc0gwm3z.jpg)

Click the Generate new private key button shown in the image above, to download a service key. Place this file in your project folder.

## Code

First start by initializing the Firebase SDK using the credentials we created in the previous step

```
import firebase_admin
from firebase_admin import credentials, firestore

cred = credentials.Certificate("path/to/serviceAccountKey.json")
firebase_admin.initialize_app(cred)

db= firestore.client()
```

Next we will loop through all the JSON files in the `data/` directory and import each of them. Remember, the filenames will be used as the collection names (`categories.json` will import into the categories collection).

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/dctvlxlypngxjkhizqep.png)

Line 12 makes sure we are only using JSON files.

Line 13 gets the name of the collection from the filename

Line 15 converts the JSON data to a list of objects

For each object (line 17), we remove (pop) the `id` field and store it in a variable which are going to use as the document ID (line 20). If the object has no `id` field, we will let Firestore auto-generate an ID for us (line 22).




Below is the full code

```
import firebase_admin

import os

import json

from firebase_admin import credentials, firestore

cred = credentials.Certificate("path/to/serviceAccountKey.json")

firebase_admin.initialize_app(cred)

db = firestore.client()

for filename in os.listdir('data'):

    if filename.endswith('.json'):

        collectionName = filename.split('.')[0] # filename minus ext will be used as collection name

        f = open('data/' + filename, 'r')

        docs = json.loads(f.read())

        for doc in docs:

            id = doc.pop('id', None)

            if id:

                db.collection(collectionName).document(id).set(doc, merge=True)

            else:

                db.collection(collectionName).add(doc)

```
That's it! All done.

Please reach out with questions and suggestions.

Happy coding.