Crunch Pokemon Data with Python and Deta Base
Tags: programming, database, python
Published: 2021-04-18 21:09:50

_This article was originally posted on [Medium](https://python.plainenglish.io/crunch-pokemon-data-with-deta-base-bd9c97d96ad3). If you prefer reading it from there, please do check it out._

## Introduction

Performance is a must when it comes to building software. However, in certain situations, the speed to set up and integrate new services for proof of concept evaluation or infrastructure is overlooked. Specifically, in the realm of databases, there are many options, but I find [Deta](https://www.deta.sh/) to be most seamless to set up and use. Deta offers Deta Base (I will refer to it as Base), a NoSQL data store optimized for developer simplicity. In this article, I aim to show how to set up a Deta project and interact with your Base to store and manipulate Pokemon data.

![Team Rocket](https://media.giphy.com/media/139JVKQ8m6GeuQ/giphy.gif)

**Let’s get started!**

## Agenda

* Setup

* CRUD

* Querying

## Setup

To begin, navigate to [Sign Up](https://web.deta.sh/), create a new account, and verify your account. Once you sign in, you should be on the following page:

![Photo by Author](https://cdn-images-1.medium.com/max/3828/1*TgHKkBFeHYKwAqqRvEuaXg.png)

Click the arrow in the top left, we will create a new project from scratch.

![Photo by Author](https://cdn-images-1.medium.com/max/3840/1*O1mNYmNDwpWs8PAHjxTN1w.png)

Enter a name of your choice and hit create! A popup with your project key and project id will appear. Make sure you save the key!

With the project key saved, create a new directory, walk into it and run the following command.

   ```pip install flask deta```

[Flask](https://flask.palletsprojects.com/en/1.1.x/) is a web framework that we will use to create endpoints to listen to incoming requests. Based on the requests, we will interact with our remote database using the deta package.

Here is how the folder structure will look like:

    app.py

    config.py

In `config.py`, we will store our project key

```python 
DETA_KEY= "YOUR_COPIED_KEY"
```

In our project, we will create a Base called pokemon and utilize it to store, access, and manipulate Pokemon data. Before that, let’s go over the schema for a record in pokemon:

{% gist https://gist.github.com/Ramko9999/26a37768975bc7c7f219b74411e8ef5a 

%}

Each record in a Base must correspond to a unique identifier called key. When we insert a Pokemon record into our Base, we will provide name as our key. As a result, if we needed to get information on Charizard, we just have to ask pokemon to find the associated record with Charizard as its key.

In `app.py`, let’s set up the Flask app and our connection to our Base, pokemon.
```python
    from config import DETA_KEY
    from flask import Flask, request
    from deta import Deta
    from json.decoder import JSONDecoder

    app = Flask(__name__)

    deta_project = Deta(DETA_KEY)
    db = deta_project.Base("pokemon")
    decoder = JSONDecoder()

    if __name__ == "__main__":
        app.run()
```
All the data access and manipulations will occur through `db`.

Our setup is now complete! Onto the CRUD!

## CRUD

CRUD is an acronym for creating, reading, updating, and deleting data in a database. We will explore how to perform each of the above operations in our pokemon Base. All the work will be done in `app.py`.

### Insertion

Deta provides 2 ways of inserting data. The first is through the [`put`](https://docs.deta.sh/docs/base/sdk#put) method. `put` is the faster method of insertion. If you call `put` on a record that already exists in the base, `put` will overwrite the record. In contrast, [`insert`](https://docs.deta.sh/docs/base/sdk#put) is 2x slower than `put`. In the case that you try to insert into the Base with an already existing key, it will throw an error.

Let’s create an endpoint to insert a new Pokemon with `put`.

{% gist https://gist.github.com/Ramko9999/9c322cc3d4313352432939147452aa63

%}

Here is what the endpoint would look like using `insert` :

{% gist https://gist.github.com/Ramko9999/e2330f1c4673a11d6ab46f950fd24c93

%}

Deta also supports inserting multiple records at the same time with put_many. According to the documentation, it is possible to insert at most 25 items into the Base in a single call ([Deta Base SDK](https://docs.deta.sh/docs/base/sdk#put-many)).

{% gist https://gist.github.com/Ramko9999/2f143f9253e3b37525eb4e5d6d83734d

%}

Let’s test out what we have so far. Let’s insert the data for Pichu into the Base via an HTTP POST through `/pokemon` on Postman.

![](https://cdn-images-1.medium.com/max/2688/1*jXuk1uvzHgw_Oc54okwsqg.png)

Once you hit send, navigate back to [Deta](https://web.deta.sh/home/Ramko9999/default).

![Photo by Author](https://cdn-images-1.medium.com/max/3822/1*kelCdjWliRDu4ORQ2KUJSw.png)

Click on your Base under the **Bases** section. You will now be able to view the data in your Base.

![Photo by Author](https://cdn-images-1.medium.com/max/2988/1*saJ77wMEGlkTT5rq2vl_fw.png)

As you can see, Pichu is in our pokemon Base!

### Access

We can use the [`get`](https://docs.deta.sh/docs/base/sdk/#get) method to access the record of a given key.

{% gist https://gist.github.com/Ramko9999/f38bbd33264c23c7827221b05ac9bf42

%}

Since our Pokemon name is the key, we can directly access its record by providing its name. We can test it with the following GET request.

![Photo by Author](https://cdn-images-1.medium.com/max/2686/1*97_qFgHmoKagFg7Dtknp0w.png)

### Updation

As stated previously in **Insertion**, the `put` function can be used for overwriting records. However, `put` completely overwrites the record and can remove prior fields that are not part of the new updates. As result, if we want to partially update the record, we can use the [`update`](https://docs.deta.sh/docs/base/sdk#update) function. In fact, `update` also allows for fine-grained updations like incrementing values and appending, prepending, and removing elements in a list ([Deta Base SDK](https://docs.deta.sh/docs/base/sdk#update)).

{% gist https://gist.github.com/Ramko9999/ad7c5b9f7f61e70d5b5d44643c5ee990

%}

Let’s make sure updating works with an example. First, I will insert the following data for Charizard.
```python
    {
        "region": "Johto", # Charizard is from Kanto region
        "name": "Charizard",
        "height": 1.7,
        "weight": 90.5,
        "type": ["Fire"], # Charizard also Flying type
        "evolution": ""
    }
```
`region` should be Kanto, and Charizard is also a flying type. Let’s update Charizard with a POST request to `/pokemon/update/Charizard`.

![Photo by Author](https://cdn-images-1.medium.com/max/2682/1*2Os_Wp85latc_fOM2FQ0Lg.png)

After a GET request to `/pokemon/Charizard`, it is clear that `region` and `type` are updated.

![Photo by Author](https://cdn-images-1.medium.com/max/2710/1*mm-besOPWb3zpVmVk2NjhA.png)

### Deletion

Deta provides [`delete`](https://docs.deta.sh/docs/base/sdk#delete), a function that takes in a key and deletes the record associated with the key.

{% gist https://gist.github.com/Ramko9999/6b3796a7d3256c747c770073a610f158

%}

If I wanted to delete Charizard, I would make an HTTP DELETE request to `/pokemon/delete/Charizard`.

We have now explored how to insert, access, update and delete from our Base. Let’s learn how to query our Base.

## Querying

Prior to querying, make sure to fill up your Base with some more Pokemon.

Querying is done through the [`fetch`](https://docs.deta.sh/docs/base/sdk#fetch) method. To elaborate, fetch takes in a [query](https://docs.deta.sh/docs/base/sdk/#queries) or a list of queries and accumulates a list of records whose fields match the query or queries.

A query is nothing more than a dictionary where the mapping between the keys and values represents the query condition. For instance, suppose I wanted to get Blastoise’s record with a query instead of `get`, here is how it would work:
```python
    query = {"name": "Blastoise"} 
    results = next(db.fetch(query))
    blastoise = results[0]
```
We accumulate all the records with `name` equal to Blastoise.

It is also possible to query based on inequalities. For instance, we can query for all Pokemon that weigh greater than 100kg and are less than 1 meter tall.

```python
    query = {"weight?gt": 100, "height?lt": 1}
    pokemon = next(db.fetch(query))
```

We can append “?gt” and “?lt” at the end of numerical fields to query for records with respective values greater than or less than a threshold. There are a lot more suffixes that can be added to the end of a query field, so I recommend reading the documentation for your specific use case ([Deta Base SDK](https://docs.deta.sh/docs/base/sdk/#queries)).

Let’s create an endpoint that will return Pokemon which are of a parameter type.

{% gist https://gist.github.com/Ramko9999/95fb49178555be48eab8fde64ee40edb

%}

“?contains” checks if a provided query element exists in the list associated with the field. To provide an example, if I wanted all the fire-type Pokemon, I would send a GET request to `/pokemon/type/Fire`.

![Photo by Author](https://i.imgur.com/MO2GmcG.gif)

*There are 13 Pokemon returned from the above request.*

This is a little overwhelming since this query will return all the fire-type Pokemon in our Base. Deta also provides us with the ability to limit the number of query results with [buffer](https://docs.deta.sh/docs/base/sdk/#example-using-buffer-pages-1). Let’s set the buffer arg in fetch to 2.

{% gist https://gist.github.com/Ramko9999/2b79b1ef8e8c4d58b2710fec1c7ba23c

%}

Now, we only get two records back.

![Photo by Author](https://cdn-images-1.medium.com/max/2692/1*Z-BgZLjSFIdWVceL8tqY-Q.png)

Finally, by changing the `pages` argument in `fetch`, it is possible to spread the result data over multiple pages.

## Conclusion

My primary goal with this writing was to shed greater light on a database that you can set up and work on in the blink of an eye. It took hardly much time to create the project and based on the above examples, CRUD and Querying are as simple as they can get. For these reasons, Deta Base is perfect for proof of concepts, serverless applications, hackathons, and many more situations and projects that require simplicity and speed.

### Resources

[**Deta (Home)**]
(https://www.deta.sh/)

[**Flask**]
(https://flask.palletsprojects.com/en/1.1.x/)

[**Deta SDK**]
(https://docs.deta.sh/docs/base/sdk)

[**Bulbapedia**](https://bulbapedia.bulbagarden.net/wiki/Main_Page)