Modelling with NDB
Tags: ndb, datastore, database, relationship
Published: 2021-06-08 18:43:41

Python ORMs(Object Relationship Mappers) for SQL databases, such as Django ORM and SQLAlchemy, ship with properties that explicitly model database relationships between different database tables. 
However, `ndb` - the python ORM built for Google Cloud Datastore does not provide such properties. Fortunately, using existing `ndb` properties, database relationships can still be modelled with `ndb`. 
There are three types of relationships in a database, namely: **one-to-one**, **one-to-many**, and **many-to-many**. Let's dive into each relationship's implementation using `ndb` as well as their pitfalls.

**Assumptions**:
1) Readers are familiar with [Database Relationships](https://condor.depaul.edu/gandrus/240IT/accesspages/relationships.htm).
2) Readers have basic familiarity with python.
3) Readers are familiar with various column [properties](https://cloud.google.com/appengine/docs/standard/python/ndb/entity-property-reference) of `ndb`.

## One-to-One
![One-to-One](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/v1h3slaw5cvzh8gotev7.jpeg)

Famous examples of one-to-one relationships include:
* Western Countries and their presidents.
* Personal bank account and its owners.
* Monogamous marriage between two partners.

To implement a one-to-one relationship, `ndb` provides the `KeyProperty` for storing datastore keys. The `KeyProperty` also accepts an optional `Kind` argument to validate keys assigned to this property.
 One-To-One relationships between two ndb Kinds can be implemented by creating a `KeyProperty` on one of the Kinds and setting its optional `kind` argument to be the other Kind.
Let's write some code to illustrate how `KeyProperty` can be used to implement a one-to-one relationship:

```python
Class Person(ndb.Model):
    name = ndb.StringProperty()
    nationality = ndb.StringProperty(repeated=true)
    email = ndb.StringProperty()
    age = ndb.IntegerProperty()

Class Country(ndb.Model):
    name = ndb.StringProperty()
    location = ndb.GeoProperty()
    citizen_count = ndb.IntegerProperty()
    currency_name = ndb.StringProperty()
    president = ndb.KeyProperty(Person)

# Creating
obama = Person(name='Barack Obama', 
               nationality=['American',  'Kenya'], 
               email='darasimiajewole@gmail.com', 
               age=61)
obama_key = obama.put()

america = Country(name='America', 
                  citizen_count=1*10**6, 
                  currency_name='dollars', 
                  president=obama_key)

america_key = america.put()
america_id = america_key.id()

# Reading
america = Country.query(Country.president=obama_key).get()
obama = america.president.get()
```

**Pitfalls**
* KeyProperty does not enforce uniqueness like the one-to-one fields of traditional SQL ORMs. Unless a uniqueness check is being made before the entities of the Two Kinds are created/updated, a one-to-many relationship can be inadvertently created instead of one-to-one.

## One-to-Many
![One-to-Many](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ziwcd1rh2cxoy6k53gl2.jpeg)
Famous examples of one-to-many relationships include:
* Polygamous marriages.
* Father and his Children.
* Planets and their Moons
* Companies and their products
* Countries and their States

You have two ways to model a one-to-many relationship with ndb:
### 1) Using KeyProperty
This is quite similar to modeling a one-to-one relationship except the uniqueness check is omitted. 
To implement this relationship between two Kinds using `KeyProperty`, simply create a field on one of the Kinds with the property type as the KeyProperty type and pass the other Kind class as the optional type argument. Let's dive into some code to illustrate the modeling:

```python
Class Country(ndb.Model):
    name = ndb.StringProperty()
    location = ndb.GeoProperty()
    citizen_count = ndb.IntegerProperty()
    currency_name = ndb.StringProperty()

Class State(ndb.Model):
    name = ndb.StringProperty
    location = ndb.GeoProperty()
    population = ndb.IntegerProperty()
    country = ndb.KeyProperty(Country)

# Creating
## parent entity
nigeria = Country(name='Nigeria',       
                  citizen_count=2*10**7,   
                  currency_name='Naira')
nigeria_key = country.put()

## child entities
lagos = State(name='Lagos', 
              population=30*10**6, 
              country=nigeria_key)
lagos.put()

abj = State(name='Abuja',  
            population=10*10**6, 
            country=nigeria_key)
abj.put()

# Reading all states in a country
all_state = State.query(State.country == nigeria_key).fetch()
#[lagos, abj]


# Updating
china = Country(name='China',        
                citizen_count=2*10**9,   
                currency_name='yen')
china_key = china.put()

lagos.country = china_key
lagos.put()
```

**Pitfalls**
* Querying for the child entities is not supported within a datastore transaction because they don't have the same "datastore ancestor".


### 2) Modelling using Ancestor Key

Just a bit of background on datastore and ancestor keys culled from the official docs:
_Datastore entities are identified by a key, an identifier unique within the application's datastore. The key can have an ancestor, which is another key. This parent can have an ancestor, and so on; at the top of this chain of parents is a key with no ancestor, called the root. Entities whose keys have the same root form an entity group or group._
Leveraging on the ancestor key relationship between root entities and entity groups, one-to-many relationships can be established between two Kinds by making the Key of the parent Kind the ancestor to the Key of the child Kind. Let's show some code to illustrate this:
```python

Class Company(ndb.Model):
    name = ndb.StringProperty()
    staff_num = ndb.StringProperty()
    headquarter = ndb.StringProperty()

Class Product(ndb.Model):
    name = ndb.StringProperty()
    market_share = ndb.IntegerProperty()
    users_num = ndb.StringProperty()
    

# Creating
## parent
google = Company(name='Google', 
                 staff_num=20000, 
                 headquarter='Yabacon Valley')
## parent key
google_key = google.put()

## child key will have parent as google_key
gmail_key = ndb.Key(Product, 'Gmail Service', parent=google_key)
chrome_key = ndb.Key(Product, 'Chrome Browser', parent=google_key)
## child
gmail = Product(key=gmail_key, 
                name='Gmail', 
                market_share=80, 
                users_num=200)
gmail.put()
chrome = Product(key=chrome_key, 
                 name='Chrome Browser',
                 market_share=70,
                 users_num=34)
chrome.put()

# Reading
## reading parent entity
google_key = chrome.key.parent()

## reading child entities
all_product = Product.query(ancestor=google_key).fetch()
print(all_product) # [chrome, gmail]
```

To change the parent entity in a one-to-many relationship modeled using an Ancestor key, the child entity is deleted and recreated with a new key formed from the new parent key. These extra steps are needed because entity keys are immutable in the datastore after been created. Let's show some code:

```python
# updating
## new parent entity
microsoft = Company(name='Microsoft', 
                    staff_num=5000, 
                    headquarter='Palo Alto')
microsoft_key = microsoft.put()

# child entity
chrome = chrome_key.get()
chrome_key.delete()

## creating new child entity key from new parent key
new_chrome_key = ndb.Key(Product,
                         'Chrome Explorer', 
                          parent=microsoft_key)
new_chrome = Product(key=new_chrome_key)
new_chrome.name = chrome.name
new_chrome.market_share = chrome.market_share
new_chrome.users_num = chrome.users_num
new_chrome.put()
```

**Pitfalls**:
* Datastore writes take longer, due to replication of changes across the entity groups stored on many Datastore servers. This could lead to inconsistent reads seconds after a datastore write.

* It's highly advisable not to use the keys of the child entities as parent keys of entities of another Kind. This is due to the volatility of the child entity keys during the update of its parent.

* The relationship is not explicit enough from the model definition.

* Multiple One-to-Many relationships cannot be formed by a child entity to multiple parents of a different kind. So the relationship between a User and their home country, and the same User and their Job, cannot be modeled using `ndb` ancestor keys.

## Many-to-Many relationships
![Many-to-Many](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/hndaqdpiwzsymnbnt770.jpeg)
Famous examples of this relationship include:
1) Pizza and their toppings. Pizzas can have multiple toppings and a topping can be on multiple pizzas.
2) Citizens with multiple countries.

The `KeyProperty` type from NDB has earlier been used to implement the one-to-one relationship and also one-to-many relationship,  it should not be surprising that it can also be used to model the many-to-many relationship.
That is possible because the `KeyProperty` type supports the `repeated` argument, which means a list of Datastore keys can be attached to the `KeyProperty` fields.
Leveraging on that, many-to-many relationships between two NDB kinds such as Pizza and Toppings could be modeled using a repeated KeyProperty on the Pizza kind with the Kind argument as the Toppings kind. Below is some code to illustrate how the modeling works:

```python
class Pizza(ndb.Model):
    name = ndb.StringProperty()
    steps = ndb.TextProperty(repeated=true)
    toppings = ndb.KeyProperty(Toppings, repeated=true)

Class Toppings(ndb.Model):
    name = ndb.StringProperty()
    ingredients= ndb.StringProperty(repeated=true)
    date_created = ndb.DatetimeProperty(auto_now=true)

# Creating
cheese = Toppings(name='cheese', 
                  ingredients= ['cheese', 'random seasoning'])
onions = Toppings(name='onion', 
                  ingredients= ['onions', 'chili seasonings'])
bacon = Toppings(name='bacon', 
                 ingredients= ['bacon', 'garlic seasoning'])
ndb.put_multi(cheese, onions, bacon)

macaroni_cheese = Pizza(name='Macaroni Cheese', 
                        steps=['A', 'B', 'C'], 
                        toppings=[cheese.key, bacon.key])
barbeque = Pizza(name='Barbeque', 
                 steps=['H', 'L', 'O'], 
                 toppings=[cheese.key, onion.key])
ndb.put_multi(macaroni_cheese, barbeque)

# Reading
barbeque = Pizza.get_by_id(barbeque_id)
toppings = barbeque.toppings
### [cheese.key, onion.key]

cheese_pizzas = Pizza.query(Pizza.toppings == cheese.key).fetch()
### [macaroni_cheese_key, barbeque_key]


# Updating
sweet_corn = Toppings(name='sweet corn', 
                      ingredients=['sweet corn', 'honey'])
sweet_corn.put()


macaroni_cheese.toppings.append(sweet_corn.key)
macaroni_cheese.put()
```

**Pitfalls**
* Having more than one indexed repeated properties on a Kind can lead to exploding index. Due to this behavior, it's advisable for ndb kinds not to have more than a single Many-to-Many relationships.