DynamoDB Table Design Guidelines. Simple list of steps that me and my…
By: Varsha Venkatesh
Tags: Software Design, Dynamodb, NoSQL, Nosql Database, Aws Dynamodb

Here are some steps that me and my team try to follow when we are designing DynamoDB tables. This list is based on our knowledge of using DDB for the past year and half and also from reading great articles on the internet, some of which are listed here.

Guidelines

Start by listing down the use cases we are trying to solve and convert them to DB queries required

As opposed to SQL where we define one table per entity, in DDB/NoSQL the idea is to have single table for all related entities. Hence we start with trying to reuse any existing table by modifying the existing table’s design instead of creating a new table. As a general rule, you should maintain as few tables as possible in a DynamoDB application. Exceptions are cases where high-volume time series data are involved, or datasets that have very different access patterns

Modeling many-to-many relationships/ graph data is explained here

Consider the data size and expected WCU/RCU to decide the partitions to avoid any hot partitions/throttling. More details on key design here

and expected to decide the partitions to avoid any hot partitions/throttling. More details on key design here Consider options if data is not distributed more evenly just by the partition key design

Key Selection

Primary Key

Primary Key = Partition Key + Sort Key

Partition Key — input to Hash function that DDB uses to decide the physical partition in to which the data goes into.

Sort Key/Range Key — used to store data in sorted order within a partition based on this key

Primary Key needs to be globally unique

Why should you care about partition key

Better distributes data across different physical partitions without which there could be hot partitions/throttling

Fast lookups on frequently used access patterns

What if Partition + Sort Key is not enough to distribute the data evenly?

Write Sharding using suffixes/prefixes

Composed Partition Keys by adding other values along with Partition key

Using DAX — a caching layer in front of DynamoDB which would prevent queries hitting DDB directly so in turn would prevent hot partitions or throttling

Global Secondary Index (GSI)

When to use GSI?

When you have frequent query pattern that is not based on the primary key of the table. Naive approach for those use cases would be to use Scan + Filter Expression , but we want to avoid this approach due to latency and scanning cost on the table.

, but we want to avoid this approach due to latency and scanning cost on the table. By creating a GSI on the fields that are frequently queried but not the primary key of the table, look ups on those fields are much more efficient.

How does GSI work?

Creating a GSI creates a new table in the background with a new GSI’s PK (Primary Key) and this table is kept in sync with the actual table

GSI has its own RCU/WCU — which is different from the actual table. It is recommended to keep WCU of GSI >= WCU of original table

GSI Partition Key requires uniform distribution for the same reasons discussed here

Cost of using a GSI

Since GSI is a clone of original table, now write costs to original table is doubled by adding a GSI. For example, if there are 3 GSIs — 1 write update to the table = 4 writes (1 write to original table + 3 writes to GSIs)

Single table design

Why should you care about single-table design

Its highly recommended to have single table for all related items in DDB for the following reasons

Main benefit

There is no concept of joins in DDB. If there is one table for per entity — during join use cases, it translates into writing multiple queries to different tables. This is bad for performance due to network hops and we want to avoid that for latency.

Marginal benefits

Multiple tables also means operational overhead to monitor metrics of multiple tables.

Cost savings by combining entities into single table. If you have one or two entity types in your single table that are accessed much more frequently than the others, you can hide some of the extra capacity for less-frequently accessed items in the buffer for the other items.

This might be minimal or maybe even zero if on-demand pricing is used.

Downsides of single table design

Learning curve to design properly (this can be overcome once we establish a couple of patterns)

Inflexibility in adding new access patterns (its not impossible to re-design tables and do a data migration if required)

More work in exporting data to any analytics platform (this can also be overcome once we establish a couple of patterns)

Many-to-Many relationships

Context of joins

In SQL we create one table per entity and for queries involving multiple entities we use joins to get results. Joins are expensive and one of the reasons that SQL does not scale well. Hence in NoSQL the problem of joins is solved by removing the need for it altogether.

Advantage of joins: Getting heterogenous data in a single request

Disadvantage of joins: expensive and does not scale well

Goal of many-to-many relationships design

In modeling many-to-many relationships in DDB the goal is to have single query -> single request / getting heterogenous data in a single request ie, to not have multiple tables per entity and make two requests to achieve what we would have achieved by joins in SQL (similar to GraphQL optimization in the frontend)

We achieve this by pre-joining data into a collection , ie items sharing same partition key in a single table

Design Guidelines

Identify entities and relationship between entities

Identify access patterns for all the entities and start with the most important ones

Entities are identified by the partition key

Relationships are identified by the sort key

Model using adjacency graph patterns

Reference