DynamoDB OneTable API Overview
Tags: aws, serverless, dynamodb, nosql
Published: 2021-07-05 22:09:13

---
title: DynamoDB OneTable API Overview
published: true
description: A quick tour of the OneTable API for single-table DynamoDB designs.
tags: aws, serverless, dynamodb, nosql
cover_image: https://www.sensedeep.com/images/on-top-of-the-world.jpg
canonical_url: https://www.sensedeep.com/blog/posts/2021/dynamodb-onetable-tour.html
---

DynamoDB OneTable (OneTable) is an access library for [DynamoDB](https://aws.amazon.com/dynamodb/) applications that use one-table design patterns with NodeJS. OneTable makes dealing with DynamoDB and one-table design patterns dramatically easier while still providing easy access to the full DynamoDB API.

We've worked very hard to make the OneTable API expressive, terse and elegant to use. I hope you will find that it makes your DynamoDB developments proceed faster and more easily than every before.

This article takes a quick tour through the OneTable API and demonstrates basic calls and patterns when using the API.

## OneTable Overview

![OneTable](https://www.sensedeep.com/images/ring-bare.png)

A good way to read this article is to checkout the OneTable Overview working sample and follow along in VS Code. If you are using TypeScript, use the TypeScript sample instead.

* [OneTable Overview Sample](https://github.com/sensedeep/dynamodb-onetable/tree/main/samples/overview) &mdash; A quick tour through OneTable.
* [OneTable TypeScript Sample](https://github.com/sensedeep/dynamodb-onetable/tree/main/samples/typescript) &mdash; A TypeScript quick tour.

## Installation

    npm i dynamodb-onetable

## Quick Tour

Import the OneTable library. If you are not using ES modules or Typescript, use `require` to import the libraries.

```javascript
import {Table} from 'dynamodb-onetable'
```

If you are using the AWS SDK V2, import the AWS `DynamoDB` class and create a `DocumentClient` instance.

```javascript
import DynamoDB from 'aws-sdk/clients/dynamodb'
const client = new DynamoDB.DocumentClient(params)
```

This version includes prototype support for the AWS SDK v3.

If you are using the AWS SDK v3, import the AWS v3 `DynamoDBClient` class and the OneTable `Dynamo` helper. Then create a `DynamoDBClient` instance and Dynamo wrapper instance.

```javascript
import {DynamoDBClient} from '@aws-sdk/client-dynamodb'
import Dynamo from 'dynamodb-onetable/Dynamo'
const client = new Dynamo({client: new DynamoDBClient(params)})
```

Initialize your your OneTable `Table` instance and define your models via a schema. The schema defines your single-table entities, attributes and indexes.

```javascript
import MySchema from './Schema'

const table = new Table({
    client: client,
    name: 'MyTable',
    schema: MySchema,
})
```

## Schemas
Schemas define your `models` (entities), keys, indexes and attributes. Schemas look like this:

```javascript
export default {
    indexes: {
        primary: { hash: 'pk', sort: 'sk' },
        gs1:     { hash: 'gs1pk', sort: 'gs1sk', project: ['gs1pk', 'gs1sk', 'data'] },
    },
    models: {

        Account: {
            pk:         { type: String, value: 'account#${id}' },
            sk:         { type: String, value: 'account#' },
            id:         { type: String, uuid: true, validate: Match.ulid },
            name:       { type: String, required: true, unique: true, validate: Match.name },
            balance:    { type: Number, default: 0 },

            //  Search by account name or by type
            gs1pk:      { type: String, value: 'account#' },
            gs1sk:      { type: String, value: 'account#${name}${id}' },
        },

        User: {
            pk:         { type: String, value: 'account#${accountId}' },
            sk:         { type: String, value: 'user#${email}' },
            accountId:  { type: String, required: true },
            id:         { type: String, uuid: true, validate: Match.ulid },
            name:       { type: String, required: true, validate: Match.name },
            email:      { type: String, required: true, validate: Match.email, crypt: true },

            address:    { type: Object, default: {}, schema: {
                street: { type: String, /* map: 'data.street' */ },
                city:   { type: String, /* map: 'data.city' */ },
                zip:    { type: String, /* map: 'data.zip' */ },
            } },

            status:     { type: String, required: true, default: 'active', enum: ['active', 'inactive'] },
            balance:    { type: Number, default: 0 },

            //  Search by user name or by type
            gs1pk:      { type: String, value: 'user#' },
            gs1sk:      { type: String, value: 'user#${name}#${id}' },
        },

        Product: {
            pk:         { type: String, value: 'product#${id}' },
            sk:         { type: String, value: 'product#' },
            id:         { type: String, uuid: true, validate: Match.ulid },
            name:       { type: String, required: true },
            price:      { type: Number, required: true },

            //  Search by product name or by type
            gs1pk:      { type: String, value: 'product#' },
            gs1sk:      { type: String, value: 'product#${name}#${id}' },
        },

        Invoice: {
            pk:         { type: String, value: 'account#${accountId}' },
            sk:         { type: String, value: 'invoice#${id}' },

            accountId:  { type: String, required: true },
            date:       { type: Date, default: () => new Date() },
            id:         { type: String, uuid: true },
            product:    { type: String },
            count:      { type: Number },
            total:      { type: Number },

            //  Search by invoice date or by type
            gs1pk:      { type: String, value: 'invoice#' },
            gs1sk:      { type: String, value: 'invoice#${date}#${id}' },
        }
    }
```

This schema has models for: Account, User, Product and Invoice.

The schema also defines your indexes. The index key values are defined for each model and can derive their values from other attributes via value templating.

Most interactions with the OneTable API are via model objects that correspond to each schema model. For example,

```javascript
const Account = table.getModel('Account')
```

For typescript, we use a slightly different formulation that provides type safety for your model references at compile time.

```typescript
type AccountType = Entity<typeof Schema.models.Account>
const Account = table.getModel<AccountType>('Account')
```

### Create a Table

Normally you would create your tables separately, hopefully via infrastructure-as-code. But for development, you can create a table directly from the schema via:

```
await table.createTable()
```


### Creating an Item

Once the table is prepared, you can create an item:

```javascript
let account = await Account.create({name: 'Acme Airplanes'})
```

This will write the following to DynamoDB:
```javascript
{
    pk:         'account:8e7bbe6a-4afc-4117-9218-67081afc935b',
    sk:         'account:98034',
    name:       'Acme Airplanes',
    id:         '8e7bbe6a-4afc-4117-9218-67081afc935b',
    balance:    0,
    created:    '2021-07-05T04:58:55.136Z',
    updated:    '2021-07-05T04:58:55.136Z',
}
```

This API will return the following, and omit the underlying hidden properties: pk, sk, gs1pk, gs1sk.

```
{
    name:       'Acme Airplanes',
    id:         '8e7bbe6a-4afc-4117-9218-67081afc935b',
    balance:    0,
    created:    '2021-07-05T04:58:55.136Z',
    updated:    '2021-07-05T04:58:55.136Z',
}
```

To return the hidden properties, set `{hidden: true}` in the params:

```javascript
let account = await Account.create({name: 'Acme Airplanes'}, {hidden: true})
```

## Context Properties

If creating a multi-tenant application, it us helpful to have the account ID be part of the primary key. Rather than having each API specify this account ID, it is best to provide this via a context property that ensures it is never omitted. Context properties are blended with the API properties and take precedence.

```javascript
table.setContext({accountId: account.id})
```

Now when we create a user, it will automatically set the user account ID to the context property value.

```javascript
let user = await User.create({name: 'Road Runner', email: 'roadrunner@acme.com'})
```

### Getting Items

To get an item, we provide the properties require to construct the key. In the schema, the Account keys were specified as:

```javascript
{
    pk: { type: String, value: 'account#${id}' },
    sk: { type: String, value: 'account#' },
}
```

So we need to provide the account `id` and then OneTable will construct the keys for you.

This will retrieve an account, using the primary key.

```javascript
account = await Account.get({id: '8e7bbe6a-4afc-4117-9218-67081afc935b'})
```

## Using a secondary index

We can also fetch items using a secondary index which allows us to retrieve items by other attributes such as the Account name or User name. An overloaded secondary index can be used to retrieve different items using alternative properties. The schema defines a Global Secondary Index (GSI) and then each model entity uses different values from their value templates into the GSI key fields.

For example, the User type constructs the GSI sort key using the user name and user ID.

```javascript
{
    gs1pk: { type: String, value: 'user#' },
    gs1sk: { type: String, value: 'user#${name}#${id}' },
}
```

So we can retrieve a user by name from the GSI 1 using:

```javascript
user = await User.get({name: 'Road Runner'}, {index: 'gs1', follow: true, log: true})
```

This will use the secondary index and construct a query to search for users that have the gs1sk set to `user#Road Runner`. Because the user ID portion is not provided, OneTable searches for a user that begins with the provided string.

To save storage costs, the GSI in this example only projects the `gs1pk`, `gs1sk` and `data` attributes to the GSI. That means the other user attributes are not returned when querying only from the GSI. OneTable addresses this by providing the `follow: true` parameter which follows the GSI read with a read from the primary index to fetch the full user record.

The `{log: true}` is useful to see the actual DynamoDB request logged to the console.


## Finding items

To find all the users for an account:

```javascript
let users = await User.find()
```

Remember, the account ID is provided by the context, so we don't need to explicitly provide it.

If we want to filter items, we can add non-key properties to use as filters. Here we find the users that have a role of 'admin'.

```javascript
let adminUsers = await User.find({accountId: account.id, role: 'admin'})
```

Alternatively, we can use a `where` clause. This will find users with an account balance over $100. Where clauses are useful for more complex filtering expressions. A where clause supplies property references by enclosing the name in `${}`. Similarly, values are enclosed in `{}` without the dollar.

```javascript
users = await User.find({}}, {
    where: '${balance} > {100}'
})
```

Note: these are not Javascript string templates, so do not use back-ticks when specifying.


## Updating Items

To update items, you can provide the keys and new values via the properties. Remember that when updating you cannot change an item's keys in DynamoDB.

```javascript
user = await User.update({email: 'roadrunner@acme.com', balance: 0, role: 'admin'})
```

To help prevent accidental creation when updating, OneTable adds a constraint to ensure your items already exist and you don't accidentally create new items by providing incorrect key ingredient properties.

To update nested properties, provide them via the `params.set`. This will update just the property you specify.

```javascript
user = await User.update({email: 'roadrunner@acme.com'}, {set: {'address.zip': '{"98034"}'}})
```

You can also use expressions to atomically update your attributes. For example:

```javascript
user = await User.update({email: 'roadrunner@acme.com'}, {add: {balance: 10}})
user = await User.update({email: 'roadrunner@acme.com'}, {set: {balance: '${balance} - {20}'}})
```

### Item Collections

A key power of DynamoDB is to fetch multiple related items in a single efficient request.

To fetch an account, the account users and invoices in one request, use `table.fetch`.

```javascript
let collection = await table.fetch(['Account', 'User', 'Invoice'], {pk: `account#${account.id}`})
```

This returns an object map with lists for Account, User and Invoice.

In this request, we need to provide the actual primary key value as each model type may have different value templates to construct their pk value.

### Batch Adding

To efficiently add many items, DynamoDB has batch operations. OneTable makes these easy. This code snippet creates 200 users in batches of 25.

```javascript
let batch = {}
let i = 0, count = 0
while (i++ < 200) {
    User.create({name: `user${i}`, email: `user${i}@acme.com`}, {batch})
    if (++count >= 25) {
        await table.batchWrite(batch)
        batch = {}
        count = 0
    }
}
```

### Transactions

OneTable transactions are expressed similarly to batches.

```javascript
let transaction = {}
await Account.update({id: account.id, status: 'active'}, {transaction})
await User.update({id: user.id, role: 'user'}, {transaction})
await table.transact('write', transaction)
```

### Pagination

The DynamoDB API has limits on the maximum size of request response data. So you often need to use pagination to step through large result sets.

OneTable uses the `start` param on request APIs and in result sets to mark the current reading position.

```javascript
let start
do {
    users = await User.find({}, {start, limit: 25})
    //  Operate on this batch of 25 users ...
    start = users.start
} while (users.start)

```

## Summary

Checkout the full overview sample at:

* [OneTable Overview Sample](https://github.com/sensedeep/dynamodb-onetable/tree/main/samples/overview) &mdash; A quick tour through OneTable.
* [OneTable TypeScript Sample](https://github.com/sensedeep/dynamodb-onetable/tree/main/samples/typescript) &mdash; A TypeScript quick tour.
* [All OneTable Samples](https://github.com/sensedeep/dynamodb-onetable/tree/main/samples)

You can read more in the detailed documentation at:

* [GitHub OneTable](https://github.com/sensedeep/dynamodb-onetable) or
* [NPM OneTable](https://www.npmjs.com/package/dynamodb-onetable).



## Why OneTable?

DynamoDB is a great [NoSQL](https://en.wikipedia.org/wiki/NoSQL) database that comes with a steep learning curve. Folks migrating from SQL often have a hard time adjusting to the NoSQL paradigm and especially to DynamoDB which offers exceptional scalability but with a fairly low-level API.

The standard DynamoDB API requires a lot of boiler-plate syntax and expressions. This is tedious to use and can unfortunately can be error prone at times. I doubt that creating complex attribute type expressions, key, filter, condition and update expressions are anyone's idea of a good time.

Net/Net: it is not easy to write terse, clear, robust Dynamo code for one-table patterns.

Our goal with OneTable for DynamoDB was to keep all the good parts of DynamoDB and to remove the tedium and provide a more natural, "JavaScripty / TypeScripty" way to interact with DynamoDB without obscuring any of the power of DynamoDB itself.

## Other Working Samples

* [OneTable Overview Sample](https://github.com/sensedeep/dynamodb-onetable/tree/main/samples/overview) &mdash; A quick tour through OneTable.
* [OneTable CRUD Sample](https://github.com/sensedeep/dynamodb-onetable/tree/main/samples/crud) &mdash; Basic CRUD.
* [OneTable TypeScript Sample](https://github.com/sensedeep/dynamodb-onetable/tree/main/samples/typescript) &mdash; Basic TypeScript use.
* [OneTable Migrate Sample](https://github.com/sensedeep/dynamodb-onetable/tree/main/samples/migrate) &mdash; how to use OneTable Migrate and the Migrate CLI.
* [OneTable Packed Attributes Sample](https://github.com/sensedeep/dynamodb-onetable/tree/main/samples/packed) &mdash; How to use packed attributes.
* [OneTable SenseDeep Sample](https://github.com/sensedeep/dynamodb-onetable/tree/main/samples/sensedeep) &mdash; How to access SenseDeep log data.
* [All OneTable Samples](https://github.com/sensedeep/dynamodb-onetable/tree/main/samples)


## SenseDeep with OneTable

At SenseDeep, we've used the OneTable module extensively with our [SenseDeep](https://app.sensedeep.com) serverless developer studio. All data is stored in a single DynamoDB table and we extensively use one-table design patterns. We could not be more satisfied with DynamoDB implementation. Our storage and database access costs are insanely low and access/response times are excellent.

### Contact

You can contact me (Michael O'Brien) on Twitter at: [@mobstream](https://twitter.com/mobstream), or [email](mob-pub-18@sensedeep.com) and read my[Blog](https://www.sensedeep.com/blog).

To learn more about SenseDeep and how to use our serverless developer studio, please visit [https://www.sensedeep.com/](https://www.sensedeep.com/).

### Links

* [GitHub OneTable](https://github.com/sensedeep/dynamodb-onetable)
* [NPM OneTable](https://www.npmjs.com/package/dynamodb-onetable)
* [SenseDeep Web Site](https://www.sensedeep.com/)
* [SenseDeep App](https://app.sensedeep.com/)