DynamoDB Attribute Packing for Single-Table Designs
Tags: aws, serverless, dynamodb, nosql
Published: 2021-04-08 03:07:46

---
title: DynamoDB Attribute Packing for Single-Table Designs
published: true
description: DynamoDB Attribute Packing for Single-Table Designs
tags: aws, serverless, dynamodb, nosql
cover_image: https://www.sensedeep.com/images/blog/blocks.jpg
canonical_url: https://www.sensedeep.com/blog/posts/2021/attribute-packing.html
---

A DynamoDB secondary index can select which attribute to project (replicate) to the index. It can project all item attributes, a subset of the attributes or only the key attributes.

If you project only the keys, then a read from the secondary index will return the key attributes. With these keys you can read all the remaining attributes from the primary index, but that will incur an additional read request and require code to manage the second request.

With [DynamoDB single-table Designs](https://www.sensedeep.com/blog/posts/2021/dynamodb-singletable-design.html) choosing which attributes to project to secondary indexes can be a challenge. With single-table designs, you store multiple entities with different named attributes in a single table. This means the set of attributes to project to GSIs may be large and diverse. Furthermore, once defined, you cannot change the names of projected attributes after you create the GSI. These issues can make efficient use GSIs difficult and evolving and changing your data design problematic.

[OneTable](https://www.sensedeep.com/blog/posts/2020/dynamodb-onetable.html) solves this problem by supporting the mapping and package of entity attributes into a single GSI attribute. This makes the task of defining which attributes to project to the GSI relatively easy and also permits changing your data design without having to recreate the GSIs.

## OneTable Attribute Mapping

OneTable schemas can define an attribute mapping via the `map` schema property. This defines a physical table attribute name for the schema field.

```javascript
{
    User: {
        email: { type: String, map: 'data'}
    }
}
```

This will store the User.name property in the `data` table attribute.


### One to One Mapping

OneTable mapping definitions can also map multiple different entities onto the same attribute name.

```javascript
{
    Account: {
        name: { type: String, map: 'data', }
    }
    User: {
        email: { type: String, map: 'data', }
    }
}
```

This will store both the Account.name and User.email values in the GSI 'data' attribute.

### Many to One Packing

Sometimes, you may need to project multiple field properties into a GSI. By using OneTable mappings, you can map and pack multiple attributes from a single entity to a single GSI attribute.

By specifying a mapped name that contains the period character, you can pack property values into an object stored in a single attribute. OneTable will transparently pack and unpack values on read/write operations.

```javascript
const Schema = {
    models: {
        User: {
            pk:          { value: 'user:${email}' },
            sk:          { value: 'user' },
            id:          { type: String },
            email:       { type: String, map: 'data.email' },
            firstName:   { type: String, map: 'data.first' },
            lastName:    { type: String, map: 'data.last' },
        }
    }
}
```

This will pack the User.email, User.firstName and User.lastName properties under the GSI `data` attribute. The data attribute will have the values:

```javascript
{
    email: 'email-name',
    first: 'firstName',
    last: 'lastName',
}
```
You can also map and pack the properties from multiple entities into a single attribute name.

By using the `map` facility, you can create a single GSI `data` attribute that contains all the required attributes for access patterns that use the GSI. By modifying the OneTable schema and using the OneTable CLI for migrations, you can easily evolve your design without recreating your GSIs.


## Using Mapped Attributes

When issuing APIs that write to a mapped attribute, you must provide all the properties that map to that attribute for the entity.

For example, the following will fail because the lastName is not provided and the API must provide all three properties: email, firstName and lastName that map to the `data` attribute.

```javascript
await User.update({email: 'coyote@acme.com', firstName: 'Peter'})
```


## Value Templates

There is one other technique you can use for one-way attribute packing.

A OneTable schema field can define a `value` property which operates like a JavaScript template string. Embedded `${field}` references are expanded to create the attribute value. For example:

```javascript
{
    sk:      { type: String, value: 'user:${country}:${zip}:${state}:${address}' },
    country: { type: String },
    zip:     { type: String },
    ...
}
```

This will create a sort-key (sk) attribute with the values of country, zip, state and address catenated after a 'user:' prefix. This is useful for queries that can search on varying segments of the sort key using begins_with.

Note: this technique replicates the attributes in the `value` template and is thus not a technique to reduce overall data storage.


## OneTable Follow

If reading from a secondary index that projects a subset of attributes and you wish to fetch the entire item, you would normally have to issue a second read to fetch the full item from the primary index.

OneTable makes this easier by using the `follow` option where OneTable will transparently follow the retrieved primary keys from a GSI and fetch the full item from the primary index so that you do not have to issue the second read manually.

```javascript
let account = await Account.find({name: 'acme'}, {index: 'gs1', follow: true})
```

Under the hood, OneTable is still performing two reads to retrieve the item but your code is much cleaner. For situations where the storage costs are a concern, this approach allows minimal cost, keys-only secondary indexes to be used without the complexity of multiple requests in your code.

## SenseDeep with OneTable

At SenseDeep, we've used OneTable and the OneTable CLI extensively with our [SenseDeep](https://app.sensedeep.com) serverless Developer Studio. All data is stored in a single DynamoDB table and we extensively use single-table design patterns. We could not be more satisfied with DynamoDB implementation. Our storage and database access costs are insanely low and access/response times are excellent.

Please try our Serverless developer studio [SenseDeep](https://www.sensedeep.com/).

### Contact

You can contact me (Michael O'Brien) on Twitter at: [@mobstream](https://twitter.com/mobstream), or [email](mob-pub-18@sensedeep.com) and ready my [Blog](https://www.sensedeep.com/blog).

To learn more about SenseDeep and how to use our serverless developer studio, please visit [https://www.sensedeep.com/](https://www.sensedeep.com/).

### Links

* [OneTable Post](https://www.sensedeep.com/blog/posts/2020/dynamodb-onetable.html)
* [OneTable Migrate CLI](https://www.npmjs.com/package/onetable-cli)
* [DynamoDB Single Table Design](https://www.sensedeep.com/blog/posts/2021/dynamodb-singletable-design.html)
* [OneTable Migrate CLI](https://www.npmjs.com/package/onetable-cli)
* [SenseDeep Web Site](https://www.sensedeep.com/)
* [SenseDeep App](https://app.sensedeep.com/)