Create a NoSQL database with Amazon DynamoDB and optimize cloud spend
Tags: beginners, tutorial, aws, database
Published: 2022-03-02 13:54:32


## What is Amazon DynamoDB?

Amazon DynamoDB was internally developed in 2006 to manage workloads for the online Amazon store, DynamoDB was officially launched to the general public in 2012 as a fully managed, serverless non-relational(NoSQL) database service for fast and reliable performance e.g. internet performance at scale for online stores.
 
### DynamoDB turns 10 🎉🎂- Join us on 1 March - A celebration with special guests

On 1 March, Amazon celebrates a decade of innovating with DynamoDB, there is a full of learning with special guest appearances including Amazon Chief Evangelist Jeff Barr, AWS Vice President of Analytics, Database and Machine Learning Dr Swami Sivasubramanian, AWS Data Hero Alex DeBrie the author of DynamoDB book and Jeremy Daly who is an AWS Serverless Hero and DynamoDB modeling instructor.

**You may watch the replay from Severless Land: A decade of innovation with DynamoDB**

{% embed https://www.youtube.com/watch?v=YI67mWmjbZ4 %}


![A decade of dynamodb](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/vlu2bdufqnes9sa45brt.png)

You may register and view the agenda at this link [here](https://pages.awscloud.com/NAMER-field-LS-DynamoDB-10-Year-Anniversary-2022-reg-event.html) and watch the celebration and day of learning with live streaming from Twitch.

{% embed https://twitter.com/dynamodb/status/1496839080239267840  %}

### Why was Amazon DynamoDB created?

After experiencing e-commerce platform outages from the online store Amazon.com and upon conducting a root cause analysis, former Amazon Intern Dr Swaminathan Sivasubramanian who is now the Vice President of Analytics, Databases and Machine Learning noticed that relational databases were an inefficient solution. 

Other observations included:

* Relational databases only store and retrieve data with a primary key and cannot handle internet scale with millions of requests per second from Amazon Prime Days
* Replication technologies are not highly available
* Databases could not scale or use partitioning for load balancing

![reseARCH PAPER](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/hl32qrmtu3vmvub3ac6a.png)

In 2007 Peter Vosshall presented ***Dynamo: Amazon's Highly Available Key-value Store*** at the prestigious ACM Symposium on Operating System Principles (SOSP), a tehnical paper that was co-authored by Giuseppe DeCandia, Deniz Hastorun, Madan Jampani, Gunavardhan Kakulapati, Avinash Lakshman, Alex Pilchin, Swaminathan Sivasubramanian, Peter Vosshall, Werner Vogels that proposed that DynamoDB would be a novel **solution** that included:

* Highly available key-value data storage system
* Object versioning
* Efficient in resource usage
* Simple to scale as the data size grows or API requests increase

You may download the Amazon research paper [here](https://www.amazon.science/publications/dynamo-amazons-highly-
available-key-value-store)

### Key Features of Amazon DynamoDB

**a) Performance at scale:** 

* Data and traffic for tables is spread across servers to achieve microsecond latency with in-memory cache system
* Key-value model
* Global replication of global tables
* Amazon Kinesis Firehose can deliver Dynamo data to other AWS Services

**b) Serverless:** 

* Database can be scaled on demand and increment with auto scaling
* Read/write capacity

**c) Enterprise ready:** 

* ACID transactions support
* Encrypts customer data at rest
* Point in time recovery of accidental deletion of data from tables

### Comparison of Databases: SQL vs NoSQL

**DynamoDB**

* Database type: Nonrelational(NoSQL)
* Structure: JSON, key-value,graph-based, document and column
* Schema: Dynamic
* Scale: Can run single-digit millisecond performance and at any scale. You can start small and then increment. Can handle petabytes of data
* Performance: built for online transaction processing (OLTP) at scale e.g. e-commerce platform for online stores
* Optimization: Optimized for read or write
* Query type: Simple queries

**SQL Server**

* Database type: Relational (SQL)
* Structure: Tables with rows and columns
* Schema: Predefined
* Scale: Limited
* Performance: good for online analytical processing (OLAP)
* Optimization: Optimized for storage
* Query type: Real-time, complex queries

### High Level Architecture

Amazon DynamoDB's design is based on architecture that is [highly decentralized, loosely coupled service architecture](https://www.allthingsdistributed.com/files/amazon-dynamo-sosp2007.pdf) that links multiple AWS services and data storage is always available.

DynamoDB's tables can be global and replicated across multiple regions.

DynamoDB reflects Amazon engineers' continued focus on the [Well-Architected Framework](https://aws.amazon.com/architecture/well-architected/?wa-lens-whitepapers.sort-by=item.additionalFields.sortDate&wa-lens-whitepapers.sort-order=desc) **Performance Efficiency pillar** to incorporate design considerations that include:

a) Democratize advanced technologies for greater flexibility to configure the data store
b) Go global in minutes with high availability 
c) Use serverless architectures to achieve reliability,consistency, cost-effectiveness and performance
d) Experiment more often to decide when to resolve any update conflicts during read or write to deliver on customer experience 

Hence DynamoDB has a writeable data store with an 'always on' customer experience to manage different states of service.

![architecgture](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/mvt5v7exhxqk0an0a1g4.png)

Please find below the **Service-oriented architecture** of  Amazon's platform to render web page content from client requests delivered with a response rate of 300 milliseconds for 99% of requests in the technical paper published in 2007 in [Figure 1](https://www.allthingsdistributed.com/files/amazon-dynamo-sosp2007.pdf)


![service](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/m1fie68evyu8azjwvrnj.png)

### DynamoDB Use Case: Orange Theory Fitness architecture

In the 'This Is My Architecture' episode introducing heart-rate monitoring workout at Orangetheory Fitness: Using Chime SDK to Power Coach-led Virtual Workouts with OTlive. Other AWS Services such as AWS Cognito together with AWS API Gateway work to enhance the member and coaching experience with the ability to retrieve infrequently accessed historical member data when required using
DynamoDB with low latency and high performance.

{% embed https://www.youtube.com/watch?v=Eoq7E6jMtBs %}

Other use cases for Amazon DynamoDB include:

* Super Bowl advertising
* AWS Prime Day shopping online
* Website clicks
* Voting
* e-commerce order history
* Online gaming
* Store time-series data
* Apps
* Social Media posts e.g. Tweets

### Amazon DynamoDB -  New feature releases

At AWS re:invent 2021 the DynamoDB key announcements included:

**a) Amazon DynamoDB Standard-Infrequent Access (Standard-IA)**

***What's new in DynamoDB? with speaker: Chad Tindel, Principal NoSQL Solution Architect for Amazon***

{% embed https://www.youtube.com/watch?v=KdAMSVyOsJ4&t=2s %}

Amazon DynamoDB Standard-Infrequent Access(Standard-IA) table class is a new product feature that was introduced at 2021 AWS re:invent helps you reduce your DynamoDB costs by up to 60 percent for tables that store infrequently accessed data.

The DynamoDB Standard-IA table class offers lower storage costs compared to the DynamoDB Standard tables, making it the most cost-effective option for tables used for storage. The DynamoDB Standard table class offers lower throughput costs than the DynamoDB Standard-IA table class and you can always switch back to your preferred table class and not compromise on table  performance, availability or durability.

***Amazon DynamoDB Standard-Infrequent Access launch recap with the team at re:Invent 2021 with speakers Pete Naylor - Senior Tech Product Manager for Amazon DynamoDB and Elie Gharios, Senior Tech Product Manager for Amazon DynamoDB***

{% embed https://youtu.be/Tfswg2mcSoE %}

**Do you require long term storage of your data?** 

***Check if you have a good use case to use NEW table class Amazon DynamoDB Standard-Infrequent Access (Standard-IA) to save your company some money compared to the standard table class for DynamoDB***

Stephanie Gooch wrote in her blog [here](https://aws.amazon.com/blogs/aws-cloud-financial-management/putting-aws-reinvent-2021-cost-optimization-announcements-into-practice/?nc1=b_rp) things to consider for your business use case:

a) Does your storage exceed 50% of your throughput cost (reads and writes). This usage pattern is a good indicator that you have storage that you are not reading or writing on a regular basis.

b) If you have ETL to archive data to services like Amazon S3. Rather than continuing to move data, it can be kept in the table at a lower cost.

c) If you use rolling tables (creating new tables day/month). In this case, the older tables may be less frequently read; therefore, perfect for infrequent access.

There are two tools that you may use to evaluate and support your use case to use this new feature for Amazon DynamoDB Standard-Infrequent Access (Standard-IA):

* [AWS Cost and Usage Report](https://aws.amazon.com/aws-cost-management/aws-cost-and-usage-reporting/)
* [AWS Well-Architected Lab for Cost Optimization for DynamoDB](https://wellarchitectedlabs.com/cost/300_labs/300_cur_queries/queries/database/#amazon-dynamodb)

**b) AWS meets compliance and business continuity requirements with AWS Backup**

Announced in November 2021, AWS Backup supports DynamoDB with full backups of DynamoDB tables without compromising performance and high availability. This new feature supports compliance and business continuity for industries such as finance and banking that require an audit trail to meet regulatory obligations and might have been reluctant to use DynamoDB in the past.


### Tutorial: Create an Amazon DynamoDB table using new table class Amazon DynamoDB Standard-Infrequent Access (Standard-IA)


**Free Tier**: If you do not have an existing AWS account, you may be able to take advantage of Free Tier and access Amazon DynamoDB  for free for the first 12 months with up to 25 GB of storage.

Step 1: Sign in to the **AWS Management Console** with your [IAM user credentials](https://dev.to/aws-builders/creating-your-first-iam-admin-user-and-user-group-in-your-aws-account-machine-learning-part-1-3cne)

![management console](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/p3c1whehu8452h2759na.png)


![IAM](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/5bijksa5pyaw9fkeonj8.png)

Step 2: Type the word **DynamoDB** into the navigation bar and click DynamoDB

![DB](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/jc8zzs5n47gxtzzq787h.png)

Step 3: On the new DynamoDB console, click **Create table**


![create table](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/rsdial42952j2qb0t0hq.png)


Step 4: Provide a name for the table and type in the box **Tennis**


![Tennis](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/b3xhc0btbw95pjwa5diu.png)

Step 5: The partition key is used to spread data across partitions for scalability. It’s important to choose an attribute with a wide range of values and that is likely to have evenly distributed access patterns. Type the word **TennisPlayer** in the Partition key box.


![TennisPlayer](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/tbofke8jt8g9qldgp1n9.png)

Step 6: Because each tennis player may compete in many competitions throughout the year, you can enable easy sorting with a sort key. Select the Add sort key check box. Type **CompetitionName** in the Add sort key box.


![competition](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/q2aboekc2za3wcefk29z.png)

Step 7: Tables will be created under the default setting for a DynamoDB standard table class.


![default](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/688d0h68dvlo7hvvprj6.png)

Step 8: Select **Customize Settings** to update the default settings


![cutomize](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/gpdsl5d04hn0xcya2hb7.png)

Step 9: Select the new table class **DynamoDB Standard-IA** if your storage is greater than 60% and the table is infrequently accessed


![Standard IA](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/i6z1bts54tbssqkhty0d.png)

Step 10: Estimated costs capacity is shown below when choosing **On-demand** for new tables with unknown workloads and unpredictable traffic.You may inspect the estimated cost and adjust the inputs for the read, write and also average item size.



![ondemand](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/891buibwifgu9pcelicr.png)

Step 11: And the final step is **Create Table**


![create table2](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/lr12fc2i6dx8bt1mmtxr.png)

It will take a few seconds for the table to be created and you will receive a banner message that the status of the job has been successful.


![table created](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/6ekhs61hf363mgpw2riu.png)

Step 12: To add data to the Tennis table you may follow this [tutorial](https://aws.amazon.com/getting-started/hands-on/create-nosql-table/)


Step 13: This is an optional step for AWS Back up integration with DynamoDB announced in November 2021 that you can create on-demand backup of tables if you need to maintain data for compliance or business continuity e.g. an end of year tax audit.


![create backup](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/f2x66tb9qvpj0lqvnjvy.png)

Select the option **On demand backup**

Step 14: Search for the name of the existing table **Tennis** that was previously created.



![ondemnd](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/thof0ri4ik4mu7z7jcdu.png)
Step 15: Choose **Customize Settings** to allow you to move backup to cold storage and the option to delete items.


![backup cust](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/37xp4x5gscorijzl2elx.png)

Step 16: Click **Create Backup**


![create backup](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/81fl1w9ph3yr9ykpxw7y.png)

Note: Ensure you have already created an IAM role for service role AWSBackupDefaultServiceRole to perform AWS Backup.



![permissions](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/al0ckcyf92x4i5onms7k.png)

You may inspect what the permissions allow you to do within the IAM policy with AWS Backup such as Createbackup and DescribeBackup within DynamoDB:


![backup role](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/x3m11df5yoqp8dwpxl1s.png)


Step 17: The table **Tennis** is successfully backed up with AWS Backup integration with DynamoDB



![baxup tenns](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/uphca9bj2vkcryoya3ri.png)

### AWS re:invent 2021 re-cap: Key Sessions


**DynamoDB deep dive: Advanced design patterns**

{% embed https://www.youtube.com/watch?v=xfxBhvGpoa0 %}


**Amazon DynamoDB: Driving innovation at any scale**

{% embed https://www.youtube.com/watch?v=pA6nET3YAqU&t=4s %}

**Data Modeling with DynamoDB**

{% embed https://www.youtube.com/watch?v=yNOVamgIXGQ&list=PLUfg41qH0VgsgY8A5PkTCnm8wN5j3aDuu&t=4s %}



### References

[Amazon DynamoDB Developer Guide](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Introduction.html)

[Happy 10th Birthday DynamoDB](https://aws.amazon.com/blogs/aws/happy-birthday-dynamodb/)

[Amazon DynamoDB Features](https://aws.amazon.com/dynamodb/features/?pg=dynamodbt&sec=hs)

[Amazon DynamoDB Standard-Infrequent Access (Standard-IA)](https://aws.amazon.com/blogs/aws-cloud-financial-management/putting-aws-reinvent-2021-cost-optimization-announcements-into-practice/?nc1=b_rp)

[Amazon DynamoDB meets compliance and business continuity with Amazon Backup](https://aws.amazon.com/about-aws/whats-new/2021/11/amazon-dynamodb-requirements-aws-backup/)

[Tutorial Create a NoSQL table in the console](https://aws.amazon.com/getting-started/hands-on/create-nosql-table/)

[Tutorial Create a NoSQL table using the Command Line Interface (CLI)](https://boto3.amazonaws.com/v1/documentation/api/latest/guide/dynamodb.html)


Happy Learning! 😁