How to create an AWS DynamoDB database and add data using Python Boto3 library.
Tags: boto3, python, aws, devops
Published: 2024-05-15 19:30:05

[Amazon DynamoDB](https://docs.aws.amazon.com/dynamodb/) is a managed NoSQL serverless database service providing fast and predictable performance. With Amazon DynamoDB you do not need to provision or manage servers since no installation or operating software is needed. It also scales in and out to adjust the client’s usage in capacity. 

The benefits of adding your data programmatically to DynamoDB is that it reduces errors that may occur when adding data manually and also saves a lot of time because tasks are automated. 

[Boto3](https://docs.aws.amazon.com/pythonsdk/) is the official AWS SDK for Python. With well structured Python Scripts, the user can comfortably create, update and delete AWS resources. Boto3 is widely used to manage other AWS services such as creating instances, uploading and downloading files among others. 

## Prerequisites
 
AWS Account (Create one [here](https://portal.aws.amazon.com/billing/signup?refid=em_127222&redirect_url=https%3A%2F%2Faws.amazon.com%2Fregistration-confirmation#/start/email) if not created)
Python installed (Click this [link](https://www.python.org/downloads/) if Python is not installed)
VS Studio Code installed (Install using this [link](https://code.visualstudio.com/download) if not installed)

## Setting up Boto3 and AWS DynamoDB
On your AWS root account, create an IAM user. Avoid using the AWS root user account for security purposes because, in AWS, the customer is responsible for security in the cloud. 

### Create an IAM user

- Search for IAM users on the AWS management console and on users, create a new user and a new password to sign in. 
- Attach necessary policies such as Administrative Access to the user. 
- Review your policies and click the create user button. 
- View the new user you have created and create a new access key.
- Follow the steps and select the use of the Access key by choosing the Command Line Interface (CLI). 
- Download .csv file for ease of use. 
### Configuring AWS credentials on your terminal
Open a new VS Code editor and open a new terminal. 

Type this command on your terminal:

```
aws configure
```

Enter your `Access Key ID` and `Secret access key` as prompted. 

If you are okay with the default `region` listed, press `Enter` on your keyboard otherwise, use this command to change the `region` 

```
aws configure --region 'your_region'
```

Press `Enter` on the default format `[json]`.
### Creating a DynamoDB table
- Search `DynamoDB` on the `AWS console` and create a new table. 
- Give your table a new name e.g. `graduates`
- Set the partition key (primary key) as `grad_id`
- Proceed to creating the table. 
## Writing Python Code with Boto3
Install `boto3` library by running this command on your terminal:

```
pip install boto3
```

After successfully installing `boto3`, create a new `Python` file and name it `python_boto3.py` and import the necessary libraries and write the whole python code as follows;

```
import boto3

# Create a DynamoDB resource object
db = boto3.resource('dynamodb')

# Specify the DynamoDB table
table = db.Table('graduates')

# Add the first item
table.put_item(
    Item = {
        'grad_id': "1001",
        'Name': "John Doe",
        'Gender': "Male",
        'Phone' : "030 903",
        'email' : "johndoe@mail.com",
        'address' : "Nairobi, Kenya"
    }
)

# Add another Item
table.put_item(
    Item = {
        'grad_id': "1002",
        'Name': "Precious Ellie",
        'Gender': "Female",
        'Phone' : "0789 90878",
        'email' : "elliep@mail.com",
        'address' : "NewYork, USA"
    }
)

print ("Data added successfully")
```
On the above code snippet;

`Import` the necessary libraries to provide the interface to work with `AWS services`.

Create a DynamoDB resource object named `db`.

Specify the `DynamoDB` table named `graduates` using the `Table` method of the `DynamoDB resource object`.

Add an item to the `DynamoDB table` using the `put_item` method, providing the item details as a dictionary. 

Finally, set the print function to output `Data added successfully` after effectively adding the item to the `table`. 

To add the data above directly to the DynamoDB database on AWS you created, run the following command on your terminal. _(python then followed by the name of your python file)_ 

```
python python_boto3.py
```
When the data has been uploaded successfully, you should see the data clearly after opening the `graduates` table on AWS DynamoDB. For example,


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/qyftd7xabeooaasvi85o.png)
## Best Practices and Considerations for using Boto3 with DynamoDB
### Configuring AWS IAM with proper Permissions
Since you are responsible for security in the cloud, ensure your AWS IAM users have the minimum permissions necessary to perform their tasks. Use the `Least Privilege Principle` to grant specific permissions only limited to DynamoDB operations and regularly reviewing and updating the IAM policies to ensure alignment with the principle of least privilege. Do not forget to grant temporary access roles rather than long term access to enhance security. 
### Optimizing Performance with Batch Operations
Not only, when you need to add multiple items to a DynamoDB table, use the `BatchWriteItem` operation to reduce the number of HTTP requests and improve performance. With this operation, you can delete up to 20 items in a single request. 

But also, use `BatchGetItem` for retrieving multiple items at once. This will help minimize the number of requests and reduce latency.
### Handling Large Datasets Efficiently 
Use `Parallel Processing` to distribute the load across multiple threads or processes. This operation will reduce the time required to process large amounts of data.
### Managing DynamoDB Capacity and Throughput
Avoid throttling and ensure optimal performance by using Amazon CloudWatch and use Auto-Scaling for your DynamoDB tables to automatically adjust capacity based on traffic patterns. This ensures your table can handle variable workloads without manual intervention. 

Make sure to configure appropriate scaling policies to balance cost and performance. 
### Security considerations
It is a necessity to protect sensitive data on your DynamoDB. Use AWS server side encryption (SSE) with AWS key Management Service (KMS). You can also restrict access to specific items to enhance data security and privacy.
### Data Modelling and Design
Design your DynamoDB tables and indexes to support efficient queries. Plan your data access patterns in advance to avoid inefficient queries by appropriately using primary keys, sort keys and secondary indexes to optimize data retrieval.
## Conclusion
We have explored how to add data to Amazon DynamoDB using Boto3 library in Python which started with the basics of properly installing necessary features and configuring AWS credentials and regions settings. We also explained the Python code in detail and finalized our project by highlighting the best practices to ensure efficient usage of DynamoDB.

To ensure your DynamoDB operations are efficient, scalable and secure, adhere to best practices carefully considering performance. Using Boto3 to manage DynamoDB allows powerful and flexible top tier data handling capabilities.

I encourage you to further explore other Boto3 functionalities to fully leverage its capabilities on DynamoDB for your specific use.