semvec/pypi-packages huggingface dataset:
ddbimp 0.4
Easily load data from CSV to test out your DynamoDB table design
License: Apache

# dynamodb-dev-importer (ddbimp)

_Easily load data from CSV to test out your DynamoDB table design._

When working with DynamoDB, it is common practice to minimise the number of tables used, ideally down to just one.

Techniques such as sparse indexes and GSI overloading allow a lot of flexibility and efficiency.

Designing a good schema that supports your query patterns can be challenging. Often it is nice to try things out with a small amount of data.
I personally find it convenient to enter data into a spreadsheet and play around with it there.

This utility eases the process of populating a DynamoDB table from a CSV file, exported from a spreadsheet, that follows a specific format common to DynamoDB modelling patterns.

## Install
You can install it with

```
$ pip3 install ddbimp
```

## Run
Assuming table `people(pk:S, sk:S)` is provisioned in your default region.
```
$ ddbimp --table people --skip 1 example.csv
```

### Expected input format

| pk       | sk            | data |            |           |
| -------- | ------------- | ---- | ---------- | --------- | 
| PERSON-1 | sales-Q1-2019 | Alex | jan: 12012 | feb: 1927 |

Your spreadsheet (and exported CSV) should contain columns for:
- pk
- sk
- data (optional)
- anything after those three can contain arbitrary attributes of form `attribute_name: value` i.e. `city: Edinburgh`

Example row:
```
PERSON-1,sales-Q1-2019,Alex,jan: 12012,feb: 1927
```

Will yield an item like this:
```
{
    pk: 'PERSON-1',
    sk: 'sales-Q1-2019',
    data: 'Alex',
    jan: 12012,
    feb 1927
}
```

For a full example CSV, take a look at [example.csv](https://github.com/AlexJReid/dynamodb-dev-importer/blob/master/example.csv).

## Key ideas
- Table consists of partition key `pk: S` and sort key `sk: S` - their meaning varies depending on the item
- A secondary index swaps the sort and partition keys, so the partition key is `sk: S` and sort key `pk: S`
- A final secondary index uses `sk: S` and `data: S` where data is an arbitrary value you might want to search for, the meaning of `data` depends on the item it is part of
- Group items through a shared partition key, store _sub_ items with a sort key e.g. 
    - e.g. `pk:PERSON-1, sk:sales-Q1-2019, jan:12012, feb:1927`

AWS recently [released a preview build of a tool called NoSQL Workbench](https://aws.amazon.com/blogs/aws/nosql-workbench-for-amazon-dynamodb-available-in-preview/). It builds on the above ideas. I've tried it out and it seems pretty good, but I am a luddite and am faster working in a spreadsheet right now. I'd certainly recommend giving it a try.

## Useful resources
- https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/bp-indexes.html
- https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/workbench.html
- https://www.youtube.com/watch?v=6yqfmXiZTlM

## Caveats, TODO
- Uses your default AWS profile
- Region needs to be set
- Make work directly with a Google Sheets via sheets API