Package: b-dynamodb-common
B.DynamoDbCommonA python library that makes it easier to interact with AWS DynamoDB tables.DescriptionThis library extends functionality ofpynamodb,boto3,json, etc. modules
to make it easier to interact with AWS DynamoDB tables. It contains various
useful functionalities:Attributes (PynamoDB custom attributes);Encoders (JSON encoders to work with pynamo/dynamo data types);Models (PynamoDB custom models);Seeds (Various data seeding functions);Utils (Other cool functions);RemarksBiomapasaims to modernise life-science industry by sharing its IT knowledge with other companies and the community.
This is an open source library intended to be used by anyone.
Improvements and pull requests are welcome.Related technologyPython3Boto3AWS DynamoDBPynamoDBAssumptionsThis project assumes you know about DynamoDB service, and you prefer
using PynamoDB ORM to interact with DynamoDB tables.Good Python skills and basis of OOP.Good PynamoDB/DynamoDB skills.Useful sourcesDynamoDB API reference:https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/HowItWorks.API.htmlDynamoDB boto3:https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/dynamodb.htmlPynamoDB ORM documentation:https://pynamodb.readthedocs.io/en/latest/PynamoDB ORM PyPi:https://pypi.org/project/pynamodb/InstallBefore installing this library, ensure you have these tools setup:Python / PipTo install this project from source run:pip install .Or you can install it from a PyPi repository:pip install b-dynamodb-commonUsage & ExamplesThis section shows various examples on how to use this library.AttributesAttributes module. Contains various custom PynamoDB ORM attribtues.Fernet attributeCustom PynamoDB attribute that encrypts data in the database.
Using Fernet algorithmclassUser(Model):SECRET_ENCRYPTION_KEY=b'123456'first_name=FernetAttribute(SECRET_ENCRYPTION_KEY)last_name=FernetAttribute(SECRET_ENCRYPTION_KEY)KMS attributeCustom PynamoDB attribute that encrypts data in the database.
Using AWS KMS key to encrypt/decrypt data.boto_client=boto3.client('kms')kms_key_arn='arn:of:the:custom:kms:key'classUser(Model):first_name=KmsAttribute(boto_client,kms_key_arn)last_name=KmsAttribute(boto_client,kms_key_arn)EncodersEncoders module. Contains various encoding functionality.DynamoDbEncoderCustom JSON encoder to handle DynamoDB data types.data={'key1':'RandomData','key2':OrderedSet([1,2,3]),'key3':Decimal(1.1)}json.dumps(data,cls=DynamoDbEncoder)PynamoDbEncoderCustom JSON encoder to handle PynamoDB ORM and DynamoDB data types.data={'key1':MapAttribute(map_key_1='RandomData'),'key2':OrderedSet([1,2,3]),'key3':Decimal(1.1)}json.dumps(data,cls=PynamoDbEncoder)ModelsModels module. Contains various PynamoDB-based custom models.Permission modelModel that contains permissions attribute.entity=PermissionsModel()entity.pk='PK'entity.permissions=['list','of','permissions']entity.save()# Add more permissions and save.entity.add_permission('permission')entity.save()Model type factoryAllows to use the same pynamodb Model against multiple databases.frompynamodb.modelsimportModel# Create your own model. Example, User model.classUser(Model):pass# Specify table 1 against which an example user will be saved.user_model_table_1=ModelTypeFactory(User).create('user_table_1','eu-west-1')user_model_table_1(hash_key='hash',range_key='range').save()# Specify another table and save user in different table.user_model_table_2=ModelTypeFactory(User).create('user_table_2','eu-east-1')user_model_table_2(hash_key='hash',range_key='range').save()SeedsCurrently this module is empty.UtilsUtilities module. Contains lots of cool functions.List functionWraps PynamoDBqueryandscanfunctions for better management.list_function:PynamoDBListFunction[DummyEntity]=PynamoDBListFunction(DummyEntity.query,'PK')items=list(list_function())# You can also specify a transformer function to transform results before returning.list_function:PynamoDBListFunction[DummyEntity]=PynamoDBListFunction(# PynamoDB list function (query).DummyEntity.query,# Positional arguments.'PK',# Transformer function.transformer=lambdaitem:item.pk)# Will contain only pks.ids=list(list_function())List resultsWraps PynamoDBqueryandscanfunctions to handle recursivelast_evaluated_keytokens.list_function:PynamoDBListFunction[DummyEntity]=PynamoDBListFunction(DummyEntity.scan,limit=10,filter_condition=DummyEntity.pk.is_in([...]))result=PynamoDBListResult(list_function)# Fetch one time.result.fetch(recursive=False)# Check whether all results have been fetched.result.finished# If not, feel free to call it one more time and not worry about last_evaluated_key.result.fetch(recursive=False)# If you want to retrieve absolutely all results in one call:result.fetch(recursive=True)# Simple!TestingThis package has integration tests based onpytest.
To run tests simply run:pytest b_dynamodb_common_test/integration/testsContributionFound a bug? Want to add or suggest a new feature?
Contributions of any kind are gladly welcome.
You may contact us directly, create a pull-request or an issue in github platform.
Lets modernize the world together.Release history0.4.1Improvepynamodb_list_resultclass.0.4.0Model type factory for dynamic table use now supportsIndexes(global or local). Previously if you had e.g. GSI in your model,
you would get a "Missing Meta" error.0.3.0Exposevalidate_permissionsmethod.0.2.0Add model type factory to allow dynamictable_nameandregionspecification.0.1.0Add transformer functions to transform returned results.P.S. still not a stable version.0.0.6Add documentation.0.0.5Add Fernet and KMS attribute tests.0.0.4Add Dynamo and Pynamo encoders tests.0.0.3Add tests to test PermissionModel base class.0.0.2Improvepynamodb_list_resultclass.0.0.1Initial build.