Saravanan666/pypi-org huggingface dataset:
Package: dynamodb-garbage-collector
DynamoDB Garbage CollectorThe DynamoDB Garbage Collector is a Python library that allows you to delete garbage items in DynamoDB tables.Table of ContentsInstallationUsageContributingLicenseInstallationTo install the DynamoDB Garbage Collector, usepip:$pipinstalldynamodb-garbage-collectorUsageThe DynamoDB Garbage Collector currently provides a single function calledpurge_orphan_items, which allows you to delete orphan items in a child table that reference a non-existent item in a parent table. If optional timestamp attributes are provided only will be delete orphan items earlier than a specified maximum time (by default, one hour ago).To usepurge_orphan_items, you need to provide the following parameters:logger: a logger object to log messages during the execution of the function.region: the AWS region where the parent and child tables are located.parent_table: the name of the parent table.child_table: the name of the child table.key_attribute: the name of the key attribute for both tables.child_reference_attribute: the name of the reference attribute in the child table.max_workers(optional): the maximum number of workers to use for concurrent operations. If not provided, a default value of 100 will be used.timestamp_attribute(optional): the name of the attribute that contains the timestamp of the records in the child table. If not provided, timestamp will not be taken into account when deleting items.timestamp_format(optional): the format of the timestamp attribute. If not provided, timestamp will not be taken into account when deleting items.Here is an example of how to use thepurge_orphan_itemsfunction:importloggingfromdynamodb_garbage_collectorimportpurge_orphan_items# Set up the loggerlogging.basicConfig()logger=logging.getLogger()logger.setLevel(logging.INFO)# Set the AWS region where the parent and child tables are locatedregion='eu-west-1'# Set the names of the parent and child tables, and the key and reference attributesparent_table='ParentTable'child_table='ChildTable'key_attribute='id'child_reference_attribute='parentId'# Set the maximum number of workersmax_workers=50# Set the name of the timestamp attribute and the timestamp formattimestamp_attribute='createdAt'timestamp_format='%Y-%m-%dT%H:%M:%S.%fZ'# Call the functionpurge_orphan_items(logger,region,parent_table,child_table,key_attribute,child_reference_attribute,max_workers,timestamp_attribute,timestamp_format)ContributingWe welcome contributions to the DynamoDB Garbage Collector. To contribute, please fork the repository and create a pull request with your changes.LicenseThe DynamoDB Garbage Collector is released under the MIT License.
Package: dynamodb-ghost
DynamoDB GhostDynamoDB Ghost is a small Python library providing the ability to create transient, metadata-preserving copies of DynamoDB tables on AWS.InstallationAs easy as pip.pipinstalldynamodb-ghostUsageThe main intention of DynamoDB Ghost is to be used as a testing aid in cases where we wish to apply a set of tests to an existing table without making modifications to the original (e.g. as part of setup)Using pytests fixtures, a sample test such as:importpytestimportboto3@pytest.fixturedefmy_table():returnboto3.resource('dynamodb').Table('my_table')deftest_table(my_table):assert...insead becomes:importpytestimportboto3fromdynamodb_ghostimportghost@pytest.fixturedefmy_table():withghost(boto3.client('dynamodb'),'my_table')asghost_table:yieldboto3.resource('dynamodb').Table(ghost_table)deftest_table(my_table):assert...ContributingPull requests are welcome. For major changes, please open an issue first to discuss what you would like to change.Please make sure to update tests as appropriate.LicenseMIT
Package: dynamodb-json
# DynamoDB JsonDynamoDB json util to load and dump strings of Dynamodb json format to python object and vise-versa# Installjust use pip:```pip install dynamodb-json```# UseThe dynamodb-json util works the same as json loads and dumps functions:```pythonimport timeimport uuidfrom datetime import datetimefrom decimal import Decimalfrom dynamodb_json import json_util as jsonjson_ = {"MyString": "a","num": 4,"MyBool": False,"my_dict": {"my_date": datetime.utcnow()},"MyNone": None,"MyZero": 0,"myDecimal": Decimal("19.2"),  # converts Decimal to float, load it as float"myLong": long(1938475658493),"MyNestedDict": {"my_other_nested": {"name": "John","surname": "Lennon","MyOtherNone": None,"floaty": float(29.4),"myList": [1, 3, 4, 5, 6, "This Is Sparta!"],"mySet": {1, 3, 4, 5, 6},  # converts set to list, returns as list"myUUID": uuid.uuid4(),  # converts uuid to string, loads it as string"time": time.time()  # converts it to seconds python float, loads it as float}}}dynamodb_json = json.dumps(json_)# {# "my_dict": {"M": {"my_date": {"S": "2017-04-22T14:41:35.780000"}}},# "MyBool": {"BOOL": false}, "MyNone": {"NULL": true},# "MyNestedDict": {#   "M": {"my_other_nested": {#       "M": {"myUUID": {"S": "2f4ad21e098f49b18e22ad209779048b"},#             "surname": {"S": "Lennon"}, "name": {"S": "John"},#             "mySet": {"L": [{"N": "1"}, {"N": "3"}, {"N": "4"}, {"N": "5"}, {"N": "6"}]},#             "floaty": {"N": "29.4"}, "time": {"N": "1492872095.78"},#             "myList": {"L": [{"N": "1"}, {"N": "3"}, {"N": "4"}, {"N": "5"}, {"N": "6"}, {"S": "This Is Sparta!"}]},#             "MyOtherNone": {"NULL": true}}#             }#       }#   },# "myDecimal": {"N": "19.2"}, "num": {"N": "4"},# "MyString": {"S": "a"},# "myLong": {"N": "1938475658493"},# "MyZero": {"N": "0"}# }json.loads(dynamodb_json)# {'my_dict': {'my_date': datetime.datetime(2017, 4, 22, 14, 41, 35, 780000)}, 'MyBool': False, 'MyNone': None,#  'MyNestedDict': {#      'my_other_nested': {'myUUID': '2f4ad21e098f49b18e22ad209779048b',#                          'surname': 'Lennon', 'name': 'John',#                          'mySet': [1, 3, 4, 5, 6],#                          'floaty': 29.4,#                          'time': 1492872095.78,#                          'myList': [1, 3, 4, 5, 6, 'This Is Sparta!'],#                          'MyOtherNone': None#                          }#              },#  'myDecimal': 19.2,#  'num': 4,#  'MyString': 'a',#  'myLong': 1938475658493L,#  'MyZero': 0# }```
Package: dynamodb-mapper
Dynamodb-mapper – a DynamoDB object mapper, based on boto.PresentationDynamoDBis a minimalistic NoSQL engine
provided by Amazon as a part of their AWS product.DynamoDBallows you to stores documents composed of unicode strings or numbers
as well as sets of unicode strings and numbers. Each tables must define a hash
key and may define a range key. All other fields are optional.Dynamodb-mapperbrings a tiny abstraction layer over DynamoDB to overcome some
of the limitations with no performance compromise. It is highly inspired by the
matureMoongoKit projectFull documentation:http://dynamodb-mapper.readthedocs.org/en/latest/Report bugs:https://bitbucket.org/Ludia/dynamodb-mapper/issuesDownload:http://pypi.python.org/pypi/dynamodb-mapperRequirementsBoto = 2.6.0AWS accountHighlightsPython <–> DynamoDB type mappingDeep schema definition and validation withOnctuous(new in 1.8.0)Multi-target transaction (new in 1.6.0)Sub-transactions (new in 1.6.2)Migration engine (new in 1.7.0)Smart conflict detection (new in 1.7.0)Full low-level chunking abstraction forscan,queryandget_batchDefault valuesAuto-inc hash_keyFramework agnosticExample usageWe assume you’ve correctly set your Boto credentials or useddbmock.Quick install$ pip install dynamodb-mapperIf you have not yet configured Boto, you may simply$ export AWS_ACCESS_KEY_ID=<your id key here>
$ export AWS_SECRET_ACCESS_KEY=<your secret key here>First Modelfrom dynamodb_mapper.model import DynamoDBModel


class DoomMap(DynamoDBModel):
    __table__ = u"doom_map"
    __hash_key__ = u"episode"
    __range_key__ = u"map"
    __schema__ = {
        u"episode": int,
        u"map": int,
        u"name": unicode,
        u"cheats": set,
    }
    __defaults__ = {
        "cheats": set([u"Konami"]),
    }Initial Table creationfrom dynamodb_mapper.model import ConnectionBorg

conn = ConnectionBorg()
conn.create_table(DoomMap, 10, 10, wait_for_active=True)Model Usagee1m1 = DoomMap()
e1m1.episode = 1
e1m1.map = 1
e1m1.name = u"Hangar"
e1m1.cheats = set([u"idkfa", u"iddqd", u"idclip"])
e1m1.save()


# Later on, retrieve that same object from the DB...
e1m1 = DoomMap.get(1, 1)

# query all maps of episode 1
e1_maps = DoomMap.query(hash_key=1)

# query all maps of episode 1 with 'map' hash_key > 5
from boto.dynamodb.condition import GT
e1_maps_after_5 = DoomMap.query(
    hash_key=1,
    range_key_condition=GT(5))ContributeWant to contribute, report a but of request a feature ? The development goes on
at Ludia’s BitBucket account:Dynamodb-mapperReport bugs:https://bitbucket.org/Ludia/dynamodb-mapper/issuesFork the code:https://bitbucket.org/Ludia/dynamodb-mapper/overviewDownload:http://pypi.python.org/pypi/dynamodb-mapperOnctuousFull documentation:https://onctuous.readthedocs.org/en/latestReport bugs:https://bitbucket.org/Ludia/onctuous/issuesDownload:http://pypi.python.org/pypi/onctuous
Package: dynamodb-migrator
dynamodb-migratorA library that helps you create and migrate DynamoDB databases.As performant DynamoDB is, that does come with the trade-off of being inflexible. Changing column names or adding secondary indexes is impossible.The recommended approach is to create a new table with the desired properties, and migrate the existing data.This library will help  you do just that.UsageWrite a migration scriptExecute the migration script as a step in the build pipelineAdd to the migration-script as requiredExample Scriptfrommigrator.dynamodb_migratorimportMigratormigrator=Migrator()@migrator.version(1)@migrator.create(AttributeDefinitions=[{'AttributeName':'hash_key','AttributeType':'N'}],TableName='my_new_table',KeySchema=[{'AttributeName':'hash_key','KeyType':'HASH'}],BillingMode='PAY_PER_REQUEST')defv1(created_table):print("Table created using the kwargs provided")print("Note that the keyword-args are passed onto boto as is")print(created_table)@migrator.version(2)@migrator.add_index(AttributeDefinitions=[{'AttributeName':'postcode','AttributeType':'S'}],LocalSecondaryIndexes=[{'IndexName':'string','KeySchema':[{'AttributeName':'customer_nr','KeyType':'HASH'},{'AttributeName':'postcode','KeyType':'RANGE'}],'Projection':{'ProjectionType':'ALL'}}])defv2(created_table):print("Created a new table with the new index")print("Created a DynamoDB stream that sends all updates to the old table to a custom Lambda-function")print("The custom Lambda-function sends all updates to the new table")print(created_table)@NotYetImplemented@migrator.version(3)@migrator.delete_table("first_table")defv3(migrate):print("About to delete table")print("Ensure that all upstream applications point to the new table, before adding this part to the pipeline!")migrate()print("Table deleted")@NotYetImplemented@migrator.version(4)@migrator.convert(lambdaitem->{'id':translate(item.id)})defv4(migrate):print("About to:")print(" - Create new table (first_table_v4)")print(" - Create a AWS Lambda script that will execute the above lambda, and write the result int he new table")print(" - Create DynamoDB Stream on 'first_table' that triggers the new Lambda")print(" - Execute a script that automatically updates all existing data")print("   (This will trigger all data in 'first_table' to be converted and copied into the new table")migrate()print("Table with new data is ready to use")ExamplesSee theexamples-folder.
Package: dynamodb-prettyparser
Dynamodb Pretty ParserTHIS PACKAGE IS NOW PART OF the appsync-tools package Use it insteadhttps://github.com/QuiNovas/appsync-toolsParses Dynamodb responses into a more usable format. Results are returned as a list of dictionaries with the attribute names being the key. Results are parsed recursively.
One use is to return results directly to Appsync from a Lambda function.
Parses all scalar types that Dynamo supportsN (number scalar) - cast as integer or Decimal typesS (string scalar) - stays stringSS (string set scalar) - list of stringsNS (number set scalar) - list of Decimal/integerL (list scalar) - listBOOL (boolean scalar) - booleanB (binary scalar) - string of base64BS (binary set scalar) - list of base64 stringsNULL (null scalar) - NoneM (map scalar) - DictionaryResponses are returned formated as:[{"attributeOneName":{"S":"value1"},"attributeTwoName":{"S":"value2"}},{"attributeOneName":{"S":"value1"},"attributeTwoName":{"S":"value2"}}]Becomes[{"attributeOneName":"value1","attributeTwoName":"value2"},{"attributeOneName":"value1","attributeTwoName":"value2"}]Where each dictionary in the top level array is a separate item.MethodsparseResults(records) -> dict
Arguments:
items – Response from any boto3 call to Dynamodb that returns Items in the responseExamplefromdynamodbPrettyParserimportparseResultsimportboto3dynamodb=boto3.client('dynamodb')response=dynamodb.query(TableName='mytable',KeyConditionExpression='my_attribute = :foo',ExpressionAttributeValues={':foo':'bar'})print(parseResults(response))
Package: dynamodb-python
dynamodb-pythonInstallationpipinstalldynamodb-pythonRequirementsDependenciespython = "^3.9"ddbcerealbotocoreboto3requestsCredentialsYou will first need to set your AWS credentials. Since this library usesboto3under the hood, you can use the same methods as described in theboto3 documentation. In short, AWS looks for credentials in these places:Environment variablesShared credentials file (~/.aws/credentials)AWS config file (~/.aws/config)You cacn also pass a dictionary toDynamoDBclass:fromdynamodb_pythonimportDynamoDBdynamodb=DynamoDB(credentials={"aws_access_key_id":ACCESS_KEY,"aws_secret_access_key":SECRET_KEY,"aws_session_token":SESSION_TOKEN})How to useHaving a table calledtable_name, you can access it like this:fromdynamodb_pythonimportDynamoDBdynamodb=DynamoDB()table=dynamodb.table_nameAnd then you can get an item like this:table.read_item(key={"partition_key":"key","sort_key":"sort_key"})# NOTE that sort_key is optionalOr you can get a list of items with the same key like this:table.read_items(key={"partition_key":"key"})# NOTE that you can also pass in boto3 condition keyOr you can write an item like this:table.write(key={"partition_key":"key","sort_key":"sort_key"},data={"data":"data"})
Package: dynamodb-serialise
DynamoDB value serialisation and deserialisationConvert values from AWS DynamoDB to native Python types.Makes more sensible decisions about numbers and binary values, at the cost of
floating-point precision. Very lightweight.Installationpipinstalldynamodb-serialiseUsageimportdynamodb_serialisedynamodb_serialise.deserialise({"M":{"foo":{"N":"42"},"bar":{"B":"c3BhbQ=="}}})# {'foo': 42, 'bar': b'spam'}dynamodb_serialise.serialise({'foo':42,'bar':b'spam'},bytes_to_base64=True)# {"M": {"foo": {"N": "42"}, "bar": {"B": "c3BhbQ=="}}}
Package: dynamodb_stream_parser
A sample project that exists as an aid to thePython Packaging User Guide’sTutorial on Packaging and Distributing
Projects.This projects does not aim to cover best practices for Python project
development as a whole. For example, it does not provide guidance or tool
recommendations for version control, documentation, or testing.This is the README file for the project.The file should use UTF-8 encoding and be written using ReStructured Text. It
will be used to generate the project webpage on PyPI and will be displayed as
the project homepage on common code-hosting services, and should be written for
that purpose.Typical contents for this file would include an overview of the project, basic
usage examples, etc. Generally, including the project changelog in here is not
a good idea, although a simple “What’s New” section for the most recent version
may be appropriate.
Package: dynamodb-traverse
dynamodb-traverseHigh performance, thread safe, hackable, general purpose traversing tool for AWS DynamoDB based on aioboto3.Why manually traverse dynamodb table?There're tens of ways to consume dynamodb data, for example, dynamodb stream, emr dynamodb connector, kinesis stream... they are good for different use cases. Manual traverse has following benefits comparing to these solutions:Deal with "small data"Schema evolution, table migrationCustom TTL mechanismFull control over offline traversingWork with complicated nosql schemaCross AWS account data replication/transformationIrrelevant use casesSincedynamodb-traverseis not native to AWS, do not use if your use cases like:Real time streamingSimple nosql schema that maps one primary key value to one sort key valueBig data (~TB) workload that requires dedicated emr clustersData backupInstallation/UninstallationPrerequisite: python 3.8+ and aioboto3>=6.4.1 (bleeding edge)Run following command to install requirements:$pipinstallaioboto3Next, install dynamodb-traverse by running:$pipinstalldynamodb-traverseTo uninstall dynamodb-traverse, run:$pipuninstalldynamodb-traverseSetupdynamodb-traverseby default looks at~/.aws/credentialsfor profiles you specified in the client. Make sure you have created profile to access dynamodb.You can specify audit log location when initializing client. By default it writes to/tmp/dynamodb_traverse_xxx.log.We recommend using35as default scan batch size because ofdynamodb limitationsBenchmark (in progress)
Package: dynamodb_utils
A toolchain for Amazon’sDynamoDBto make
common operations (backup, restore backups) easier. Currently contains:dynamodb-dumper- backup tables out of DynamoDB with ease.dynamodb-loader- restore tables dumped bydynamodb-dumperwith ease.All tools are built to work with both the main DynamoDB service and DynamoDB
Local so you can test them out, and also move data easily between production
and your test environment, or between Dynamo regions.Install with:pip install dynamodb_utilsUsage ExamplesYou can get the (hopefully detailed enough) help with e.g.:dynamodb-dumper --helpTo dump a table, with compression:dynamodb-dumper mytable.name --compressTo dump just a few hash-key values from a table (for example, if you want a
restricted subset of data for development):dynamodb-dumper mytable.name --compress --hash-keys 101 104 404To load some compressed dump files on a DynamoDB Local instance running locally
on port 3232:dynamodb-loader mytable.name --host http://localhost:3232 --region localhost --load mytable.name.*.dump.gzThe tools are based uponPynamoDB,
which is based uponbotocore. Therefore,
credentials will be read from one of many places - IAM Role, environment
variables, or a ‘boto file’. See theboto documentationfor
more details.
Package: dynamodm
DynamoDB Object Document Mapper (ODM) for PythonInstallationpipinstalldynamodmUsagefromdynamodmimportDynaModel,FieldclassUser(DynaModel):sub:str=Field(pk=True)name:str=Field(sk=True)age:intuser=User(sub="user",name="John Doe",age=42)asyncdefmain():awaituser.put()# User(sub='user', name='John Doe', age=42)awaitUser.get(pk="user",sk="John Doe")# User(sub='user', name='John Doe', age=42)awaitUser.query(pk="user",sk="John Doe",operator="begins_with")# [User(sub='user', name='John Doe', age=42)]awaitUser.delete(pk="user",sk="John Doe")# NoneFeaturesType hintsAutomatic schema generationAutomatic table creationAutomatic table updatesAutomatic table deletion
Package: dynamodol
dynamodoldynamodb (through boto3) with a simple (dict-like or list-like) interfaceTo install:pip install dynamodolA basic DynamoDb via Boto3 persister demo:>>>s=DynamoDbPersister()>>>k={'key':'777'}# Each collection will happily accept user-defined _key values.>>>v={'val':'bar'}>>>for_keyins:...dels[_key]...>>>kinsFalse>>>len(s)0>>>s[k]=v>>>len(s)1>>>s[k]{'val':'bar'}>>>s.get(k){'val':'bar'}>>>s.get({'not':'a key'},{'default':'val'})# testing s.get with default{'default':'val'}>>>list(s.values())[{'val':'bar'}]>>>kins# testing __contains__ againTrue>>>dels[k]>>>len(s)0>>>s=DynamoDbPersister(table_name='dynamodol2',key_fields=('name',))>>>for_keyins:...dels[_key]>>>len(s)0>>>s[{'name':'guido'}]={'yob':1956,'proj':'python','bdfl':False}>>>s[{'name':'guido'}]{'proj':'python','yob':Decimal('1956'),'bdfl':False}>>>s[{'name':'vitalik'}]={'yob':1994,'proj':'ethereum','bdfl':True}>>>s[{'name':'vitalik'}]{'proj':'ethereum','yob':Decimal('1994'),'bdfl':True}>>>forkey,valins.items():...print(f"{key}:{val}"){'name':'vitalik'}:{'proj':'ethereum','yob':Decimal('1994'),'bdfl':True}{'name':'guido'}:{'proj':'python','yob':Decimal('1956'),'bdfl':False}
Package: dynamo-fuse
dynamo-fuse===========Linux FUSE file system implementation with AWS DynamoDB as the storageInstallation============Simply download the dynamofs.py or install it using `pip install dynamo-fuse`Usage=====1. Install python-fuse:pip install python-fuse2. Create an AWS Dynamo DB table in the region of your choice. The table must have Hash key named `path` and Range key named `name` (case matters, both Strings)3. Define environment variables for AWS key - `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY`. At the moment only the configuration by environment variables is supported.The user with these keys must have read/write access to the specified AWS Dynamo DB table.4. Execute python command to mount the filesystem:python -m dynamofuse.fs <aws region> <dynamo table> <mount point>This will mount the table to the mount point. After that you will be able to execute normal Linux file commands, such as "ls" or "mkdir".Limitations===========**Note**: This project is in its early R&D stage. Various designs and implementation strategies are being tried for file system operationsso any mission-critical usage is not yet recommended. However, the implementation is already almost POSIX-compliant (only 9 out of 1957 from the [fstest](http://www.tuxera.com/community/posix-test-suite/)test suite are failing).However, the behavior during concurrent access to the same files or directories by different clients (different server instances) is not well handled at the moment and may result in inconsistent results.Also, locking is not supported yet.License=======[GNU General Public License, version 3](http://opensource.org/licenses/gpl-3.0.html)
Package: dynamo-json
dynamo-jsonSick of DynamoDB using its own data type descriptors? Swap between DynamoDB and normal JSON!InstallationInstall from PyPI viapip:pip install dynamo_jsonUsageUse as a libraryimportdynamo_jsondynamo_json.marshall({"some":["json","document"]})# {"some": {"L": [{"S": "json"}, {"S": "document"}]}}dynamo_json.unmarshall({"some":{"L":[{"S":"json"},{"S":"document"}]}})# {"some": ["json", "document"]}or as a CLI tool$ dynamo-json '{"my": "json"}'
{"my": {"S": "json"}}

$ dynamo-json '{"my": {"S": "dynamo-json"}}'
{"my": "dynamo-json"}
Package: dynamongo
dynamongois Python ORM/framework-agnostic library for DynamoDB.
It is highly inspired by the PyMongo project.
This documentation attempts to explain everything you need to know to use dynamongo.importdatetimefromdynamongoimportModel,ConnectionfromdynamongoimportIntField,StringField,ListField,EmailField,DateTimeField# This only need be called once. Alternatively, it can be set using env variablesConnection.set_config(access_key_id='<KEY>',secret_access_key='<SECRET>',table_prefix='test-')classUser(Model):__table__='users'__hash_key__='email'email=EmailField(required=True)name=StringField(required=True)year_of_birth=IntField(max_value=2018,min_value=1900)cities_visited=ListField(StringField)created_at=DateTimeField(default=datetime.datetime.now)# store data to DynamoDBjohn=User.save_one({'email':'johndoe@gmail.com','name':'John Doe','year_of_birth':1990,'cities_visited':['Nairobi','New York']})# year_of_birth, cities_visited & created_at are all optionaljane=User.save_one({'email':'jane@gmail.com','name':'Jane Doe'})# Access attribute valuesprint(john.name)# Fetch data from dynamoDBuser=User.get_one(User.email=='johndoe@gmail.com')print(user.to_dict())In short, dynamongo models can be used to easily:validateinput datasaveserialized data to DynamoDBreadand deserialize data from DynamoDBdeleteitems from DynamoDBupdatedata in DynamoDBGet It Now$ pip install dynamongoDocumentationFull documentation is available athttp://dynamongo.readthedocs.io/.RequirementsPython >= 3.5