Package: orjson-ddb
orjson-ddborjson-ddb is a fast DynamoDB native JSON library for Python.
It is a fork/reboot oforjson(from which it inherits thefast performance)
adapted to serialize and deserialize DDB native JSON format in Python.
Compared toboto3DynamoDBTypeDeserializer,
it deserializes DynamoDB response 10x faster and deserialize float numbers (e.g.{"N": "0.13"}) to float instead of Decimal.orjson-ddb supports CPython 3.7, 3.8, 3.9, 3.10, and 3.11. It distributes x86_64/amd64,
aarch64/armv8, and arm7 wheels for Linux, amd64 and aarch64 wheels for macOS,
and amd64 wheels for Windows. orjson-ddb does not support PyPy. Releases
follow semantic versioning and serializing a new object type
without an opt-in flag is considered a breaking change.The repository and issue tracker isgithub.com/MattFanto/orjson-ddb,
and patches may be submitted there. There is aCHANGELOGavailable in the repository.UsageInstallQuickstartDeserializeSerializeTestingPerformanceLatencyMemoryReproducingQuestionsPackagingLicenseUsageInstallTo install a wheel from PyPI:pipinstall--upgrade"pip>=20.3"# manylinux_x_y, universal2 wheel supportpipinstall--upgradeorjson-ddbTo build a wheel, seepackaging.DeserializeThis library exposes a functionloadswhich can deserializer DynamoDB response into a Python dictionary.
This can be used to parse the API response from DynamoDB, as an example if you are using the REST API:importrequestsimportorjson_ddbresponse=requests.post("https://dynamodb.us-east-1.amazonaws.com/",data={"TableName":"some_table","Key":{"pk":{"S":"pk1"},"sk":{"S":"sk1"}}},headers={# some headers})data=orjson_ddb.loads(response.content)The same results can be achieved when using boto3 via a context manager provided by the library:importboto3fromorjson_ddbimportddb_json_parserdynamodb_client=boto3.client("dynamodb",region_name="us-east-1")withddb_json_parser():resp=dynamodb_client.get_item(TableName="some_table",Key={"pk":{"S":"pk1"},"sk":{"S":"sk1"}})print(resp["Items"])# {'sk': 'sk1', 'pk': 'pk1', 'data': {'some_number': 0.123, 'some_string': 'hello'}}This context manager tells boto3 to useorjson_ddb.loadsto deserialize the DynamoDB response.
The output dictionary doesn't contain any reference to the DynamoDB Native format and the result is the same of
what you would get withboto3.resource('dynamodb').Table('some_table').get_item(Key={"pk": "pk1", "sk": "sk1"})["Item"]except for "N" type being translated directly to int or float instead of Decimal.N.B. Unfortunately at the moment it is not possible to use this library withboto3.resource.SerializeSerialization of python dictionary to DynamoDB Native JSON format is not available yet.PerformanceDeserialization performance of orjson-ddb is better than
boto3, dynamodb-json-util. The benchmarks are done on
fixtures of real data converted to DynamoDB native format:twitter.json, 631.5KiB, results of a search on Twitter for "一", containing
CJK strings, dictionaries of strings and arrays of dictionaries, indented.github.json, 55.8KiB, a GitHub activity feed, containing dictionaries of
strings and arrays of dictionaries, not indented.citm_catalog.json, 1.7MiB, concert data, containing nested dictionaries of
strings and arrays of integers, indented.canada.json, 2.2MiB, coordinates of the Canadian border in GeoJSON
format, containing floats and arrays, indented.Latencytwitter.json deserializationLibraryMedian latency (milliseconds)Operations per secondRelative (latency)orjson-ddb2.17459.71boto3-json18.6154.18.57dynamodb-json-util54.1318.424.92citm_catalog.json deserializationLibraryMedian latency (milliseconds)Operations per secondRelative (latency)orjson-ddb4.43240.31boto3-json53.318.612.03dynamodb-json-util57.2717.512.93canada.json deserializationLibraryMedian latency (milliseconds)Operations per secondRelative (latency)orjson-ddb19.22521boto3-json221.834.511.54dynamodb-json-util244.144.112.7ReproducingThe above was measured using Python 3.9.13 on Linux (amd64) with
orjson-ddb 0.1.1, boto3==1.21.27, dynamodb-json==1.3The latency results can be reproduced using thepybenchandgraphscripts.QuestionsWhy can't I install it from PyPI?Probablypipneeds to be upgraded to version 20.3 or later to support
the latest manylinux_x_y or universal2 wheel formats."Cargo, the Rust package manager, is not installed or is not on PATH."This happens when there are no binary wheels (like manylinux) for your
platform on PyPI. You can installRustthroughrustupor a package manager and then it will compile.Will it support PyPy?Probably not.PackagingTo package orjson-ddb requires at leastRust1.57
and thematurinbuild tool. The recommended
build command is:maturinbuild--release--stripIt benefits from also having a C build environment to compile a faster
deserialization backend. See this project'smanylinux_2_28builds for an
example using clang and LTO.The project's own CI tests againstnightly-2022-07-26and stable 1.54. It
is prudent to pin the nightly version because that channel can introduce
breaking changes.orjson-ddb is tested for amd64, aarch64, and arm7 on Linux. It is tested for
amd64 on macOS and cross-compiles for aarch64. For Windows it is tested on
amd64.There are no runtime dependencies other than libc.Tests are included in the source distribution on PyPI. The
requirements to run the tests are specified intest/requirements.txt. The
tests should be run as part of the build. It can be run withpytest -q test.Licenseorjson was written by ijlijl@mailbox.org, copyright 2018 - 2021, licensed under both the Apache 2 and MIT licenses.orjson-ddb was forked from orjson and is maintained by Mattia Fantonimattia.fantoni@gmail.com, licensed same as orjson.