Package: dynamodbgeo
Geo Library for Amazon DynamoDBThis project is an unofficial port ofawslabs/dynamodb-geo, bringing creation and querying of geospatial data to Python developers usingAmazon DynamoDB.FeaturesBox Queries:Return all of the items that fall within a pair of geo points that define a rectangle as projected onto a sphere.Radius Queries:Return all of the items that are within a given radius of a geo point.Basic CRUD Operations:Create, retrieve, update, and delete geospatial data items.Customizable:Access to raw request and result objects from the AWS SDK for python.Installationpipinstalls2spherepipinstallboto3pipinstalldynamodbgeoGetting startedFirst you'll need to import the AWS sdk and set up your DynamoDB connection:importboto3importdynamodbgeoimportuuiddynamodb=boto3.client('dynamodb',region_name='us-east-2')Next you must create an instance ofGeoDataManagerConfigurationfor each geospatial table you wish to interact with. This is a container for various options (see API below), but you must always provide aDynamoDBinstance and a table name.config=dynamodbgeo.GeoDataManagerConfiguration(dynamodb,'geo_test_8')Finally, you should instantiate a manager to query and write to the table using this config object.geoDataManager=dynamodbgeo.GeoDataManager(config)Choosing ahashKeyLength(optimising for performance and cost)ThehashKeyLengthis the number of most significant digits (in base 10) of the 64-bit geo hash to use as the hash key. Larger numbers will allow small geographical areas to be spread across DynamoDB partitions, but at the cost of performance as morequeriesneed to be executed for box/radius searches that span hash keys. Seethese tests from the JS version(TODO) for an idea of how query performance scales withhashKeyLengthfor different search radii.If your data is sparse, a large number will mean more RCUs since more empty queries will be executed and each has a minimum cost. However if your data is dense andhashKeyLengthtoo short, more RCUs will be needed to read a hash key and a higher proportion will be discarded by server-side filtering.From theAWSQuerydocumentationDynamoDB calculates the number of read capacity units consumed based on item size, not on the amount of data that is returned to an application. ...The number will also be the same whether or not you use aFilterExpressionOptimally, you should pick the largesthashKeyLengthyour usage scenario allows. The wider your typical radius/box queries, the smaller it will need to be.Note that theJava versionuses ahashKeyLengthof6by default. The same value will need to be used if you access the same data with both clients.This is an important early choice, since changing yourhashKeyLengthwill mean recreating your data.From theAWSQuerydocumentationDynamoDB calculates the number of read capacity units consumed based on item size, not on the amount of data that is returned to an application. ...The number will also be the same whether or not you use aFilterExpressionOptimally, you should pick the largesthashKeyLengthyour usage scenario allows. The wider your typical radius/box queries, the smaller it will need to be.Note that theJava versionuses ahashKeyLengthof6by default. The same value will need to be used if you access the same data with both clients.This is an important early choice, since changing yourhashKeyLengthwill mean recreating your data.Creating a tableGeoTableUtilhas a static methodgetCreateTableRequestfor helping you prepare aDynamoDB CreateTable requestrequest, given aGeoDataManagerConfiguration.You can modify this request as desired before executing it using AWS's DynamoDB SDK.Example:# Pick a hashKeyLength appropriate to your usageconfig.hashKeyLength=3# Use GeoTableUtil to help construct a CreateTableInput.table_util=dynamodbgeo.GeoTableUtil(config)create_table_input=table_util.getCreateTableRequest()#tweaking the base table parameters as a dictcreate_table_input["ProvisionedThroughput"]['ReadCapacityUnits']=5# Use GeoTableUtil to create the tabletable_util.create_table(create_table_input)Adding data#preparing non key attributes for the item to addPutItemInput={'Item':{'Country':{'S':"Tunisia"},'Capital':{'S':"Tunis"},'year':{'S':'2020'}},'ConditionExpression':"attribute_not_exists(hashKey)"# ... Anything else to pass through to `putItem`, eg ConditionExpression}geoDataManager.put_Point(dynamodbgeo.PutPointInput(dynamodbgeo.GeoPoint(36.879163,10.243120),# latitude then latitude longitudestr(uuid.uuid4()),# Use this to ensure uniqueness of the hash/range pairs.PutItemInput# pass the dict here))See alsoDynamoDB PutItem requestUpdating a specific pointNote that you cannot update the hash key, range key, geohash or geoJson. If you want to change these, you'll need to recreate the record.You must specify aRangeKeyValue, aGeoPoint, and anUpdateItemInput dictmatching theDynamoDB UpdateItemrequest (TableNameandKeyare filled in for you).Note : You must NOT update geoJson and geohash attributes.#define a dict of the item to updateUpdateItemDict={# Dont provide TableName and Key, they are filled in for you"UpdateExpression":"set Capital = :val1","ConditionExpression":"Capital = :val2","ExpressionAttributeValues":{":val1":{"S":"Tunis"},":val2":{"S":"Ariana"}},"ReturnValues":"ALL_NEW"}geoDataManager.update_Point(dynamodbgeo.UpdateItemInput(dynamodbgeo.GeoPoint(36.879163,10.24312),# latitude then latitude longitude"1e955491-d8ba-483d-b7ab-98370a8acf82",# Use this to ensure uniqueness of the hash/range pairs.UpdateItemDict# pass the dict that contain the remaining parameters here))Deleting a specific pointYou must specify aRangeKeyValueand aGeoPoint. Optionally, you can passDeleteItemInputmatchingDynamoDB DeleteItemrequest (TableNameandKeyare filled in for you).# Preparing dict of the item to deleteDeleteItemDict={"ConditionExpression":"attribute_exists(Country)","ReturnValues":"ALL_OLD"# Don't put keys here, they will be generated for you implecitly}geoDataManager.delete_Point(dynamodbgeo.DeleteItemInput(dynamodbgeo.GeoPoint(36.879163,10.24312),# latitude then latitude longitude"0df9742f-619b-49e5-b79e-9fb94279d30c",# Use this to ensure uniqueness of the hash/range pairs.DeleteItemDict# pass the dict that contain the remaining parameters here))Rectangular queriesQuery by rectangle by specifying aMinPointandMaxPoint. You can also pass filtring criteria in a dictionary as explained in the example.NOTE: You cannot add filtring criteria related to the key attributes as they're used in the geo spacial filtring.# Querying a rectangleQueryRectangleInput={"FilterExpression":"Country = :val1","ExpressionAttributeValues":{":val1":{"S":"Italy"},}}print(geoDataManager.queryRectangle(dynamodbgeo.QueryRectangleRequest(dynamodbgeo.GeoPoint(36.878184,10.242358),dynamodbgeo.GeoPoint(36.879317,10.243648),QueryRectangleInput)))Radius queriesQuery by radius by specifying aCenterPointandRadiusInMeter. You can also pass filtring criteria in a dictionary as explained in the example.NOTE:Same as in query rectangle, you cannot add filtring criteria related to the key attributes as they're used in the geo spacial filtring.# Querying 95 meter from the center point (36.879131, 10.243057)QueryRadiusInput={"FilterExpression":"Country = :val1","ExpressionAttributeValues":{":val1":{"S":"Italy"},}}query_reduis_result=geoDataManager.queryRadius(dynamodbgeo.QueryRadiusRequest(dynamodbgeo.GeoPoint(36.879131,10.243057),# center point95,QueryRadiusInput,sort=True))# diameterBatch operationsTODO:Configuration referenceThese are public properties of aGeoDataManagerConfigurationinstance. After creating the config object you may modify these properties.geohashAttributeName: string = "geohash"The name of the attribute storing the full 64-bit geohash. Its value is auto-generated based on item coordinates.hashKeyAttributeName: string = "hashKey"The name of the attribute storing the firsthashKeyLengthdigits (default 2) of the geo hash, used as the hash (aka partition) part of ahash/range primary key pair. Its value is auto-generated based on item coordinates.hashKeyLength: number = 2Seeabove.rangeKeyAttributeName: string = "rangeKey"The name of the attribute storing the range key, used as the range (aka sort) part of ahash/range key primary key pair. Its value must be specified by you (hash-range pairs must be unique).geoJsonAttributeName: string = "geoJson"The name of the attribute which will contain the longitude/latitude pair in a GeoJSON-style point (see alsolongitudeFirst).geohashIndexName: string = "geohash-index"The name of the index to be created against the geohash. Only used for creating new tables.ExampleTODOLimitationsNo composite key supportCurrently, the library does not support composite keys. You may want to add tags such as restaurant, bar, and coffee shop, and search locations of a specific category; however, it is currently not possible. You need to create a table for each tag and store the items separately.Queries retrieve all paginated dataAlthough low levelDynamoDB Queryrequests return paginated results, this library automatically pages through the entire result set. When querying a large area with many points, a lot of Read Capacity Units may be consumed.More Read Capacity UnitsThe library retrieves candidate Geo points from the cells that intersect the requested bounds. The library then post-processes the candidate data, filtering out the specific points that are outside the requested bounds. Therefore, the consumed Read Capacity Units will be higher than the final results dataset. Typically 8 queries are exectued per radius or box search.High memory consumptionBecause all paginatedQueryresults are loaded into memory and processed, it may consume substantial amounts of memory for large datasets.Dataset density limitationThe Geohash used in this library is roughly centimeter precision. Therefore, the library is not suitable if your dataset has much higher density.