Package: eib-aws-utils
eib-aws-utilsHelper used by Engie Impact Sustainability Solution EMEAI to develop services based on AWS.This package is in alpha versionBackground jobsFor lambdas and batch background job, meaning not exposed though API-Gateway, you can use
theentry_pointdecorator to manage the error handling and to configure the logging.
This decorator must be used only on the main handler.
It will:configure the standard logging package to use JSON format (see).catch all exception and log them correctly.Usagefromeib_aws_utils.backgound_jobimportentry_point@entry_point("my_package_name")defmy_handler(event,context):passAPI-EndpointsLambda exposing data behind API-Gateway can use thehttp_endpointdecorator.
This decorator manage the error handling, the logging configuration and the serialization of the output.Error handlingThere is two category of error.TheBackendErrorwho are internal errors.
Meaning errors occurring inside the application and don't have any link with the end users.
For example, missing data on an external source. These errors should be catch somewhere in the application.
If they are not catch then the decorator will log the error and return a500 - Internal Server Errorto the end user.TheClientErrorare error that must be return to the end user.
They will be catch by the decorator and logged as warning.
The error will then be serialized and returned as a JSON.
For example, if the resource didn't exist you can raise aNotFoundError.Logging configurationThe decorator will use theconfigure_loggingfunction to configure properly the logging.
For the detailssee the logging section.SerializationThe decorator also manage the serialisation of the lambda output to match the api-gateway format.
Your function can return only the body or a tuple with the body and the http status code.The body can be an object or a dictionary. If it's an object then it must have ato_dict()function
that return a dictionary representation of the object.
If it's a dictionary then it will be serialized as a JSON string.
In addition to the python basic data types, the dictionary can contain the following types:Decimal -> floatdatetime -> ISO stringClientError -> dictionaryobject withto_dictfunction -> dictionaryUsagefromeib_aws_utils.apiimporthttp_endpoint@http_endpoint("my_package_name")defmy_handler(event,context):passLogging configurationThis library also manage the logging configuration.Theconfigure_loggingfunction will configure the standard logging package to use JSON format.
In addition, it get theaws_request_idfrom the context to add it in every log.
This allow use to track every logs that belong to the same run.The root level of logging will be set toINFOand for the package gave in parameter
it will use the level defined by theLOGGING_LEVELenvironment variable.This function must be used at the beginning of the handler if you don't use one of the decorator.
Otherwise the decorator manage the configuration for you.Usagefromeib_aws_utils.loggingimportconfigure_loggingdefmy_handler(event,context):configure_logging("my_package_name",context)DynamoDB UtilsFloat compatible resourceThe dynamoDB serializer can be changed to map Decimal values to float. And the reverse.
By patching the default serializer you can work with float for decimal values instead of Decimal.For this you can patch yourself the serializer when you create your boto3 resource.
Or you can use thecreate_resource_with_float_serializerfunction to get the patched version of the resource.UsageIf you patch the dynamo resource manually when you create it:importboto3fromunittest.mockimportpatchfromeib_aws_utils.dynamo_utilsimportFloatSerializer,FloatDeserializerwithpatch("boto3.dynamodb.types.TypeSerializer",new=FloatSerializer),\patch("boto3.dynamodb.types.TypeDeserializer",new=FloatDeserializer):dynamodb=boto3.resource("dynamodb")Or get directly the patched version:fromeib_aws_utilsimportdynamo_utilsdynamodb=dynamo_utils.create_resource_with_float_serializer()Query all results page at onceThequery_all_itemsfunction will iterate over all results page of the query and give you all the items.
You need to provide the DynamoDB resource, the table name and then you can use all the arguments
available with thequeryfunction of the DynamoDB Table.Usagefromeib_aws_utilsimportdynamo_utilsfromboto3.dynamodb.conditionsimportKeyall_customer=dynamo_utils.query_all_items(dynamodb_resource,"my-table-name",KeyConditionExpression=Key('partition_key').eq('123456'),ScanIndexForward=False,...)Http UtilsYou can create easily a requests session with the retry policy and
API key authentication with thecreate_requests_sessionfunction.
You can pass the API key in parameter. This key will be set in thex-api-keyheader.
You can also provide other headers in the parameters, configure the exponential backoff or change the default timeout.
The default timeout can always be override on each call.Usagefromeib_aws_utilsimporthttp_utils# The most simple session with authenticationsession=http_utils.create_requests_session(api_gateway_key="MY-API-KEY")# A more complex examplesession=http_utils.create_requests_session(headers={"Accept":"application/json"},api_gateway_key="MY-API-KEY",max_retry=25,backoff_factor=1.5,timeout=30)Change logs2.0.0Migration to python 3.91.0.5New backend error for invalid event:InvalidEventError1.0.4http_utils: improve sessionAdd a timeout on http callAdd retry on connection issue1.0.3Fix dynamoquery_all_items1.0.2Fix dynamo patching1.0.1Improve function wrapping in the decorators1.0.0Breaking changesDynamo_serializer has been renamed to dynamo_utils.The entry_point and http_endpoint decorator need themain_module_nameparameter.
And they configure the logging automatically.New featuresCreation of float compatible DynamoDB resource trough dynamo_utils module.Query all result pages trough dynamo_utils module.Get requests session with retry policy and api-key module.Generate a context id if there is no aws_request_id in the logging context.0.0.3Bug fixes0.0.2Bug fixes