Package: jeffy
Jeffy is Serverless Application Framework for Python AWS Lambda.DescriptionJeffy is Serverless"Application"Framework for Python, which is
suite of Utilities for Lambda functions to make it easy to develop serverless applications.Mainly, Jeffy is focusing on three things.Logging: Providing easy to see JSON format logging. All decorators are capturing all events, responses and errors. And you can configure to inject additional attributes what you want to see to logs.Decorators: To save time to implement common things for Lambda functions, providing some useful decorators and utiliies.Tracing: Traceable events within related functions and AWS services with generating and passingcorrelation_id.Configurable: You can customize the framework settings easily.Logging1.1.Basic Usage1.2.Injecting additional attributes to logs1.3.Change the attribute name of correlation id1.4.Change the log lervelEvent handlers2.1.common2.2.rest_api2.3.sqs2.4.sns2.5.kinesis_streams2.6.dynamodb_streams2.7.s32.8.scheduleSDK3.1.Kinesis Clinent3.2.SNS Client3.3.SQS Client3.4.S3 ClientEncodingValidation5.1.JSONSchemaValidatorInstall$pipinstalljeffyFeatures1.Logging1.1.Basic UsageJeffy logger automatically inject some Lambda contexts to CloudWatchLogs.fromjeffy.frameworkimportget_appapp=get_app()defhandler(event,context):app.logger.info({'foo':'bar'})Output in CloudWatchLogs{"msg":{"foo":"bar"},"aws_region":"us-east-1","function_name":"jeffy-dev-hello","function_version":"$LATEST","function_memory_size":"1024","log_group_name":"/aws/lambda/jeffy-dev-hello","log_stream_name":"2020/01/21/[$LATEST]d7729c0ea59a4939abb51180cda859bf","correlation_id":"f79759e3-0e37-4137-b536-ee9a94cd4f52"}1.2.Injecting additional attributes to logsYou can inject some additional attributes what you want to output with usingupdate_contextmethod.fromjeffy.frameworkimportget_appapp=get_app()app.logger.update_context({'username':'user1','email':'user1@example.com'})defhandler(event,context):app.logger.info({'foo':'bar'})Output in CloudWatchLogs{"msg":{"foo":"bar"},"username":"user1","email":"user1@example.com","aws_region":"us-east-1","function_name":"jeffy-dev-hello","function_version":"$LATEST","function_memory_size":"1024","log_group_name":"/aws/lambda/jeffy-dev-hello","log_stream_name":"2020/01/21/[$LATEST]d7729c0ea59a4939abb51180cda859bf","correlation_id":"f79759e3-0e37-4137-b536-ee9a94cd4f52"}1.3.Change the attribute name of correlation idYou can change the attribute name of correlation id in the setting options.fromjeffy.frameworkimportget_appfromjeffy.settingsimportLoggingapp=get_app(logging=Logging(correlation_attr_name='my-trace-id'))defhandler(event,context):app.logger.info({'foo':'bar'})Output in CloudWatchLogs{"msg":{"foo":"bar"},"aws_region":"us-east-1","function_name":"jeffy-dev-hello","function_version":"$LATEST","function_memory_size":"1024","log_group_name":"/aws/lambda/jeffy-dev-hello","log_stream_name":"2020/01/21/[$LATEST]d7729c0ea59a4939abb51180cda859bf","my-trace-id":"f79759e3-0e37-4137-b536-ee9a94cd4f52"}1.4.Change the log lervelYou can change the log level of Jeffy logger.importloggingfromjeffy.frameworkimportget_appfromjeffy.settingsimportLoggingapp=get_app(logging=Logging(log_level=logging.DEBUG))defhandler(event,context):app.logger.info({'foo':'bar'})2.Event handlersDecorators make simple to implement common lamdba tasks, such as parsing array from Kinesis, SNS, SQS events etc.Here are provided decorators2.1.commoncommondecorator allows you to outputevent,responseand error infomations when you face Exceptionsfromjeffy.frameworkimportget_appapp=get_app()app.logger.update_context({'username':'user1','email':'user1@example.com'})@app.handlers.common()defhandler(event,context):...Error output with auto_logging{"msg":"JSONDecodeError('Expecting value: line 1 column 1 (char 0)')","exec_info":"Traceback (most recent call last):File '/var/task/jeffy/decorators.py', line 41, in wrapperraise eFile '/var/task/jeffy/decorators.py', line 36, in wrapperresult = func(event, context)File '/var/task/handler.py', line 8, in hellojson.loads('s')File '/var/lang/lib/python3.8/json/__init__.py', line 357, in loadsreturn _default_decoder.decode(s)File '/var/lang/lib/python3.8/json/decoder.py', line 337, in decodeobj, end = self.raw_decode(s, idx=_w(s, 0).end())File '/var/lang/lib/python3.8/json/decoder.py', line 355, in raw_decoderaise JSONDecodeError('Expecting value', s, err.value) from None","function_name":"jeffy-dev-hello","function_version":"$LATEST","function_memory_size":"1024","log_group_name":"/aws/lambda/jeffy-dev-hello","log_stream_name":"2020/01/21/[$LATEST]90e1f70f6e774e07b681e704646feec0","correlation_id":"f79759e3-0e37-4137-b536-ee9a94cd4f52"}2.2.rest_apiDecorator for API Gateway event. Automatically get the correlation id from request header and set the correlation id to response header.Default encoding isjeffy.encoding.json.JsonEncoding.fromjeffy.frameworkimportget_appapp=get_app()@app.handlers.rest_api()defhandler(event,context):return{'statusCode':200,'headers':'Content-Type':'application/json','body':json.dumps({'resutl':'ok.'})}Default header name is 'x-jeffy-correlation-id'.
You can change this name in the setting option.fromjeffy.frameworkimportget_appfromjeffy.settingsimportRestApiapp=get_app(rest_api=RestApi(correlation_id_header='x-foo-bar'))@app.handlers.rest_api()defhandler(event,context):return{'statusCode':200,'headers':'Content-Type':'application/json','body':json.dumps({'resutl':'ok.'})}2.3.sqsDecorator for sqs event. Automaticlly parse"event.Records"list from SQS event source to each items for making it easy to treat it inside main process of Lambda.Default encoding isjeffy.encoding.json.JsonEncoding.fromjeffy.frameworkimportget_appapp=get_app()@app.handlers.sqs()defhandler(event,context):returnevent['foo']""""event.Records" list from SQS event source was parsed each itemsif event.Records value is the following,[{'foo': 1},{'foo': 2}]event['foo'] value is 1 and 2, event['correlation_id'] is correlation_id you should pass to next event"""2.4.snsDecorator for sns event. Automaticlly parseevent.Recordslist from SNS event source to each items for making it easy to treat it inside main process of Lambda.Default encoding isjeffy.encoding.json.JsonEncoding.fromjeffy.frameworkimportget_appapp=get_app()@app.handlers.sns()defhandler(event,context):returnevent['foo']""""event.Records" list from SNS event source was parsed each itemsif event.Records value is the following,[{'foo': 1},{'foo': 2}]event['foo'] value is 1 and 2, event['correlation_id'] is correlation_id you should pass to next event"""2.5.kinesis_streamsDecorator for kinesis stream event. Automaticlly parseevent.Recordslist from Kinesis event source to each items and decode it with base64 for making it easy to treat it inside main process of Lambda.Default encoding isjeffy.encoding.json.JsonEncoding.fromjeffy.frameworkimportget_appapp=get_app()@app.handlers.kinesis_streams()defhandler(event,context):returnevent['foo']""""event.Records" list from Kinesis event source was parsed each itemsand decoded with base64 if event.Records value is the following,[<base64 encoded value>,<base64 encoded value>]event['foo'] value is 1 and 2, event['correlation_id'] is correlation_id you should pass to next event"""2.6.dynamodb_streamsDecorator for dynamodb stream event. Automaticlly parseevent.Recordslist from Dynamodb event source to  items for making it easy to treat it inside main process of Lambda.fromjeffy.frameworkimportget_appapp=get_app()@app.handlers.dynamodb_streams()defhandler(event,context):returnevent['foo']""""event.Records" list from Dynamodb event source was parsed each itemsif event.Records value is the following,[{'foo': 1},{'foo': 2}]event['foo'] value is 1 and 2, event['correlation_id'] is correlation_id you should pass to next event"""2.7.s3Decorator for S3 event. Automatically parse body stream from triggered S3 object and S3 bucket and key name to Lambda.This handler requiress3:GetObjectpermission.Default encoding isjeffy.encoding.bytes.BytesEncoding.fromjeffy.frameworkimportget_appapp=get_app()@app.handlers.s3()defhandler(event,context):event['key']# S3 bucket keyevent['bucket_name']# S3 bucket nameevent['body']# Bytes data of the objectevent['correlation_id']# correlation_idevent['metadata']# object matadata2.8.scheduleDecorator for schedule event. just captures correlation id before main Lambda process. do nothing other than that.fromjeffy.frameworkimportsetupapp=setup()@app.handlers.schedule()defhandler(event,context):...3.SDKJeffy has the original wrapper clients of AWS SDK(boto3). The clients automatically injectcorrelation_idin the event payload and encode it to the specified(or default) encoding.3.1.Kinesis ClinentDefault encoding isjeffy.encoding.json.JsonEncoding.fromjeffy.frameworkimportget_appfromjeffy.sdk.kinesisimportKinesisapp=get_app()@app.handlers.kinesis_streams()defhandler(event,context):Kinesis().put_record(stream_name=os.environ['STREAM_NAME'],data={'foo':'bar'},partition_key='your_partition_key')3.2.SNS ClientDefault encoding isjeffy.encoding.json.JsonEncoding.fromjeffy.frameworkimportget_appfromjeffy.sdk.snsimportSnsapp=get_app()@app.handlers.sns()defhandler(event,context):Sns().publish(topic_arn=os.environ['TOPIC_ARN'],subject='hello',message='world')3.3.SQS ClientDefault encoding isjeffy.encoding.json.JsonEncoding.fromjeffy.frameworkimportget_appfromjeffy.sdk.sqsimportSqsapp=get_app()@app.handlers.sqs()defhandler(event,context):Sqs().send_message(queue_url=os.environ['QUEUE_URL'],message='hello world')3.4.S3 ClientDefault encoding isjeffy.encoding.bytes.BytesEncoding.fromjeffy.frameworkimportget_appfromjeffy.sdk.s3importS3app=get_app()@app.handlers.s3()defhandler(event,context):S3().upload_file(file_path='/path/to/file',bucket_name=os.environ['BUCKET_NAME'],key='key/of/object')4.EncodingEach handler and SDK client has a default encoding and automatically encode/decode the data from/to python object. And you can change the encoding.Currently, the encodings you can choose are:jeffy.encoding.bytes.BytesEncodingjeffy.encoding.json.JsonEncodingEach encoding class also hasencodemethods to encodebytesdata into own encoding.fromjeffy.frameworkimportget_appfromjeffy.encoding.bytesimportBytesEncodingfromjeffy.sdk.kinesisimportKinesisapp=get_app()bytes_encoding=BytesEncoding()@app.handlers.kinesis_streams(encoding=bytes_encoding)defhandler(event,context):kinesis=Kinesis(encoding=bytes_encoding)kinesis.put_record(stream_name=os.environ['STREAM_NAME'],data=bytes_encoding.encode('foo'.encode('utf-8)),partition_key='your-partition-key')5.Validation5.1.JSONSchemaValidatorJsonSchemaValidatoris automatically validate event payload with following json schema you define. raiseValidationErrorexception if the validation fails.fromjeffy.frameworkimportget_appfromjeffy.validator.jsonschemaimportJsonSchemaValidatorapp=get_app()@app.handlers.rest_api(validator=JsonSchemaValidator(schema={'type':'object','properties':{'message':{'type':'string'}}}))defhandler(event,context):return{'statusCode':200,'headers':'Content-Type':'application/json','body':json.dumps({'message':'ok.'})}RequirementsPython 3.6 or higherDevelopmentSource hosted atGitHubReport issues/questions/feature requests onGitHub
IssuesPull requests are very welcome! Make sure your patches are well tested.
Ideally create a topic branch for every separate change you make. For
example:Fork the repoCreate your feature branch (git checkout -b my-new-feature)Commit your changes (git commit -am"Added some feature")Push to the branch (git push origin my-new-feature)Create new Pull RequestAuthorsBought up initial idea byMasashi Terui(marcy9114@gmail.com)Created and maintained byServerless Operations, IncCreditsJeffy is inspired by the following products.Lambda PowertoolsDAZN Lambda Powertoolslambda_decoratorsLicenseMIT License (seeLICENSE)