Package: sap-cf-logging
This is a collection of support libraries for Python applications running on Cloud Foundry that
serve two main purposes: provide (a) means to emit structured application log messages and (b)
instrument web applications of your application stack to collect request metrics.For details on the concepts and log formats, please look at the sibling project forjava logging
support.FeaturesLightweight, no dependencies. Support of Python 2.7 & 3.5.Compatible with the Pythonloggingmodule. Minimal configuration needed.Emits JSON logs (format
details).Supportscorrelation-id.Supports request instrumentation. Built in support for:Flask 0.1xSanic 0.5.xFalconDjangoExtensible to support othersIncludes CF-specific information (space id, app id, etc.) to logs.Supports adding extra properties to JSON log object.InstallationInstall the package with pip:pip install sap_cf_loggingUsageSetting up your applicationLogging library needs to be initialized. Depending on you application type, different initialization
is used. You should usually do this in your application entrypoint.For CLI applications you just need to callcf_logging.init()onceto configure the library.
The library will try to configure future loggers to emit logs in JSON format.If you are using one of the supported frameworks, check theConfigurationsection to see how to configure it.Setting up the CloudFoundry environmentIn order for your logs to appear in the Kibana dashboard, you have to create anapplication-logsservice instance and bind it to your application.ConfigurationAfter installation use the following guide to configure the Python cf logging library.FlaskFirst import thecf_logginglibrary and setup Flask logging on the application.fromsap.cf_loggingimportflask_loggingapp=flask.Flask(__name__)flask_logging.init(app,logging.INFO)Next use Python’s logging library@app.route('/')defroot_route():logger=logging.getLogger('my.logger')logger.info('Hi')return'ok'Note the logs generated by the applicationSanicimportsanicimportloggingfromsanic.responseimportHTTPResponsefromsap.cf_loggingimportsanic_loggingfromsap.cf_logging.core.constantsimportREQUEST_KEYapp=sanic.Sanic('test.cf_logging')sanic_logging.init(app)@app.route('/')asyncdeftwo(request):extra={REQUEST_KEY:request}logging.getLogger('my.logger').debug('Hi',extra=extra)returnHTTPResponse(body='ok')Note: With Sanic you need to pass the request with anextraparameter in the logging API.
This is needed in order to get thecorrelation_idgenerated at the beginning of the request or
fetched from the HTTP headers.Falconimportfalconfromsap.cf_loggingimportfalcon_loggingfromsap.cf_logging.core.constantsimportREQUEST_KEYclassResource:defon_get(self,req,resp):extra={REQUEST_KEY:req}logging.getLogger('my.logger').log('Resource requested',extra=extra)resp.media={'name':'Cloud Foundry'}app=falcon.API(middleware=[falcon_logging.LoggingMiddleware()])app.add_route('/resource',Resource())falcon_logging.init(app)Djangodjango-adminstartprojectexample# example/settings.pyMIDDLEWARES=[# ...,'sap.cf_logging.django_logging.LoggingMiddleware']# example/wsgi.py# ...fromsap.cf_loggingimportdjango_loggingos.environ.setdefault("DJANGO_SETTINGS_MODULE","sap_logtesting.settings")django_logging.init()# ...Create a new apppythonmanage.pystartappexample_app# example_app/views.pyimportloggingfromdjango.httpimportHttpResponsefromsap.cf_logging.core.constantsimportREQUEST_KEYdefindex(request):extra={REQUEST_KEY:request}logger=logging.getLogger('my.logger')logger.info("Resource requested",extra=extra)returnHttpResponse("ok")# example_app/urls.pyfromdjango.conf.urlsimporturlfrom.importviewsurlpatterns=[url('^$',views.index)]# example/urls.pyfromdjango.contribimportadminfromdjango.conf.urlsimporturl,includeurlpatterns=[url('admin/',admin.site.urls),url('example/',include('example_app.urls'))]Generalimportloggingfromsapimportcf_loggingcf_logging.init()logger=logging.getLogger("cli.logger")logger.info('hi')Notes: All loggers set up and created before the initialization of the Cloud Foundry logging library will
be left untouched. When using Flask and Sanic with the logging library before and
after request middleware is attached, and it will capture response times for each request.Custom FieldsTo use custom fields. Pass a dictionary property custom_fields to the initialize method:importloggingfromsapimportcf_loggingcf_logging.init(custom_fields={"foo":"default","bar":None})Here we mark the two fields: foo and bar as custom_fields. Logging with:logging.getLogger('my.logger').debug('Hi')The property foo will be output as a custom field with a value “default”. The property bar will not be logged, as it does not have a value.To log bar, provide a value when logging:logging.getLogger('my.logger').debug('Hi',extra={"bar":"new_value"})It is also possible to log foo with a different value:logging.getLogger('my.logger').debug('Hi',extra={"foo":"hello"})Setting and getting correlation IDWhen using cf_logging in a web application you don’t need to set the correlation ID, because the logging library will fetch it from the HTTP headers and set it.
For non web applications you could set the correlation ID manually, so that the log entries can be filtered later on based on thecorrelation_idlog property.
In this case the correlation ID is kept in a thread local variable and each thread should set its own correlation ID.Setting and getting the correlation_id can be done via:cf_logging.FRAMEWORK.context.get_correlation_id()cf_logging.FRAMEWORK.context.set_correlation_id(value)If you need to get the correlation ID in a web application, take into account the framework you are using.
In async frameworks like Sanic and Falcon the context is stored into the request object and you need to provide the request to the call:cf_logging.FRAMEWORK.context.get_correlation_id(request)Logging sensitive dataThe logging library does not log sensitive fields by default. Those fields are replaced with ‘redacted’ instead of their original content.
The following fields are considered sensitive data:remote_ip,remote_host,remote_port,x_forwarded_for,remote_user,referer.
Logging of all or some of these fields can be activated by setting the following environment variables:Environment variableValueEnables sensitive fieldLOG_SENSITIVE_CONNECTION_DATAtrueremote_ip,remote_host,remote_port,x_forwarded_forLOG_REMOTE_USERtrueremote_userLOG_REFERERtruerefererThis behavior matches the corresponding mechanism in theCF Java Logging Support library.ExamplesFor more examples please see the tests within the./tests/directory.RequirementsNo external requirements are needed to run the package.LimitationsNAKnown IssuesNAHow to obtain supportPlease open an issue on the github page.ContributingPlease create a pull request and briefly describe the nature of the change. Please submit a test
case along with your pull request.To-Do (upcoming changes)NAChangelogSeeCHANGELOG file.LicenseCopyright (c) 2017-2021 SAP SE or an SAP affiliate company and cf-python-logging-support contributors.  Please see ourLICENSE filefor copyright and license information. Detailed information including third-party components and their licensing/copyright information is availablevia the REUSE tool.