Package: logging-azure
Python Logging Azure Workspace OMS ExtensionThis package provides an asynchronous solution for uploading application logs to an Azure Log Workspace using their
provided REST API, all supplied as a handler and service extension for python builtinloggingmodule.The service works by instantiating an always-alive (non-daemonized) thread connected to a log request pool in which
logs will be queued, then periodically running through the pool in order to send the requests asynchronously in bulk
using thegrequestspackage. This is all to intend logging safely without interrupting or slowing down the main
process execution as transparently as possible.InstallationFor installing via the distributed package via PyPi:$ pip install logging-azureOr if you which to install from the source, you can checkout the git repository and install usingsetuptools:$ python setup.py installSpecial conditionsInstalling GEventIf you require using this package within a docker distributed application image for example, you will
be required to have an available compiler and necessary libraries in order to buildcythonandgeventneeded forgrequeststo run.If using an Alpine image for example, this can be accomplished with the following:FROMpython:3.7.4-alpine3.9# Install required dependencies for building like git etc.RUNapkadd--virtual.build-depbuild-base[...]# Install the python packageRUNpipinstall--no-cache-dirlogging-azure# Cleanup no-longer required dependencies for a lighter imageRUNapkdel.build-depGEvent monkey patching sslAs this package leverages thegrequestspackage, which itself usesgevent, in several cases,
like running in a debug Flask server, you may need to monkey patchgeventfor things to work correctly.For this you just need to run the followingas early as possiblewithin your application:fromgeventimportmonkeymonkey.patch_all()Usually, you are warned bygeventitself when running your application if this is needed or not done early enough,
so it shouldn't be hard to miss.UsagePackage ConfigurationThe following environment variables are read to configure the extension and are required:AZURE_LOG_CUSTOMER_ID: Customer ID for the Azure Log WorkspaceAZURE_LOG_SHARED_KEY: Customer shared key for the Azure Log WorkspaceAZURE_LOG_DEFAULT_NAME: The default "log type" name to indicate where the logs are stored.
This will be suffixed with "_CL" within the Azure Log Workspace.The following environment variables are read to tweak some parameters of the extension,
they all have default values and therefore are optional:AZURE_LOG_MAX_CONCURRENT_REQUESTS:Default: 10The maximum number of asyncronous requests to handle at once.
Used bygrequestsAZURE_LOG_SEND_FREQUENCY:Default: 5How many seconds the service thread should wait before sending pooled logs.Logging ConfigurationAs you would any other handler, you only require to define a handler using thelogging_azure.handler.AzureLogServiceHandlerclass:[...]"handlers":{"console":{"level":logging.DEBUG,"class":"logging.StreamHandler","formatter":"colorize"},"azure_log_oms":{"level":logging.INFO,"class":"logging_azure.handler.AzureLogServiceHandler","formatter":"azure"},"default":{"level":logging.INFO,"class":"logging.StreamHandler","formatter":"default"},},[...]Then add the handler to your selected logger instance:[...]"loggers":{LOGGER_NAME:{"handlers":["console"]ifIS_LOCAL_DEV_INSTANCEelse["default","azure_log_oms"],"level":LOG_LEVEL,"propagate":True,}},[...]RecommendationsSet an appropriate log level minimumIn order to prevent additional cost from Azure Log OMS ingestion for talkative applications, it is recommended to set
an adequate LOG_LEVEL for your application (avoidlogging.DEBUGfor example).It is also recommended to configure your loggers appropriately so theAzureLogServiveHandlerisn't used during
local development or CI environments for example.You are warned.Set a clear formatter for messagesThis package supplies a decent amount of information already to complement the message in a clean way by reading the
log record in order to facilitate Azure Log OMS Queries and ingestion, so you may only require using a specific
formatter that will only format themessageitself as follows:[...]"formatters":{"default":{"format":'[%(asctime)s]%(levelname)s%(name)s%(module)s"%(message)s"',"datefmt":"%d/%b/%Y %H:%M:%S",},"azure":{"format":'%(message)s',},[...]This provides the following output within the Azure Portal for example:This allows for easily creating custom queries and alerts for your application directly from the Azure Log OMS
solution.Useful linksCreate a Log Analytics workspace in the Azure portalCreate a Log Analytics workspace with Azure CLI 2.0Manage access to log data and workspaces in Azure MonitorAzure Monitor pricingPackage requirementsgrequestsgeventcythoninjector