Package: boston-logger
Boston LoggerSmaht logging solutions for Django applications.Basic UsageAdd theRequestResponseLoggerMiddlewaremiddleware in your Django project's
settings file to log all request/responses that Django handles.Importboston_logger.requests_monkey_patch(or set theENABLE_REQUESTS_LOGGINGflag toTrue) to log all requests made through therequestslibrary.For Django settings fileIf you want to configure in settings.py start with this:BOSTON_LOGGER = {
    "ENABLE_OUTBOUND_REQUEST_LOGGING": True,
    "ENABLE_LOGGING_MIDDLEWARE": True,
    "ENABLE_SENSITIVE_PATHS_PROCESSOR": False,
    "ENABLE_REQUESTS_LOGGING": True,  # Enable the requests monkey patch
    "MAX_VERBOSE_OUTPUT_LENGTH": 500,
    "MAX_JSON_DATA_TO_LOG": 0,  # Do not limit json output size, by default
    "MIDDLEWARE_BLOCKLIST": ["admin:index", "swagger-docs"],
    "LOGGER_NAME": "boston_logger",
    "LOG_RESPONSE_CONTENT": False,
    "PREFER_TEXT_FALLBACK_MASKING": False,
    "SHOW_NESTED_KEYS_IN_SENSITIVE_PATHS": False,
}LOGGING = {
    'disable_existing_loggers': True,
    'version': 1,
    'formatters': {
        'json_formatter': {
            '()': 'boston_logger.logger.JsonFormatter',
            'default_extra': {
                '_logging_metadata': {
                    'category': '/app/environment',
                },
            },
        },
        'smart_formatter': {
            '()': 'boston_logger.logger.SmartFormatter',
        },
    },
    'filters': {
        'request_edge': {
            '()': 'boston_logger.logger.RequestEdgeEndFilter',
        },
    },
    'handlers': {
        'console': {
            'class': 'logging.StreamHandler',
            'level': 'DEBUG',
            'formatter': 'smart_formatter',
            'filters': ['request_edge'],
        },
        'json': {
            'class': 'logging.StreamHandler',
            'level': 'DEBUG',
            'formatter': 'json_formatter',
            'filters': ['request_edge'],
        },
    },
    'loggers': {
        '': {
            'handlers': ['console'],
            'level': os.environ.get('DJANGO_LOG_LEVEL', 'INFO'),
        },
        'boston_logger': {
            'handlers': ['json'],
            'level': os.environ.get('DJANGO_LOG_LEVEL', 'DEBUG'),
            'propagate': False,
        },
    },
}All config options, with defaultsENABLE_OUTBOUND_REQUEST_LOGGING: True - Requests lib requests will be
captured (Only if requests has been patched by the monkey patch, orENABLE_REQUESTS_LOGGINGsettings).ENABLE_LOGGING_MIDDLEWARE: True - The django middleware will log START/END
events for incoming requests.ENABLE_SENSITIVE_PATHS_PROCESSOR: False -sensitive_paths.SensitivePathsobjects will mask dataENABLE_REQUESTS_LOGGING: False - Monkey patches requests lib at startupMAX_VERBOSE_OUTPUT_LENGTH: 500 - Character length for request, header, and
response data in SmartFormatter (console logs).MAX_JSON_DATA_TO_LOG: 0 - If greater than zero, truncate json payloads to
this sizeMIDDLEWARE_BLOCKLIST: [admin:index,swagger-docs] - Middleware will not
log requests that match these named URLs, must be a list.LOGGER_NAME:boston_logger- Default name of the logger that all request logs
will be sent to.LOG_RESPONSE_CONTENT: False - Log the json responses the site is sending.PREFER_TEXT_FALLBACK_MASKING: False - If parsing text data to sanitize as a
query string fails, mask the whole value.SHOW_NESTED_KEYS_IN_SENSITIVE_PATHS: False - When True, the objects in a
sensitve path will show keys, but all values will be masked. When False, the
entire object will be replaced with the masked string.Filtering Sensitive DataENABLE_SENSITIVE_PATHS_PROCESSORis set toFalseby default. If enabled, you
must define filtering rules and activate them.Example:importrequestsfromdjango.httpimportJsonResponsefromboston_logger.context_managersimportSensitivePathContextfromboston_logger.sensitive_pathsimportSensitivePaths,add_mask_processor# Global processors are applied to all log messages# A SensitivePaths instance is the only implemented processor at this time.# It will mask json and querystring data that matches the given paths.add_mask_processor("GlobalRuleName",SensitivePaths("obj1/key1","list1"),is_global=True)# Non Global rules must be activated per log messageadd_mask_processor("SpecialRule",SensitivePaths("specific_obj1/key1"),is_global=False)withSensitivePathContext("SpecialRule"):# SpecialRule will apply to logging triggered for this requestrequests.get("https://example.com")# But not this onerequests.get("https://example.com")defdjango_view(request):response=JsonResponse({})# SpecialRule will apply to logging triggered for this responseresponse._apply_mask_processors=["SpecialRule"]returnresponse# "ALL" is a built-in rule which masks the entire payloadwithSensitivePathContext("ALL"):# All request data, response data, and headers will be masked.requests.get("https://example.com")Reducing Log SizeMAX_JSON_DATA_TO_LOGtries to ensure that json log messages don't get beyond
a certain size. If it is set to a non-zero value, we use it as a message
length limit. If the message is too long, we truncate the response data.MiddlewareIf you're using theRequestResponseMiddlewarein your Django application, you
can overrideBOSTON_LOGGER.MIDDLEWARE_BLOCKLISTin your settings.py if you
don't want to generate logs for given URLs. The argument expected is a list of
URLs names. By default it won't log admin and swagger requests.Log JSONResponsesLOG_RESPONSE_CONTENTis set toFalseby default. Setting it toTruewill
add JSON data to the log responses ofdjango.JsonResponse.Adding NotesBoston Logger allows you to add custom notes data to log messages.  This can be
any type of data that is JSON serializable (recommended to be a string or simple
dictionary).To add notes to anINCOMINGrequest (i.e. one handled by the middleware),
simply add an attribute to the WSGI request object called_request_notes; theJsonFormatterwill include it in the JSON log output.Notethat if you are using Django REST Framework, the incomingrequestobject in yourViewSetor view method(s) will be an abstraction of the
original WSGI request.  You'll need to set the attribute onrequest._request,
e.g.:setattr(request._request,'_request_notes','Some extra log data here.')To add notes toOUTGOINGrequests (i.e. you're using therequestslibrary to
send an external request to another service/system), it's recommend you leverage
therequests_monkey_patchfunctionality described above.  This will allow you
to specify anoteskeyword argument which will attach your notes metadata onto
theOUTGOINGlog message emitted by theJsonFormatter:requests.post(url,data,notes='Some extra log data here.')Also, remember that you can always use theRequestLogContextand attach your
notes that way:withRequestLogContext(method='post',notes='Some extra log data here.')aslog_context:response=requests.post(url,data)request=response.requestlog_context.request=requestlog_context.response=response