Repository: hynek/structlog

Issue:
Strip LogRecord.args Attribute After foreign_pre_chain Processors Run in ProcessorFormatter
First off, thank you to everyone involved in maintaining this project!

I love the chained approach it provides to building up a log message context. Great work!

I want to start using a structlog-based formatter to output logs as json for several small to medium size code bases I manage.

In particular, I want to be able to write a `processor` that takes the *keys* from the `LogRecord.args` attribute and insert them into my `event_dict`.

Right now, when the `ProcessorFormatter` class receives a log record from `logging` rather than `structlib`, it sets `LogRecord.args = ()` after generating a message [here](https://github.com/hynek/structlog/blob/0e209d95652c0f9b2943bf0334d74f4964fc7f13/src/structlog/stdlib.py#L519).

This means that none of my downstream processors defined in `foreign_pre_chain` can work with the original `LogRecord.args` keys.

Would the maintainers be open to moving the `record.args = ()` setting to *after* the `foreign_pre_chain` processors run? I have a working version locally and can add a test if it's a PR you would accept.

Speaking more broadly, would you be open to me adding some kind of `migration` docs for projects that currently use the stdlib `logging` package but would like to switch to `structlog`?

Comments:

Patch:
diff --git a/src/structlog/stdlib.py b/src/structlog/stdlib.py
index 86ada28..601a302 100644
--- a/src/structlog/stdlib.py
+++ b/src/structlog/stdlib.py
@@ -469,12 +469,14 @@ class ProcessorFormatter(logging.Formatter):
     :param logger: Logger which we want to push through the ``structlog``
         processor chain. This parameter is necessary for some of the
         processors like `filter_by_level`. (default: None)
-
+    :param bool pass_foreign_args: If `True`, pass a foreign log record's
+        `args` to the event_dict under `positional_args` key. (default: False)
     :rtype: str
 
     .. versionadded:: 17.1.0
     .. versionadded:: 17.2.0 *keep_exc_info* and *keep_stack_info*
     .. versionadded:: 19.2.0 *logger*
+    .. versionadded:: 19.2.0 *pass_foreign_args*
     """
 
     def __init__(
@@ -484,6 +486,7 @@ class ProcessorFormatter(logging.Formatter):
         keep_exc_info=False,
         keep_stack_info=False,
         logger=None,
+        pass_foreign_args=False,
         *args,
         **kwargs
     ):
@@ -495,6 +498,7 @@ class ProcessorFormatter(logging.Formatter):
         # The and clause saves us checking for PY3 in the formatter.
         self.keep_stack_info = keep_stack_info and PY3
         self.logger = logger
+        self.pass_foreign_args = pass_foreign_args
 
     def format(self, record):
         """
@@ -516,6 +520,10 @@ class ProcessorFormatter(logging.Formatter):
             logger = self.logger
             meth_name = record.levelname.lower()
             ed = {"event": record.getMessage(), "_record": record}
+
+            if self.pass_foreign_args:
+                ed["positional_args"] = record.args
+
             record.args = ()
 
             # Add stack-related attributes to event_dict and unset them


Test patch:
diff --git a/tests/test_stdlib.py b/tests/test_stdlib.py
index e9abb08..b34e21a 100644
--- a/tests/test_stdlib.py
+++ b/tests/test_stdlib.py
@@ -455,7 +455,7 @@ def configure_for_pf():
     reset_defaults()
 
 
-def configure_logging(pre_chain, logger=None):
+def configure_logging(pre_chain, logger=None, pass_foreign_args=False):
     """
     Configure logging to use ProcessorFormatter.
     """
@@ -470,6 +470,7 @@ def configure_logging(pre_chain, logger=None):
                     "foreign_pre_chain": pre_chain,
                     "format": "%(message)s [in %(funcName)s]",
                     "logger": logger,
+                    "pass_foreign_args": pass_foreign_args,
                 }
             },
             "handlers": {
@@ -527,6 +528,29 @@ class TestProcessorFormatter(object):
             "hello world. [in test_clears_args]\n",
         ) == capsys.readouterr()
 
+    def test_pass_foreign_args_true_sets_positional_args_key(
+        self, configure_for_pf, capsys
+    ):
+        """
+        Test that when `pass_foreign_args` is `True` we set the
+        `positional_args` key in the `event_dict` before clearing args.
+        """
+        test_processor = call_recorder(lambda l, m, event_dict: event_dict)
+        configure_logging((test_processor,), pass_foreign_args=True)
+        configure(
+            processors=[ProcessorFormatter.wrap_for_formatter],
+            logger_factory=LoggerFactory(),
+            wrapper_class=BoundLogger,
+        )
+
+        positional_args = {"foo": "bar"}
+        logging.getLogger().info("okay %(foo)s", positional_args)
+
+        event_dict = test_processor.calls[0].args[2]
+
+        assert "positional_args" in event_dict
+        assert positional_args == event_dict["positional_args"]
+
     def test_log_dict(self, configure_for_pf, capsys):
         """
         Test that dicts can be logged with std library loggers.