Repository: hynek/structlog

Issue:
Is there a way to do threadlocal context for only certain binds?
I looked at the `threadlocal` module to solve a problem I have, but from what I can tell, it has undesirable behavior in that it changes all my existing logging code. From what I can tell, all binds become thread-global. But what I want is: "bind this request user and request ID to all messages during this request", and have all the rest of my existing logging code work as it would otherwise.


Comments:

Patch:
diff --git a/CHANGELOG.rst b/CHANGELOG.rst
index 84961d0..dfe1236 100644
--- a/CHANGELOG.rst
+++ b/CHANGELOG.rst
@@ -51,7 +51,9 @@ Changes:
   So far, the configuration proxy, ``structlog.processor.TimeStamper``, ``structlog.BoundLogger``, ``structlog.PrintLogger`` and ``structlog.dev.ConsoleLogger`` have been made pickelable.
   Please report if you need any another class ported.
   `#126 <https://github.com/hynek/structlog/issues/126>`_
-
+- Added a new thread-local API that allows binding values to a thread-local context explicitly without affecting the default behavior of ``bind()``.
+  `#222 <https://github.com/hynek/structlog/issues/222>`_,
+  `#225 <https://github.com/hynek/structlog/issues/225>`_,
 
 ----
 
diff --git a/docs/api.rst b/docs/api.rst
index 6e01152..231ee8c 100644
--- a/docs/api.rst
+++ b/docs/api.rst
@@ -77,6 +77,12 @@ API Reference
 
 .. automodule:: structlog.threadlocal
 
+.. autofunction:: merge_threadlocal_context
+
+.. autofunction:: clear_threadlocal
+
+.. autofunction:: bind_threadlocal
+
 .. autofunction:: wrap_dict
 
 .. autofunction:: tmp_bind(logger, **tmp_values)
diff --git a/docs/thread-local.rst b/docs/thread-local.rst
index 4314463..1816210 100644
--- a/docs/thread-local.rst
+++ b/docs/thread-local.rst
@@ -30,7 +30,57 @@ If you are willing to do that, you should stick to it because `immutable state <
 Sooner or later, global state and mutable data lead to unpleasant surprises.
 
 However, in the case of conventional web development, we realize that passing loggers around seems rather cumbersome, intrusive, and generally against the mainstream culture.
-And since it's more important that people actually *use* ``structlog`` than to be pure and snobby, ``structlog`` contains a dirty but convenient trick: thread local context storage which you may already know from `Flask <http://flask.pocoo.org/docs/design/#thread-locals>`_:
+And since it's more important that people actually *use* ``structlog`` than to be pure and snobby, ``structlog`` contains a couple of mechanisms to help here.
+
+
+The ``merge_threadlocal_context`` processor
+-------------------------------------------
+
+``structlog`` provides a simple set of functions that allow explicitly binding certain fields to a global (thread-local) context.
+These functions are :func:`structlog.threadlocal.merge_threadlocal_context`, :func:`structlog.threadlocal.clear_threadlocal`, and :func:`structlog.threadlocal.bind_threadlocal`.
+
+The general flow of using these functions is:
+
+- Use :func:`structlog.configure` with :func:`structlog.threadlocal.merge_threadlocal_context` as your first processor.
+- Call :func:`structlog.threadlocal.clear_threadlocal` at the beginning of your request handler (or whenever you want to reset the thread-local context).
+- Call :func:`structlog.threadlocal.bind_threadlocal` as an alternative to :func:`structlog.BoundLogger.bind` when you want to bind a particular variable to the thread-local context.
+- Use ``structlog`` as normal.
+  Loggers act as the always do, but the :func:`structlog.threadlocal.merge_threadlocal_context` processor ensures that any thread-local binds get included in all of your log messages.
+
+.. doctest::
+
+   >>> from structlog.threadlocal import (
+   ...     bind_threadlocal,
+   ...     clear_threadlocal,
+   ...     merge_threadlocal_context,
+   ... )
+   >>> from structlog import configure
+   >>> configure(
+   ...     processors=[
+   ...         merge_threadlocal_context,
+   ...         structlog.processors.KeyValueRenderer(),
+   ...     ]
+   ... )
+   >>> log = structlog.get_logger()
+   >>> # At the top of your request handler (or, ideally, some general
+   >>> # middleware), clear the threadlocal context and bind some common
+   >>> # values:
+   >>> clear_threadlocal()
+   >>> bind_threadlocal(a=1)
+   >>> # Then use loggers as per normal
+   >>> # (perhaps by using structlog.get_logger() to create them).
+   >>> log.msg("hi")
+   a=1 event='hi'
+   >>> # And when we clear the threadlocal state again, it goes away.
+   >>> clear_threadlocal()
+   >>> log.msg("hi there")
+   event='hi there'
+
+
+Thread-local contexts
+---------------------
+
+``structlog`` also provides thread local context storage which you may already know from `Flask <http://flask.pocoo.org/docs/design/#thread-locals>`_:
 
 Thread local storage makes your logger's context global but *only within the current thread*\ [*]_.
 In the case of web frameworks this usually means that your context becomes global to the current request.
diff --git a/src/structlog/threadlocal.py b/src/structlog/threadlocal.py
index d6eea3a..edbd6cf 100644
--- a/src/structlog/threadlocal.py
+++ b/src/structlog/threadlocal.py
@@ -9,6 +9,7 @@ Primitives to keep context global but thread (and greenlet) local.
 from __future__ import absolute_import, division, print_function
 
 import contextlib
+import threading
 import uuid
 
 from structlog._config import BoundLoggerLazyProxy
@@ -162,3 +163,46 @@ class _ThreadLocalDictWrapper(object):
     def __getattr__(self, name):
         method = getattr(self._dict, name)
         return method
+
+
+_CONTEXT = threading.local()
+
+
+def merge_threadlocal_context(logger, method_name, event_dict):
+    """
+    A processor that merges in a global (thread-local) context.
+
+    Use this as your first processor in :func:`structlog.configure` to ensure
+    thread-local context is included in all log calls.
+    """
+    context = _get_context().copy()
+    context.update(event_dict)
+    return context
+
+
+def clear_threadlocal():
+    """
+    Clear the thread-local context.
+
+    The typical use-case for this function is to invoke it early in
+    request-handling code.
+    """
+    _CONTEXT.context = {}
+
+
+def bind_threadlocal(**kwargs):
+    """
+    Put keys and values into the thread-local context.
+
+    Use this instead of :func:`~structlog.BoundLogger.bind` when you want some
+    context to be global (thread-local).
+    """
+    _get_context().update(kwargs)
+
+
+def _get_context():
+    try:
+        return _CONTEXT.context
+    except AttributeError:
+        _CONTEXT.context = {}
+        return _CONTEXT.context


Test patch:
diff --git a/tests/test_threadlocal.py b/tests/test_threadlocal.py
index be37949..c9b527d 100644
--- a/tests/test_threadlocal.py
+++ b/tests/test_threadlocal.py
@@ -13,7 +13,14 @@ import pytest
 from structlog._base import BoundLoggerBase
 from structlog._config import wrap_logger
 from structlog._loggers import ReturnLogger
-from structlog.threadlocal import as_immutable, tmp_bind, wrap_dict
+from structlog.threadlocal import (
+    as_immutable,
+    bind_threadlocal,
+    clear_threadlocal,
+    merge_threadlocal_context,
+    tmp_bind,
+    wrap_dict,
+)
 
 
 try:
@@ -262,3 +269,42 @@ class TestThreadLocalDict(object):
         The context of a new wrapped class is empty.
         """
         assert 0 == len(D())
+
+
+class TestNewThreadLocal(object):
+    def test_bind_and_merge(self):
+        """
+        Binding a variable causes it to be included in the result of
+        merge_threadlocal_context.
+        """
+        bind_threadlocal(a=1)
+        assert {"a": 1, "b": 2} == merge_threadlocal_context(
+            None, None, {"b": 2}
+        )
+
+    def test_clear(self):
+        """
+        The thread-local context can be cleared, causing any previously bound
+        variables to not be included in merge_threadlocal_context's result.
+        """
+        bind_threadlocal(a=1)
+        clear_threadlocal()
+        assert {"b": 2} == merge_threadlocal_context(None, None, {"b": 2})
+
+    def test_merge_works_without_bind(self):
+        """
+        merge_threadlocal_context returns values as normal even when there has
+        been no previous calls to bind_threadlocal.
+        """
+        assert {"b": 2} == merge_threadlocal_context(None, None, {"b": 2})
+
+    def test_multiple_binds(self):
+        """
+        Multiple calls to bind_threadlocal accumulate values instead of
+        replacing them.
+        """
+        bind_threadlocal(a=1, b=2)
+        bind_threadlocal(c=3)
+        assert {"a": 1, "b": 2, "c": 3} == merge_threadlocal_context(
+            None, None, {"b": 2}
+        )