Repository: DataDog/dd-trace-py

Issue:
Bug in logs. %(threadName)s prints "Dummy-X"
### Summary of problem

We use `ddtrace-run` to report metrics to Datadog for our https://github.com/Workable apps and during a common minor patch from `1.9.0` to `1.9.1` (or later) we got our loggers broken printing `Dummy-X` in the thread name as follows:

#### ddtrace <=1.9.0

<img width="1537" alt="Screenshot 2023-04-11 at 14 59 26" src="https://user-images.githubusercontent.com/42936184/231159499-8540116e-8d38-41b9-91d6-b1b4ff4cb445.png">


#### ddtrace >=1.9.1

<img width="1536" alt="Screenshot 2023-04-11 at 15 05 56" src="https://user-images.githubusercontent.com/42936184/231159516-9d481fed-6235-493c-93a5-ea3c25cb7bad.png">

We use python's logger with [jsonformatter](https://github.com/madzak/python-json-logger) including `%(threadName)s` in our logs as follows:
```
logger = logging.getLogger(name)
handler = logging.StreamHandler(sys.stdout)
formatter = JsonFormatter(
    {
        "timestamp": "asctime",
        "severity": "levelname",
        "name": "name",
        "message": "message",
        "path": "%(pathname)s:%(lineno)d",
        "thread": "%(threadName)s",
        "ctx": "ctx",
    }
)
handler.setFormatter(formatter)
logger.addHandler(handler)
```

### Which version of dd-trace-py are you using?

I tested with `1.9.1`, `1.9.3` and the latest `1.11.2`. All versions after `1.9.0` seem broken.

### Which version of pip are you using?

* PIP `22.3.1`
* Poetry `1.1.15`

### Which libraries and their versions are you using?

<details>
  <summary>`pip freeze`</summary>
aiohttp==3.8.4
aiosignal==1.3.1
arrow==1.2.3
async-timeout==4.0.2
attrs==22.1.0
bleach==5.0.1
boto3==1.26.18
botocore==1.29.18
bravado==11.0.3
bravado-core==5.17.1
certifi==2022.9.24
cfgv==3.3.1
charset-normalizer==2.1.1
click==8.1.3
commonmark==0.9.1
distlib==0.3.6
distro==1.8.0
docutils==0.19
elastic-transport==8.4.0
elasticsearch==8.6.2
filelock==3.8.0
fire==0.5.0
fqdn==1.5.1
frozenlist==1.3.3
future==0.18.2
gitdb==4.0.10
GitPython==3.1.29
huggingface-hub==0.10.1
identify==2.5.8
idna==3.4
importlib-metadata==5.0.0
iniconfig==1.1.1
isoduration==20.11.0
jaraco.classes==3.2.3
jmespath==0.10.0
jsonpointer==2.3
jsonref==1.0.1
jsonschema==4.17.1
keyring==23.9.3
kube-run @ git+ssh://git@github.com/Workable/kube-run.git@0d8d214465799a968916131f0f87fb38aa7a4c5c
kube-secrets @ git+ssh://git@github.com/Workable/kube-secrets.git@9069b6d8f5c6a1e618af2071bbde74595c56d3c6
markdown-it-py==2.2.0
mdurl==0.1.2
monotonic==1.6
more-itertools==8.14.0
msgpack==1.0.4
multidict==6.0.4
neptune-client==0.16.13
nodeenv==1.7.0
numpy==1.23.5
oauthlib==3.2.2
packaging==21.3
pandas==1.5.2
Pillow==9.3.0
pkginfo==1.8.3
platformdirs==2.5.2
pluggy==1.0.0
pre-commit==2.20.0
psutil==5.9.4
py==1.11.0
Pygments==2.14.0
PyJWT==2.6.0
pyparsing==3.0.9
pyrsistent==0.19.2
pystache==0.5.4
pytest==7.1.3
python-dateutil==2.8.2
pytz==2022.6
PyYAML==5.4.1
readme-renderer==37.2
requests==2.28.2
requests-oauthlib==1.3.1
requests-toolbelt==0.9.1
rfc3339-validator==0.1.4
rfc3986==2.0.0
rfc3987==1.3.8
rich==13.3.1
ruamel.yaml==0.17.21
ruamel.yaml.clib==0.2.6
s3transfer==0.6.0
shell-gpt==0.7.3
simplejson==3.18.0
six==1.16.0
smmap==5.0.0
swagger-spec-validator==3.0.3
tabulate==0.8.10
termcolor==2.2.0
toml==0.10.2
tomli==2.0.1
tqdm==4.64.1
twine==4.0.1
typer==0.7.0
typing_extensions==4.4.0
uri-template==1.2.0
urllib3==1.26.12
virtualenv==20.16.6
webcolors==1.12
webencodings==0.5.1
websocket-client==1.4.2
yarl==1.8.2
zipp==3.8.1
</details>

### How can we reproduce your problem?

Include `"%(threadName)s"` attribute in your logs and check the name running a python script using ddtrace-run wrapper. In our case we run a Kafka consumer as follows:
```
poetry run ddtrace-run python porch/kafka/consumers/jobs_consumer.py
```
You can also verify the issue removing `ddtrace-run` from the command as follows:
```
poetry run python porch/kafka/consumers/jobs_consumer.py
```

### What is the result that you get?

As explained, the `threadName` in our logs prints "Dummy-1", "Dummy-2", etc instead of the expected thread names:
<img width="157" alt="Screenshot 2023-04-11 at 15 26 37" src="https://user-images.githubusercontent.com/42936184/231161832-c7e47912-5e63-4687-ac14-1e6089a59dd2.png">


### What is the result that you expected?

The thread names I get when I skip `ddtrace-run` from the command. For example `MainThread` or `JobsConsumer-1`:

<img width="215" alt="Screenshot 2023-04-11 at 15 26 27" src="https://user-images.githubusercontent.com/42936184/231161846-0d58c654-9648-413e-9121-4207c6999a57.png">





Discussion:
Hi @antonakospanos, thanks for reaching out! 

Does your application use gevent by any chance? We introduced a major gevent-related change in 1.9.1 that fixes how ddtrace works with gevent, which includes keeping gevent-patched and non-gevent patched versions of libraries, so this could be why you're seeing the thread names revert to their gevent DummyThread names. Could you try running your application with `DD_UNLOAD_MODULES_FROM_SITECUSTOMIZE=false` to see if this resolves your issue?
Reproducible with
~~~python
import logging
from threading import Thread
import sys
from jsonformatter import JsonFormatter


logger = logging.getLogger(__name__)
handler = logging.StreamHandler(sys.stdout)
formatter = JsonFormatter(
    {
        "timestamp": "asctime",
        "severity": "levelname",
        "name": "name",
        "message": "message",
        "path": "%(pathname)s:%(lineno)d",
        "thread": "%(threadName)s",
        "ctx": "ctx",
    }
)
handler.setFormatter(formatter)
logger.addHandler(handler)



def target(i):
	logger.error(f"Hello from {i}")


ts = []
for _ in range(10):
	t = Thread(target=target, args=(_,))
	t.start()
	ts.append(t)



for t in ts:
	t.join()
~~~
and running with
~~~console
DD_UNLOAD_MODULES_FROM_SITECUSTOMIZE=1 ddtrace-run python test.py
~~~
The internal unloading mechanism keeps the original import of `logging`. Including this module hierarchy in the clean-up process (that is, removing the exclusion on `logging` in the injected sitecustomize script) seems to fix this issue.

Resolving patch:
diff --git a/ddtrace/bootstrap/sitecustomize.py b/ddtrace/bootstrap/sitecustomize.py
--- a/ddtrace/bootstrap/sitecustomize.py
+++ b/ddtrace/bootstrap/sitecustomize.py
@@ -7,6 +7,7 @@
 
 LOADED_MODULES = frozenset(sys.modules.keys())
 
+from functools import partial  # noqa
 import logging  # noqa
 import os  # noqa
 from typing import Any  # noqa
@@ -17,6 +18,7 @@
 from ddtrace.debugging._config import config as debugger_config  # noqa
 from ddtrace.internal.compat import PY2  # noqa
 from ddtrace.internal.logger import get_logger  # noqa
+from ddtrace.internal.module import ModuleWatchdog  # noqa
 from ddtrace.internal.module import find_loader  # noqa
 from ddtrace.internal.runtime.runtime_metrics import RuntimeWorker  # noqa
 from ddtrace.internal.utils.formats import asbool  # noqa
@@ -151,6 +153,14 @@ def drop(module_name):
             if m == u or m.startswith(u + "."):
                 drop(m)
 
+    # Because we are not unloading it, the logging module requires a reference
+    # to the newly imported threading module to allow it to retrieve the correct
+    # thread object information, like the thread name. We register a post-import
+    # hook on the threading module to perform this update.
+    @partial(ModuleWatchdog.register_module_hook, "threading")
+    def _(threading):
+        logging.threading = threading
+
 
 try:
     from ddtrace import tracer
@@ -179,7 +189,6 @@ def drop(module_name):
             from ddtrace.appsec.iast._ast.ast_patching import _should_iast_patch
             from ddtrace.appsec.iast._loader import _exec_iast_patched_module
             from ddtrace.appsec.iast._taint_tracking import setup
-            from ddtrace.internal.module import ModuleWatchdog
 
             setup(bytes.join, bytearray.join)
 


Test patch:
diff --git a/tests/tracer/test_logger.py b/tests/tracer/test_logger.py
--- a/tests/tracer/test_logger.py
+++ b/tests/tracer/test_logger.py
@@ -1,6 +1,7 @@
 import logging
 
 import mock
+import pytest
 
 from ddtrace.internal.logger import DDLogger
 from ddtrace.internal.logger import get_logger
@@ -378,3 +379,24 @@ def get_key(record):
         for record in (record3, record4, record5, record6):
             bucket = buckets[get_key(record)]
             self.assertEqual(bucket.skipped, 0)
+
+
+@pytest.mark.subprocess(
+    ddtrace_run=True,
+    env={"DD_UNLOAD_MODULES_FROM_SITECUSTOMIZE": "true"},
+    out=lambda _: "MyThread - ERROR - Hello from thread" in _ and "Dummy" not in _,
+)
+def test_logger_no_dummy_thread_name_after_module_cleanup():
+    import logging
+    import sys
+    from threading import Thread
+
+    logger = logging.getLogger(__name__)
+    handler = logging.StreamHandler(sys.stdout)
+    formatter = logging.Formatter("%(asctime)s - %(threadName)s - %(levelname)s - %(message)s")
+    handler.setFormatter(formatter)
+    logger.addHandler(handler)
+
+    t = Thread(target=logger.error, args=("Hello from thread",), name="MyThread")
+    t.start()
+    t.join()