Repository: DataDog/dd-trace-py

Issue:
botocore gets monkey patched before gevent when using pynamoDB
In [0.43.0 ssl libs are patched on import](https://github.com/DataDog/dd-trace-py/pull/1629) to allow `ddtrace-run` and `gevent` to exist in harmony.

`pynamodb` imports `botocore` and PynamoDB is patched by default. The result of this is that `ddtrace-run` ends up monkey patching `botocore` before `gevent` does.

I believe PynamoDB should be listed in the SSL libs that only get patched on import.

### Which version of dd-trace-py are you using?
0.43.0

### Which version of the libraries are you using?

ddtrace==0.43.0
gevent==20.9.0
greenlet==0.4.17
gunicorn==20.0.4
pynamodb==4.3.3

### How can we reproduce your problem?

1. Create new virtualenv
```
$ mkdir temp
$ cd temp
$ virtualenv .
$ . ./bin/active
```

2. Install libs
```
pip install ddtrace gunicorn[gevent] pynamodb
```

3. Create empty `app.py`
```
import time
while True:
    time.sleep(1)
```

Run the failing command
```
 ddtrace-run gunicorn -k gevent app
```

The following warning is displayed, which will turn into a SSL recursion error if you try and use urllib3.

```
$ ddtrace-run gunicorn -k gevent app
- DATADOG TRACER DIAGNOSTIC - Agent not reachable. Exception raised: [Errno 61] Connection refused
- DATADOG TRACER DIAGNOSTIC - Agent not reachable. Exception raised: [Errno 61] Connection refused
[2020-10-12 16:46:09 +1100] [69996] [INFO] Starting gunicorn 20.0.4
[2020-10-12 16:46:09 +1100] [69996] [INFO] Listening at: http://127.0.0.1:8000 (69996)
[2020-10-12 16:46:09 +1100] [69996] [INFO] Using worker: gevent
[2020-10-12 16:46:09 +1100] [70004] [INFO] Booting worker with pid: 70004
/private/tmp/venv/lib/python3.7/site-packages/gunicorn/workers/ggevent.py:53: MonkeyPatchWarning: Monkey-patching ssl after ssl has already been imported may lead to errors, including RecursionError on Python 3.6. It may also silently lead to incorrect behaviour on Python 3.7. Please monkey-patch earlier. See https://github.com/gevent/gevent/issues/1016. Modules that had direct imports (NOT patched): ['botocore.httpsession (/private/tmp/venv/lib/python3.7/site-packages/botocore/httpsession.py)', 'urllib3.util.ssl_ (/private/tmp/venv/lib/python3.7/site-packages/urllib3/util/ssl_.py)', 'urllib3.util (/private/tmp/venv/lib/python3.7/site-packages/urllib3/util/__init__.py)'].
  monkey.patch_all()
```

Disable pynamodb tracing to fix
```
DD_TRACE_PYNAMODB_ENABLED=False ddtrace-run gunicorn -k gevent app
```

Which gives the following output
```
$ DD_TRACE_PYNAMODB_ENABLED=False ddtrace-run gunicorn -k gevent app
- DATADOG TRACER DIAGNOSTIC - Agent not reachable. Exception raised: [Errno 61] Connection refused
- DATADOG TRACER DIAGNOSTIC - Agent not reachable. Exception raised: [Errno 61] Connection refused
[2020-10-12 16:48:11 +1100] [70038] [INFO] Starting gunicorn 20.0.4
[2020-10-12 16:48:11 +1100] [70038] [INFO] Listening at: http://127.0.0.1:8000 (70038)
[2020-10-12 16:48:11 +1100] [70038] [INFO] Using worker: gevent
[2020-10-12 16:48:11 +1100] [70046] [INFO] Booting worker with pid: 70046
```



Discussion:

Resolving patch:
diff --git a/ddtrace/monkey.py b/ddtrace/monkey.py
--- a/ddtrace/monkey.py
+++ b/ddtrace/monkey.py
@@ -85,6 +85,7 @@
     "requests": ("requests",),
     "botocore": ("botocore",),
     "elasticsearch": ("elasticsearch",),
+    "pynamodb": ("pynamodb",),
 }
 
 


Test patch:
diff --git a/tests/contrib/gevent/test_tracer.py b/tests/contrib/gevent/test_tracer.py
--- a/tests/contrib/gevent/test_tracer.py
+++ b/tests/contrib/gevent/test_tracer.py
@@ -457,6 +457,7 @@ def test_ddtracerun(self):
         import botocore  # noqa
         import requests  # noqa
         import elasticsearch  # noqa
+        import pynamodb  # noqa
 
         p = subprocess.Popen(
             ["ddtrace-run", "python", "tests/contrib/gevent/monkeypatch.py"],