Repository: getsentry/sentry-python

Issue:
Memory leak introduced in 0.12.3
Dear maintainers,

I recently noticed that my application containers started going out of memory fairly quickly. Upon investigation, I found that recent upgrade of sentry-sdk was causing this.

Tested with a few versions of sentry-sdk using Python 3.7.6:
sentry-sdk==0.12.2 -> memory usage stays at ~70 MiB, does not raise

sentry-sdk==0.12.3
sentry-sdk==0.13.5
sentry-sdk @ master
-> memory usage starts from ~70MiB and raises steadily until reaching memory limit and crashes container.

Please see if you can reproduce this behaviour. If not, I'll try to put up a minimal reproducible example, but this should be fairly simple to see. My application does not do anything special really, just receives some millions of messages over MQ and out of those, some few thousands end up getting logged to Sentry due to exceptions occurring. This is using pretty much the default Sentry config, no integrations.

Thanks for your help!


Discussion:
Sorry @mikmatko but I think we definetly need a minimal repro case here. We have been using versions newer than 0.12.3 in production for quite a while and know of a few high-volume enterprise customers who do as well.
Hi @untitaker, please see if are you able to reproduce or pinpoint a possible issue based on this:

main.py:
```python
import asyncio
import logging
from sentry_sdk import init

logger = logging.getLogger(__name__)

logging.basicConfig(level=logging.DEBUG)


async def main():
    init("http://...@.../2")

    for x in range(200):
        logger.exception("foo %s", x)

    await asyncio.sleep(60)


asyncio.run(main())
```

Dockerfile:
```
FROM python:3.7-slim-buster
RUN pip install sentry-sdk==0.12.2
COPY main.py .
```

Build:
```
$ docker build -t mem_issue --no-cache .
Sending build context to Docker daemon  7.868MB
Step 1/3 : FROM python:3.7-slim-buster
 ---> 1521a961d9c2
Step 2/3 : RUN pip install sentry-sdk==0.12.2
 ---> Using cache
 ---> 419d7b9d7357
Step 3/3 : COPY main.py .
 ---> Using cache
 ---> 3f7facbbf13b
Successfully built 3f7facbbf13b
Successfully tagged mem_issue:latest
```

Monitor memory usage with docker stats:
```
$ docker stats
```

Running:
```
$ docker run --name="mem_issue" --memory 50m -it mem_issue python main.py
WARNING: Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap.
DEBUG:asyncio:Using selector: EpollSelector
ERROR:__main__:foo 0
NoneType: None
ERROR:__main__:foo 1
NoneType: None
DEBUG:urllib3.connectionpool:Starting new HTTP connection (1): 10.180.1.5:9000
ERROR:__main__:foo 2
NoneType: None
ERROR:__main__:foo 3
NoneType: None
DEBUG:urllib3.connectionpool:http://10.180.1.5:9000 "POST /api/2/store/ HTTP/1.1" 200 41
DEBUG:urllib3.connectionpool:Resetting dropped connection: 10.180.1.5
ERROR:__main__:foo 4
NoneType: None
DEBUG:urllib3.connectionpool:http://10.180.1.5:9000 "POST /api/2/store/ HTTP/1.1" 200 41
DEBUG:urllib3.connectionpool:Resetting dropped connection: 10.180.1.5
ERROR:__main__:foo 5
NoneType: None
ERROR:__main__:foo 6
NoneType: None
DEBUG:urllib3.connectionpool:http://10.180.1.5:9000 "POST /api/2/store/ HTTP/1.1" 200 41
DEBUG:urllib3.connectionpool:Resetting dropped connection: 10.180.1.5
ERROR:__main__:foo 7
NoneType: None
DEBUG:urllib3.connectionpool:http://10.180.1.5:9000 "POST /api/2/store/ HTTP/1.1" 200 41
DEBUG:urllib3.connectionpool:Resetting dropped connection: 10.180.1.5
ERROR:__main__:foo 8
NoneType: None
DEBUG:urllib3.connectionpool:http://10.180.1.5:9000 "POST /api/2/store/ HTTP/1.1" 200 41
DEBUG:urllib3.connectionpool:Starting new HTTP connection (2): 10.180.1.5:9000
ERROR:__main__:foo 9
NoneType: None
ERROR:__main__:foo 10
``` 
and so forth until last one

``` 
ERROR:__main__:foo 199
NoneType: None
DEBUG:urllib3.connectionpool:Resetting dropped connection: 10.180.1.5
DEBUG:urllib3.connectionpool:http://10.180.1.5:9000 "POST /api/2/store/ HTTP/1.1" 200 41
DEBUG:urllib3.connectionpool:Resetting dropped connection: 10.180.1.5
DEBUG:urllib3.connectionpool:http://10.180.1.5:9000 "POST /api/2/store/ HTTP/1.1" 200 41
```

Memory usage stays below 20 MB.

Now with sentry-sdk==0.13.5
```
$ docker build -t mem_issue --no-cache .
Sending build context to Docker daemon  7.868MB
Step 1/3 : FROM python:3.7-slim-buster
 ---> 1521a961d9c2
Step 2/3 : RUN pip install sentry-sdk==0.13.5
 ---> Running in d5c043553373
Collecting sentry-sdk==0.13.5
  Downloading https://files.pythonhosted.org/packages/b4/48/a38ff26af174ef2d03c5754c93b7a18818951734fdaa53bf3c2b80e497e7/sentry_sdk-0.13.5-py2.py3-none-any.whl (91kB)
Collecting urllib3>=1.10.0
  Downloading https://files.pythonhosted.org/packages/b4/40/a9837291310ee1ccc242ceb6ebfd9eb21539649f193a7c8c86ba15b98539/urllib3-1.25.7-py2.py3-none-any.whl (125kB)
Collecting certifi
  Downloading https://files.pythonhosted.org/packages/b9/63/df50cac98ea0d5b006c55a399c3bf1db9da7b5a24de7890bc9cfd5dd9e99/certifi-2019.11.28-py2.py3-none-any.whl (156kB)
Installing collected packages: urllib3, certifi, sentry-sdk
Successfully installed certifi-2019.11.28 sentry-sdk-0.13.5 urllib3-1.25.7
Removing intermediate container d5c043553373
 ---> c3a9282761e4
Step 3/3 : COPY main.py .
 ---> ec654b0cd1d8
Successfully built ec654b0cd1d8
Successfully tagged mem_issue:latest

$ docker run --name="mem_issue" --memory 50m -it mem_issue python main.py
WARNING: Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap.
DEBUG:asyncio:Using selector: EpollSelector
ERROR:__main__:foo 0
NoneType: None
ERROR:__main__:foo 1
NoneType: None
DEBUG:urllib3.connectionpool:Starting new HTTP connection (1): 10.180.1.5:9000
ERROR:__main__:foo 2
NoneType: None
ERROR:__main__:foo 3
NoneType: None
ERROR:__main__:foo 4
NoneType: None
ERROR:__main__:foo 5
NoneType: None
DEBUG:urllib3.connectionpool:http://10.180.1.5:9000 "POST /api/2/store/ HTTP/1.1" 200 41
DEBUG:urllib3.connectionpool:Resetting dropped connection: 10.180.1.5
ERROR:__main__:foo 6
NoneType: None
ERROR:__main__:foo 7
NoneType: None
ERROR:__main__:foo 8
NoneType: None
ERROR:__main__:foo 9
NoneType: None
ERROR:__main__:foo 10
NoneType: None
ERROR:__main__:foo 11
NoneType: None
DEBUG:urllib3.connectionpool:http://10.180.1.5:9000 "POST /api/2/store/ HTTP/1.1" 200 41
ERROR:__main__:foo 12
NoneType: None
DEBUG:urllib3.connectionpool:Resetting dropped connection: 10.180.1.5
ERROR:__main__:foo 13
NoneType: None
ERROR:__main__:foo 14
NoneType: None
ERROR:__main__:foo 15
NoneType: None
ERROR:__main__:foo 16
NoneType: None
ERROR:__main__:foo 17
NoneType: None
DEBUG:urllib3.connectionpool:http://10.180.1.5:9000 "POST /api/2/store/ HTTP/1.1" 200 41
DEBUG:urllib3.connectionpool:Resetting dropped connection: 10.180.1.5
ERROR:__main__:foo 18
NoneType: None
ERROR:__main__:foo 19
NoneType: None
ERROR:__main__:foo 20
NoneType: None
ERROR:__main__:foo 21
NoneType: None
DEBUG:urllib3.connectionpool:http://10.180.1.5:9000 "POST /api/2/store/ HTTP/1.1" 200 41
DEBUG:urllib3.connectionpool:Resetting dropped connection: 10.180.1.5
ERROR:__main__:foo 22
NoneType: None
ERROR:__main__:foo 23
NoneType: None
ERROR:__main__:foo 24
NoneType: None
ERROR:__main__:foo 25
NoneType: None
ERROR:__main__:foo 26
NoneType: None
ERROR:__main__:foo 27
NoneType: None
DEBUG:urllib3.connectionpool:http://10.180.1.5:9000 "POST /api/2/store/ HTTP/1.1" 200 41
DEBUG:urllib3.connectionpool:Resetting dropped connection: 10.180.1.5

...

NoneType: None
DEBUG:urllib3.connectionpool:http://10.180.1.5:9000 "POST /api/2/store/ HTTP/1.1" 200 41
DEBUG:urllib3.connectionpool:Resetting dropped connection: 10.180.1.5
ERROR:__main__:foo 197
NoneType: None
ERROR:__main__:foo 198
NoneType: None
ERROR:__main__:foo 199
NoneType: None
DEBUG:urllib3.connectionpool:http://10.180.1.5:9000 "POST /api/2/store/ HTTP/1.1" 200 41
DEBUG:urllib3.connectionpool:Resetting dropped connection: 10.180.1.5
DEBUG:urllib3.connectionpool:http://10.180.1.5:9000 "POST /api/2/store/ HTTP/1.1" 200 41
DEBUG:urllib3.connectionpool:Starting new HTTP connection (9): 10.180.1.5:9000
DEBUG:urllib3.connectionpool:http://10.180.1.5:9000 "POST /api/2/store/ HTTP/1.1" 200 41
DEBUG:urllib3.connectionpool:Resetting dropped connection: 10.180.1.5
DEBUG:urllib3.connectionpool:http://10.180.1.5:9000 "POST /api/2/store/ HTTP/1.1" 200 41
DEBUG:urllib3.connectionpool:Resetting dropped connection: 10.180.1.5
DEBUG:urllib3.connectionpool:http://10.180.1.5:9000 "POST /api/2/store/ HTTP/1.1" 200 41
DEBUG:urllib3.connectionpool:Starting new HTTP connection (10): 10.180.1.5:9000
DEBUG:urllib3.connectionpool:http://10.180.1.5:9000 "POST /api/2/store/ HTTP/1.1" 200 41
DEBUG:urllib3.connectionpool:Starting new HTTP connection (11): 10.180.1.5:9000
DEBUG:urllib3.connectionpool:http://10.180.1.5:9000 "POST /api/2/store/ HTTP/1.1" 200 41
DEBUG:urllib3.connectionpool:Starting new HTTP connection (12): 10.180.1.5:9000
DEBUG:urllib3.connectionpool:http://10.180.1.5:9000 "POST /api/2/store/ HTTP/1.1" 200 41
DEBUG:urllib3.connectionpool:Starting new HTTP connection (13): 10.180.1.5:9000
DEBUG:urllib3.connectionpool:http://10.180.1.5:9000 "POST /api/2/store/ HTTP/1.1" 200 41
DEBUG:urllib3.connectionpool:Starting new HTTP connection (14): 10.180.1.5:9000
DEBUG:urllib3.connectionpool:http://10.180.1.5:9000 "POST /api/2/store/ HTTP/1.1" 200 41
DEBUG:urllib3.connectionpool:Resetting dropped connection: 10.180.1.5
DEBUG:urllib3.connectionpool:http://10.180.1.5:9000 "POST /api/2/store/ HTTP/1.1" 200 41
DEBUG:urllib3.connectionpool:Starting new HTTP connection (15): 10.180.1.5:9000
DEBUG:urllib3.connectionpool:http://10.180.1.5:9000 "POST /api/2/store/ HTTP/1.1" 200 41
DEBUG:urllib3.connectionpool:Resetting dropped connection: 10.180.1.5
DEBUG:urllib3.connectionpool:http://10.180.1.5:9000 "POST /api/2/store/ HTTP/1.1" 200 41
DEBUG:urllib3.connectionpool:Resetting dropped connection: 10.180.1.5
DEBUG:urllib3.connectionpool:http://10.180.1.5:9000 "POST /api/2/store/ HTTP/1.1" 200 41
DEBUG:urllib3.connectionpool:Resetting dropped connection: 10.180.1.5
DEBUG:urllib3.connectionpool:http://10.180.1.5:9000 "POST /api/2/store/ HTTP/1.1" 200 41
DEBUG:urllib3.connectionpool:Resetting dropped connection: 10.180.1.5
DEBUG:urllib3.connectionpool:http://10.180.1.5:9000 "POST /api/2/store/ HTTP/1.1" 200 41
DEBUG:urllib3.connectionpool:Resetting dropped connection: 10.180.1.5
DEBUG:urllib3.connectionpool:http://10.180.1.5:9000 "POST /api/2/store/ HTTP/1.1" 200 41
DEBUG:urllib3.connectionpool:Starting new HTTP connection (16): 10.180.1.5:9000
DEBUG:urllib3.connectionpool:http://10.180.1.5:9000 "POST /api/2/store/ HTTP/1.1" 200 41
DEBUG:urllib3.connectionpool:Starting new HTTP connection (17): 10.180.1.5:9000
DEBUG:urllib3.connectionpool:http://10.180.1.5:9000 "POST /api/2/store/ HTTP/1.1" 200 41
DEBUG:urllib3.connectionpool:Starting new HTTP connection (18): 10.180.1.5:9000
DEBUG:urllib3.connectionpool:http://10.180.1.5:9000 "POST /api/2/store/ HTTP/1.1" 200 41
DEBUG:urllib3.connectionpool:Resetting dropped connection: 10.180.1.5
DEBUG:urllib3.connectionpool:http://10.180.1.5:9000 "POST /api/2/store/ HTTP/1.1" 200 41
DEBUG:urllib3.connectionpool:Resetting dropped connection: 10.180.1.5
DEBUG:urllib3.connectionpool:http://10.180.1.5:9000 "POST /api/2/store/ HTTP/1.1" 200 41
DEBUG:urllib3.connectionpool:Resetting dropped connection: 10.180.1.5
DEBUG:urllib3.connectionpool:http://10.180.1.5:9000 "POST /api/2/store/ HTTP/1.1" 200 41
DEBUG:urllib3.connectionpool:Starting new HTTP connection (19): 10.180.1.5:9000
DEBUG:urllib3.connectionpool:http://10.180.1.5:9000 "POST /api/2/store/ HTTP/1.1" 200 41
DEBUG:urllib3.connectionpool:Resetting dropped connection: 10.180.1.5
DEBUG:urllib3.connectionpool:http://10.180.1.5:9000 "POST /api/2/store/ HTTP/1.1" 200 41
DEBUG:urllib3.connectionpool:Resetting dropped connection: 10.180.1.5
DEBUG:urllib3.connectionpool:http://10.180.1.5:9000 "POST /api/2/store/ HTTP/1.1" 200 41
DEBUG:urllib3.connectionpool:Resetting dropped connection: 10.180.1.5
DEBUG:urllib3.connectionpool:http://10.180.1.5:9000 "POST /api/2/store/ HTTP/1.1" 200 41
DEBUG:urllib3.connectionpool:Starting new HTTP connection (20): 10.180.1.5:9000
DEBUG:urllib3.connectionpool:http://10.180.1.5:9000 "POST /api/2/store/ HTTP/1.1" 200 41
DEBUG:urllib3.connectionpool:Resetting dropped connection: 10.180.1.5
DEBUG:urllib3.connectionpool:http://10.180.1.5:9000 "POST /api/2/store/ HTTP/1.1" 200 41
DEBUG:urllib3.connectionpool:Starting new HTTP connection (21): 10.180.1.5:9000
DEBUG:urllib3.connectionpool:http://10.180.1.5:9000 "POST /api/2/store/ HTTP/1.1" 200 41
DEBUG:urllib3.connectionpool:Resetting dropped connection: 10.180.1.5
DEBUG:urllib3.connectionpool:http://10.180.1.5:9000 "POST /api/2/store/ HTTP/1.1" 200 41
DEBUG:urllib3.connectionpool:Starting new HTTP connection (22): 10.180.1.5:9000
DEBUG:urllib3.connectionpool:http://10.180.1.5:9000 "POST /api/2/store/ HTTP/1.1" 200 41
DEBUG:urllib3.connectionpool:Starting new HTTP connection (23): 10.180.1.5:9000
DEBUG:urllib3.connectionpool:http://10.180.1.5:9000 "POST /api/2/store/ HTTP/1.1" 200 41
DEBUG:urllib3.connectionpool:Starting new HTTP connection (24): 10.180.1.5:9000
DEBUG:urllib3.connectionpool:http://10.180.1.5:9000 "POST /api/2/store/ HTTP/1.1" 200 41
DEBUG:urllib3.connectionpool:Starting new HTTP connection (25): 10.180.1.5:9000
DEBUG:urllib3.connectionpool:http://10.180.1.5:9000 "POST /api/2/store/ HTTP/1.1" 200 41
DEBUG:urllib3.connectionpool:Starting new HTTP connection (26): 10.180.1.5:9000
DEBUG:urllib3.connectionpool:http://10.180.1.5:9000 "POST /api/2/store/ HTTP/1.1" 200 41
DEBUG:urllib3.connectionpool:Starting new HTTP connection (27): 10.180.1.5:9000
DEBUG:urllib3.connectionpool:http://10.180.1.5:9000 "POST /api/2/store/ HTTP/1.1" 200 41
DEBUG:urllib3.connectionpool:Starting new HTTP connection (28): 10.180.1.5:9000
DEBUG:urllib3.connectionpool:http://10.180.1.5:9000 "POST /api/2/store/ HTTP/1.1" 200 41
DEBUG:urllib3.connectionpool:Starting new HTTP connection (29): 10.180.1.5:9000
DEBUG:urllib3.connectionpool:http://10.180.1.5:9000 "POST /api/2/store/ HTTP/1.1" 200 41
DEBUG:urllib3.connectionpool:Starting new HTTP connection (30): 10.180.1.5:9000
DEBUG:urllib3.connectionpool:http://10.180.1.5:9000 "POST /api/2/store/ HTTP/1.1" 200 41
DEBUG:urllib3.connectionpool:Resetting dropped connection: 10.180.1.5
DEBUG:urllib3.connectionpool:http://10.180.1.5:9000 "POST /api/2/store/ HTTP/1.1" 200 41
DEBUG:urllib3.connectionpool:Starting new HTTP connection (31): 10.180.1.5:9000
DEBUG:urllib3.connectionpool:http://10.180.1.5:9000 "POST /api/2/store/ HTTP/1.1" 200 41
DEBUG:urllib3.connectionpool:Starting new HTTP connection (32): 10.180.1.5:9000
DEBUG:urllib3.connectionpool:http://10.180.1.5:9000 "POST /api/2/store/ HTTP/1.1" 200 41
DEBUG:urllib3.connectionpool:Resetting dropped connection: 10.180.1.5
DEBUG:urllib3.connectionpool:http://10.180.1.5:9000 "POST /api/2/store/ HTTP/1.1" 200 41
DEBUG:urllib3.connectionpool:Starting new HTTP connection (33): 10.180.1.5:9000
DEBUG:urllib3.connectionpool:http://10.180.1.5:9000 "POST /api/2/store/ HTTP/1.1" 200 41
DEBUG:urllib3.connectionpool:Resetting dropped connection: 10.180.1.5
DEBUG:urllib3.connectionpool:http://10.180.1.5:9000 "POST /api/2/store/ HTTP/1.1" 200 41
DEBUG:urllib3.connectionpool:Starting new HTTP connection (34): 10.180.1.5:9000
DEBUG:urllib3.connectionpool:http://10.180.1.5:9000 "POST /api/2/store/ HTTP/1.1" 200 41
DEBUG:urllib3.connectionpool:Resetting dropped connection: 10.180.1.5
DEBUG:urllib3.connectionpool:http://10.180.1.5:9000 "POST /api/2/store/ HTTP/1.1" 200 41
DEBUG:urllib3.connectionpool:Resetting dropped connection: 10.180.1.5
DEBUG:urllib3.connectionpool:http://10.180.1.5:9000 "POST /api/2/store/ HTTP/1.1" 200 41
DEBUG:urllib3.connectionpool:Starting new HTTP connection (35): 10.180.1.5:9000
DEBUG:urllib3.connectionpool:http://10.180.1.5:9000 "POST /api/2/store/ HTTP/1.1" 200 41
DEBUG:urllib3.connectionpool:Resetting dropped connection: 10.180.1.5
DEBUG:urllib3.connectionpool:http://10.180.1.5:9000 "POST /api/2/store/ HTTP/1.1" 200 41
DEBUG:urllib3.connectionpool:Resetting dropped connection: 10.180.1.5
DEBUG:urllib3.connectionpool:http://10.180.1.5:9000 "POST /api/2/store/ HTTP/1.1" 200 41
DEBUG:urllib3.connectionpool:Starting new HTTP connection (36): 10.180.1.5:9000
DEBUG:urllib3.connectionpool:http://10.180.1.5:9000 "POST /api/2/store/ HTTP/1.1" 200 41
DEBUG:urllib3.connectionpool:Resetting dropped connection: 10.180.1.5
DEBUG:urllib3.connectionpool:http://10.180.1.5:9000 "POST /api/2/store/ HTTP/1.1" 200 41
DEBUG:urllib3.connectionpool:Resetting dropped connection: 10.180.1.5
DEBUG:urllib3.connectionpool:http://10.180.1.5:9000 "POST /api/2/store/ HTTP/1.1" 200 41
DEBUG:urllib3.connectionpool:Starting new HTTP connection (37): 10.180.1.5:9000
DEBUG:urllib3.connectionpool:http://10.180.1.5:9000 "POST /api/2/store/ HTTP/1.1" 200 41
DEBUG:urllib3.connectionpool:Starting new HTTP connection (38): 10.180.1.5:9000
DEBUG:urllib3.connectionpool:http://10.180.1.5:9000 "POST /api/2/store/ HTTP/1.1" 200 41
DEBUG:urllib3.connectionpool:Starting new HTTP connection (39): 10.180.1.5:9000
DEBUG:urllib3.connectionpool:http://10.180.1.5:9000 "POST /api/2/store/ HTTP/1.1" 200 41
DEBUG:urllib3.connectionpool:Starting new HTTP connection (40): 10.180.1.5:9000
DEBUG:urllib3.connectionpool:http://10.180.1.5:9000 "POST /api/2/store/ HTTP/1.1" 200 41
DEBUG:urllib3.connectionpool:Starting new HTTP connection (41): 10.180.1.5:9000
DEBUG:urllib3.connectionpool:http://10.180.1.5:9000 "POST /api/2/store/ HTTP/1.1" 200 41
DEBUG:urllib3.connectionpool:Starting new HTTP connection (42): 10.180.1.5:9000
DEBUG:urllib3.connectionpool:http://10.180.1.5:9000 "POST /api/2/store/ HTTP/1.1" 200 41
DEBUG:urllib3.connectionpool:Resetting dropped connection: 10.180.1.5
DEBUG:urllib3.connectionpool:http://10.180.1.5:9000 "POST /api/2/store/ HTTP/1.1" 200 41
DEBUG:urllib3.connectionpool:Starting new HTTP connection (43): 10.180.1.5:9000
DEBUG:urllib3.connectionpool:http://10.180.1.5:9000 "POST /api/2/store/ HTTP/1.1" 200 41
DEBUG:urllib3.connectionpool:Starting new HTTP connection (44): 10.180.1.5:9000
DEBUG:urllib3.connectionpool:http://10.180.1.5:9000 "POST /api/2/store/ HTTP/1.1" 200 41
DEBUG:urllib3.connectionpool:Starting new HTTP connection (45): 10.180.1.5:9000
DEBUG:urllib3.connectionpool:http://10.180.1.5:9000 "POST /api/2/store/ HTTP/1.1" 200 41
DEBUG:urllib3.connectionpool:Starting new HTTP connection (46): 10.180.1.5:9000
DEBUG:urllib3.connectionpool:http://10.180.1.5:9000 "POST /api/2/store/ HTTP/1.1" 200 41
DEBUG:urllib3.connectionpool:Starting new HTTP connection (47): 10.180.1.5:9000
DEBUG:urllib3.connectionpool:http://10.180.1.5:9000 "POST /api/2/store/ HTTP/1.1" 200 41
DEBUG:urllib3.connectionpool:Resetting dropped connection: 10.180.1.5
DEBUG:urllib3.connectionpool:http://10.180.1.5:9000 "POST /api/2/store/ HTTP/1.1" 200 41
DEBUG:urllib3.connectionpool:Resetting dropped connection: 10.180.1.5
DEBUG:urllib3.connectionpool:http://10.180.1.5:9000 "POST /api/2/store/ HTTP/1.1" 200 41
DEBUG:urllib3.connectionpool:Starting new HTTP connection (48): 10.180.1.5:9000
DEBUG:urllib3.connectionpool:http://10.180.1.5:9000 "POST /api/2/store/ HTTP/1.1" 200 41
DEBUG:urllib3.connectionpool:Starting new HTTP connection (49): 10.180.1.5:9000
DEBUG:urllib3.connectionpool:http://10.180.1.5:9000 "POST /api/2/store/ HTTP/1.1" 200 41
DEBUG:urllib3.connectionpool:Starting new HTTP connection (50): 10.180.1.5:9000
DEBUG:urllib3.connectionpool:http://10.180.1.5:9000 "POST /api/2/store/ HTTP/1.1" 200 41
DEBUG:urllib3.connectionpool:Starting new HTTP connection (51): 10.180.1.5:9000
DEBUG:urllib3.connectionpool:http://10.180.1.5:9000 "POST /api/2/store/ HTTP/1.1" 200 41
DEBUG:urllib3.connectionpool:Resetting dropped connection: 10.180.1.5
DEBUG:urllib3.connectionpool:http://10.180.1.5:9000 "POST /api/2/store/ HTTP/1.1" 200 41
DEBUG:urllib3.connectionpool:Resetting dropped connection: 10.180.1.5
DEBUG:urllib3.connectionpool:http://10.180.1.5:9000 "POST /api/2/store/ HTTP/1.1" 200 41
DEBUG:urllib3.connectionpool:Resetting dropped connection: 10.180.1.5
DEBUG:urllib3.connectionpool:http://10.180.1.5:9000 "POST /api/2/store/ HTTP/1.1" 200 41
DEBUG:urllib3.connectionpool:Starting new HTTP connection (52): 10.180.1.5:9000
DEBUG:urllib3.connectionpool:http://10.180.1.5:9000 "POST /api/2/store/ HTTP/1.1" 200 41
DEBUG:urllib3.connectionpool:Starting new HTTP connection (53): 10.180.1.5:9000
DEBUG:urllib3.connectionpool:http://10.180.1.5:9000 "POST /api/2/store/ HTTP/1.1" 200 41
DEBUG:urllib3.connectionpool:Starting new HTTP connection (54): 10.180.1.5:9000
DEBUG:urllib3.connectionpool:http://10.180.1.5:9000 "POST /api/2/store/ HTTP/1.1" 200 41
DEBUG:urllib3.connectionpool:Resetting dropped connection: 10.180.1.5
DEBUG:urllib3.connectionpool:http://10.180.1.5:9000 "POST /api/2/store/ HTTP/1.1" 200 41
DEBUG:urllib3.connectionpool:Resetting dropped connection: 10.180.1.5
DEBUG:urllib3.connectionpool:http://10.180.1.5:9000 "POST /api/2/store/ HTTP/1.1" 200 41
DEBUG:urllib3.connectionpool:Starting new HTTP connection (55): 10.180.1.5:9000
DEBUG:urllib3.connectionpool:http://10.180.1.5:9000 "POST /api/2/store/ HTTP/1.1" 200 41
DEBUG:urllib3.connectionpool:Starting new HTTP connection (56): 10.180.1.5:9000
DEBUG:urllib3.connectionpool:http://10.180.1.5:9000 "POST /api/2/store/ HTTP/1.1" 200 41
DEBUG:urllib3.connectionpool:Starting new HTTP connection (57): 10.180.1.5:9000
DEBUG:urllib3.connectionpool:http://10.180.1.5:9000 "POST /api/2/store/ HTTP/1.1" 200 41
```

The events are queued very differently. Memory usage constantly raises and is never cleared/garbage collected even after the sending has stopped.
@mikmatko to rule out the most obvious candidate, could you run with `init(transport=print)` instead of providing a DSN? Also could you run `pip freeze` in that container? (I am not seeing anything significant even when increasing 200 to a much larger nr)
Hi @untitaker, I've put it in gist here: https://gist.github.com/mikmatko/4e9b58b26d8a7136f10aa3f5afc30b56

Same issue still with sentry-sdk==0.14.0

pip freeze from container:
```
certifi==2019.11.28
sentry-sdk==0.14.0
urllib3==1.25.7
```
I was wondering if my Sentry / Postgres / Redis was somehow messed up and resulted in this. So I did a fresh installation from https://hub.docker.com/_/sentry/ ("How to setup a full Sentry instance"). No luck, same issue persists.
>I was wondering if my Sentry / Postgres / Redis was somehow messed up and resulted in this

I'm sorry, is your Sentry installation or your instrumented application going out of memory? This seems wholly unrelated to what we discussed above.
To clarify: My Python application is going out of memory. Not the Sentry installation.

(But I was wondering if there was some possible link to my Sentry installation, so I tried a fresh installation on that too. As expected, it did not affect this at all. My idea on this was to check if the Sentry installation somehow resulted in e.g. connection errors that were not seen, and that resulted in urllib leaking memory. But that was not the case.)
I ran my application for a while with tracemalloc with sentry-sdk==0.14.0. After a few minutes, top 9 memory usage was caused in:

```
/usr/local/lib/python3.7/site-packages/sentry_sdk/serializer.py:288: size=22.0 MiB, count=72739, average=317 B
/usr/local/lib/python3.7/site-packages/sentry_sdk/serializer.py:267: size=14.0 MiB, count=174459, average=84 B
/usr/local/lib/python3.7/site-packages/sentry_sdk/serializer.py:106: size=11.3 MiB, count=101865, average=117 B
/usr/local/lib/python3.7/site-packages/sentry_sdk/serializer.py:269: size=10.7 MiB, count=67354, average=166 B
/usr/local/lib/python3.7/site-packages/sentry_sdk/utils.py:711: size=10.3 MiB, count=19272, average=561 B
/usr/local/lib/python3.7/site-packages/sentry_sdk/utils.py:714: size=7151 KiB, count=113435, average=65 B
/usr/local/lib/python3.7/site-packages/sentry_sdk/serializer.py:260: size=5345 KiB, count=72018, average=76 B
/usr/local/lib/python3.7/site-packages/sentry_sdk/serializer.py:112: size=4426 KiB, count=35913, average=126 B
/usr/local/lib/python3.7/site-packages/sentry_sdk/utils.py:291: size=4034 KiB, count=43493, average=95 B
 ```

Do those give any hints?
Could you monitor `Hub.current.client.transport._worker._queue.qsize()` while running your thing? I wonder whether the Sentry installation is just slow.
@untitaker Thanks for the advice. This was indeed the issue. My guess is that sentry-sdk==0.12.3 saw some performance improvements over 0.12.2 and that lead to sentry-sdk being able to serialize exceptions faster, meaning that the events-to-be-sent are getting queued faster. And with slow Sentry installation, the queue is not getting cleared quickly enough.

Now I'm monitoring `Hub.current.client.transport._worker._queue.qsize()` and periodically doing a flush with `Hub.current.client.flush()` after too many events are queued up. This seems to work nicely :)

I appreciate your help in this. Thanks a lot!


Resolving patch:
diff --git a/sentry_sdk/worker.py b/sentry_sdk/worker.py
--- a/sentry_sdk/worker.py
+++ b/sentry_sdk/worker.py
@@ -22,7 +22,7 @@ class BackgroundWorker(object):
     def __init__(self):
         # type: () -> None
         check_thread_support()
-        self._queue = queue.Queue(-1)  # type: Queue[Any]
+        self._queue = queue.Queue(30)  # type: Queue[Any]
         self._lock = Lock()
         self._thread = None  # type: Optional[Thread]
         self._thread_for_pid = None  # type: Optional[int]
@@ -86,10 +86,18 @@ def start(self):
 
     def kill(self):
         # type: () -> None
+        """
+        Kill worker thread. Returns immediately. Not useful for
+        waiting on shutdown for events, use `flush` for that.
+        """
         logger.debug("background worker got kill request")
         with self._lock:
             if self._thread:
-                self._queue.put_nowait(_TERMINATOR)
+                try:
+                    self._queue.put_nowait(_TERMINATOR)
+                except queue.Full:
+                    logger.debug("background worker queue full, kill failed")
+
                 self._thread = None
                 self._thread_for_pid = None
 
@@ -114,7 +122,10 @@ def _wait_flush(self, timeout, callback):
     def submit(self, callback):
         # type: (Callable[[], None]) -> None
         self._ensure_thread()
-        self._queue.put_nowait(callback)
+        try:
+            self._queue.put_nowait(callback)
+        except queue.Full:
+            logger.debug("background worker queue full, dropping event")
 
     def _target(self):
         # type: () -> None


Test patch: