Repository: obspy/obspy

Issue:
fdsn EIDA routed client: problem with spammed subprocesses (memory leak)
Issue reported on the users mailing list:

```
I am facing a potential issue with the ObsPy FDSN client. I am using the EIDA
routing client to download waveform data and metadata for many stations. The 
way I do it is a simple loop over all stations, using the get_waveforms()
function in the loop. In principle it looks like this:

from obspy.clients.fdsn import RoutingClient as Client
client = Client('eida-routing')
for station in long_list :
    data = client.get_waveforms(...)
    data.write(filename)

The problem is, that each time data is actually downloaded, ObsPy seems to
start several new processes in the background, which are not being closed until
the end of the python script. Thus, for each station in the for-loop new 
additional processes are started but not closed again. Since I have a very long
stationlist and many days that I loop over, basically I end up spawning more
processes than the systen can handle, ultimately resulting in the message:
"Can't start new thread"

This problem only occurs with the fdsn routing client, but e.g. not with the 
ArcLink client. I did not try the non-routed fdsn client. Same behaviour with
ObsPy versions 1.1.0 and 1.1.1 ... 
```


Discussion:
I cannot reproduce this on OSX and Python 3.7. I used this code snippet and the threads are cleaned up again afterwards. Maybe it depends on the Python version/operating system?

```python
import obspy
from obspy.clients.fdsn import RoutingClient as Client

client = Client('eida-routing', debug=True)

starttime = obspy.UTCDateTime(2018, 1, 1)

for i in range(1000):
    print(client.get_waveforms(network="*", station="B*", location="", channel="LHZ",
                               starttime=starttime + i * 5, endtime=starttime + i * 5 + 2))
```

Can anyone reproduce it? Also the only use of threads we make is here which should be nicely cleaned by Python's GC.

https://github.com/obspy/obspy/blob/ce2f41e8a1efdc0c07855d7e48b24b1887e5f85c/obspy/clients/fdsn/routing/routing_client.py#L267-L286


Using the code snippet above I can reproduce the issue, on Linux Mint 18 Sarah (basically Ubuntu 16.04) and on CentOS 6.10 with a clean and new Python 2.7 + ObsPy 1.1.1 Conda environment and in several existing Python 2.7 or 3.7 Conda environments. As soon as I execute the script I observe (with htop) plenty of new threads being started and not getting closed. Only with a clean and new Python 3.7 + ObsPy 1.1.1 Conda environment it's closing the threads again.

Even a recently installed clean Python 3.7 + ObsPy 1.1.0 + yam environment, that I updated to ObsPy 1.1.1 shows the same unwanted behavior. I suspect it might be either a conflict with some Python 2.7 packages or incompatible package versions within Python 3.7. I will investigate this further ...

I can provide a list of the packages, if that could be helpful ...

Resolving patch:
diff --git a/obspy/clients/fdsn/routing/routing_client.py b/obspy/clients/fdsn/routing/routing_client.py
--- a/obspy/clients/fdsn/routing/routing_client.py
+++ b/obspy/clients/fdsn/routing/routing_client.py
@@ -282,6 +282,10 @@ def _download_parallel(self, split, data_type, **kwargs):
                 continue
             collection += _i
 
+        # Explitly close the thread pool as somehow this does not work
+        # automatically under linux. See #2342.
+        pool.close()
+
         return collection
 
     def _handle_requests_http_error(self, r):


Test patch: