Repository: obspy/obspy

Issue:
Hanging fdsn network test
We have some hanging fdsn network tests due to server problems. These do not always occur. Note the command line option `--full-traceback`, the test duration 42 minutes and the keybord interuppt to end the tests. Maybe a timeout needs to be added somewehere?

Edit: Here is an example github actions run: https://github.com/obspy/obspy/runs/5539170773

Reproduce with

```
obspy-runtests --network --full-trace clients/fdsn
```

I deleted some uninteresting stuff from the beginning of the traceback

```
============================================================================================================ test session starts =============================================================================================================
platform linux -- Python 3.8.6, pytest-6.2.4, py-1.10.0, pluggy-0.13.1
rootdir: /home/eule/dev/obspy/obspy, configfile: pytest.ini
plugins: sphinx-0.3.1, html-3.1.1, cov-2.10.1, metadata-1.11.0, json-report-1.4.1
collected 150 items / 13 deselected / 137 selected                                                                                                                                                                                           

clients/fdsn/__init__.py .                                                                                                                                                                                                             [  0%]
clients/fdsn/client.py ........                                                                                                                                                                                                        [  6%]
clients/fdsn/wadl_parser.py .                                                                                                                                                                                                          [  7%]
clients/fdsn/mass_downloader/__init__.py .                                                                                                                                                                                             [  8%]
clients/fdsn/mass_downloader/domain.py ...                                                                                                                                                                                             [ 10%]
clients/fdsn/mass_downloader/restrictions.py .                                                                                                                                                                                         [ 10%]
clients/fdsn/routing/routing_client.py .                                                                                                                                                                                               [ 11%]
clients/fdsn/tests/test_base_routing_client.py ......                                                                                                                                                                                  [ 16%]
clients/fdsn/tests/test_client.py .........s................................s...                                                                                                                                                       [ 49%]
clients/fdsn/tests/test_eidaws_routing_client.py ............                                                                                                                                                                          [ 58%]
clients/fdsn/tests/test_federator_routing_client.py ....^C

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! KeyboardInterrupt !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

item = <TestCaseFunction test_get_stations_integration_test>

    def pytest_runtest_call(item: Item) -> None:
        _update_current_test_var(item, "call")
        try:
            del sys.last_type
            del sys.last_value
            del sys.last_traceback
        except AttributeError:
            pass
        try:
>           item.runtest()

../../../anaconda/envs/workhorse/lib/python3.8/site-packages/_pytest/runner.py:162: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <TestCaseFunction test_get_stations_integration_test>

    def runtest(self) -> None:
        from _pytest.debugging import maybe_wrap_pytest_function_for_tracing
    
        assert self._testcase is not None
    
        maybe_wrap_pytest_function_for_tracing(self)
    
        # Let the unittest framework handle async functions.
        if is_async_function(self.obj):
            # Type ignored because self acts as the TestResult, but is not actually one.
            self._testcase(result=self)  # type: ignore[arg-type]
        else:
            # When --pdb is given, we want to postpone calling tearDown() otherwise
            # when entering the pdb prompt, tearDown() would have probably cleaned up
            # instance variables, which makes it difficult to debug.
            # Arguably we could always postpone tearDown(), but this changes the moment where the
            # TestCase instance interacts with the results object, so better to only do it
            # when absolutely needed.
            if self.config.getoption("usepdb") and not _is_skipped(self.obj):
                self._explicit_tearDown = self._testcase.tearDown
                setattr(self._testcase, "tearDown", lambda *args: None)
    
            # We need to update the actual bound method with self.obj, because
            # wrap_pytest_function_for_tracing replaces self.obj by a wrapper.
            setattr(self._testcase, self.name, self.obj)
            try:
>               self._testcase(result=self)  # type: ignore[arg-type]

../../../anaconda/envs/workhorse/lib/python3.8/site-packages/_pytest/unittest.py:321: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <obspy.clients.fdsn.tests.test_federator_routing_client.FederatorRoutingClientTestCase testMethod=test_get_stations_integration_test>, args = (), kwds = {'result': <TestCaseFunction test_get_stations_integration_test>}

    def __call__(self, *args, **kwds):
>       return self.run(*args, **kwds)

../../../anaconda/envs/workhorse/lib/python3.8/unittest/case.py:736: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <obspy.clients.fdsn.tests.test_federator_routing_client.FederatorRoutingClientTestCase testMethod=test_get_stations_integration_test>, result = <TestCaseFunction test_get_stations_integration_test>

    def run(self, result=None):
        orig_result = result
        if result is None:
            result = self.defaultTestResult()
            startTestRun = getattr(result, 'startTestRun', None)
            if startTestRun is not None:
                startTestRun()
    
        result.startTest(self)
    
        testMethod = getattr(self, self._testMethodName)
        if (getattr(self.__class__, "__unittest_skip__", False) or
            getattr(testMethod, "__unittest_skip__", False)):
            # If the class or method was skipped.
            try:
                skip_why = (getattr(self.__class__, '__unittest_skip_why__', '')
                            or getattr(testMethod, '__unittest_skip_why__', ''))
                self._addSkip(result, self, skip_why)
            finally:
                result.stopTest(self)
            return
        expecting_failure_method = getattr(testMethod,
                                           "__unittest_expecting_failure__", False)
        expecting_failure_class = getattr(self,
                                          "__unittest_expecting_failure__", False)
        expecting_failure = expecting_failure_class or expecting_failure_method
        outcome = _Outcome(result)
        try:
            self._outcome = outcome
    
            with outcome.testPartExecutor(self):
                self._callSetUp()
            if outcome.success:
                outcome.expecting_failure = expecting_failure
                with outcome.testPartExecutor(self, isTest=True):
>                   self._callTestMethod(testMethod)

../../../anaconda/envs/workhorse/lib/python3.8/unittest/case.py:676: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <obspy.clients.fdsn.tests.test_federator_routing_client.FederatorRoutingClientTestCase testMethod=test_get_stations_integration_test>
method = <bound method FederatorRoutingClientTestCase.test_get_stations_integration_test of <obspy.clients.fdsn.tests.test_federator_routing_client.FederatorRoutingClientTestCase testMethod=test_get_stations_integration_test>>

    def _callTestMethod(self, method):
>       method()

../../../anaconda/envs/workhorse/lib/python3.8/unittest/case.py:633: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <obspy.clients.fdsn.tests.test_federator_routing_client.FederatorRoutingClientTestCase testMethod=test_get_stations_integration_test>

    def test_get_stations_integration_test(self):
        """
        Integration test that does not mock anything but actually downloads
        things.
        """
>       inv = self.client.get_stations(
            starttime=obspy.UTCDateTime(2017, 1, 1),
            endtime=obspy.UTCDateTime(2017, 1, 1, 0, 1),
            latitude=35.0, longitude=-110, maxradius=0.3, network="TA",
            channel="LHZ", level="station")

clients/fdsn/tests/test_federator_routing_client.py:335: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <obspy.clients.fdsn.routing.federator_routing_client.FederatorRoutingClient object at 0x7fd2dbf5ba60>, kwargs = {'channel': 'LHZ', 'endtime': UTCDateTime(2017, 1, 1, 0, 1), 'latitude': 35.0, 'level': 'station', ...}

>   ???

<decorator-gen-62>:2: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

f = <function FederatorRoutingClient.get_stations at 0x7fd2b5cb00d0>, args = (<obspy.clients.fdsn.routing.federator_routing_client.FederatorRoutingClient object at 0x7fd2dbf5ba60>,)
kwargs = {'channel': 'LHZ', 'endtime': UTCDateTime(2017, 1, 1, 0, 1), 'latitude': 35.0, 'level': 'station', ...}

    @decorator.decorator
    def _assert_filename_not_in_kwargs(f, *args, **kwargs):
        if "filename" in kwargs:
            raise ValueError("The `filename` argument is not supported")
>       return f(*args, **kwargs)

clients/fdsn/routing/routing_client.py:78: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <obspy.clients.fdsn.routing.federator_routing_client.FederatorRoutingClient object at 0x7fd2dbf5ba60>, kwargs = {'channel': 'LHZ', 'endtime': UTCDateTime(2017, 1, 1, 0, 1), 'latitude': 35.0, 'level': 'station', ...}

    @_assert_filename_not_in_kwargs
    def get_stations(self, **kwargs):
        """
        Get stations from multiple data centers.
    
        It will pass on most parameters to the federated routing service.
        They will also be passed on to the individual FDSNWS implementations
        if a service supports them.
    
        The ``filename`` parameter of the single provider FDSN client is not
        supported.
    
        This can route on a number of different parameters, please see the
        web site of the
        `IRIS Federator  <https://service.iris.edu/irisws/fedcatalog/1/>`_
        for details.
        """
>       return super(FederatorRoutingClient, self).get_stations(**kwargs)

clients/fdsn/routing/federator_routing_client.py:105: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <obspy.clients.fdsn.routing.federator_routing_client.FederatorRoutingClient object at 0x7fd2dbf5ba60>, kwargs = {'channel': 'LHZ', 'endtime': UTCDateTime(2017, 1, 1, 0, 1), 'latitude': 35.0, 'level': 'station', ...}

>   ???

<decorator-gen-55>:2: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

f = <function BaseRoutingClient.get_stations at 0x7fd2b6e63700>, args = (<obspy.clients.fdsn.routing.federator_routing_client.FederatorRoutingClient object at 0x7fd2dbf5ba60>,)
kwargs = {'channel': 'LHZ', 'endtime': UTCDateTime(2017, 1, 1, 0, 1), 'latitude': 35.0, 'level': 'station', ...}

    @decorator.decorator
    def _assert_filename_not_in_kwargs(f, *args, **kwargs):
        if "filename" in kwargs:
            raise ValueError("The `filename` argument is not supported")
>       return f(*args, **kwargs)

clients/fdsn/routing/routing_client.py:78: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <obspy.clients.fdsn.routing.federator_routing_client.FederatorRoutingClient object at 0x7fd2dbf5ba60>, kwargs = {'latitude': 35.0, 'level': 'station', 'longitude': -110, 'maxradius': 0.3}
bulk = ['TA', '*', '*', 'LHZ', UTCDateTime(2017, 1, 1, 0, 0), UTCDateTime(2017, 1, 1, 0, 1)]

    @_assert_filename_not_in_kwargs
    def get_stations(self, **kwargs):
        """
        Get stations from multiple data centers.
    
        It will pass on most parameters to the underlying routed service.
        They will also be passed on to the individual FDSNWS implementations
        if a service supports them.
    
        The ``filename`` parameter of the single provider FDSN client is not
        supported.
    
        This can route on a number of different parameters, please see the
        web sites of the
        `IRIS Federator  <https://service.iris.edu/irisws/fedcatalog/1/>`_
        and of the `EIDAWS Routing Service
        <http://www.orfeus-eu.org/data/eida/webservices/routing/>`_ for
        details.
        """
        # Just pass these to the bulk request.
        bulk = [kwargs.pop(key, '*') for key in (
                "network", "station", "location", "channel", "starttime",
                "endtime")]
>       return self.get_stations_bulk([bulk], **kwargs)

clients/fdsn/routing/routing_client.py:367: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <obspy.clients.fdsn.routing.federator_routing_client.FederatorRoutingClient object at 0x7fd2dbf5ba60>, bulk = [['TA', '*', '*', 'LHZ', UTCDateTime(2017, 1, 1, 0, 0), UTCDateTime(2017, 1, 1, 0, 1)]]
kwargs = {'latitude': 35.0, 'level': 'station', 'longitude': -110, 'maxradius': 0.3}

>   ???

<decorator-gen-63>:2: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

f = <function FederatorRoutingClient.get_stations_bulk at 0x7fd2b5cb0430>
args = (<obspy.clients.fdsn.routing.federator_routing_client.FederatorRoutingClient object at 0x7fd2dbf5ba60>, [['TA', '*', '*', 'LHZ', UTCDateTime(2017, 1, 1, 0, 0), UTCDateTime(2017, 1, 1, 0, 1)]])
kwargs = {'latitude': 35.0, 'level': 'station', 'longitude': -110, 'maxradius': 0.3}

    @decorator.decorator
    def _assert_filename_not_in_kwargs(f, *args, **kwargs):
        if "filename" in kwargs:
            raise ValueError("The `filename` argument is not supported")
>       return f(*args, **kwargs)

clients/fdsn/routing/routing_client.py:78: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <obspy.clients.fdsn.routing.federator_routing_client.FederatorRoutingClient object at 0x7fd2dbf5ba60>, bulk = [['TA', '*', '*', 'LHZ', UTCDateTime(2017, 1, 1, 0, 0), UTCDateTime(2017, 1, 1, 0, 1)]]
kwargs = {'latitude': 35.0, 'level': 'station', 'longitude': -110, 'maxradius': 0.3}, bulk_params = ['network', 'station', 'location', 'channel', 'starttime', 'endtime'], _i = 'endtime'
params = OrderedDict([('level', 'station'), ('latitude', '35.0'), ('longitude', '-110'), ('maxradius', '0.3'), ('format', 'request')]), k = 'maxradius'
bulk_str = b'level=station\nlatitude=35.0\nlongitude=-110\nmaxradius=0.3\nformat=request\nTA * * LHZ 2017-01-01T00:00:00.000000 2017-01-01T00:01:00.000000'

    @_assert_filename_not_in_kwargs
    def get_stations_bulk(self, bulk, **kwargs):
        """
        Get stations from multiple data centers.
    
        It will pass on most parameters to the federated routing service.
        They will also be passed on to the individual FDSNWS implementations
        if a service supports them.
    
        The ``filename`` parameter of the single provider FDSN client is not
        supported.
    
        This can route on a number of different parameters, please see the
        web site of the
        `IRIS Federator  <https://service.iris.edu/irisws/fedcatalog/1/>`_
        for details.
        """
        bulk_params = ["network", "station", "location", "channel",
                       "starttime", "endtime"]
        for _i in bulk_params:
            if _i in kwargs:
                raise ValueError("`%s` must not be part of the optional "
                                 "parameters in a bulk request." % _i)
    
        params = collections.OrderedDict()
        for k in self.kwargs_of_interest:
            if k in kwargs:
                params[k] = str(kwargs[k])
        params["format"] = "request"
    
        bulk_str = get_bulk_string(bulk, params)
>       r = self._download(self._url + "/query", data=bulk_str)

clients/fdsn/routing/federator_routing_client.py:138: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <obspy.clients.fdsn.routing.federator_routing_client.FederatorRoutingClient object at 0x7fd2dbf5ba60>, url = 'http://service.iris.edu/irisws/fedcatalog/1/query', params = None, filename = None
data = b'level=station\nlatitude=35.0\nlongitude=-110\nmaxradius=0.3\nformat=request\nTA * * LHZ 2017-01-01T00:00:00.000000 2017-01-01T00:01:00.000000'

    def _download(self, url, params=None, filename=None, data=None):
        """
        Download the URL with GET or POST and the chosen parameters.
    
        Will call the ``_handle_requests_http_error()`` method if the response
        comes back with an HTTP code other than 200. Returns the response
        object if successful and ``filename`` is not given - if given it will
        save the response to the specified file and return ``None``.
    
        By default it will send a GET request - if data is given it will
        send a POST request.
    
        :param url: The URL to download from.
        :type url: str
        :param params: Additional URL parameters.
        :type params: dict
        :param filename: String or file like object. Will download directly
            to the file. If specified, this function will return nothing.
        :type filename: str or file-like object
        :param data: If specified, a POST request will be sent with the data in
            the body of the request.
        :type data: dict, bytes, or file-like object
        :return: The response object assuming ``filename`` is ``None``.
        :rtype: :class:`requests.Response`
        """
        if params:
            params = {k: v for k, v in params.items()}
    
        _request_args = {"url": url,
                         "headers": {"User-Agent": self._user_agent},
                         "params": params}
    
        # Stream to file - no need to keep it in memory for large files.
        if filename:
            _request_args["stream"] = True
    
        if self._debug:
            # Construct the same URL requests would construct.
            from requests import PreparedRequest  # noqa
            p = PreparedRequest()
            p.prepare(method="GET", **_request_args)
            print("Downloading %s ..." % p.url)
            if data is not None:
                print("Sending along the following payload:")
                print("-" * 70)
                print(data.decode() if hasattr(data, "decode") else data)
                print("-" * 70)
    
        # Workaround for old request versions.
        try:
            if data is None:
                r = requests.get(**_request_args)
            else:
                # Compatibility with old request versions.
                if hasattr(data, "read"):
                    data = data.read()
                _request_args["data"] = data
>               r = requests.post(**_request_args)

clients/base.py:225: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

url = 'http://service.iris.edu/irisws/fedcatalog/1/query', data = b'level=station\nlatitude=35.0\nlongitude=-110\nmaxradius=0.3\nformat=request\nTA * * LHZ 2017-01-01T00:00:00.000000 2017-01-01T00:01:00.000000', json = None
kwargs = {'headers': {'User-Agent': 'ObsPy/1.3.0.post0+15.gc5d0911cd9.obspy.master (Linux-4.15.0-171-generic-x86_64-with-glibc2.10, Python 3.8.6)'}, 'params': None}

    def post(url, data=None, json=None, **kwargs):
        r"""Sends a POST request.
    
        :param url: URL for the new :class:`Request` object.
        :param data: (optional) Dictionary, list of tuples, bytes, or file-like
            object to send in the body of the :class:`Request`.
        :param json: (optional) json data to send in the body of the :class:`Request`.
        :param \*\*kwargs: Optional arguments that ``request`` takes.
        :return: :class:`Response <Response>` object
        :rtype: requests.Response
        """
    
>       return request('post', url, data=data, json=json, **kwargs)

../../../anaconda/envs/workhorse/lib/python3.8/site-packages/requests/api.py:119: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

method = 'post', url = 'http://service.iris.edu/irisws/fedcatalog/1/query'
kwargs = {'data': b'level=station\nlatitude=35.0\nlongitude=-110\nmaxradius=0.3\nformat=request\nTA * * LHZ 2017-01-01T00:00:00...c5d0911cd9.obspy.master (Linux-4.15.0-171-generic-x86_64-with-glibc2.10, Python 3.8.6)'}, 'json': None, 'params': None}
session = <requests.sessions.Session object at 0x7fd2dbf10c70>

    def request(method, url, **kwargs):
        """Constructs and sends a :class:`Request <Request>`.
    
        :param method: method for the new :class:`Request` object: ``GET``, ``OPTIONS``, ``HEAD``, ``POST``, ``PUT``, ``PATCH``, or ``DELETE``.
        :param url: URL for the new :class:`Request` object.
        :param params: (optional) Dictionary, list of tuples or bytes to send
            in the query string for the :class:`Request`.
        :param data: (optional) Dictionary, list of tuples, bytes, or file-like
            object to send in the body of the :class:`Request`.
        :param json: (optional) A JSON serializable Python object to send in the body of the :class:`Request`.
        :param headers: (optional) Dictionary of HTTP Headers to send with the :class:`Request`.
        :param cookies: (optional) Dict or CookieJar object to send with the :class:`Request`.
        :param files: (optional) Dictionary of ``'name': file-like-objects`` (or ``{'name': file-tuple}``) for multipart encoding upload.
            ``file-tuple`` can be a 2-tuple ``('filename', fileobj)``, 3-tuple ``('filename', fileobj, 'content_type')``
            or a 4-tuple ``('filename', fileobj, 'content_type', custom_headers)``, where ``'content-type'`` is a string
            defining the content type of the given file and ``custom_headers`` a dict-like object containing additional headers
            to add for the file.
        :param auth: (optional) Auth tuple to enable Basic/Digest/Custom HTTP Auth.
        :param timeout: (optional) How many seconds to wait for the server to send data
            before giving up, as a float, or a :ref:`(connect timeout, read
            timeout) <timeouts>` tuple.
        :type timeout: float or tuple
        :param allow_redirects: (optional) Boolean. Enable/disable GET/OPTIONS/POST/PUT/PATCH/DELETE/HEAD redirection. Defaults to ``True``.
        :type allow_redirects: bool
        :param proxies: (optional) Dictionary mapping protocol to the URL of the proxy.
        :param verify: (optional) Either a boolean, in which case it controls whether we verify
                the server's TLS certificate, or a string, in which case it must be a path
                to a CA bundle to use. Defaults to ``True``.
        :param stream: (optional) if ``False``, the response content will be immediately downloaded.
        :param cert: (optional) if String, path to ssl client cert file (.pem). If Tuple, ('cert', 'key') pair.
        :return: :class:`Response <Response>` object
        :rtype: requests.Response
    
        Usage::
    
          >>> import requests
          >>> req = requests.request('GET', 'https://httpbin.org/get')
          >>> req
          <Response [200]>
        """
    
        # By using the 'with' statement we are sure the session is closed, thus we
        # avoid leaving sockets open which can trigger a ResourceWarning in some
        # cases, and look like a memory leak in others.
        with sessions.Session() as session:
>           return session.request(method=method, url=url, **kwargs)

../../../anaconda/envs/workhorse/lib/python3.8/site-packages/requests/api.py:61: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <requests.sessions.Session object at 0x7fd2dbf10c70>, method = 'post', url = 'http://service.iris.edu/irisws/fedcatalog/1/query', params = None
data = b'level=station\nlatitude=35.0\nlongitude=-110\nmaxradius=0.3\nformat=request\nTA * * LHZ 2017-01-01T00:00:00.000000 2017-01-01T00:01:00.000000'
headers = {'User-Agent': 'ObsPy/1.3.0.post0+15.gc5d0911cd9.obspy.master (Linux-4.15.0-171-generic-x86_64-with-glibc2.10, Python 3.8.6)'}, cookies = None, files = None, auth = None, timeout = None, allow_redirects = True, proxies = {}
hooks = None, stream = None, verify = None, cert = None, json = None

    def request(self, method, url,
            params=None, data=None, headers=None, cookies=None, files=None,
            auth=None, timeout=None, allow_redirects=True, proxies=None,
            hooks=None, stream=None, verify=None, cert=None, json=None):
        """Constructs a :class:`Request <Request>`, prepares it and sends it.
        Returns :class:`Response <Response>` object.
    
        :param method: method for the new :class:`Request` object.
        :param url: URL for the new :class:`Request` object.
        :param params: (optional) Dictionary or bytes to be sent in the query
            string for the :class:`Request`.
        :param data: (optional) Dictionary, list of tuples, bytes, or file-like
            object to send in the body of the :class:`Request`.
        :param json: (optional) json to send in the body of the
            :class:`Request`.
        :param headers: (optional) Dictionary of HTTP Headers to send with the
            :class:`Request`.
        :param cookies: (optional) Dict or CookieJar object to send with the
            :class:`Request`.
        :param files: (optional) Dictionary of ``'filename': file-like-objects``
            for multipart encoding upload.
        :param auth: (optional) Auth tuple or callable to enable
            Basic/Digest/Custom HTTP Auth.
        :param timeout: (optional) How long to wait for the server to send
            data before giving up, as a float, or a :ref:`(connect timeout,
            read timeout) <timeouts>` tuple.
        :type timeout: float or tuple
        :param allow_redirects: (optional) Set to True by default.
        :type allow_redirects: bool
        :param proxies: (optional) Dictionary mapping protocol or protocol and
            hostname to the URL of the proxy.
        :param stream: (optional) whether to immediately download the response
            content. Defaults to ``False``.
        :param verify: (optional) Either a boolean, in which case it controls whether we verify
            the server's TLS certificate, or a string, in which case it must be a path
            to a CA bundle to use. Defaults to ``True``. When set to
            ``False``, requests will accept any TLS certificate presented by
            the server, and will ignore hostname mismatches and/or expired
            certificates, which will make your application vulnerable to
            man-in-the-middle (MitM) attacks. Setting verify to ``False``
            may be useful during local development or testing.
        :param cert: (optional) if String, path to ssl client cert file (.pem).
            If Tuple, ('cert', 'key') pair.
        :rtype: requests.Response
        """
        # Create the Request.
        req = Request(
            method=method.upper(),
            url=url,
            headers=headers,
            files=files,
            data=data or {},
            json=json,
            params=params or {},
            auth=auth,
            cookies=cookies,
            hooks=hooks,
        )
        prep = self.prepare_request(req)
    
        proxies = proxies or {}
    
        settings = self.merge_environment_settings(
            prep.url, proxies, stream, verify, cert
        )
    
        # Send the request.
        send_kwargs = {
            'timeout': timeout,
            'allow_redirects': allow_redirects,
        }
        send_kwargs.update(settings)
>       resp = self.send(prep, **send_kwargs)

../../../anaconda/envs/workhorse/lib/python3.8/site-packages/requests/sessions.py:542: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <requests.sessions.Session object at 0x7fd2dbf10c70>, request = <PreparedRequest [POST]>, kwargs = {'cert': None, 'proxies': OrderedDict(), 'stream': False, 'timeout': None, ...}, allow_redirects = True, stream = False
hooks = {'response': []}, adapter = <requests.adapters.HTTPAdapter object at 0x7fd2dbf106d0>, start = 1647269807.87868

    def send(self, request, **kwargs):
        """Send a given PreparedRequest.
    
        :rtype: requests.Response
        """
        # Set defaults that the hooks can utilize to ensure they always have
        # the correct parameters to reproduce the previous request.
        kwargs.setdefault('stream', self.stream)
        kwargs.setdefault('verify', self.verify)
        kwargs.setdefault('cert', self.cert)
        kwargs.setdefault('proxies', self.proxies)
    
        # It's possible that users might accidentally send a Request object.
        # Guard against that specific failure case.
        if isinstance(request, Request):
            raise ValueError('You can only send PreparedRequests.')
    
        # Set up variables needed for resolve_redirects and dispatching of hooks
        allow_redirects = kwargs.pop('allow_redirects', True)
        stream = kwargs.get('stream')
        hooks = request.hooks
    
        # Get the appropriate adapter to use
        adapter = self.get_adapter(url=request.url)
    
        # Start time (approximately) of the request
        start = preferred_clock()
    
        # Send the request
>       r = adapter.send(request, **kwargs)

../../../anaconda/envs/workhorse/lib/python3.8/site-packages/requests/sessions.py:655: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <requests.adapters.HTTPAdapter object at 0x7fd2dbf106d0>, request = <PreparedRequest [POST]>, stream = False, timeout = Timeout(connect=None, read=None, total=None), verify = '/etc/ssl/certs/ca-certificates.crt', cert = None
proxies = OrderedDict()

    def send(self, request, stream=False, timeout=None, verify=True, cert=None, proxies=None):
        """Sends PreparedRequest object. Returns Response object.
    
        :param request: The :class:`PreparedRequest <PreparedRequest>` being sent.
        :param stream: (optional) Whether to stream the request content.
        :param timeout: (optional) How long to wait for the server to send
            data before giving up, as a float, or a :ref:`(connect timeout,
            read timeout) <timeouts>` tuple.
        :type timeout: float or tuple or urllib3 Timeout object
        :param verify: (optional) Either a boolean, in which case it controls whether
            we verify the server's TLS certificate, or a string, in which case it
            must be a path to a CA bundle to use
        :param cert: (optional) Any user-provided SSL certificate to be trusted.
        :param proxies: (optional) The proxies dictionary to apply to the request.
        :rtype: requests.Response
        """
    
        try:
            conn = self.get_connection(request.url, proxies)
        except LocationValueError as e:
            raise InvalidURL(e, request=request)
    
        self.cert_verify(conn, request.url, verify, cert)
        url = self.request_url(request, proxies)
        self.add_headers(request, stream=stream, timeout=timeout, verify=verify, cert=cert, proxies=proxies)
    
        chunked = not (request.body is None or 'Content-Length' in request.headers)
    
        if isinstance(timeout, tuple):
            try:
                connect, read = timeout
                timeout = TimeoutSauce(connect=connect, read=read)
            except ValueError as e:
                # this may raise a string formatting error.
                err = ("Invalid timeout {}. Pass a (connect, read) "
                       "timeout tuple, or a single float to set "
                       "both timeouts to the same value".format(timeout))
                raise ValueError(err)
        elif isinstance(timeout, TimeoutSauce):
            pass
        else:
            timeout = TimeoutSauce(connect=timeout, read=timeout)
    
        try:
            if not chunked:
>               resp = conn.urlopen(
                    method=request.method,
                    url=url,
                    body=request.body,
                    headers=request.headers,
                    redirect=False,
                    assert_same_host=False,
                    preload_content=False,
                    decode_content=False,
                    retries=self.max_retries,
                    timeout=timeout
                )

../../../anaconda/envs/workhorse/lib/python3.8/site-packages/requests/adapters.py:439: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <urllib3.connectionpool.HTTPConnectionPool object at 0x7fd2dbf103a0>, method = 'POST', url = '/irisws/fedcatalog/1/query'
body = b'level=station\nlatitude=35.0\nlongitude=-110\nmaxradius=0.3\nformat=request\nTA * * LHZ 2017-01-01T00:00:00.000000 2017-01-01T00:01:00.000000'
headers = {'User-Agent': 'ObsPy/1.3.0.post0+15.gc5d0911cd9.obspy.master (Linux-4.15.0-171-generic-x86_64-with-glibc2.10, Python 3.8.6)', 'Accept-Encoding': 'gzip, deflate', 'Accept': '*/*', 'Connection': 'keep-alive', 'Content-Length': '136'}
retries = Retry(total=0, connect=None, read=False, redirect=None, status=None), redirect = False, assert_same_host = False, timeout = Timeout(connect=None, read=None, total=None), pool_timeout = None, release_conn = False, chunked = False
body_pos = None, response_kw = {'decode_content': False, 'preload_content': False}, parsed_url = Url(scheme=None, auth=None, host=None, port=None, path='/irisws/fedcatalog/1/query', query=None, fragment=None), destination_scheme = None
conn = None, release_this_conn = True, http_tunnel_required = False, err = None, clean_exit = False

    def urlopen(
        self,
        method,
        url,
        body=None,
        headers=None,
        retries=None,
        redirect=True,
        assert_same_host=True,
        timeout=_Default,
        pool_timeout=None,
        release_conn=None,
        chunked=False,
        body_pos=None,
        **response_kw
    ):
        """
        Get a connection from the pool and perform an HTTP request. This is the
        lowest level call for making a request, so you'll need to specify all
        the raw details.
    
        .. note::
    
           More commonly, it's appropriate to use a convenience method provided
           by :class:`.RequestMethods`, such as :meth:`request`.
    
        .. note::
    
           `release_conn` will only behave as expected if
           `preload_content=False` because we want to make
           `preload_content=False` the default behaviour someday soon without
           breaking backwards compatibility.
    
        :param method:
            HTTP request method (such as GET, POST, PUT, etc.)
    
        :param url:
            The URL to perform the request on.
    
        :param body:
            Data to send in the request body, either :class:`str`, :class:`bytes`,
            an iterable of :class:`str`/:class:`bytes`, or a file-like object.
    
        :param headers:
            Dictionary of custom headers to send, such as User-Agent,
            If-None-Match, etc. If None, pool headers are used. If provided,
            these headers completely replace any pool-specific headers.
    
        :param retries:
            Configure the number of retries to allow before raising a
            :class:`~urllib3.exceptions.MaxRetryError` exception.
    
            Pass ``None`` to retry until you receive a response. Pass a
            :class:`~urllib3.util.retry.Retry` object for fine-grained control
            over different types of retries.
            Pass an integer number to retry connection errors that many times,
            but no other types of errors. Pass zero to never retry.
    
            If ``False``, then retries are disabled and any exception is raised
            immediately. Also, instead of raising a MaxRetryError on redirects,
            the redirect response will be returned.
    
        :type retries: :class:`~urllib3.util.retry.Retry`, False, or an int.
    
        :param redirect:
            If True, automatically handle redirects (status codes 301, 302,
            303, 307, 308). Each redirect counts as a retry. Disabling retries
            will disable redirect, too.
    
        :param assert_same_host:
            If ``True``, will make sure that the host of the pool requests is
            consistent else will raise HostChangedError. When ``False``, you can
            use the pool on an HTTP proxy and request foreign hosts.
    
        :param timeout:
            If specified, overrides the default timeout for this one
            request. It may be a float (in seconds) or an instance of
            :class:`urllib3.util.Timeout`.
    
        :param pool_timeout:
            If set and the pool is set to block=True, then this method will
            block for ``pool_timeout`` seconds and raise EmptyPoolError if no
            connection is available within the time period.
    
        :param release_conn:
            If False, then the urlopen call will not release the connection
            back into the pool once a response is received (but will release if
            you read the entire contents of the response such as when
            `preload_content=True`). This is useful if you're not preloading
            the response's content immediately. You will need to call
            ``r.release_conn()`` on the response ``r`` to return the connection
            back into the pool. If None, it takes the value of
            ``response_kw.get('preload_content', True)``.
    
        :param chunked:
            If True, urllib3 will send the body using chunked transfer
            encoding. Otherwise, urllib3 will send the body using the standard
            content-length form. Defaults to False.
    
        :param int body_pos:
            Position to seek to in file-like body in the event of a retry or
            redirect. Typically this won't need to be set because urllib3 will
            auto-populate the value when needed.
    
        :param \\**response_kw:
            Additional parameters are passed to
            :meth:`urllib3.response.HTTPResponse.from_httplib`
        """
    
        parsed_url = parse_url(url)
        destination_scheme = parsed_url.scheme
    
        if headers is None:
            headers = self.headers
    
        if not isinstance(retries, Retry):
            retries = Retry.from_int(retries, redirect=redirect, default=self.retries)
    
        if release_conn is None:
            release_conn = response_kw.get("preload_content", True)
    
        # Check host
        if assert_same_host and not self.is_same_host(url):
            raise HostChangedError(self, url, retries)
    
        # Ensure that the URL we're connecting to is properly encoded
        if url.startswith("/"):
            url = six.ensure_str(_encode_target(url))
        else:
            url = six.ensure_str(parsed_url.url)
    
        conn = None
    
        # Track whether `conn` needs to be released before
        # returning/raising/recursing. Update this variable if necessary, and
        # leave `release_conn` constant throughout the function. That way, if
        # the function recurses, the original value of `release_conn` will be
        # passed down into the recursive call, and its value will be respected.
        #
        # See issue #651 [1] for details.
        #
        # [1] <https://github.com/urllib3/urllib3/issues/651>
        release_this_conn = release_conn
    
        http_tunnel_required = connection_requires_http_tunnel(
            self.proxy, self.proxy_config, destination_scheme
        )
    
        # Merge the proxy headers. Only done when not using HTTP CONNECT. We
        # have to copy the headers dict so we can safely change it without those
        # changes being reflected in anyone else's copy.
        if not http_tunnel_required:
            headers = headers.copy()
            headers.update(self.proxy_headers)
    
        # Must keep the exception bound to a separate variable or else Python 3
        # complains about UnboundLocalError.
        err = None
    
        # Keep track of whether we cleanly exited the except block. This
        # ensures we do proper cleanup in finally.
        clean_exit = False
    
        # Rewind body position, if needed. Record current position
        # for future rewinds in the event of a redirect/retry.
        body_pos = set_file_position(body, body_pos)
    
        try:
            # Request a connection from the queue.
            timeout_obj = self._get_timeout(timeout)
            conn = self._get_conn(timeout=pool_timeout)
    
            conn.timeout = timeout_obj.connect_timeout
    
            is_new_proxy_conn = self.proxy is not None and not getattr(
                conn, "sock", None
            )
            if is_new_proxy_conn and http_tunnel_required:
                self._prepare_proxy(conn)
    
            # Make the request on the httplib connection object.
>           httplib_response = self._make_request(
                conn,
                method,
                url,
                timeout=timeout_obj,
                body=body,
                headers=headers,
                chunked=chunked,
            )

../../../anaconda/envs/workhorse/lib/python3.8/site-packages/urllib3/connectionpool.py:699: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <urllib3.connectionpool.HTTPConnectionPool object at 0x7fd2dbf103a0>, conn = <urllib3.connection.HTTPConnection object at 0x7fd2dbf10df0>, method = 'POST', url = '/irisws/fedcatalog/1/query'
timeout = Timeout(connect=None, read=None, total=None), chunked = False
httplib_request_kw = {'body': b'level=station\nlatitude=35.0\nlongitude=-110\nmaxradius=0.3\nformat=request\nTA * * LHZ 2017-01-01T00:00:00...hon 3.8.6)', 'Accept-Encoding': 'gzip, deflate', 'Accept': '*/*', 'Connection': 'keep-alive', 'Content-Length': '136'}}
timeout_obj = Timeout(connect=None, read=None, total=None), read_timeout = None

    def _make_request(
        self, conn, method, url, timeout=_Default, chunked=False, **httplib_request_kw
    ):
        """
        Perform a request on a given urllib connection object taken from our
        pool.
    
        :param conn:
            a connection from one of our connection pools
    
        :param timeout:
            Socket timeout in seconds for the request. This can be a
            float or integer, which will set the same timeout value for
            the socket connect and the socket read, or an instance of
            :class:`urllib3.util.Timeout`, which gives you more fine-grained
            control over your timeouts.
        """
        self.num_requests += 1
    
        timeout_obj = self._get_timeout(timeout)
        timeout_obj.start_connect()
        conn.timeout = timeout_obj.connect_timeout
    
        # Trigger any extra validation we need to do.
        try:
            self._validate_conn(conn)
        except (SocketTimeout, BaseSSLError) as e:
            # Py2 raises this as a BaseSSLError, Py3 raises it as socket timeout.
            self._raise_timeout(err=e, url=url, timeout_value=conn.timeout)
            raise
    
        # conn.request() calls http.client.*.request, not the method in
        # urllib3.request. It also calls makefile (recv) on the socket.
        try:
            if chunked:
                conn.request_chunked(method, url, **httplib_request_kw)
            else:
                conn.request(method, url, **httplib_request_kw)
    
        # We are swallowing BrokenPipeError (errno.EPIPE) since the server is
        # legitimately able to close the connection after sending a valid response.
        # With this behaviour, the received response is still readable.
        except BrokenPipeError:
            # Python 3
            pass
        except IOError as e:
            # Python 2 and macOS/Linux
            # EPIPE and ESHUTDOWN are BrokenPipeError on Python 2, and EPROTOTYPE is needed on macOS
            # https://erickt.github.io/blog/2014/11/19/adventures-in-debugging-a-potential-osx-kernel-bug/
            if e.errno not in {
                errno.EPIPE,
                errno.ESHUTDOWN,
                errno.EPROTOTYPE,
            }:
                raise
    
        # Reset the timeout for the recv() on the socket
        read_timeout = timeout_obj.read_timeout
    
        # App Engine doesn't have a sock attr
        if getattr(conn, "sock", None):
            # In Python 3 socket.py will catch EAGAIN and return None when you
            # try and read into the file pointer created by http.client, which
            # instead raises a BadStatusLine exception. Instead of catching
            # the exception and assuming all BadStatusLine exceptions are read
            # timeouts, check for a zero timeout before making the request.
            if read_timeout == 0:
                raise ReadTimeoutError(
                    self, url, "Read timed out. (read timeout=%s)" % read_timeout
                )
            if read_timeout is Timeout.DEFAULT_TIMEOUT:
                conn.sock.settimeout(socket.getdefaulttimeout())
            else:  # None or a value
                conn.sock.settimeout(read_timeout)
    
        # Receive the response from the server
        try:
            try:
                # Python 2.7, use buffering of HTTP responses
                httplib_response = conn.getresponse(buffering=True)
            except TypeError:
                # Python 3
                try:
                    httplib_response = conn.getresponse()
                except BaseException as e:
                    # Remove the TypeError from the exception chain in
                    # Python 3 (including for exceptions like SystemExit).
                    # Otherwise it looks like a bug in the code.
>                   six.raise_from(e, None)

../../../anaconda/envs/workhorse/lib/python3.8/site-packages/urllib3/connectionpool.py:445: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

value = None, from_value = None

>   ???

<string>:3: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <urllib3.connectionpool.HTTPConnectionPool object at 0x7fd2dbf103a0>, conn = <urllib3.connection.HTTPConnection object at 0x7fd2dbf10df0>, method = 'POST', url = '/irisws/fedcatalog/1/query'
timeout = Timeout(connect=None, read=None, total=None), chunked = False
httplib_request_kw = {'body': b'level=station\nlatitude=35.0\nlongitude=-110\nmaxradius=0.3\nformat=request\nTA * * LHZ 2017-01-01T00:00:00...hon 3.8.6)', 'Accept-Encoding': 'gzip, deflate', 'Accept': '*/*', 'Connection': 'keep-alive', 'Content-Length': '136'}}
timeout_obj = Timeout(connect=None, read=None, total=None), read_timeout = None

    def _make_request(
        self, conn, method, url, timeout=_Default, chunked=False, **httplib_request_kw
    ):
        """
        Perform a request on a given urllib connection object taken from our
        pool.
    
        :param conn:
            a connection from one of our connection pools
    
        :param timeout:
            Socket timeout in seconds for the request. This can be a
            float or integer, which will set the same timeout value for
            the socket connect and the socket read, or an instance of
            :class:`urllib3.util.Timeout`, which gives you more fine-grained
            control over your timeouts.
        """
        self.num_requests += 1
    
        timeout_obj = self._get_timeout(timeout)
        timeout_obj.start_connect()
        conn.timeout = timeout_obj.connect_timeout
    
        # Trigger any extra validation we need to do.
        try:
            self._validate_conn(conn)
        except (SocketTimeout, BaseSSLError) as e:
            # Py2 raises this as a BaseSSLError, Py3 raises it as socket timeout.
            self._raise_timeout(err=e, url=url, timeout_value=conn.timeout)
            raise
    
        # conn.request() calls http.client.*.request, not the method in
        # urllib3.request. It also calls makefile (recv) on the socket.
        try:
            if chunked:
                conn.request_chunked(method, url, **httplib_request_kw)
            else:
                conn.request(method, url, **httplib_request_kw)
    
        # We are swallowing BrokenPipeError (errno.EPIPE) since the server is
        # legitimately able to close the connection after sending a valid response.
        # With this behaviour, the received response is still readable.
        except BrokenPipeError:
            # Python 3
            pass
        except IOError as e:
            # Python 2 and macOS/Linux
            # EPIPE and ESHUTDOWN are BrokenPipeError on Python 2, and EPROTOTYPE is needed on macOS
            # https://erickt.github.io/blog/2014/11/19/adventures-in-debugging-a-potential-osx-kernel-bug/
            if e.errno not in {
                errno.EPIPE,
                errno.ESHUTDOWN,
                errno.EPROTOTYPE,
            }:
                raise
    
        # Reset the timeout for the recv() on the socket
        read_timeout = timeout_obj.read_timeout
    
        # App Engine doesn't have a sock attr
        if getattr(conn, "sock", None):
            # In Python 3 socket.py will catch EAGAIN and return None when you
            # try and read into the file pointer created by http.client, which
            # instead raises a BadStatusLine exception. Instead of catching
            # the exception and assuming all BadStatusLine exceptions are read
            # timeouts, check for a zero timeout before making the request.
            if read_timeout == 0:
                raise ReadTimeoutError(
                    self, url, "Read timed out. (read timeout=%s)" % read_timeout
                )
            if read_timeout is Timeout.DEFAULT_TIMEOUT:
                conn.sock.settimeout(socket.getdefaulttimeout())
            else:  # None or a value
                conn.sock.settimeout(read_timeout)
    
        # Receive the response from the server
        try:
            try:
                # Python 2.7, use buffering of HTTP responses
                httplib_response = conn.getresponse(buffering=True)
            except TypeError:
                # Python 3
                try:
>                   httplib_response = conn.getresponse()

../../../anaconda/envs/workhorse/lib/python3.8/site-packages/urllib3/connectionpool.py:440: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <urllib3.connection.HTTPConnection object at 0x7fd2dbf10df0>

    def getresponse(self):
        """Get the response from the server.
    
        If the HTTPConnection is in the correct state, returns an
        instance of HTTPResponse or of whatever object is returned by
        the response_class variable.
    
        If a request has not been sent or if a previous response has
        not be handled, ResponseNotReady is raised.  If the HTTP
        response indicates that the connection should be closed, then
        it will be closed before the response is returned.  When the
        connection is closed, the underlying socket is closed.
        """
    
        # if a prior response has been completed, then forget about it.
        if self.__response and self.__response.isclosed():
            self.__response = None
    
        # if a prior response exists, then it must be completed (otherwise, we
        # cannot read this response's header to determine the connection-close
        # behavior)
        #
        # note: if a prior response existed, but was connection-close, then the
        # socket and response were made independent of this HTTPConnection
        # object since a new request requires that we open a whole new
        # connection
        #
        # this means the prior response had one of two states:
        #   1) will_close: this connection was reset and the prior socket and
        #                  response operate independently
        #   2) persistent: the response was retained and we await its
        #                  isclosed() status to become true.
        #
        if self.__state != _CS_REQ_SENT or self.__response:
            raise ResponseNotReady(self.__state)
    
        if self.debuglevel > 0:
            response = self.response_class(self.sock, self.debuglevel,
                                           method=self._method)
        else:
            response = self.response_class(self.sock, method=self._method)
    
        try:
            try:
>               response.begin()

../../../anaconda/envs/workhorse/lib/python3.8/http/client.py:1347: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <http.client.HTTPResponse object at 0x7fd2dbf10f70>

    def begin(self):
        if self.headers is not None:
            # we've already started reading the response
            return
    
        # read until we get a non-100 response
        while True:
>           version, status, reason = self._read_status()

../../../anaconda/envs/workhorse/lib/python3.8/http/client.py:307: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <http.client.HTTPResponse object at 0x7fd2dbf10f70>

    def _read_status(self):
>       line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")

../../../anaconda/envs/workhorse/lib/python3.8/http/client.py:268: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <socket.SocketIO object at 0x7fd2dbf101f0>, b = <memory at 0x7fd2b4139640>

    def readinto(self, b):
        """Read up to len(b) bytes into the writable buffer *b* and return
        the number of bytes read.  If the socket is non-blocking and no bytes
        are available, None is returned.
    
        If *b* is non-empty, a 0 return value indicates that the connection
        was shutdown at the other end.
        """
        self._checkClosed()
        self._checkReadable()
        if self._timeout_occurred:
            raise OSError("cannot read from timed out object")
        while True:
            try:
>               return self._sock.recv_into(b)
E               KeyboardInterrupt

../../../anaconda/envs/workhorse/lib/python3.8/socket.py:669: KeyboardInterrupt
========================================================================================= 82 passed, 2 skipped, 13 deselected in 2520.70s (0:42:00) ==========================================================================================
```


Discussion:
We could maybe try to adjust timeout, the default of 120s might be OK for user land, but certainly not for testing and especially not for routed clients (like above) that might talk to a dozen FDSN / EIDA nodes.

I'm not sure how to do this most elegantly though.. maybe there's a nice trick in pytest somewhere with these fixture thingies? We'd kinda want to do a `client.timeout = 10` after any client initialization in tests or override the default `timeout=120` in their `__init__` methods..
That being said, there's no guarantee setting `timeout` fixes all hanging network problems.. it should in principle ideally, but can never be sure
I think either timeout is completely missing or something else is not working. A time out of 120 s cannot explain the 6 hour run time for the action run linked above (before it gets canceled).
> I think either timeout is completely missing or something else is not working. A time out of 120 s cannot explain the 6 hour run time for the action run linked above (before it gets canceled).

An issue could be the configuration of the FDSN web server itself that wants to serve the seedlink data, allowing request for "the future" and waiting for potential data. But the timeout should work (except it has to be implemented in a separate thread, to be "killable" properly, I think I had this issue before too)
> 6 hour run time for the action run

GitHub lets an action run for 6 hours before aborting it? 0_o :rofl: 

> A time out of 120 s cannot explain the 6 hour run time

Agreed, but that fail is in a FDSN routing test, so there are tons of individual requests involved, at least a few dozens I'd say, and timeout applies to one single request, so in the worst case you could have `120s * no. of requests`. But I totally agree, it's really fishy. I could only imagine the runner losing network connection completely or something, but it's hard to imagine that happening and if it would, I'd really expect github to cancel the run after a short while
![Screenshot from 2022-03-15 11-51-10](https://user-images.githubusercontent.com/1842780/158362203-1833e011-7a5d-449e-9b95-6d1836b141cb.png)

Maybe, we could do verbose on all test run actions? Then we could see exactly what test method takes that long, although it's probably safe to assume the last method that ran is responsible
> Maybe, we could do verbose on all test run actions?

:+1:

> Then we could see exactly what test method takes that long, although it's probably safe to assume the last method that ran is responsible

Yes, I think its the same method as in the above traceback.  `test_get_stations_integration_test`




Resolving patch:
diff --git a/obspy/clients/base.py b/obspy/clients/base.py
--- a/obspy/clients/base.py
+++ b/obspy/clients/base.py
@@ -195,7 +195,8 @@ def _download(self, url, params=None, filename=None, data=None):
 
         _request_args = {"url": url,
                          "headers": {"User-Agent": self._user_agent},
-                         "params": params}
+                         "params": params,
+                         "timeout": self._timeout}
 
         # Stream to file - no need to keep it in memory for large files.
         if filename:


Test patch:
diff --git a/obspy/clients/fdsn/tests/test_eidaws_routing_client.py b/obspy/clients/fdsn/tests/test_eidaws_routing_client.py
--- a/obspy/clients/fdsn/tests/test_eidaws_routing_client.py
+++ b/obspy/clients/fdsn/tests/test_eidaws_routing_client.py
@@ -26,7 +26,7 @@
 
 class EIDAWSRoutingClientTestCase(unittest.TestCase):
     def setUp(self):
-        self.client = EIDAWSRoutingClient()
+        self.client = EIDAWSRoutingClient(timeout=20)
         self._cls = ("obspy.clients.fdsn.routing.eidaws_routing_client."
                      "EIDAWSRoutingClient")