Repository: obspy/obspy

Issue:
KeyError: 'station' using RoutingClient
I wanted to create a plot of all available permanent stations.

What would be the best way to do it?

On my way I stumbled upon the following Exception.

Code:
```
from obspy.clients.fdsn import RoutingClient
from obspy import UTCDateTime as UTC

client = RoutingClient('iris-federator', debug=True)
inv1 = client.get_stations(
        starttime=UTC('2018-01-01'),
        endtime=UTC('2018-02-01'),
        level='station')
inv1.write('station.txt', 'STATIONTXT', level='station')
```

Tail of the output:

```
Error while downloading: http://eida-service.koeri.boun.edu.tr/fdsnws/dataselect/1/application.wadl
Error while downloading: http://eida-service.koeri.boun.edu.tr/fdsnws/event/1/application.wadl
Error while downloading: http://eida-service.koeri.boun.edu.tr/fdsnws/station/1/application.wadl
Error while downloading: http://eida-service.koeri.boun.edu.tr/fdsnws/event/1/contributors
Error while downloading: http://eida-service.koeri.boun.edu.tr/fdsnws/event/1/catalogs
/home/eule/dev/obspy/obspy/clients/fdsn/routing/routing_client.py:103: UserWarning: No FDSN services could be discovered at 'http://eida-service.koeri.boun.edu.tr'. This could be due to a temporary service outage or an invalid FDSN service address.It will not be used for routing. Try again later?
  warnings.warn(msg)
Downloaded http://ws.resif.fr/fdsnws/station/1/query with HTTP code: 200
Error while downloading: http://service.scedc.caltech.edu/fdsnws/dataselect/1/application.wadl
Error while downloading: http://service.scedc.caltech.edu/fdsnws/event/1/catalogs
Error while downloading: http://service.scedc.caltech.edu/fdsnws/event/1/contributors
Error while downloading: http://service.scedc.caltech.edu/fdsnws/station/1/application.wadl
Discovered event service
Storing discovered services in cache.
Uncompressing gzipped response for http://service.iris.edu/fdsnws/station/1/query
Downloaded http://service.iris.edu/fdsnws/station/1/query with HTTP code: 200
Uncompressing gzipped response for http://webservices.ingv.it/fdsnws/station/1/query
Downloaded http://webservices.ingv.it/fdsnws/station/1/query with HTTP code: 200
HTTP error 504, reason Gateway Time-out, while downloading 'http://geofon.gfz-potsdam.de/fdsnws/station/1/query': b'<html>\r\n<head><title>504 Gateway Time-out</title></head>\r\n<body bgcolor="white">\r\n<center><h1>504 Gateway Time-out</h1></center>\r\n<hr><center>nginx/1.12.2</center>\r\n</body>\r\n</html>\r\n'
Error while downloading: http://service.ncedc.org/fdsnws/station/1/query
Traceback (most recent call last):
  File "test.py", line 25, in <module>
    inv1 = get_stations()
  File "test.py", line 12, in get_stations
    level='station')
  File "<decorator-gen-61>", line 2, in get_stations
  File "/home/eule/dev/obspy/obspy/clients/fdsn/routing/routing_client.py", line 79, in _assert_filename_not_in_kwargs
    return f(*args, **kwargs)
  File "/home/eule/dev/obspy/obspy/clients/fdsn/routing/federator_routing_client.py", line 110, in get_stations
    return super(FederatorRoutingClient, self).get_stations(**kwargs)
  File "<decorator-gen-58>", line 2, in get_stations
  File "/home/eule/dev/obspy/obspy/clients/fdsn/routing/routing_client.py", line 79, in _assert_filename_not_in_kwargs
    return f(*args, **kwargs)
  File "/home/eule/dev/obspy/obspy/clients/fdsn/routing/routing_client.py", line 344, in get_stations
    return self.get_stations_bulk([bulk], **kwargs)
  File "<decorator-gen-62>", line 2, in get_stations_bulk
  File "/home/eule/dev/obspy/obspy/clients/fdsn/routing/routing_client.py", line 79, in _assert_filename_not_in_kwargs
    return f(*args, **kwargs)
  File "/home/eule/dev/obspy/obspy/clients/fdsn/routing/federator_routing_client.py", line 147, in get_stations_bulk
    return self._download_stations(split, **kwargs)
  File "/home/eule/dev/obspy/obspy/clients/fdsn/routing/routing_client.py", line 228, in _download_stations
    return self._download_parallel(split, data_type="station", **kwargs)
  File "/home/eule/dev/obspy/obspy/clients/fdsn/routing/routing_client.py", line 254, in _download_parallel
    results = pool.map(_download_bulk, dl_requests)
  File "/home/eule/anaconda/envs/workhorse/lib/python3.6/multiprocessing/pool.py", line 266, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
  File "/home/eule/anaconda/envs/workhorse/lib/python3.6/multiprocessing/pool.py", line 644, in get
    raise self._value
  File "/home/eule/anaconda/envs/workhorse/lib/python3.6/multiprocessing/pool.py", line 119, in worker
    result = (True, func(*args, **kwds))
  File "/home/eule/anaconda/envs/workhorse/lib/python3.6/multiprocessing/pool.py", line 44, in mapstar
    return list(map(*args))
  File "/home/eule/dev/obspy/obspy/clients/fdsn/routing/routing_client.py", line 115, in _download_bulk
    service = c.services["station"]
KeyError: 'station'
```




FDSN fedcatalog query problem
On current master there is one failing FDSN fedcatalog test.

Assuming that that test used to work when it was added, there must have been some change server side. The test is doing the following POST query:

```
$ cat fed.txt 
level=network
latitude=35.0
longitude=-110
maxradius=0.3
format=request
* * * LHZ 2017-01-01T00:00:00.000000 2017-01-01T00:05:00.000000
$ wget --post-file=fed.txt -O fedcatalog.txt http://service.iris.edu/irisws/fedcatalog/1/query
--2018-04-26 10:44:08--  http://service.iris.edu/irisws/fedcatalog/1/query
Resolving service.iris.edu (service.iris.edu)... 192.5.99.2
Connecting to service.iris.edu (service.iris.edu)|192.5.99.2|:80... connected.
HTTP request sent, awaiting response... 400 Bad Request
2018-04-26 10:44:09 ERROR 400: Bad Request.
```

When replacing `level=network` with `level=station`, the request succeeds and returns data instead of the error code.


Discussion:

Resolving patch:
diff --git a/obspy/clients/fdsn/routing/routing_client.py b/obspy/clients/fdsn/routing/routing_client.py
--- a/obspy/clients/fdsn/routing/routing_client.py
+++ b/obspy/clients/fdsn/routing/routing_client.py
@@ -18,6 +18,9 @@
 from multiprocessing.dummy import Pool as ThreadPool
 
 import decorator
+import io
+import sys
+import traceback
 import warnings
 
 from obspy.core.compatibility import (urlparse, string_types,
@@ -86,6 +89,17 @@ def _assert_attach_response_not_in_kwargs(f, *args, **kwargs):
     return f(*args, **kwargs)
 
 
+def _try_download_bulk(r):
+    try:
+        return _download_bulk(r)
+    except Exception:
+        reason = "".join(traceback.format_exception(*sys.exc_info()))
+        warnings.warn(
+            "Failed to download data of type '%s' from '%s' due to: \n%s" % (
+                r["data_type"], r["endpoint"], reason))
+        return None
+
+
 def _download_bulk(r):
     # Figure out the passed credentials, if any. Two possibilities:
     # (1) User and password, given explicitly for the base URLs (or an
@@ -251,7 +265,7 @@ def _download_parallel(self, split, data_type, **kwargs):
                 "kwargs": kwargs,
                 "credentials": self.credentials})
         pool = ThreadPool(processes=len(dl_requests))
-        results = pool.map(_download_bulk, dl_requests)
+        results = pool.map(_try_download_bulk, dl_requests)
 
         # Merge all results into a single object.
         if data_type == "waveform":
@@ -277,7 +291,16 @@ def _handle_requests_http_error(self, r):
 
         Please overwrite this method in a child class if necessary.
         """
-        raise_on_error(r.status_code, get_reason_from_response(r))
+        reason = get_reason_from_response(r)
+        if hasattr(r, "content"):
+            c = r.content
+            try:
+                c = c.encode()
+            except Exception:
+                pass
+            reason += b" -- " + c
+        with io.BytesIO(reason) as buf:
+            raise_on_error(r.status_code, buf)
 
     @_assert_filename_not_in_kwargs
     @_assert_attach_response_not_in_kwargs


Test patch:
diff --git a/obspy/clients/fdsn/tests/test_base_routing_client.py b/obspy/clients/fdsn/tests/test_base_routing_client.py
--- a/obspy/clients/fdsn/tests/test_base_routing_client.py
+++ b/obspy/clients/fdsn/tests/test_base_routing_client.py
@@ -13,6 +13,7 @@
 
 import collections
 import unittest
+import warnings
 
 import obspy
 from obspy.core.compatibility import mock
@@ -203,6 +204,32 @@ def test_downloading_stations(self):
         for _i in wf_bulk.call_args_list:
             self.assertEqual(_i[1], {})
 
+    def test_unexpected_exception_handling(self):
+        split = {
+            "https://example.com": "1234"
+        }
+
+        with mock.patch("obspy.clients.fdsn.client.Client") as p:
+            mock_instance = p.return_value
+            mock_instance.get_stations_bulk.side_effect = ValueError("random")
+            c = self._cls_object(debug=False, timeout=240)
+            # Should not fail, but a warning should be raised.
+            with warnings.catch_warnings(record=True) as w:
+                warnings.simplefilter("always")
+                inv = c._download_stations(split=split)
+
+        # Returns an empty inventory.
+        self.assertIsInstance(inv, obspy.core.inventory.Inventory)
+        self.assertEqual(len(inv), 0)
+
+        # Raises a nice warning.
+        self.assertEqual(len(w), 1)
+        msg = w[0].message.args[0]
+        self.assertTrue(msg.startswith(
+            "Failed to download data of type 'station' from "
+            "'https://example.com' due to:"))
+        self.assertIn("ValueError: random", msg)
+
 
 def suite():  # pragma: no cover
     return unittest.makeSuite(BaseRoutingClientTestCase, 'test')
diff --git a/obspy/clients/fdsn/tests/test_eidaws_routing_client.py b/obspy/clients/fdsn/tests/test_eidaws_routing_client.py
--- a/obspy/clients/fdsn/tests/test_eidaws_routing_client.py
+++ b/obspy/clients/fdsn/tests/test_eidaws_routing_client.py
@@ -146,8 +146,8 @@ def test_error_handling(self):
                 network="XX", station="XXXXX", location="XX", channel="XXX",
                 starttime=obspy.UTCDateTime(2017, 1, 1),
                 endtime=obspy.UTCDateTime(2017, 1, 2))
-        self.assertEqual(e.exception.args[0],
-                         "No data available for request.")
+        self.assertTrue(e.exception.args[0].startswith(
+            "No data available for request."))
 
     def test_get_waveforms(self):
         """
diff --git a/obspy/clients/fdsn/tests/test_federator_routing_client.py b/obspy/clients/fdsn/tests/test_federator_routing_client.py
--- a/obspy/clients/fdsn/tests/test_federator_routing_client.py
+++ b/obspy/clients/fdsn/tests/test_federator_routing_client.py
@@ -338,8 +338,8 @@ def test_get_stations_integration_test(self):
         inv = self.client.get_stations(
             starttime=obspy.UTCDateTime(2017, 1, 1),
             endtime=obspy.UTCDateTime(2017, 1, 1, 0, 1),
-            latitude=35.0, longitude=-110, maxradius=0.3,
-            channel="LHZ", level="network")
+            latitude=35.0, longitude=-110, maxradius=0.3, network="TA",
+            channel="LHZ", level="station")
         # This yields 1 network at the time of writing this test - I assume
         # it is unlikely to every yield less. So this test should be fairly
         # stable.
@@ -350,8 +350,9 @@ def test_get_stations_integration_test(self):
             [["*", "*", "*", "LHZ", obspy.UTCDateTime(2017, 1, 1),
               obspy.UTCDateTime(2017, 1, 1, 0, 1)]],
             latitude=35.0, longitude=-110, maxradius=0.3,
-            level="network")
+            level="station")
         self.assertGreaterEqual(len(inv2), 1)
+        inv2 = inv2.select(network="TA")
 
         # The results should be basically identical - they will still differ
         # because times stamps and also order might change slightly.