Repository: obspy/obspy

Issue:
massdownloader not finding stations
If you run the code below, the massdownloader wont find any stations/data in the circular region.  However I know there are stations there with data, and in the bottom of attached code, I show that the normal fdsn  client.get_stations  returns what I expect.

Am I doing something wrong? Or is there something broken in the massdownloader?

```

import obspy
from obspy.clients.fdsn.mass_downloader import CircularDomain, Restrictions, MassDownloader
from obspy.clients.fdsn import Client
from obspy.core import UTCDateTime
from obspy.clients.fdsn.header import URL_MAPPINGS

lat = 39.883
lon = -88.901
minR = 0.0
maxR = 1.0


st = obspy.UTCDateTime(2014,1,1, 1,0,0)
et = st+(3600*24*14)       

# via the massdownloader
print('\n\n')
print('------------- MASSDOWNLOADER OUTPUT -------------------')
domain = domain = CircularDomain(latitude=lat, longitude=lon, minradius=minR, maxradius=maxR)
restrictions = Restrictions( starttime=st, endtime=et, network='GS', minimum_length=0, reject_channels_with_gaps=False, sanitize=False)

mdl = MassDownloader(providers=['IRIS'])
mdl.download(domain, restrictions, mseed_storage="tmp/waveforms", stationxml_storage="tmp/stations") 



#now via the regular fdsn
print('\n\n\n\n')
print('--------------- FDSN Client.getStations OUTPUT ----------' )
client = Client('IRIS')
inv = client.get_stations(network='GS', starttime=st, endtime=et, latitude=lat, longitude=lon, minradius=minR, maxradius=maxR)

print(inv)

```

Here is the output...

```
------------- MASSDOWNLOADER OUTPUT -------------------
[2018-05-21 17:25:02,308] - obspy.clients.fdsn.mass_downloader - INFO: Initializing FDSN client(s) for IRIS.
[2018-05-21 17:25:06,285] - obspy.clients.fdsn.mass_downloader - INFO: Successfully initialized 1 client(s): IRIS.
[2018-05-21 17:25:06,285] - obspy.clients.fdsn.mass_downloader - INFO: Total acquired or preexisting stations: 0
[2018-05-21 17:25:06,285] - obspy.clients.fdsn.mass_downloader - INFO: Client 'IRIS' - Requesting reliable availability.
[2018-05-21 17:25:06,461] - obspy.clients.fdsn.mass_downloader - INFO: Client 'IRIS' - Successfully requested availability (0.18 seconds)
[2018-05-21 17:25:06,467] - obspy.clients.fdsn.mass_downloader - INFO: Client 'IRIS' - Found 0 stations (0 channels).
[2018-05-21 17:25:06,468] - obspy.clients.fdsn.mass_downloader - INFO: Client 'IRIS' - No data available.
[2018-05-21 17:25:06,468] - obspy.clients.fdsn.mass_downloader - INFO: ============================== Final report
[2018-05-21 17:25:06,468] - obspy.clients.fdsn.mass_downloader - INFO: 0 MiniSEED files [0.0 MB] already existed.
[2018-05-21 17:25:06,468] - obspy.clients.fdsn.mass_downloader - INFO: 0 StationXML files [0.0 MB] already existed.
[2018-05-21 17:25:06,468] - obspy.clients.fdsn.mass_downloader - INFO: Client 'IRIS' - Acquired 0 MiniSEED files [0.0 MB].
[2018-05-21 17:25:06,468] - obspy.clients.fdsn.mass_downloader - INFO: Client 'IRIS' - Acquired 0 StationXML files [0.0 MB].
[2018-05-21 17:25:06,468] - obspy.clients.fdsn.mass_downloader - INFO: Downloaded 0.0 MB in total.



--------------- FDSN Client.getStations OUTPUT ----------
Inventory created at 2018-05-21T23:25:06.000000Z
	Created by: IRIS WEB SERVICE: fdsnws-station | version: 1.1.33
		    http://service.iris.edu/fdsnws/station/1/query?starttime=2014-01-01...
	Sending institution: IRIS-DMC (IRIS-DMC)
	Contains:
		Networks (1):
			GS
                Stations (12):
			GS.DEC01 (Heartland Borehole/Surface)
			GS.DEC02 (ADM Borehole/Surface)
			GS.DEC03 (Forest Parkway Borehole/Surface)
			GS.DEC04 (Sangamon Road Surface Site)
			GS.DEC05 (Apollo Drive Surface Site)
			GS.DEC06 (Neely Ave Surface Site)
			GS.DEC07 (Harrison Ave Surface Site)
			GS.DEC08 (Union Local 159 Surface Site)
			GS.DEC09 (Kile Street Surface Site)
			GS.DEC10 (Finch Street Surface Site)
			GS.DEC11 (Phillips Surface Site)
			GS.DEC12 (Boyd Road Surface Site)
		Channels (0):
```



Discussion:
You're unlucky because of the default of `location_priorities` in the `Restrictions` class.. try something like `Restrictions(..., location_priorities=('', '00', '10', '01', '02'))` and it fetches the data.

@krischer, I don't full understand the reasoning of this (default) `location_priorities` setting. As you even mention in the docstring of `Restriction`, `"Location codes are arbitrary and there is no rule as to which location is best."` I totally agree and I would strongly suggest to change the default behavior to regard channels with *any* location code.

(Furthermore, I think it's usual to use `01`, `02` when incrementing location code, one more point why I don't understand the default of `location_priorities=("", "00", "10")`)
yep, that worked.  Thanks for the quick reply guys  
I'll reopen this, since I want to hear opinions on the `location_priorities` issue. I think it needs a change.
> @krischer, I don't full understand the reasoning of this (default) location_priorities setting. As you even mention in the docstring of Restriction, "Location codes are arbitrary and there is no rule as to which location is best." I totally agree and I would strongly suggest to change the default behavior to regard channels with any location code.
> 
> (Furthermore, I think it's usual to use 01, 02 when incrementing location code, one more point why I don't understand the default of location_priorities=("", "00", "10"))

Yes I think I agree with changing this to a better default. Accepting anything by default would probably be a bad idea as some stations have a very large number of locations and the mass downloader would then just download everything with the way things currently are.

How about expanding the default values to `["", "00", "10", "01", "02", "03", "20"]`? I'd be happy to add other locations codes as well.
> the mass downloader would then just download everything with the way things currently are.

maybe that should be changed in the long run.. but yeah, low priority I guess


> How about expanding the default values to ["", "00", "10", "01", "02", "03", "20"]? I'd be happy to add other locations codes as well.

Much better than current situation, but also just a quick fix that doesn't really solve the underlying problem just works around it for some common cases.
For what it's worth, what I was expecting was that the default location (and channel) code would be '*' (everything), and that if someone wanted just a subset of the station's data, then they would specify what they wanted.
> maybe that should be changed in the long run.. but yeah, low priority I guess

I don't see how to be honest. There must be some kind of filtering happening. The only possibility is to allow for some kind of callback function to be passed as a `location_priority` which would enable these kind of things but that seems like a bit of over-engineering.

> I was expecting was that the default location (and channel) code would be '*' (everything)

For some stations, e.g. `TA` network, this would download a fairly large amount of data and one could just directly use the normal `obspy.clients.fdsn.client.Client` for that. The mass downloader has been designed to offer a solution to exactly this problem - it tries to find the "most suitable" location + channel code combination for any station part of the selection. For the location codes this is of course fairly arbitrary as they have no semantic meaning.


As I cannot think of another solution I propose to just use the 20 most common location codes in the whole IRIS inventory:

`"", "01", "00", "EP", "S1", "S3", "02", "10", "09", "08", "03", "04", "06", "07", "05", "20", "T0", "2C", "40", "50"`

Any objections?
What are the `EP`, `S1`, `S3`, `T0`, and `2C` location code supposed to be?
If someone cares - here are the counts for encountered location codes in the IRIS inventory:

```python
[('', 428516),
 ('01', 39859),
 ('00', 28661),
 ('EP', 16816),
 ('S1', 15558),
 ('S3', 15540),
 ('02', 11431),
 ('10', 9125),
 ('09', 8768),
 ('08', 8368),
 ('03', 5390),
 ('04', 5169),
 ('06', 4643),
 ('07', 4622),
 ('05', 4458),
 ('20', 4226),
 ('T0', 1804),
 ('2C', 768),
 ('40', 762),
 ('50', 587)]
```

After this arbitrary cut-off it becomes pretty obscure but there are 507 different location codes all in all...
> As I cannot think of another solution I propose to just use the 20 most common location codes in the whole IRIS inventory:

Sounds good, and nice idea to just do statistics. For more crazed stats, we could multiply location codes by their encountered duration.. ;-)
Hmm, maybe we move this to 1.2.0? Changes behavior a bit, and then we could do 1.1.1 now and people are supposed to move to 1.2.0 soonish anyway..
> What are the EP, S1, S3, T0, and 2C location code supposed to be?

Interesting that there are so strange ones among the top list..
Okay - so `EP` seems to be either barometers, the clock phase, or some state of health channel, `S1`/`S3` either accelerometers. fiberoptic strain sensors, or synthetics seismograms,  `T0` strainmeters, and `2C` are accelerometers.

I'll not include these in the default list but only all the "numerical" codes and the empty one.

Resolving patch:
diff --git a/obspy/clients/fdsn/mass_downloader/download_helpers.py b/obspy/clients/fdsn/mass_downloader/download_helpers.py
--- a/obspy/clients/fdsn/mass_downloader/download_helpers.py
+++ b/obspy/clients/fdsn/mass_downloader/download_helpers.py
@@ -1219,9 +1219,20 @@ def get_loc(x):
                 if self.restrictions.location is None:
                     # Filter to remove unwanted locations according to the
                     # priority list.
+                    has_channels_before_filtering = bool(channels)
                     channels = utils.filter_channel_priority(
                         channels, key="location",
                         priorities=self.restrictions.location_priorities)
+                    # This has been a point of confusion for users so raise a
+                    # warning in case this removed all channels and is still
+                    # using the default settings.
+                    if not channels and has_channels_before_filtering and \
+                            self.restrictions._loc_prios_are_default_values:
+                        self.logger.warning(
+                            "Client '%s' - No channel at station %s.%s has "
+                            "been selected due to the `location_priorities` "
+                            "settings." % (self.client_name, network.code,
+                                           station.code))
 
                 if not channels:
                     continue
diff --git a/obspy/clients/fdsn/mass_downloader/restrictions.py b/obspy/clients/fdsn/mass_downloader/restrictions.py
--- a/obspy/clients/fdsn/mass_downloader/restrictions.py
+++ b/obspy/clients/fdsn/mass_downloader/restrictions.py
@@ -201,7 +201,21 @@ def __init__(self, starttime, endtime,
                                      "BL[ZNE12]", "ML[ZNE12]",
                                      "EL[ZNE12]", "LL[ZNE12]",
                                      "SH[ZNE12]"),
-                 location_priorities=("", "00", "10")):
+                 location_priorities=("", "00", "10", "01", "20", "02", "30",
+                                      "03", "40", "04", "50", "05", "60",
+                                      "06", "70", "07", "80", "08", "90",
+                                      "09")):
+        # Awkward logic to keep track whether or not the location priorities
+        # are equal to the default values. This "solution" keeps the function
+        # signature intact and it also located close to where the location
+        # priorities are set.
+        if location_priorities == (
+                "", "00", "10", "01", "20", "02", "30", "03", "40", "04", "50",
+                "05", "60", "06", "70", "07", "80", "08", "90", "09"):
+            self._loc_prios_are_default_values = True
+        else:
+            self._loc_prios_are_default_values = False
+
         self.starttime = obspy.UTCDateTime(starttime)
         self.endtime = obspy.UTCDateTime(endtime)
         self.station_starttime = station_starttime and \


Test patch:
diff --git a/obspy/clients/fdsn/tests/data/channel_level_fdsn_obscure_location_code.txt b/obspy/clients/fdsn/tests/data/channel_level_fdsn_obscure_location_code.txt
new file mode 100644
--- /dev/null
+++ b/obspy/clients/fdsn/tests/data/channel_level_fdsn_obscure_location_code.txt
@@ -0,0 +1,2 @@
+#Network | Station | Location | Channel | Latitude | Longitude | Elevation | Depth | Azimuth | Dip | SensorDescription | Scale | ScaleFreq | ScaleUnits |  SampleRate | StartTime | EndTime
+AK|BAGL|XY|LHZ|60.4896|-142.0915|1470.0|0.0|0.0|-90.0|Nanometrics Trillium 240 Sec Response sn 400 and a|4.88233E8|0.02|M/S|1.0|2013-01-01T00:00:00|2599-12-31T23:59:59
\ No newline at end of file
diff --git a/obspy/clients/fdsn/tests/test_mass_downloader.py b/obspy/clients/fdsn/tests/test_mass_downloader.py
--- a/obspy/clients/fdsn/tests/test_mass_downloader.py
+++ b/obspy/clients/fdsn/tests/test_mass_downloader.py
@@ -2571,6 +2571,37 @@ def test_parse_miniseed_filenames(self):
             with open(filename, "w") as buf:
                 buf.write("obspy")
 
+    def test_warning_when_location_prios_excludes_all_channels(self):
+        """
+        Tests that the logger raises a warning when the location_priorities
+        settings excludes all channels.
+        """
+        # No warning should have been raised yet.
+        self.assertEqual(self.logger.warning.call_count, 0)
+        c = self._init_client()
+        c.client.get_stations.return_value = obspy.read_inventory(
+            os.path.join(self.data,
+                         "channel_level_fdsn_obscure_location_code.txt"))
+        c.get_availability()
+        # Nothing should have been selected.
+        self.assertEqual(c.stations, {})
+        # But a warning should have been raised.
+        self.assertEqual(self.logger.warning.call_count, 1)
+        self.assertEqual(
+            self.logger.warning.call_args[0][0],
+            "Client 'Test' - No channel at station AK.BAGL has been selected "
+            "due to the `location_priorities` settings.")
+
+        self.logger.warning.reset_mock()
+        self.assertEqual(self.logger.warning.call_count, 0)
+        # Having non-default location priorities should not warn.
+        self.restrictions = Restrictions(
+            starttime=obspy.UTCDateTime(2001, 1, 1),
+            endtime=obspy.UTCDateTime(2015, 1, 1),
+            location_priorities=["00"])
+        self.assertEqual(c.stations, {})
+        self.assertEqual(self.logger.warning.call_count, 0)
+
 
 class DownloadHelperTestCase(unittest.TestCase):
     """