Repository: obspy/obspy

Issue:
read_nlloc_hyp issue when phase hour and minute is '2400' 
Hi,
obspy version: >1.1
platform: linux
install: frrom source

issue in `io.nlloc.core.read_nlloc_hyp` line (334)

When reading a nlloc phase file the problem arises when the phase hour and time is '2400', `UTCDateTime.strptime` can't handle it.
Proposed fix:
```
date, hourmin, sec = map(str, line[6:9])
sec=float(sec)
 if hourmin=='2400': 
      hourmin='2359'
      sec=sec+60
t = UTCDateTime.strptime(date + hourmin, "%Y%m%d%H%M") + sec
```
I hope this helps and can be integrated in the future,

Ch.


Discussion:
Hey @ChrisBail, good find. Would you mind writing a test that captures this behavior? If you are comfortable putting it in obspy [this](https://github.com/obspy/obspy/blob/master/obspy/io/nlloc/tests/test_core.py) would probably be the right place.  If not, you could attach a code sample here that reproduces the issue and I will make sure it gets included in the test suite. Once we have a test that fails then we can work on a fix for it.

It would be great if UTCDateTime could handle this directly without having to put this kind of logic in all the format parsers where this can crop up. I have seen this several times in other formats and I have usually resorted to ad-hoc checks as well. 
I'd consider this a broken time specification, but yeah, I guess we should allow for it. Actually, looking up ISO standard, it even explicitly mentions the 24:00:00 case: https://en.wikipedia.org/wiki/ISO_8601#Times

So, yeah, I think we should just add it as a special allowed case in `UTCDateTime.__init__` directly (even though `datetime.datetime` does not allow this special case). Other opinions?
> a special allowed case in `UTCDateTime.__init__` directly

we'd also have to patch `strptime`, I guess..
Thanks Derrick & Tobias for the quick reply, I would agree with Tobias to put that directly in UTCDateTime instead of doing it in read_nlloc_hyp. I  know for sure that NLLOC and SEISAN (nordic files) use the 24:00 convention.
Just to make sure.. do NLLOC and Seisan only produce 24:00:00 or does it also happen with additional hours or seconds?
From line 4 in seisan:
 19-20 I2 Hour 	Hour can be up to 48 to indicate next day 

It doesn't say anything about minutes or seconds going beyond their assumed bounds but I believe I have seen a 60 in the seconds column before (I can double check if needed).

Arc files (Y2000 archive format from hypoinverse) on the other hand, have a line that provides information on both P and S phases. Here is a snippet of the spec:

18 4 I4 Year. *
22 8 4I2 Month, day, hour and minute.
30 5 F5.2 Second of P arrival.
35 4 F4.2 P travel time residual.
39 3 F3.2 P weight actually used.
42 5 F5.2 Second of S arrival.

So in this case, seconds (especially on the S pick) can go to 60 and beyond. 

Thinking about this more, I would be happy if `UTCDateTime` just supported hour 24, minute 60 and second 60. It may not be necessary (or even a good idea) to support any higher numbers. 
In my nlloc phase file I doesn't seem to happen with others times, only 24:00. I'm digging into NLLoc.c source code to be more specific
In Nordic io I catch hour 24 and seconds 60, but I'm not sure if I catch minutes 60. Unfortunately on my phone so hard to check. If we do go down the route of allowing in UTCDateTime I would be keen to remove those catches from Nordic io.
I spent a few hours looking at this and it is probably going to be very hard to implement. Unless we can do something clever that I am not seeing, it would require writing a `time.strptime` implementation to use when the standard one fails. I don't know that supporting this edge case would be worth doing that.
Yeah.. so maybe after all it might be better to handle this directly inside the I/O modules that encounter this? Maybe it could be done with a centralized helper method in `core/utcdatetime.py` that gets reused wherever needed?
@megies, that would work,  or if we can settle for not handling it for all input shapes for UTCDateTime (eg only works when passing kwargs with year, month, day etc.) then it could be done fairly easily.
> not handling it for all input shapes for UTCDateTime (eg only works when passing kwargs with year, month, day etc.) then it could be done fairly easily.

If we do it in I/O modules it should be easy to make sure of this, because it is not users calling `strptime` but it's an internal thing to just always go via `UTCDateTime.__init__()` where this problem applies.
@megies sounds good. I will take a shot at this in the next few days. 
Thanks a lot for looking at it @megies and @d-chambers ! Greatly appreciated!

Resolving patch:
diff --git a/obspy/io/nlloc/core.py b/obspy/io/nlloc/core.py
--- a/obspy/io/nlloc/core.py
+++ b/obspy/io/nlloc/core.py
@@ -190,9 +190,9 @@ def _read_single_hypocenter(lines, coordinate_converter, original_picks):
     # maximum likelihood origin time info line
     line = lines["GEOGRAPHIC"]
 
-    year, month, day, hour, minute = map(int, line.split()[1:6])
+    year, mon, day, hour, min = map(int, line.split()[1:6])
     seconds = float(line.split()[6])
-    time = UTCDateTime(year, month, day, hour, minute, seconds)
+    time = UTCDateTime(year, mon, day, hour, min, seconds, strict=False)
 
     # distribution statistics line
     line = lines["STATISTICS"]
@@ -328,8 +328,11 @@ def _read_single_hypocenter(lines, coordinate_converter, original_picks):
         # when reading the .hyp file.. to conform with QuakeML standard set an
         # empty network code
         wid = WaveformStreamID(network_code="", station_code=station)
+        # have to split this into ints for overflow to work correctly
         date, hourmin, sec = map(str, line[6:9])
-        t = UTCDateTime.strptime(date + hourmin, "%Y%m%d%H%M") + float(sec)
+        ymd = [int(date[:4]), int(date[4:6]), int(date[6:8])]
+        hm = [int(hourmin[:2]), int(hourmin[2:4])]
+        t = UTCDateTime(*(ymd + hm), strict=False) + float(sec)
         pick.waveform_id = wid
         pick.time = t
         pick.time_errors.uncertainty = float(line[10])


Test patch:
diff --git a/obspy/io/nlloc/tests/test_core.py b/obspy/io/nlloc/tests/test_core.py
--- a/obspy/io/nlloc/tests/test_core.py
+++ b/obspy/io/nlloc/tests/test_core.py
@@ -5,6 +5,7 @@
 from future.builtins import *  # NOQA @UnusedWildImport
 
 import inspect
+import io
 import os
 import re
 import unittest
@@ -277,6 +278,32 @@ def test_read_nlloc_6_beta_signature(self):
             cat[0].creation_info.creation_time,
             UTCDateTime(2017, 5, 9, 11, 0, 22))
 
+    def test_issue_2222(self):
+        """
+        Test that hour values of 24 don't break parser.
+        """
+
+        # modify the example file to contain an hour 24 and second 60
+        nll_str = open(get_example_file('nlloc.hyp')).read().splitlines()
+        # first add a line with hour 24
+        str_list = list(nll_str[-3])
+        str_list[37:41] = '2400'
+        nll_str[-3] = ''.join(str_list)
+        # then add a line with second 60
+        str_list = list(nll_str[-4])
+        str_list[46:48] = '60'
+        nll_str[-4] = ''.join(str_list)
+        # write to string io and read into catalog object
+        str_io = io.StringIO()
+        str_io.write('\n'.join(nll_str))
+        str_io.seek(0)
+        cat = read_nlloc_hyp(str_io)
+        # check catalog is populated and pick times are right
+        self.assertEqual(len(cat), 1)
+        pick1, pick2 = cat[0].picks[-1], cat[0].picks[-2]
+        self.assertEqual(pick1.time.hour, 0)
+        self.assertEqual(pick2.time.second, 0)
+
 
 def suite():
     return unittest.makeSuite(NLLOCTestCase, "test")