Repository: dateutil/dateutil (Python)

Issue:
Accept more than 6 digits for fractional seconds in isoparse
Currently, `isoparse` raises an exception if there are more than 6 digits in the fractional second.

```
>>> dateutil.parser.isoparse("2006-01-02T15:04:05.9999999-07:00")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/jay/.local/lib/python2.7/site-packages/dateutil/parser/isoparser.py", line 37, in func
    return f(self, str_in, *args, **kwargs)
  File "/home/jay/.local/lib/python2.7/site-packages/dateutil/parser/isoparser.py", line 136, in isoparse
    components += self._parse_isotime(dt_str[pos + 1:])
  File "/home/jay/.local/lib/python2.7/site-packages/dateutil/parser/isoparser.py", line 363, in _parse_isotime
    raise ValueError('Unused components in ISO string')
ValueError: Unused components in ISO string
```

Go, for example, includes up to 9 digits (nanosecond precision) when marshaling a time into JSON. It would be nice if `isoparse` could accept those without issue, even if precision gets truncated a bit.
Accept more than 6 digits for fractional seconds in isoparse
Currently, `isoparse` raises an exception if there are more than 6 digits in the fractional second.

```
>>> dateutil.parser.isoparse("2006-01-02T15:04:05.9999999-07:00")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/jay/.local/lib/python2.7/site-packages/dateutil/parser/isoparser.py", line 37, in func
    return f(self, str_in, *args, **kwargs)
  File "/home/jay/.local/lib/python2.7/site-packages/dateutil/parser/isoparser.py", line 136, in isoparse
    components += self._parse_isotime(dt_str[pos + 1:])
  File "/home/jay/.local/lib/python2.7/site-packages/dateutil/parser/isoparser.py", line 363, in _parse_isotime
    raise ValueError('Unused components in ISO string')
ValueError: Unused components in ISO string
```

Go, for example, includes up to 9 digits (nanosecond precision) when marshaling a time into JSON. It would be nice if `isoparse` could accept those without issue, even if precision gets truncated a bit.


Hints:
This would be nice, though I'm a bit wary of the truncating idea.  Until it gets implemented here (PRs welcome BTW), you might want to just use `pandas.Timestamp` since it retains nanoseconds.
I am willing to take a shot at implementing it. I propose that `isoparse` truncate by default, with an optional argument to raise some kind of `PrecisionError` instead. Truncation would not be unprecedented since `dateutil.parser.parse` already does so on the same input that fails with `isoparse`.
I agree with this. The ISO 8601 spec does not specify the precision of fractional components (this is left to "mutual agreement"). My design principle is that we will be liberal about anything not explicitly forbidden by the spec.

I don't think there's any way to do better than truncation while still returning standard datetimes.

Let's start with expanding the tests to cover up to 12 digits (should be very easy) and the corresponding fixes that makes the tests pass. We'll consider the UI of how to specify the precision later.
This would be nice, though I'm a bit wary of the truncating idea.  Until it gets implemented here (PRs welcome BTW), you might want to just use `pandas.Timestamp` since it retains nanoseconds.
I am willing to take a shot at implementing it. I propose that `isoparse` truncate by default, with an optional argument to raise some kind of `PrecisionError` instead. Truncation would not be unprecedented since `dateutil.parser.parse` already does so on the same input that fails with `isoparse`.
I agree with this. The ISO 8601 spec does not specify the precision of fractional components (this is left to "mutual agreement"). My design principle is that we will be liberal about anything not explicitly forbidden by the spec.

I don't think there's any way to do better than truncation while still returning standard datetimes.

Let's start with expanding the tests to cover up to 12 digits (should be very easy) and the corresponding fixes that makes the tests pass. We'll consider the UI of how to specify the precision later.

Patch:
diff --git a/dateutil/parser/isoparser.py b/dateutil/parser/isoparser.py
--- a/dateutil/parser/isoparser.py
+++ b/dateutil/parser/isoparser.py
@@ -88,7 +88,7 @@ def isoparse(self, dt_str):
         - ``hh``
         - ``hh:mm`` or ``hhmm``
         - ``hh:mm:ss`` or ``hhmmss``
-        - ``hh:mm:ss.sss`` or ``hh:mm:ss.ssssss`` (3-6 sub-second digits)
+        - ``hh:mm:ss.ssssss`` (Up to 6 sub-second digits)
 
         Midnight is a special case for `hh`, as the standard supports both
         00:00 and 24:00 as a representation. The decimal separator can be
@@ -199,10 +199,9 @@ def parse_tzstr(self, tzstr, zero_as_utc=True):
         return self._parse_tzstr(tzstr, zero_as_utc=zero_as_utc)
 
     # Constants
-    _MICROSECOND_END_REGEX = re.compile(b'[-+Z]+')
     _DATE_SEP = b'-'
     _TIME_SEP = b':'
-    _MICRO_SEPS = b'.,'
+    _FRACTION_REGEX = re.compile(b'[\\.,]([0-9]+)')
 
     def _parse_isodate(self, dt_str):
         try:
@@ -357,16 +356,14 @@ def _parse_isotime(self, timestr):
                     pos += 1
 
             if comp == 3:
-                # Microsecond
-                if timestr[pos:pos + 1] not in self._MICRO_SEPS:
+                # Fraction of a second
+                frac = self._FRACTION_REGEX.match(timestr[pos:])
+                if not frac:
                     continue
 
-                pos += 1
-                us_str = self._MICROSECOND_END_REGEX.split(timestr[pos:pos + 6],
-                                                           1)[0]
-
+                us_str = frac.group(1)[:6]  # Truncate to microseconds
                 components[comp] = int(us_str) * 10**(6 - len(us_str))
-                pos += len(us_str)
+                pos += len(frac.group())
 
         if pos < len_str:
             raise ValueError('Unused components in ISO string')


Test patch:
diff --git a/dateutil/test/test_isoparser.py b/dateutil/test/test_isoparser.py
--- a/dateutil/test/test_isoparser.py
+++ b/dateutil/test/test_isoparser.py
@@ -130,6 +130,15 @@ def test_ymd_hms_micro(dt, date_fmt, time_fmt, tzoffset, precision):
 
     _isoparse_date_and_time(dt, date_fmt, time_fmt, tzoffset, precision)
 
+###
+# Truncation of extra digits beyond microsecond precision
+@pytest.mark.parametrize('dt_str', [
+    '2018-07-03T14:07:00.123456000001',
+    '2018-07-03T14:07:00.123456999999',
+])
+def test_extra_subsecond_digits(dt_str):
+    assert isoparse(dt_str) == datetime(2018, 7, 3, 14, 7, 0, 123456)
+
 @pytest.mark.parametrize('tzoffset', FULL_TZOFFSETS)
 def test_full_tzoffsets(tzoffset):
     dt = datetime(2017, 11, 27, 6, 14, 30, 123456)
@@ -473,12 +482,11 @@ def test_isotime_midnight(isostr):
     ('14時30分15秒', ValueError),                # Not ASCII
     ('14_30_15', ValueError),                   # Invalid separators
     ('1430:15', ValueError),                    # Inconsistent separator use
-    ('14:30:15.3684000309', ValueError),        # Too much us precision
     ('25', ValueError),                         # Invalid hours
     ('25:15', ValueError),                      # Invalid hours
     ('14:60', ValueError),                      # Invalid minutes
     ('14:59:61', ValueError),                   # Invalid seconds
-    ('14:30:15.3446830500', ValueError),        # No sign in time zone
+    ('14:30:15.34468305:00', ValueError),       # No sign in time zone
     ('14:30:15+', ValueError),                  # Time zone too short
     ('14:30:15+1234567', ValueError),           # Time zone invalid
     ('14:59:59+25:00', ValueError),             # Invalid tz hours