Repository: betodealmeida/shillelagh

Issue:
ISODate parse can use std library
Currently `ISODate`'s `parse` uses `dateutil`:
https://github.com/betodealmeida/shillelagh/blob/7afaf13ec822f8c56895a8aec6ad77a7de2ea600/src/shillelagh/fields.py#L323-L328

 however as of Python 3.7, [`date.fromisoformat`](https://docs.python.org/3/library/datetime.html#datetime.date.fromisoformat) should work and be faster + less permissive (it doesn't try to guess the format)

Comments:

Patch:
diff --git a/ARCHITECTURE.rst b/ARCHITECTURE.rst
index b0e43b2..a0d7e45 100644
--- a/ARCHITECTURE.rst
+++ b/ARCHITECTURE.rst
@@ -158,7 +158,7 @@ There's another layer of type conversion. SQLite has limited support for types,
 .. code-block:: python
 
     cursor.execute(
-        "SELECT event_time FROM sometable WHERE event_time > %s", 
+        "SELECT event_time FROM sometable WHERE event_time > %s",
         (datetime.datetime.now() - datetime.timdelta(days=7),),
     )
 
diff --git a/README.rst b/README.rst
index e1bab88..cf0d5c2 100644
--- a/README.rst
+++ b/README.rst
@@ -81,20 +81,20 @@ You can even run ``INSERT``/``DELETE``/``UPDATE`` queries against the spreadshee
 
 Queries like this are supported by `adapters <https://shillelagh.readthedocs.io/en/latest/adapters.html>`_. Currently Shillelagh has the following adapters:
 
-============ ============ ========================================================================== ===================================================================================================== 
- Name         Type         URI pattern                                                                Example URI                                                                                          
-============ ============ ========================================================================== ===================================================================================================== 
- CSV          File         ``/path/to/file.csv``                                                      ``/home/user/sample_data.csv``                                                                       
- Datasette    API          ``http(s)://*``                                                            ``https://global-power-plants.datasettes.com/global-power-plants/global-power-plants``               
- GitHub       API          ``https://api.github.com/repos/${owner}/{$repo}/pulls``                    ``https://api.github.com/repos/apache/superset/pulls``                                               
- GSheets      API          ``https://docs.google.com/spreadsheets/d/${id}/edit#gid=${sheet_id}``      ``https://docs.google.com/spreadsheets/d/1LcWZMsdCl92g7nA-D6qGRqg1T5TiHyuKJUY1u9XAnsk/edit#gid=0``   
- HTML table   API          ``http(s)://*``                                                            ``https://en.wikipedia.org/wiki/List_of_countries_and_dependencies_by_population``                   
- Pandas       In memory    Any variable name (local or global)                                        ``my_df``                                                                                            
- S3           API          ``s3://bucket/path/to/file``                                               ``s3://shillelagh/sample_data.csv``                                                                  
- Socrata      API          ``https://${domain}/resource/${dataset-id}.json``                          ``https://data.cdc.gov/resource/unsk-b7fc.json``                                                     
- System       API          ``system://${resource}``                                                   ``system://cpu?interval=2``                                                                          
- WeatherAPI   API          ``https://api.weatherapi.com/v1/history.json?key=${key}&q=${location}``    ``https://api.weatherapi.com/v1/history.json?key=XXX&q=London``                                      
-============ ============ ========================================================================== ===================================================================================================== 
+============ ============ ========================================================================== =====================================================================================================
+ Name         Type         URI pattern                                                                Example URI
+============ ============ ========================================================================== =====================================================================================================
+ CSV          File         ``/path/to/file.csv``                                                      ``/home/user/sample_data.csv``
+ Datasette    API          ``http(s)://*``                                                            ``https://global-power-plants.datasettes.com/global-power-plants/global-power-plants``
+ GitHub       API          ``https://api.github.com/repos/${owner}/{$repo}/pulls``                    ``https://api.github.com/repos/apache/superset/pulls``
+ GSheets      API          ``https://docs.google.com/spreadsheets/d/${id}/edit#gid=${sheet_id}``      ``https://docs.google.com/spreadsheets/d/1LcWZMsdCl92g7nA-D6qGRqg1T5TiHyuKJUY1u9XAnsk/edit#gid=0``
+ HTML table   API          ``http(s)://*``                                                            ``https://en.wikipedia.org/wiki/List_of_countries_and_dependencies_by_population``
+ Pandas       In memory    Any variable name (local or global)                                        ``my_df``
+ S3           API          ``s3://bucket/path/to/file``                                               ``s3://shillelagh/sample_data.csv``
+ Socrata      API          ``https://${domain}/resource/${dataset-id}.json``                          ``https://data.cdc.gov/resource/unsk-b7fc.json``
+ System       API          ``system://${resource}``                                                   ``system://cpu?interval=2``
+ WeatherAPI   API          ``https://api.weatherapi.com/v1/history.json?key=${key}&q=${location}``    ``https://api.weatherapi.com/v1/history.json?key=XXX&q=London``
+============ ============ ========================================================================== =====================================================================================================
 
 There are also 3rd-party adapters:
 
@@ -134,7 +134,7 @@ You also need to install optional dependencies, depending on the adapter you wan
     $ pip install 'shillelagh[datasetteapi]'  # for Datasette
     $ pip install 'shillelagh[githubapi]'     # for GitHub
     $ pip install 'shillelagh[gsheetsapi]'    # for GSheets
-    $ pip install 'shillelagh[htmltableapi]'  # for HTML tables 
+    $ pip install 'shillelagh[htmltableapi]'  # for HTML tables
     $ pip install 'shillelagh[pandasmemory]'  # for Pandas in memory
     $ pip install 'shillelagh[s3selectapi]'   # for S3 files
     $ pip install 'shillelagh[socrataapi]'    # for Socrata API
diff --git a/src/shillelagh/adapters/api/github.py b/src/shillelagh/adapters/api/github.py
index 6574d6f..fb787c5 100644
--- a/src/shillelagh/adapters/api/github.py
+++ b/src/shillelagh/adapters/api/github.py
@@ -11,7 +11,7 @@ from jsonpath import JSONPath
 
 from shillelagh.adapters.base import Adapter
 from shillelagh.exceptions import ProgrammingError
-from shillelagh.fields import Boolean, Field, Integer, ISODateTime, String
+from shillelagh.fields import Boolean, Field, Integer, String, StringDateTime
 from shillelagh.filters import Equal, Filter
 from shillelagh.typing import RequestedOrder, Row
 
@@ -58,10 +58,10 @@ TABLES: Dict[str, Dict[str, List[Column]]] = {
             Column("username", "user.login", String()),
             Column("draft", "draft", Boolean()),
             Column("head", "head.ref", String(filters=[Equal])),  # head.label?
-            Column("created_at", "created_at", ISODateTime()),
-            Column("updated_at", "updated_at", ISODateTime()),
-            Column("closed_at", "closed_at", ISODateTime()),
-            Column("merged_at", "merged_at", ISODateTime()),
+            Column("created_at", "created_at", StringDateTime()),
+            Column("updated_at", "updated_at", StringDateTime()),
+            Column("closed_at", "closed_at", StringDateTime()),
+            Column("merged_at", "merged_at", StringDateTime()),
         ],
     },
 }
diff --git a/src/shillelagh/adapters/api/socrata.py b/src/shillelagh/adapters/api/socrata.py
index 8121a21..2c892da 100644
--- a/src/shillelagh/adapters/api/socrata.py
+++ b/src/shillelagh/adapters/api/socrata.py
@@ -15,7 +15,7 @@ from typing_extensions import TypedDict
 
 from shillelagh.adapters.base import Adapter
 from shillelagh.exceptions import ImpossibleFilterError, ProgrammingError
-from shillelagh.fields import Field, ISODate, Order, String
+from shillelagh.fields import Field, Order, String, StringDate
 from shillelagh.filters import Equal, Filter, IsNotNull, IsNull, Like, NotEqual, Range
 from shillelagh.lib import SimpleCostModel, build_sql
 from shillelagh.typing import RequestedOrder, Row
@@ -69,7 +69,7 @@ class Number(Field[str, float]):
 
 
 type_map: Dict[str, Tuple[Type[Field], List[Type[Filter]]]] = {
-    "calendar_date": (ISODate, [Range, Equal, NotEqual, IsNull, IsNotNull]),
+    "calendar_date": (StringDate, [Range, Equal, NotEqual, IsNull, IsNotNull]),
     "number": (Number, [Range, Equal, NotEqual, IsNull, IsNotNull]),
     "text": (String, [Range, Equal, NotEqual, Like, IsNull, IsNotNull]),
 }
diff --git a/src/shillelagh/fields.py b/src/shillelagh/fields.py
index 5f1e847..0741cd2 100644
--- a/src/shillelagh/fields.py
+++ b/src/shillelagh/fields.py
@@ -361,11 +361,11 @@ class ISODate(Field[str, datetime.date]):
             return None
 
         try:
-            date = dateutil.parser.parse(value)
-        except dateutil.parser.ParserError:
+            date = datetime.date.fromisoformat(value)
+        except ValueError:
             return None
 
-        return date.date()
+        return date
 
     def format(self, value: Optional[datetime.date]) -> Optional[str]:
         if value is None:
@@ -378,6 +378,23 @@ class ISODate(Field[str, datetime.date]):
         return f"'{value}'"
 
 
+class StringDate(ISODate):
+    """
+    A more permissive date format.
+    """
+
+    def parse(self, value: Optional[str]) -> Optional[datetime.date]:
+        if value is None:
+            return None
+
+        try:
+            date = dateutil.parser.parse(value)
+        except dateutil.parser.ParserError:
+            return None
+
+        return date.date()
+
+
 class Time(Field[datetime.time, datetime.time]):
     """
     A time of the day.
@@ -413,11 +430,11 @@ class ISOTime(Field[str, datetime.time]):
             return None
 
         try:
-            timestamp = dateutil.parser.parse(value)
-        except dateutil.parser.ParserError:
+            timestamp = datetime.time.fromisoformat(value)
+        except ValueError:
             return None
 
-        time = timestamp.time()
+        time = timestamp
 
         # timezone is not preserved
         return time.replace(tzinfo=timestamp.tzinfo)
@@ -433,6 +450,26 @@ class ISOTime(Field[str, datetime.time]):
         return f"'{value}'"
 
 
+class StringTime(ISOTime):
+    """
+    A more permissive time format.
+    """
+
+    def parse(self, value: Optional[str]) -> Optional[datetime.time]:
+        if value is None:
+            return None
+
+        try:
+            timestamp = dateutil.parser.parse(value)
+        except dateutil.parser.ParserError:
+            return None
+
+        time = timestamp.time()
+
+        # timezone is not preserved
+        return time.replace(tzinfo=timestamp.tzinfo)
+
+
 class DateTime(Field[datetime.datetime, datetime.datetime]):
     """
     A timestamp.
@@ -469,8 +506,8 @@ class ISODateTime(Field[str, datetime.datetime]):
             return None
 
         try:
-            timestamp = dateutil.parser.parse(value)
-        except dateutil.parser.ParserError:
+            timestamp = datetime.datetime.fromisoformat(value)
+        except ValueError:
             return None
 
         # if the timestamp has a timezone change it to UTC, so that
@@ -497,6 +534,28 @@ class ISODateTime(Field[str, datetime.datetime]):
         return f"'{value}'"
 
 
+class StringDateTime(ISODateTime):
+    """
+    A more permissive datetime format.
+    """
+
+    def parse(self, value: Optional[str]) -> Optional[datetime.datetime]:
+        if value is None:
+            return None
+
+        try:
+            timestamp = dateutil.parser.parse(value)
+        except dateutil.parser.ParserError:
+            return None
+
+        # if the timestamp has a timezone change it to UTC, so that
+        # timestamps in different timezones can be compared as strings
+        if timestamp.tzinfo is not None:
+            timestamp = timestamp.astimezone(datetime.timezone.utc)
+
+        return timestamp
+
+
 class StringDuration(Field[str, datetime.timedelta]):
     """
     A duration.


Test patch:
diff --git a/tests/fields_test.py b/tests/fields_test.py
index 96a724e..babb701 100644
--- a/tests/fields_test.py
+++ b/tests/fields_test.py
@@ -22,7 +22,10 @@ from shillelagh.fields import (
     String,
     StringBlob,
     StringBoolean,
+    StringDate,
+    StringDateTime,
     StringDuration,
+    StringTime,
     Time,
 )
 from shillelagh.filters import Equal
@@ -102,11 +105,6 @@ def test_isodate() -> None:
     Test ``ISODate``.
     """
     assert ISODate().parse("2020-01-01") == datetime.date(2020, 1, 1)
-    assert ISODate().parse("2020-01-01T00:00+00:00") == datetime.date(
-        2020,
-        1,
-        1,
-    )
     assert ISODate().parse(None) is None
     assert ISODate().parse("invalid") is None
     assert ISODate().format(datetime.date(2020, 1, 1)) == "2020-01-01"
@@ -115,6 +113,20 @@ def test_isodate() -> None:
     assert ISODate().quote(None) == "NULL"
 
 
+def test_string_date() -> None:
+    """
+    Test ``StringDate``.
+    """
+    assert StringDate().parse("2020-01-01") == datetime.date(2020, 1, 1)
+    assert StringDate().parse("2020-01-01T00:00+00:00") == datetime.date(
+        2020,
+        1,
+        1,
+    )
+    assert StringDate().parse(None) is None
+    assert StringDate().parse("invalid") is None
+
+
 def test_time() -> None:
     """
     Test ``Time``.
@@ -164,6 +176,23 @@ def test_iso_time() -> None:
     assert ISOTime().quote(None) == "NULL"
 
 
+def test_string_time() -> None:
+    """
+    Test ``StringTime``.
+    """
+    assert StringTime().parse("12:00+00:00") == datetime.time(
+        12,
+        0,
+        tzinfo=datetime.timezone.utc,
+    )
+    assert StringTime().parse("12:00") == datetime.time(
+        12,
+        0,
+    )
+    assert StringTime().parse(None) is None
+    assert StringTime().parse("invalid") is None
+
+
 def test_datetime() -> None:
     """
     Test ``DateTime``.
@@ -228,6 +257,40 @@ def test_iso_datetime() -> None:
     assert ISODateTime().quote(None) == "NULL"
 
 
+def test_string_datetime() -> None:
+    """
+    Test ``StringDateTime``.
+    """
+    assert StringDateTime().parse("2020-01-01T12:00+00:00") == datetime.datetime(
+        2020,
+        1,
+        1,
+        12,
+        0,
+        0,
+        tzinfo=datetime.timezone.utc,
+    )
+    assert StringDateTime().parse("2020-01-01T12:00Z") == datetime.datetime(
+        2020,
+        1,
+        1,
+        12,
+        0,
+        0,
+        tzinfo=datetime.timezone.utc,
+    )
+    assert StringDateTime().parse("2020-01-01T12:00") == datetime.datetime(
+        2020,
+        1,
+        1,
+        12,
+        0,
+        0,
+    )
+    assert StringDateTime().parse(None) is None
+    assert StringDateTime().parse("invalid") is None
+
+
 def test_boolean() -> None:
     """
     Test ``Boolean``.