Repository: dbt-labs/dbt-core

Issue:
Seed with custom column type of string fails with boolean like seed values
## Issue

### Issue description

Seeding fails to treat a boolean-like string as a varchar even when specified.

I have the following seed file:

```csv
true_is_correct,is_correct,choice_body
true,true,"True"
true,false,"False"
false,true,"False"
false,false,"True"
```

With the following config in `dbt_project.yml`:

```yaml
seeds:
  project_name:
    enabled: true
    seed_file_name:
      column_types:
        choice_body: varchar(5)
```

### Results

`dbt seed` errors on this file with:

```
Database Error in seed seed_file_name (data/seed_file_name.csv)
  column "choice_body" is of type character varying but expression is of type boolean
  HINT:  You will need to rewrite or cast the expression.
```

I expect this to seed properly.

### System information

- dbt 0.11.0
- Mac OS X 10.13.6 (17G65)
- Python 2.7.10
- Redshift

### Steps to reproduce

Sample seed file and config above.


Rationalize agate in dbt
List of open (or recently closed) issues pertaining to Agate:
 - #1452 
 - #1632 
 - #1626 
 - #1566 
 - #999 
 - + more
 - a database named `tomorrow` will cause dbt to fail

Let's rip out agate where it's not needed and make it generally less clever (type inference) where we can.


Discussion:
Thanks for the report @ianterrell - we'll take a look!
@drewbanin Hi Drew, we're experiencing the same issue: dbt implicitly processes values from the seed files before it loads it to a seed table, so even if data type of the column is specified via `column_types` (e.g. VARCHAR(50)), the values will be changed (e.g., leading zero is going to be stripped, if dbt thinks that values in that column are integers, not strings). Are there any plans to to fix this? Thanks in advance!
hey @borisuvarov - thanks for the update. dbt doesn't do any of this type inference itself. Instead, the work is outsourced to a library called [agate](https://github.com/wireservice/agate).

I 100% agree with you that dbt's behavior in these situations is incorrect and suboptimal, but I haven't yet been able to figure out a good way to make this work.

One option is to use `column_types` to specify both the database type, and the python type. That could look like:
```
column_types:
  some_field:
    sql: varchar(100)
    type: string
```

^ I don't at all like that syntax, but I _think_ we need both of these pieces of information! I would like to get this working at some point, but we don't plan on prioritizing a fix for this in the near future.  If you have good ideas for how this could work, I'd love to hear them!

Edit: this was a hypothetical syntax that we could use in the future, but as of <=0.15.0, it is not yet implemented.
Thanks @drewbanin! 

Adding a `type` should help in most cases. But it's also worth to try to figure out how to (optionally) leverage native COPY (Redshift), COPY INTO (Snowflake), etc. DML commands, which loads a CSV file without any transformations of values in Python.
Going to close this - it may still be an issue, but we're largely at the behest of Agate for seed types. Happy to discuss / reopen if anyone feels strongly about it!
This is tentatively prioritized for the Louisa May Alcott release. We'll need to do this in a minor release, as it will be a breaking change. See https://github.com/fishtown-analytics/dbt/pull/1541 for an implementation by @rrbarbosa 

Edit: wrong person, wrong link!

See also https://github.com/fishtown-analytics/dbt/issues/1616
I'm happy to pitch in here after dealing with this in the new `run_query()` function.
Hey @Aylr - I appreciate your support :) 

This is going to be a pretty sizable change to dbt and I want to make sure we treat it with the respect it deserves! I think a good place to start is: What should we use instead? dbt operates on tabular data (either from loaded up seed files, or from the results of queries). I think there are parts of the agate API that are sensible. I like that you can operate on columns, or iterate over rows in the data frame, for instance.

Before we picked agate, we considered using something like Pandas. In practice, Pandas is a real bear to install, so we chose not to ship that with dbt. Are you aware of any existing "data frame" libs that are easy to install, or do you think this is something that makes sense for us to build ourselves?
@drewbanin I totally agree that this should be very thoughtful!

From my perspective (which is likely different that a majority of the dbt target audience as I understand it - so I want to call out that bias), I think that choosing something other than pandas doesn't make much sense since it is effectively the de-facto dataframe library. I do want to pay particular attention that we keep installation as easy as possible for everyone who wants to try dbt. However, from where I sit (data science & engineering) installing pandas is not an issue. But again, I do want to be thoughtful about that.
And FWIW pandas has all those kinds of operators and a wide user base to draw expertise from (myself included).

Resolving patch:
diff --git a/core/dbt/clients/agate_helper.py b/core/dbt/clients/agate_helper.py
--- a/core/dbt/clients/agate_helper.py
+++ b/core/dbt/clients/agate_helper.py
@@ -1,21 +1,58 @@
 from codecs import BOM_UTF8
 
 import agate
+import datetime
+import isodate
 import json
+from typing import Iterable
 
 
 BOM = BOM_UTF8.decode('utf-8')  # '\ufeff'
 
-DEFAULT_TYPE_TESTER = agate.TypeTester(types=[
-    agate.data_types.Number(null_values=('null', '')),
-    agate.data_types.TimeDelta(null_values=('null', '')),
-    agate.data_types.Date(null_values=('null', '')),
-    agate.data_types.DateTime(null_values=('null', '')),
-    agate.data_types.Boolean(true_values=('true',),
-                             false_values=('false',),
-                             null_values=('null', '')),
-    agate.data_types.Text(null_values=('null', ''))
-])
+
+class ISODateTime(agate.data_types.DateTime):
+    def cast(self, d):
+        # this is agate.data_types.DateTime.cast with the "clever" bits removed
+        # so we only handle ISO8601 stuff
+        if isinstance(d, datetime.datetime) or d is None:
+            return d
+        elif isinstance(d, datetime.date):
+            return datetime.datetime.combine(d, datetime.time(0, 0, 0))
+        elif isinstance(d, str):
+            d = d.strip()
+            if d.lower() in self.null_values:
+                return None
+        try:
+            return isodate.parse_datetime(d)
+        except:  # noqa
+            pass
+
+        raise agate.exceptions.CastError(
+            'Can not parse value "%s" as datetime.' % d
+        )
+
+
+def build_type_tester(text_columns: Iterable[str]):
+    types = [
+        agate.data_types.Number(null_values=('null', '')),
+        agate.data_types.Date(null_values=('null', ''),
+                              date_format='%Y-%m-%d'),
+        agate.data_types.DateTime(null_values=('null', ''),
+                                  datetime_format='%Y-%m-%d %H:%M:%S'),
+        ISODateTime(null_values=('null', '')),
+        agate.data_types.Boolean(true_values=('true',),
+                                 false_values=('false',),
+                                 null_values=('null', '')),
+        agate.data_types.Text(null_values=('null', ''))
+    ]
+    force = {
+        k: agate.data_types.Text(null_values=('null', ''))
+        for k in text_columns
+    }
+    return agate.TypeTester(force=force, types=types)
+
+
+DEFAULT_TYPE_TESTER = build_type_tester(())
 
 
 def table_from_data(data, column_names):
@@ -61,8 +98,9 @@ def as_matrix(table):
     return [r.values() for r in table.rows.values()]
 
 
-def from_csv(abspath):
+def from_csv(abspath, text_columns):
+    type_tester = build_type_tester(text_columns=text_columns)
     with open(abspath, encoding='utf-8') as fp:
         if fp.read(1) != BOM:
             fp.seek(0)
-        return agate.Table.from_csv(fp, column_types=DEFAULT_TYPE_TESTER)
+        return agate.Table.from_csv(fp, column_types=type_tester)
diff --git a/core/dbt/context/common.py b/core/dbt/context/common.py
--- a/core/dbt/context/common.py
+++ b/core/dbt/context/common.py
@@ -3,7 +3,7 @@
 from typing_extensions import Protocol
 from typing import Union, Callable, Any, Dict, TypeVar, Type
 
-import dbt.clients.agate_helper
+from dbt.clients import agate_helper
 from dbt.contracts.graph.compiled import CompiledSeedNode
 from dbt.contracts.graph.parsed import ParsedSeedNode
 import dbt.exceptions
@@ -105,11 +105,11 @@ def add_macros(self, context):
 def _store_result(sql_results):
     def call(name, status, agate_table=None):
         if agate_table is None:
-            agate_table = dbt.clients.agate_helper.empty_table()
+            agate_table = agate_helper.empty_table()
 
         sql_results[name] = dbt.utils.AttrDict({
             'status': status,
-            'data': dbt.clients.agate_helper.as_matrix(agate_table),
+            'data': agate_helper.as_matrix(agate_table),
             'table': agate_table
         })
         return ''
@@ -185,8 +185,9 @@ def _build_load_agate_table(
 ) -> Callable[[], agate.Table]:
     def load_agate_table():
         path = model.seed_file_path
+        column_types = model.config.column_types
         try:
-            table = dbt.clients.agate_helper.from_csv(path)
+            table = agate_helper.from_csv(path, text_columns=column_types)
         except ValueError as e:
             dbt.exceptions.raise_compiler_error(str(e))
         table.original_abspath = os.path.abspath(path)
diff --git a/core/setup.py b/core/setup.py
--- a/core/setup.py
+++ b/core/setup.py
@@ -52,6 +52,7 @@ def read(fname):
         'requests>=2.18.0,<3',
         'colorama>=0.3.9,<0.5',
         'agate>=1.6,<2',
+        'isodate>=0.6,<0.7',
         'json-rpc>=1.12,<2',
         'werkzeug>=0.15,<0.17',
         'dataclasses==0.6;python_version<"3.7"',


Test patch:
diff --git a/test/integration/005_simple_seed_test/data-config/seed_tricky.csv b/test/integration/005_simple_seed_test/data-config/seed_tricky.csv
new file mode 100644
--- /dev/null
+++ b/test/integration/005_simple_seed_test/data-config/seed_tricky.csv
@@ -0,0 +1,7 @@
+id,id_str,a_bool,looks_like_a_bool,a_date,looks_like_a_date,relative,weekday
+1,1,true,true,2019-01-01 12:32:30,2019-01-01 12:32:30,tomorrow,Saturday
+2,2,True,True,2019-01-01 12:32:31,2019-01-01 12:32:31,today,Sunday
+3,3,TRUE,TRUE,2019-01-01 12:32:32,2019-01-01 12:32:32,yesterday,Monday
+4,4,false,false,2019-01-01 01:32:32,2019-01-01 01:32:32,tomorrow,Saturday
+5,5,False,False,2019-01-01 01:32:32,2019-01-01 01:32:32,today,Sunday
+6,6,FALSE,FALSE,2019-01-01 01:32:32,2019-01-01 01:32:32,yesterday,Monday
diff --git a/test/integration/005_simple_seed_test/macros/schema_test.sql b/test/integration/005_simple_seed_test/macros/schema_test.sql
--- a/test/integration/005_simple_seed_test/macros/schema_test.sql
+++ b/test/integration/005_simple_seed_test/macros/schema_test.sql
@@ -9,6 +9,10 @@
     {% endfor %}
 
     {% set val = 0 if col_types.get(column_name) == type else 1 %}
+    {% if val == 1 and execute %}
+        {# I'm so tired of guessing what's wrong, let's just log it #}
+        {{ log('Got a column type of ' ~ col_types.get(column_name) ~ ', expected ' ~ type, info=True) }}
+    {% endif %}
 
     select {{ val }} as pass_fail
 
diff --git a/test/integration/005_simple_seed_test/models-bq/schema.yml b/test/integration/005_simple_seed_test/models-bq/schema.yml
--- a/test/integration/005_simple_seed_test/models-bq/schema.yml
+++ b/test/integration/005_simple_seed_test/models-bq/schema.yml
@@ -10,3 +10,38 @@ models:
     tests:
     - column_type:
         type: FLOAT64
+
+- name: seed_tricky
+  columns:
+  - name: id
+    tests:
+    - column_type:
+        type: INT64
+  - name: id_str
+    tests:
+    - column_type:
+        type: STRING
+  - name: a_bool
+    tests:
+    - column_type:
+        type: BOOLEAN
+  - name: looks_like_a_bool
+    tests:
+    - column_type:
+        type: STRING
+  - name: a_date
+    tests:
+    - column_type:
+        type: DATETIME
+  - name: looks_like_a_date
+    tests:
+    - column_type:
+        type: STRING
+  - name: relative
+    tests:
+    - column_type:
+        type: STRING
+  - name: weekday
+    tests:
+    - column_type:
+        type: STRING
diff --git a/test/integration/005_simple_seed_test/models-pg/schema.yml b/test/integration/005_simple_seed_test/models-pg/schema.yml
--- a/test/integration/005_simple_seed_test/models-pg/schema.yml
+++ b/test/integration/005_simple_seed_test/models-pg/schema.yml
@@ -10,3 +10,38 @@ models:
     tests:
     - column_type:
         type: text
+
+- name: seed_tricky
+  columns:
+  - name: id
+    tests:
+    - column_type:
+        type: integer
+  - name: id_str
+    tests:
+    - column_type:
+        type: text
+  - name: a_bool
+    tests:
+    - column_type:
+        type: boolean
+  - name: looks_like_a_bool
+    tests:
+    - column_type:
+        type: text
+  - name: a_date
+    tests:
+    - column_type:
+        type: timestamp without time zone
+  - name: looks_like_a_date
+    tests:
+    - column_type:
+        type: text
+  - name: relative
+    tests:
+    - column_type:
+        type: text
+  - name: weekday
+    tests:
+    - column_type:
+        type: text
diff --git a/test/integration/005_simple_seed_test/models-rs/schema.yml b/test/integration/005_simple_seed_test/models-rs/schema.yml
--- a/test/integration/005_simple_seed_test/models-rs/schema.yml
+++ b/test/integration/005_simple_seed_test/models-rs/schema.yml
@@ -10,3 +10,38 @@ models:
     tests:
     - column_type:
         type: character varying(256)
+
+- name: seed_tricky
+  columns:
+  - name: id
+    tests:
+    - column_type:
+        type: integer
+  - name: id_str
+    tests:
+    - column_type:
+        type: character varying(256)
+  - name: a_bool
+    tests:
+    - column_type:
+        type: boolean
+  - name: looks_like_a_bool
+    tests:
+    - column_type:
+        type: character varying(256)
+  - name: a_date
+    tests:
+    - column_type:
+        type: timestamp without time zone
+  - name: looks_like_a_date
+    tests:
+    - column_type:
+        type: character varying(256)
+  - name: relative
+    tests:
+    - column_type:
+        type: character varying(9)
+  - name: weekday
+    tests:
+    - column_type:
+        type: character varying(8)
diff --git a/test/integration/005_simple_seed_test/models-snowflake/schema.yml b/test/integration/005_simple_seed_test/models-snowflake/schema.yml
--- a/test/integration/005_simple_seed_test/models-snowflake/schema.yml
+++ b/test/integration/005_simple_seed_test/models-snowflake/schema.yml
@@ -2,11 +2,46 @@ version: 2
 models:
 - name: seed_enabled
   columns:
-  - name: BIRTHDAY
+  - name: birthday
     tests:
     - column_type:
         type: character varying(16777216)
-  - name: ID
+  - name: id
     tests:
     - column_type:
         type: FLOAT
+
+- name: seed_tricky
+  columns:
+  - name: id
+    tests:
+    - column_type:
+        type: NUMBER(38,0)
+  - name: id_str
+    tests:
+    - column_type:
+        type: character varying(16777216)
+  - name: a_bool
+    tests:
+    - column_type:
+        type: BOOLEAN
+  - name: looks_like_a_bool
+    tests:
+    - column_type:
+        type: character varying(16777216)
+  - name: a_date
+    tests:
+    - column_type:
+        type: TIMESTAMP_NTZ
+  - name: looks_like_a_date
+    tests:
+    - column_type:
+        type: character varying(16777216)
+  - name: relative
+    tests:
+    - column_type:
+        type: character varying(16777216)
+  - name: weekday
+    tests:
+    - column_type:
+        type: character varying(16777216)
diff --git a/test/integration/005_simple_seed_test/test_seed_type_override.py b/test/integration/005_simple_seed_test/test_seed_type_override.py
--- a/test/integration/005_simple_seed_test/test_seed_type_override.py
+++ b/test/integration/005_simple_seed_test/test_seed_type_override.py
@@ -10,17 +10,21 @@ def schema(self):
     @property
     def project_config(self):
         return {
-            "data-paths": ['data-config'],
-            "macro-paths": ['macros'],
-            "seeds": {
-                "test": {
-                    "enabled": False,
-                    "seed_enabled": {
-                        "enabled": True,
-                        "column_types": self.seed_types()
+            'data-paths': ['data-config'],
+            'macro-paths': ['macros'],
+            'seeds': {
+                'test': {
+                    'enabled': False,
+                    'seed_enabled': {
+                        'enabled': True,
+                        'column_types': self.seed_enabled_types()
                     },
+                    'seed_tricky': {
+                        'enabled': True,
+                        'column_types': self.seed_tricky_types(),
+                    }
                 },
-                'quote_columns': False,
+                'quote_columns': True,
             }
         }
 
@@ -34,18 +38,25 @@ def models(self):
     def profile_config(self):
         return self.postgres_profile()
 
-    def seed_types(self):
+    def seed_enabled_types(self):
         return {
             "id": "text",
             "birthday": "date",
         }
 
+    def seed_tricky_types(self):
+        return {
+            'id_str': 'text',
+            'looks_like_a_bool': 'text',
+            'looks_like_a_date': 'text',
+        }
+
     @use_profile('postgres')
     def test_postgres_simple_seed_with_column_override_postgres(self):
         results = self.run_dbt(["seed", "--show"])
-        self.assertEqual(len(results),  1)
-        results = self.run_dbt(["test"])
         self.assertEqual(len(results),  2)
+        results = self.run_dbt(["test"])
+        self.assertEqual(len(results),  10)
 
 
 class TestSimpleSeedColumnOverrideRedshift(TestSimpleSeedColumnOverride):
@@ -57,18 +68,25 @@ def models(self):
     def profile_config(self):
         return self.redshift_profile()
 
-    def seed_types(self):
+    def seed_enabled_types(self):
         return {
             "id": "text",
             "birthday": "date",
         }
 
+    def seed_tricky_types(self):
+        return {
+            'id_str': 'text',
+            'looks_like_a_bool': 'text',
+            'looks_like_a_date': 'text',
+        }
+
     @use_profile('redshift')
     def test_redshift_simple_seed_with_column_override_redshift(self):
         results = self.run_dbt(["seed", "--show"])
-        self.assertEqual(len(results),  1)
-        results = self.run_dbt(["test"])
         self.assertEqual(len(results),  2)
+        results = self.run_dbt(["test"])
+        self.assertEqual(len(results),  10)
 
 
 class TestSimpleSeedColumnOverrideSnowflake(TestSimpleSeedColumnOverride):
@@ -76,17 +94,18 @@ class TestSimpleSeedColumnOverrideSnowflake(TestSimpleSeedColumnOverride):
     def models(self):
         return "models-snowflake"
 
-    def seed_types(self):
+    def seed_enabled_types(self):
         return {
-            "ID": "FLOAT",
-            "BIRTHDAY": "TEXT",
+            "id": "FLOAT",
+            "birthday": "TEXT",
         }
 
-    @property
-    def project_config(self):
-        cfg = super().project_config
-        cfg['data-paths'] = ['snowflake-data-config']
-        return cfg
+    def seed_tricky_types(self):
+        return {
+            'id_str': 'TEXT',
+            'looks_like_a_bool': 'TEXT',
+            'looks_like_a_date': 'TEXT',
+        }
 
     @property
     def profile_config(self):
@@ -95,9 +114,9 @@ def profile_config(self):
     @use_profile('snowflake')
     def test_snowflake_simple_seed_with_column_override_snowflake(self):
         results = self.run_dbt(["seed", "--show"])
-        self.assertEqual(len(results),  1)
-        results = self.run_dbt(["test"])
         self.assertEqual(len(results),  2)
+        results = self.run_dbt(["test"])
+        self.assertEqual(len(results),  10)
 
 
 class TestSimpleSeedColumnOverrideBQ(TestSimpleSeedColumnOverride):
@@ -105,12 +124,19 @@ class TestSimpleSeedColumnOverrideBQ(TestSimpleSeedColumnOverride):
     def models(self):
         return "models-bq"
 
-    def seed_types(self):
+    def seed_enabled_types(self):
         return {
             "id": "FLOAT64",
             "birthday": "STRING",
         }
 
+    def seed_tricky_types(self):
+        return {
+            'id_str': 'STRING',
+            'looks_like_a_bool': 'STRING',
+            'looks_like_a_date': 'STRING',
+        }
+
     @property
     def profile_config(self):
         return self.bigquery_profile()
@@ -118,6 +144,6 @@ def profile_config(self):
     @use_profile('bigquery')
     def test_bigquery_simple_seed_with_column_override_bigquery(self):
         results = self.run_dbt(["seed", "--show"])
-        self.assertEqual(len(results),  1)
-        results = self.run_dbt(["test"])
         self.assertEqual(len(results),  2)
+        results = self.run_dbt(["test"])
+        self.assertEqual(len(results),  10)
diff --git a/test/integration/005_simple_seed_test/test_simple_seed.py b/test/integration/005_simple_seed_test/test_simple_seed.py
--- a/test/integration/005_simple_seed_test/test_simple_seed.py
+++ b/test/integration/005_simple_seed_test/test_simple_seed.py
@@ -132,7 +132,7 @@ def project_config(self):
     @use_profile('postgres')
     def test_postgres_simple_seed_with_disabled(self):
         results = self.run_dbt(["seed"])
-        self.assertEqual(len(results),  1)
+        self.assertEqual(len(results),  2)
         self.assertTableDoesExist('seed_enabled')
         self.assertTableDoesNotExist('seed_disabled')
 
diff --git a/test/unit/test_agate_helper.py b/test/unit/test_agate_helper.py
--- a/test/unit/test_agate_helper.py
+++ b/test/unit/test_agate_helper.py
@@ -16,16 +16,25 @@
 
 
 EXPECTED = [
-    [1, 'n', 'test', Decimal('3.2'),
-     datetime(2018, 8, 6, 11, 33, 29, 320000, tzinfo=tzinfo.Utc()),
-     True, None,
+    [
+        1, 'n', 'test', Decimal('3.2'),
+        datetime(2018, 8, 6, 11, 33, 29, 320000, tzinfo=tzinfo.Utc()),
+        True, None,
     ],
-    [2, 'y', 'asdf', 900,
-     datetime(2018, 8, 6, 11, 35, 29, 320000, tzinfo=tzinfo.Utc()),
-     False, 'a string',
+    [
+        2, 'y', 'asdf', 900,
+        datetime(2018, 8, 6, 11, 35, 29, 320000, tzinfo=tzinfo.Utc()),
+        False, 'a string',
     ],
 ]
 
+
+EXPECTED_STRINGS = [
+    ['1', 'n', 'test', '3.2', '20180806T11:33:29.320Z', 'True', None],
+    ['2', 'y', 'asdf', '900', '20180806T11:35:29.320Z', 'False', 'a string'],
+]
+
+
 class TestAgateHelper(unittest.TestCase):
     def setUp(self):
         self.tempdir = mkdtemp()
@@ -37,7 +46,7 @@ def test_from_csv(self):
         path = os.path.join(self.tempdir, 'input.csv')
         with open(path, 'wb') as fp:
             fp.write(SAMPLE_CSV_DATA.encode('utf-8'))
-        tbl = agate_helper.from_csv(path)
+        tbl = agate_helper.from_csv(path, ())
         self.assertEqual(len(tbl), len(EXPECTED))
         for idx, row in enumerate(tbl):
             self.assertEqual(list(row), EXPECTED[idx])
@@ -46,11 +55,20 @@ def test_bom_from_csv(self):
         path = os.path.join(self.tempdir, 'input.csv')
         with open(path, 'wb') as fp:
             fp.write(SAMPLE_CSV_BOM_DATA.encode('utf-8'))
-        tbl = agate_helper.from_csv(path)
+        tbl = agate_helper.from_csv(path, ())
         self.assertEqual(len(tbl), len(EXPECTED))
         for idx, row in enumerate(tbl):
             self.assertEqual(list(row), EXPECTED[idx])
 
+    def test_from_csv_all_reserved(self):
+        path = os.path.join(self.tempdir, 'input.csv')
+        with open(path, 'wb') as fp:
+            fp.write(SAMPLE_CSV_DATA.encode('utf-8'))
+        tbl = agate_helper.from_csv(path, tuple('abcdefg'))
+        self.assertEqual(len(tbl), len(EXPECTED_STRINGS))
+        for expected, row in zip(EXPECTED_STRINGS, tbl):
+            self.assertEqual(list(row), expected)
+
     def test_from_data(self):
         column_names = ['a', 'b', 'c', 'd', 'e', 'f', 'g']
         data = [
@@ -63,3 +81,17 @@ def test_from_data(self):
         self.assertEqual(len(tbl), len(EXPECTED))
         for idx, row in enumerate(tbl):
             self.assertEqual(list(row), EXPECTED[idx])
+
+    def test_datetime_formats(self):
+        path = os.path.join(self.tempdir, 'input.csv')
+        datetimes = [
+            '20180806T11:33:29.000Z',
+            '20180806T11:33:29Z',
+            '20180806T113329Z',
+        ]
+        expected = datetime(2018, 8, 6, 11, 33, 29, 0, tzinfo=tzinfo.Utc())
+        for dt in datetimes:
+            with open(path, 'wb') as fp:
+                fp.write('a\n{}'.format(dt).encode('utf-8'))
+            tbl = agate_helper.from_csv(path, ())
+            self.assertEqual(tbl[0][0], expected)