Repository: IAMconsortium/pyam (python)

Issue:
Aggregate fails with time column
**Describe the bug**

If you make a dataframe with datetime times and call its `aggregate` method, you receive an error. This should work the same as a dataframe with year columns. 

```python
>>> from datetime import datetime
>>> 
>>> import pandas as pd
>>> 
>>> from pyam import IamDataFrame
>>> 
>>> eg_df = pd.DataFrame([
...     ['model_a', 'scen_a', 'World', 'Primary Energy|Gas', 'EJ/y', 1, 6.],
...     ['model_a', 'scen_a', 'World', 'Primary Energy|Coal', 'EJ/y', 0.5, 3],
...     ['model_a', 'scen_b', 'World', 'Primary Energy|Gas', 'EJ/y', 2, 7],
... ],
...     columns=['model', 'scenario', 'region', 'variable', 'unit', 2005, 2010],
... )
>>> year_df = IamDataFrame(data=eg_df)
>>> year_df.timeseries()
                                                  2005  2010
model   scenario region variable            unit            
model_a scen_a   World  Primary Energy|Coal EJ/y   0.5   3.0
                        Primary Energy|Gas  EJ/y   1.0   6.0
        scen_b   World  Primary Energy|Gas  EJ/y   2.0   7.0
>>> 
>>> year_df.aggregate('Primary Energy', append=True)
>>> year_df.timeseries()
                                                  2005  2010
model   scenario region variable            unit            
model_a scen_a   World  Primary Energy      EJ/y   1.5   9.0
                        Primary Energy|Coal EJ/y   0.5   3.0
                        Primary Energy|Gas  EJ/y   1.0   6.0
        scen_b   World  Primary Energy      EJ/y   2.0   7.0
                        Primary Energy|Gas  EJ/y   2.0   7.0
>>> 
>>> dt_df = IamDataFrame(data=eg_df.rename(
...     {2005: datetime(2005, 1, 1), 2010: datetime(2010, 6, 1)},
...     axis="columns"
... ))
>>> dt_df.timeseries()
                                                  2005-01-01  2010-06-01
model   scenario region variable            unit                        
model_a scen_a   World  Primary Energy|Coal EJ/y         0.5         3.0
                        Primary Energy|Gas  EJ/y         1.0         6.0
        scen_b   World  Primary Energy|Gas  EJ/y         2.0         7.0
>>> 
>>> dt_df.aggregate('Primary Energy', append=True)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/zebedeenicholls/Documents/AGCEC/Misc/pyam-zn-fork/pyam/core.py", line 700, in aggregate
    df_components = _aggregate_by_variables(self.data, components, units)
  File "/Users/zebedeenicholls/Documents/AGCEC/Misc/pyam-zn-fork/pyam/core.py", line 1319, in _aggregate_by_variables
    return df.groupby(GROUP_IDX).sum()['value']
  File "/Users/zebedeenicholls/miniconda3/envs/pyam-zn-fork/lib/python3.6/site-packages/pandas/core/generic.py", line 7632, in groupby
    observed=observed, **kwargs)
  File "/Users/zebedeenicholls/miniconda3/envs/pyam-zn-fork/lib/python3.6/site-packages/pandas/core/groupby/groupby.py", line 2110, in groupby
    return klass(obj, by, **kwds)
  File "/Users/zebedeenicholls/miniconda3/envs/pyam-zn-fork/lib/python3.6/site-packages/pandas/core/groupby/groupby.py", line 360, in __init__
    mutated=self.mutated)
  File "/Users/zebedeenicholls/miniconda3/envs/pyam-zn-fork/lib/python3.6/site-packages/pandas/core/groupby/grouper.py", line 578, in _get_grouper
    raise KeyError(gpr)
KeyError: 'year'
```

**Failing Test**

Change `meta_df` to `test_df` in all tests in `tests/test_feature_aggregate.py`.

**System (please complete the following information):**

 - OS: macOS
 - Python and pyam version: Python 3.6.7, 0.1.2+53.gb0e6562.dirty

**Additional context**

The failure occurs because the definition of `GROUP_IDX` is static. Hence line 1319 of `pyam/core.py` looks for a `year` column and doesn't find it, hence the failure. I think there are two possible solutions here:

1. Like was done with `LONG_IDX`, move `GROUP_IDX` to being an attribute of `IamDataFrame`. It would make sense to also move `YEAR_IDX`, `REGION_IDX` and `SORT_IDX` at the same time in order to avoid these failures in other places. (I would suggest moving all indexes to being attributes of `IamDataFrame` for consistency.)
1. Drop `year` support and always use `time` columns. Given earlier discussions I think this one is unlikely to happen, but this instance is another place where it would be a sensible solution. Just for completeness, I think dropping `year` support has a couple of extra arguments in its favour. It's always possible to recover year information from a datetime. It would make the meaning of the `time` column uniform (always being a datetime) and hence force users with other `time` information to put it in more well named columns (e.g. `Summer evening` might have to go in a `season-time-of-day` column instead to be more explicit, it would be annoying to try and filter e.g. `time` based on year only to have it explode because `time` is full of strings). 


Pull request:
Extend meta_df tests to include datetime
# Please confirm that this PR has done the following:

- [x] Tests Added
- [x] Documentation Added (N/A)
- [x] Description in RELEASE_NOTES.md Added

## Adding to RELEASE_NOTES.md (remove section after adding to RELEASE_NOTES.md)

Please add a single line in the release notes similar to the following:

```
- (#XX)[http://link-to-pr.com] Added feature which does something
```

# Description of PR

Closes #222. Good news is this was already sorted in v0.2.0 so all this does is make sure it stays that way by expanding the tests a bit.


Patch:
diff --git a/RELEASE_NOTES.md b/RELEASE_NOTES.md
index 6766a16..47ad60e 100644
--- a/RELEASE_NOTES.md
+++ b/RELEASE_NOTES.md
@@ -1,6 +1,8 @@
 
 # Next Release
 
+- [#236](https://github.com/IAMconsortium/pyam/pull/236) Add `swap_time_for_year` method and confirm datetime column is compatible with pyam features
+
 # Release v0.2.0
 
 ## Highlights
diff --git a/appveyor.yml b/appveyor.yml
index e54f789..4a69d02 100644
--- a/appveyor.yml
+++ b/appveyor.yml
@@ -25,8 +25,8 @@ install:
   - conda create -n testing python="%PYTHON_VERSION%" --yes
   - conda --version
   - activate testing
-  - conda install -y numpy pandas pyyaml xlrd xlsxwriter matplotlib==3.0.3 seaborn==0.9.0 six requests jupyter nbconvert
-  - conda install -y -c conda-forge libiconv gdal fiona "geopandas<0.5.0" cartopy
+  - conda install -y numpy pandas pyyaml xlrd xlsxwriter matplotlib==3.0.3 seaborn==0.9.0 six requests jupyter nbconvert proj4==5.2.0
+  - conda install -y -c conda-forge libiconv gdal fiona "geopandas<0.5.0" cartopy cython pyproj==1.9.6
 
 build: false
 
diff --git a/ci/environment-conda-forge.txt b/ci/environment-conda-forge.txt
index d2acba7..f5830e1 100644
--- a/ci/environment-conda-forge.txt
+++ b/ci/environment-conda-forge.txt
@@ -2,4 +2,5 @@ libiconv
 gdal
 fiona
 "geopandas<0.5.0"
+cython
 cartopy
diff --git a/pyam/core.py b/pyam/core.py
index 81f3b06..dbd88b0 100644
--- a/pyam/core.py
+++ b/pyam/core.py
@@ -309,6 +309,36 @@ class IamDataFrame(object):
         fill_values['year'] = year
         self.data = self.data.append(fill_values, ignore_index=True)
 
+    def swap_time_for_year(self, inplace=False):
+        """Convert the `time` column to `year`.
+
+        Parameters
+        ----------
+        inplace: bool, default False
+            if True, do operation inplace and return None
+
+        Raises
+        ------
+        ValueError
+            "time" is not a column of `self.data`
+        """
+        if "time" not in self.data:
+            raise ValueError("time column must be datetime to use this method")
+
+        ret = self.copy() if not inplace else self
+
+        ret.data["year"] = ret.data["time"].apply(lambda x: x.year)
+        ret.data = ret.data.drop("time", axis="columns")
+        ret._LONG_IDX = [v if v != "time" else "year" for v in ret._LONG_IDX]
+
+        if any(ret.data[ret._LONG_IDX].duplicated()):
+            error_msg = ('swapping time for year will result in duplicate '
+                         'rows in `data`!')
+            raise ValueError(error_msg)
+
+        if not inplace:
+            return ret
+
     def as_pandas(self, with_metadata=False):
         """Return this as a pd.DataFrame
 
@@ -1364,6 +1394,10 @@ def _check_rows(rows, check, in_range=True, return_test='any'):
         msg = 'Unknown checking type: {}'
         raise ValueError(msg.format(check.keys() - valid_checks))
 
+    if 'year' not in rows:
+        rows = rows.copy()
+        rows['year'] = rows['time'].apply(lambda x: x.year)
+
     where_idx = set(rows.index[rows['year'] == check['year']]) \
         if 'year' in check else set(rows.index)
     rows = rows.loc[list(where_idx)]


Test patch:
diff --git a/tests/conftest.py b/tests/conftest.py
index 6f8a6ad..e80d183 100644
--- a/tests/conftest.py
+++ b/tests/conftest.py
@@ -167,15 +167,19 @@ TEST_STACKPLOT_DF['model'] = 'IMG'
 TEST_STACKPLOT_DF['scenario'] = 'a_scen'
 
 
-TIME_AXES = [
-    [2005, 2010],
-    [datetime(2005, 6, 17), datetime(2010, 7, 21)],
-    ['2005-06-17', '2010-07-21'],
-    ['2005-06-17 00:00:00', '2010-07-21 12:00:00']
-]
-
-
-@pytest.fixture(scope="function", params=TIME_AXES)
+TEST_YEARS = [2005, 2010]
+TEST_DTS = [datetime(2005, 6, 17), datetime(2010, 7, 21)]
+
+
+@pytest.fixture(
+    scope="function",
+    params=[
+        TEST_YEARS,
+        TEST_DTS,
+        ['2005-06-17', '2010-07-21'],
+        ['2005-06-17 00:00:00', '2010-07-21 12:00:00']
+    ]
+)
 def test_df(request):
     tdf = TEST_DF.iloc[:2]
     tdf = tdf.rename({2005: request.param[0], 2010: request.param[1]},
@@ -195,9 +199,17 @@ def test_pd_df():
     yield TEST_DF.copy()
 
 
-@pytest.fixture(scope="function")
-def meta_df():
-    df = IamDataFrame(data=TEST_DF)
+@pytest.fixture(
+    scope="function",
+    params=[
+        TEST_YEARS,
+        TEST_DTS,
+    ]
+)
+def meta_df(request):
+    mdf = TEST_DF.rename({2005: request.param[0], 2010: request.param[1]},
+                         axis="columns")
+    df = IamDataFrame(data=mdf)
     yield df
 
 
diff --git a/tests/test_cast_to_iamc.py b/tests/test_cast_to_iamc.py
index c190b99..3e9e760 100644
--- a/tests/test_cast_to_iamc.py
+++ b/tests/test_cast_to_iamc.py
@@ -2,40 +2,45 @@ import pytest
 import pandas as pd
 from pyam import IamDataFrame, compare
 
+from conftest import TEST_DTS
 
 def test_cast_from_value_col(meta_df):
     df_with_value_cols = pd.DataFrame([
-        ['model_a', 'scen_a', 'World', 'EJ/y', 2005, 1, 0.5],
-        ['model_a', 'scen_a', 'World', 'EJ/y', 2010, 6., 3],
-        ['model_a', 'scen_b', 'World', 'EJ/y', 2005, 2, None],
-        ['model_a', 'scen_b', 'World', 'EJ/y', 2010, 7, None]
+        ['model_a', 'scen_a', 'World', 'EJ/y', TEST_DTS[0], 1, 0.5],
+        ['model_a', 'scen_a', 'World', 'EJ/y', TEST_DTS[1], 6., 3],
+        ['model_a', 'scen_b', 'World', 'EJ/y', TEST_DTS[0], 2, None],
+        ['model_a', 'scen_b', 'World', 'EJ/y', TEST_DTS[1], 7, None]
     ],
-        columns=['model', 'scenario', 'region', 'unit', 'year',
+        columns=['model', 'scenario', 'region', 'unit', 'time',
                  'Primary Energy', 'Primary Energy|Coal'],
     )
     df = IamDataFrame(df_with_value_cols,
                       value=['Primary Energy', 'Primary Energy|Coal'])
+    if "year" in meta_df.data.columns:
+        df = df.swap_time_for_year()
 
     assert compare(meta_df, df).empty
-    pd.testing.assert_frame_equal(df.data, meta_df.data)
+    pd.testing.assert_frame_equal(df.data, meta_df.data, check_like=True)
 
 
 def test_cast_from_value_col_and_args(meta_df):
     # checks for issue [#210](https://github.com/IAMconsortium/pyam/issues/210)
     df_with_value_cols = pd.DataFrame([
-        ['scen_a', 'World', 'EJ/y', 2005, 1, 0.5],
-        ['scen_a', 'World', 'EJ/y', 2010, 6., 3],
-        ['scen_b', 'World', 'EJ/y', 2005, 2, None],
-        ['scen_b', 'World', 'EJ/y', 2010, 7, None]
+        ['scen_a', 'World', 'EJ/y', TEST_DTS[0], 1, 0.5],
+        ['scen_a', 'World', 'EJ/y', TEST_DTS[1], 6., 3],
+        ['scen_b', 'World', 'EJ/y', TEST_DTS[0], 2, None],
+        ['scen_b', 'World', 'EJ/y', TEST_DTS[1], 7, None]
     ],
-        columns=['scenario', 'iso', 'unit', 'year',
+        columns=['scenario', 'iso', 'unit', 'time',
                  'Primary Energy', 'Primary Energy|Coal'],
     )
     df = IamDataFrame(df_with_value_cols, model='model_a', region='iso',
                       value=['Primary Energy', 'Primary Energy|Coal'])
+    if "year" in meta_df.data.columns:
+        df = df.swap_time_for_year()
 
     assert compare(meta_df, df).empty
-    pd.testing.assert_frame_equal(df.data, meta_df.data)
+    pd.testing.assert_frame_equal(df.data, meta_df.data, check_like=True)
 
 
 def test_cast_with_model_arg_raises():
@@ -58,17 +63,21 @@ def test_cast_with_model_arg(meta_df):
 
 
 def test_cast_by_column_concat(meta_df):
+    dts = TEST_DTS
     df = pd.DataFrame([
         ['scen_a', 'World', 'Primary Energy', None, 'EJ/y', 1, 6.],
         ['scen_a', 'World', 'Primary Energy', 'Coal', 'EJ/y', 0.5, 3],
         ['scen_b', 'World', 'Primary Energy', None, 'EJ/y', 2, 7],
     ],
-        columns=['scenario', 'region', 'var_1', 'var_2', 'unit', 2005, 2010],
+        columns=['scenario', 'region', 'var_1', 'var_2', 'unit'] + dts,
     )
 
     df = IamDataFrame(df, model='model_a', variable=['var_1', 'var_2'])
+    if "year" in meta_df.data.columns:
+        df = df.swap_time_for_year()
+
     assert compare(meta_df, df).empty
-    pd.testing.assert_frame_equal(df.data, meta_df.data)
+    pd.testing.assert_frame_equal(df.data, meta_df.data, check_like=True)
 
 
 def test_cast_with_variable_and_value(meta_df):
diff --git a/tests/test_core.py b/tests/test_core.py
index 222affe..d6dd724 100644
--- a/tests/test_core.py
+++ b/tests/test_core.py
@@ -8,10 +8,10 @@ import pandas as pd
 from numpy import testing as npt
 
 from pyam import IamDataFrame, validate, categorize, \
-    require_variable, filter_by_meta, META_IDX, IAMC_IDX, sort_data
+    require_variable, filter_by_meta, META_IDX, IAMC_IDX, sort_data, compare
 from pyam.core import _meta_idx, concat
 
-from conftest import TEST_DATA_DIR
+from conftest import TEST_DATA_DIR, TEST_DTS
 
 
 df_filter_by_meta_matching_idx = pd.DataFrame([
@@ -518,7 +518,11 @@ def test_validate_up(meta_df):
     obs = meta_df.validate({'Primary Energy': {'up': 6.5}},
                            exclude_on_fail=False)
     assert len(obs) == 1
-    assert obs['year'].values[0] == 2010
+    if 'year' in meta_df.data:
+        assert obs['year'].values[0] == 2010
+    else:
+        exp_time = pd.to_datetime(datetime.datetime(2010, 7, 21))
+        assert pd.to_datetime(obs['time'].values[0]) == exp_time
 
     assert list(meta_df['exclude']) == [False, False]  # assert none excluded
 
@@ -526,14 +530,24 @@ def test_validate_up(meta_df):
 def test_validate_lo(meta_df):
     obs = meta_df.validate({'Primary Energy': {'up': 8, 'lo': 2.0}})
     assert len(obs) == 1
-    assert obs['year'].values[0] == 2005
+    if 'year' in meta_df.data:
+        assert obs['year'].values[0] == 2005
+    else:
+        exp_year = pd.to_datetime(datetime.datetime(2005, 6, 17))
+        assert pd.to_datetime(obs['time'].values[0]) == exp_year
+
     assert list(obs['scenario'].values) == ['scen_a']
 
 
 def test_validate_both(meta_df):
     obs = meta_df.validate({'Primary Energy': {'up': 6.5, 'lo': 2.0}})
     assert len(obs) == 2
-    assert list(obs['year'].values) == [2005, 2010]
+    if 'year' in meta_df.data:
+        assert list(obs['year'].values) == [2005, 2010]
+    else:
+        exp_time = pd.to_datetime(TEST_DTS)
+        assert (pd.to_datetime(obs['time'].values) == exp_time).all()
+
     assert list(obs['scenario'].values) == ['scen_a', 'scen_b']
 
 
@@ -556,7 +570,11 @@ def test_validate_top_level(meta_df):
     obs = validate(meta_df, criteria={'Primary Energy': {'up': 6.0}},
                    exclude_on_fail=True, variable='Primary Energy')
     assert len(obs) == 1
-    assert obs['year'].values[0] == 2010
+    if 'year' in meta_df.data:
+        assert obs['year'].values[0] == 2010
+    else:
+        exp_time = pd.to_datetime(datetime.datetime(2010, 7, 21))
+        assert (pd.to_datetime(obs['time'].values[0]) == exp_time)
     assert list(meta_df['exclude']) == [False, True]
 
 
@@ -971,10 +989,51 @@ def test_normalize(meta_df):
     exp = meta_df.data.copy().reset_index(drop=True)
     exp['value'][1::2] /= exp['value'][::2].values
     exp['value'][::2] /= exp['value'][::2].values
-    obs = meta_df.normalize(year=2005).data.reset_index(drop=True)
+    if "year" in meta_df.data:
+        obs = meta_df.normalize(year=2005).data.reset_index(drop=True)
+    else:
+        obs = meta_df.normalize(
+            time=datetime.datetime(2005, 6, 17)
+        ).data.reset_index(drop=True)
     pd.testing.assert_frame_equal(obs, exp)
 
 
 def test_normalize_not_time(meta_df):
     pytest.raises(ValueError, meta_df.normalize, variable='foo')
     pytest.raises(ValueError, meta_df.normalize, year=2015, variable='foo')
+
+
+@pytest.mark.parametrize("inplace", [True, False])
+def test_swap_time_to_year(test_df, inplace):
+    if "year" in test_df.data:
+        return  # year df not relevant for this test
+
+    exp = test_df.data.copy()
+    exp["year"] = exp["time"].apply(lambda x: x.year)
+    exp = exp.drop("time", axis="columns")
+    exp = IamDataFrame(exp)
+
+    obs = test_df.swap_time_for_year(inplace=inplace)
+
+    if inplace:
+        assert obs is None
+        assert compare(test_df, exp).empty
+    else:
+        assert compare(obs, exp).empty
+        assert "year" not in test_df.data.columns
+
+
+@pytest.mark.parametrize("inplace", [True, False])
+def test_swap_time_to_year_errors(test_df, inplace):
+    if "year" in test_df.data:
+        with pytest.raises(ValueError):
+            test_df.swap_time_for_year(inplace=inplace)
+        return
+
+    tdf = test_df.data.copy()
+    tdf["time"] = tdf["time"].apply(
+        lambda x: datetime.datetime(2005, x.month, x.day)
+    )
+
+    with pytest.raises(ValueError):
+        IamDataFrame(tdf).swap_time_for_year(inplace=inplace)
diff --git a/tests/test_feature_aggregate.py b/tests/test_feature_aggregate.py
index 714f51a..d6511de 100644
--- a/tests/test_feature_aggregate.py
+++ b/tests/test_feature_aggregate.py
@@ -2,6 +2,7 @@ import numpy as np
 import pandas as pd
 from pyam import check_aggregate, IAMC_IDX
 
+from conftest import TEST_DTS
 
 def test_do_aggregate_append(meta_df):
     meta_df.rename({'variable': {'Primary Energy': 'Primary Energy|Gas'}},
@@ -9,13 +10,19 @@ def test_do_aggregate_append(meta_df):
     meta_df.aggregate('Primary Energy', append=True)
     obs = meta_df.filter(variable='Primary Energy').timeseries()
 
+    dts = TEST_DTS
+    times = [2005, 2010] if "year" in meta_df.data else dts
     exp = pd.DataFrame([
         ['model_a', 'scen_a', 'World', 'Primary Energy', 'EJ/y', 1.5, 9.],
         ['model_a', 'scen_b', 'World', 'Primary Energy', 'EJ/y', 2, 7],
     ],
-        columns=['model', 'scenario', 'region', 'variable', 'unit', 2005, 2010]
+        columns=['model', 'scenario', 'region', 'variable', 'unit'] + times
     ).set_index(IAMC_IDX)
-    exp.columns = list(map(int, exp.columns))
+    if "year" in meta_df.data:
+        exp.columns = list(map(int, exp.columns))
+    else:
+        exp.columns = pd.to_datetime(exp.columns)
+
     pd.testing.assert_frame_equal(obs, exp)
 
 
diff --git a/tests/test_feature_append_rename_convert.py b/tests/test_feature_append_rename_convert.py
index 391e0f2..ea1ad1f 100644
--- a/tests/test_feature_append_rename_convert.py
+++ b/tests/test_feature_append_rename_convert.py
@@ -1,13 +1,15 @@
 import copy
 import pytest
+import datetime as dt
 
 import numpy as np
 import pandas as pd
 from numpy import testing as npt
 
-
 from pyam import IamDataFrame, META_IDX, IAMC_IDX, compare
 
+from conftest import TEST_DTS
+
 
 RENAME_DF = IamDataFrame(pd.DataFrame([
     ['model', 'scen', 'region_a', 'test_1', 'unit', 1, 5],
@@ -136,13 +138,18 @@ def test_rename_index(meta_df):
     obs = meta_df.rename(mapping, scenario={'scen_a': 'scen_c'})
 
     # test data changes
+    dts = TEST_DTS
+    times = [2005, 2010] if 'year' in meta_df.data else dts
     exp = pd.DataFrame([
         ['model_b', 'scen_c', 'World', 'Primary Energy', 'EJ/y', 1, 6.],
         ['model_b', 'scen_c', 'World', 'Primary Energy|Coal', 'EJ/y', 0.5, 3],
         ['model_a', 'scen_b', 'World', 'Primary Energy', 'EJ/y', 2, 7],
-    ], columns=['model', 'scenario', 'region', 'variable', 'unit', 2005, 2010]
+    ], columns=['model', 'scenario', 'region', 'variable', 'unit'] + times
     ).set_index(IAMC_IDX).sort_index()
-    exp.columns = exp.columns.map(int)
+    if "year" in meta_df.data:
+        exp.columns = list(map(int, exp.columns))
+    else:
+        exp.columns = pd.to_datetime(exp.columns)
     pd.testing.assert_frame_equal(obs.timeseries().sort_index(), exp)
 
     # test meta changes
@@ -160,15 +167,20 @@ def test_rename_append(meta_df):
     obs = meta_df.rename(mapping, append=True)
 
     # test data changes
+    dts = [dt.datetime(2005, 6, 17), dt.datetime(2010, 7, 21)]
+    times = [2005, 2010] if "year" in meta_df.data else dts
     exp = pd.DataFrame([
         ['model_a', 'scen_a', 'World', 'Primary Energy', 'EJ/y', 1, 6.],
         ['model_a', 'scen_a', 'World', 'Primary Energy|Coal', 'EJ/y', 0.5, 3],
         ['model_a', 'scen_b', 'World', 'Primary Energy', 'EJ/y', 2, 7],
         ['model_b', 'scen_c', 'World', 'Primary Energy', 'EJ/y', 1, 6.],
         ['model_b', 'scen_c', 'World', 'Primary Energy|Coal', 'EJ/y', 0.5, 3],
-    ], columns=['model', 'scenario', 'region', 'variable', 'unit', 2005, 2010]
+    ], columns=['model', 'scenario', 'region', 'variable', 'unit'] + times
     ).set_index(IAMC_IDX).sort_index()
-    exp.columns = exp.columns.map(int)
+    if "year" in meta_df.data:
+        exp.columns = list(map(int, exp.columns))
+    else:
+        exp.columns = pd.to_datetime(exp.columns)
     pd.testing.assert_frame_equal(obs.timeseries().sort_index(), exp)
 
     # test meta changes
diff --git a/tests/test_feature_compare.py b/tests/test_feature_compare.py
index e91b679..e9813ae 100644
--- a/tests/test_feature_compare.py
+++ b/tests/test_feature_compare.py
@@ -1,4 +1,7 @@
 import copy
+import datetime as dt
+
+
 import numpy as np
 import pandas as pd
 from pyam import compare, IAMC_IDX
@@ -13,17 +16,23 @@ def test_compare(meta_df):
     obs = compare(meta_df, clone, right_label='meta_df', left_label='clone')
 
     exp = pd.DataFrame([
-        ['Primary Energy', 'EJ/y', 2005, 2, 1],
-        ['Primary Energy|Coal', 'EJ/y', 2005, np.nan, 0.5],
-        ['Primary Energy|Coal', 'EJ/y', 2010, np.nan, 3],
-        ['Primary Energy|Gas', 'EJ/y', 2005, 0.5, np.nan],
-        ['Primary Energy|Gas', 'EJ/y', 2010, 3, np.nan],
+        ['Primary Energy', 'EJ/y', dt.datetime(2005, 6, 17), 2, 1],
+        ['Primary Energy|Coal', 'EJ/y', dt.datetime(2005, 6, 17), np.nan, 0.5],
+        ['Primary Energy|Coal', 'EJ/y', dt.datetime(2010, 7, 21), np.nan, 3],
+        ['Primary Energy|Gas', 'EJ/y', dt.datetime(2005, 6, 17), 0.5, np.nan],
+        ['Primary Energy|Gas', 'EJ/y', dt.datetime(2010, 7, 21), 3, np.nan],
     ],
-        columns=['variable', 'unit', 'year', 'meta_df', 'clone'],
+        columns=['variable', 'unit', 'time', 'meta_df', 'clone'],
     )
     exp['model'] = 'model_a'
     exp['scenario'] = 'scen_a'
     exp['region'] = 'World'
-    exp = exp.set_index(IAMC_IDX + ['year'])
+    time_col = 'time'
+    if 'year' in meta_df.data.columns:
+        exp['year'] = exp['time'].apply(lambda x: x.year)
+        exp = exp.drop('time', axis='columns')
+        time_col = 'year'
+
+    exp = exp.set_index(IAMC_IDX + [time_col])
 
     pd.testing.assert_frame_equal(obs, exp)