Repository: IAMconsortium/pyam
Pull request #599 (2021-12-31 17:13:45)

Add `time_domain` and `time` attributes
**Note**: this PR is targeted to the `feature/year-and-datetime` branch that collects all developments related to #597.

# Description of PR

This PR adds the `time_domain` attribute (value: year, datetime, mixed) and a `time` attribute that returns an Index of the time domain.

Previously, there was either a `year` attribute (returning a list of integers) or a `time` attribute (returning an pandas.Index) depending on the time domain.

Now, the `time` attribute exists for any **IamDataFrame** (returning an index depending on the time domain). The `year` attribute continues to exist for **IamDataFrame** instances that have (only) yearly data. 


codecov[bot]: # [Codecov](https://codecov.io/gh/IAMconsortium/pyam/pull/599?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=IAMconsortium) Report
> Merging [#599](https://codecov.io/gh/IAMconsortium/pyam/pull/599?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=IAMconsortium) (a4062e2) into [feature/year-and-datetime](https://codecov.io/gh/IAMconsortium/pyam/commit/32f01e92e0c85c169c4b7e884938209a81b26b41?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=IAMconsortium) (32f01e9) will **increase** coverage by `0.0%`.
> The diff coverage is `100.0%`.

[![Impacted file tree graph](https://codecov.io/gh/IAMconsortium/pyam/pull/599/graphs/tree.svg?width=650&height=150&src=pr&token=ThxWlNkonJ&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=IAMconsortium)](https://codecov.io/gh/IAMconsortium/pyam/pull/599?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=IAMconsortium)

```diff
@@                    Coverage Diff                    @@
##           feature/year-and-datetime    #599   +/-   ##
=========================================================
  Coverage                       94.1%   94.2%           
=========================================================
  Files                             50      50           
  Lines                           5358    5383   +25     
=========================================================
+ Hits                            5047    5072   +25     
  Misses                           311     311           
```


| [Impacted Files](https://codecov.io/gh/IAMconsortium/pyam/pull/599?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=IAMconsortium) | Coverage Δ | |
|---|---|---|
| [pyam/time.py](https://codecov.io/gh/IAMconsortium/pyam/pull/599/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=IAMconsortium#diff-cHlhbS90aW1lLnB5) | `96.0% <ø> (-0.1%)` | :arrow_down: |
| [pyam/core.py](https://codecov.io/gh/IAMconsortium/pyam/pull/599/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=IAMconsortium#diff-cHlhbS9jb3JlLnB5) | `94.4% <100.0%> (+0.1%)` | :arrow_up: |
| [tests/conftest.py](https://codecov.io/gh/IAMconsortium/pyam/pull/599/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=IAMconsortium#diff-dGVzdHMvY29uZnRlc3QucHk=) | `100.0% <100.0%> (ø)` | |
| [tests/test\_core.py](https://codecov.io/gh/IAMconsortium/pyam/pull/599/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=IAMconsortium#diff-dGVzdHMvdGVzdF9jb3JlLnB5) | `100.0% <100.0%> (ø)` | |
| [tests/test\_time.py](https://codecov.io/gh/IAMconsortium/pyam/pull/599/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=IAMconsortium#diff-dGVzdHMvdGVzdF90aW1lLnB5) | `100.0% <100.0%> (ø)` | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/IAMconsortium/pyam/pull/599?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=IAMconsortium).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=IAMconsortium)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/IAMconsortium/pyam/pull/599?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=IAMconsortium). Last update [32f01e9...a4062e2](https://codecov.io/gh/IAMconsortium/pyam/pull/599?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=IAMconsortium). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=IAMconsortium).


Patch:
diff --git a/pyam/core.py b/pyam/core.py
index d11b3ff..67c9691 100755
--- a/pyam/core.py
+++ b/pyam/core.py
@@ -206,13 +206,17 @@ class IamDataFrame(object):
         self.plot = PlotAccessor(self)
 
     def _set_attributes(self):
-        """Utility function to set attributes"""
+        """Utility function to set attributes, called on __init__/filter/append/..."""
 
-        # add time domain as attributes
+        # add/reset internal time-index attribute (set when first using `time`)
+        setattr(self, "_time", None)
+
+        # add/reset year attribute (only if time domain is year, i.e., all integer)
         if self.time_col == "year":
             setattr(self, "year", get_index_levels(self._data, "year"))
-        else:
-            setattr(self, "time", pd.Index(get_index_levels(self._data, "time")))
+
+        #  add/reset internal time domain attribute (set when first using `time_domain`)
+        setattr(self, "_time_domain", None)
 
         # set non-standard index columns as attributes
         for c in self.meta.index.names:
@@ -373,6 +377,20 @@ class IamDataFrame(object):
             .to_dict()
         )
 
+    @property
+    def time(self):
+        """The time index, i.e., axis labels related to the time domain.
+
+        The returned type is
+        - :class:`pandas.Int64Index` if the time_domain is 'year'
+        - :class:`pandas.DatetimeIndex` if the time domain is 'datetime'
+        - :class:`pandas.Index` if the time domain is 'mixed'
+        """
+        if self._time is None:
+            self._time = pd.Index(get_index_levels(self._data, self.time_col))
+
+        return self._time
+
     @property
     def data(self):
         """Return the timeseries data as a long :class:`pandas.DataFrame`"""
@@ -401,6 +419,19 @@ class IamDataFrame(object):
         """Return the list of `data` columns (index names & data coordinates)"""
         return list(self._data.index.names)
 
+    @property
+    def time_domain(self):
+        """Indicator of the time domain: 'year', 'datetime', or 'mixed'"""
+        if self._time_domain is None:
+            if self.time_col == "year":
+                self._time_domain = "year"
+            elif isinstance(self.time, pd.DatetimeIndex):
+                self._time_domain = "datetime"
+            else:
+                self._time_domain = "mixed"
+
+        return self._time_domain
+
     @property
     def _LONG_IDX(self):
         """DEPRECATED - please use `IamDataFrame.dimensions`"""
diff --git a/pyam/time.py b/pyam/time.py
index 8f89cdf..00fafff 100644
--- a/pyam/time.py
+++ b/pyam/time.py
@@ -40,7 +40,6 @@ def swap_time_for_year(df, inplace, subannual=False):
     ret._data.index = index
     ret.time_col = "year"
     ret._set_attributes()
-    delattr(ret, "time")
 
     if not inplace:
         return ret


Test patch:
diff --git a/tests/conftest.py b/tests/conftest.py
index 4275d73..4c875bd 100644
--- a/tests/conftest.py
+++ b/tests/conftest.py
@@ -34,6 +34,7 @@ TEST_YEARS = [2005, 2010]
 TEST_DTS = [datetime(2005, 6, 17), datetime(2010, 7, 21)]
 TEST_TIME_STR = ["2005-06-17", "2010-07-21"]
 TEST_TIME_STR_HR = ["2005-06-17 00:00:00", "2010-07-21 12:00:00"]
+TEST_TIME_MIXED = [2005, datetime(2010, 7, 21)]
 
 DTS_MAPPING = {2005: TEST_DTS[0], 2010: TEST_DTS[1]}
 
diff --git a/tests/test_core.py b/tests/test_core.py
index 712f974..ab98f52 100644
--- a/tests/test_core.py
+++ b/tests/test_core.py
@@ -8,11 +8,20 @@ import pandas as pd
 from numpy import testing as npt
 from pandas import testing as pdt
 
-from pyam import IamDataFrame, filter_by_meta, META_IDX, IAMC_IDX, sort_data, compare
+from pyam import IamDataFrame, filter_by_meta, META_IDX, IAMC_IDX, sort_data
 from pyam.core import _meta_idx, concat
 from pyam.utils import isstr
 from pyam.testing import assert_iamframe_equal
 
+from conftest import (
+    TEST_YEARS,
+    TEST_DTS,
+    TEST_TIME_STR,
+    TEST_TIME_STR_HR,
+    TEST_TIME_MIXED,
+)
+
+
 df_filter_by_meta_matching_idx = pd.DataFrame(
     [
         ["model_a", "scen_a", "region_1", 1],
@@ -304,6 +313,27 @@ def test_get_item(test_df, column):
     pdt.assert_series_equal(test_df[column], test_df.data[column])
 
 
+# TODO implement this parametrization as part of `conftest.py:test_df`
+@pytest.mark.parametrize(
+    "time, domain, index",
+    [
+        (TEST_YEARS, "year", pd.Int64Index([2005, 2010])),
+        (TEST_DTS, "datetime", pd.DatetimeIndex(TEST_DTS)),
+        (TEST_TIME_STR, "datetime", pd.DatetimeIndex(TEST_DTS)),
+        (TEST_TIME_STR_HR, "datetime", pd.DatetimeIndex(TEST_TIME_STR_HR)),
+        (TEST_TIME_MIXED, "mixed", pd.Index(TEST_TIME_MIXED)),
+    ],
+)
+def test_time_domain(test_pd_df, time, domain, index):
+    # assert that the time-domain and time-index attributes are set correctly
+    mapping = dict([(i, j) for i, j in zip(TEST_YEARS, time)])
+    df = IamDataFrame(data=test_pd_df.rename(mapping, axis="columns"))
+
+    assert df.time_col == "year" if domain == "year" else "time"
+    assert df.time_domain == domain
+    pdt.assert_index_equal(df.time, index)
+
+
 def test_index(test_df_year):
     # assert that the correct index is shown for the IamDataFrame
     exp = pd.MultiIndex.from_arrays(
@@ -322,7 +352,10 @@ def test_index_attributes(test_df):
     if test_df.time_col == "year":
         assert test_df.year == [2005, 2010]
     else:
-        assert test_df.time.equals(pd.Index(test_df.data.time.unique()))
+        match = "'IamDataFrame' object has no attribute 'year'"
+        with pytest.raises(AttributeError, match=match):
+            test_df.year
+    assert test_df.time.equals(pd.Index(test_df.data[test_df.time_col].unique()))
 
 
 def test_index_attributes_extra_col(test_pd_df):
@@ -614,11 +647,11 @@ def test_filter_time_not_datetime_range_error(test_df):
 
 
 def test_filter_year_with_time_col(test_pd_df):
-    test_pd_df["time"] = ["summer", "summer", "winter"]
+    test_pd_df["subannual"] = ["summer", "summer", "winter"]
     df = IamDataFrame(test_pd_df)
-    obs = df.filter(time="summer").timeseries()
+    obs = df.filter(subannual="summer").timeseries()
 
-    exp = test_pd_df.set_index(IAMC_IDX + ["time"])
+    exp = test_pd_df.set_index(IAMC_IDX + ["subannual"])
     exp.columns = list(map(int, exp.columns))
     pd.testing.assert_frame_equal(obs, exp[0:2])
 
diff --git a/tests/test_time.py b/tests/test_time.py
index acb2e2d..b474baf 100644
--- a/tests/test_time.py
+++ b/tests/test_time.py
@@ -1,5 +1,6 @@
 import pytest
 import pandas as pd
+import pandas.testing as pdt
 from datetime import datetime
 from pyam import IamDataFrame
 from pyam.testing import assert_iamframe_equal
@@ -46,9 +47,7 @@ def test_swap_time_to_year(test_df, inplace):
         obs = test_df
 
     assert_iamframe_equal(obs, exp)
-    match = "'IamDataFrame' object has no attribute 'time'"
-    with pytest.raises(AttributeError, match=match):
-        obs.time
+    pdt.assert_index_equal(obs.time, pd.Index([2005, 2010]))
 
 
 @pytest.mark.parametrize(