Repository: IAMconsortium/pyam
Pull request #203 (2019-02-27 14:20:52)

extend use case for `format_data()` to take `variable` and `value`
# Please confirm that this PR has done the following:

- [x] Tests Added
- [x] Documentation Added

# Description of PR

Working on the `message_ix.Reporting` class, I realized that we would need a way to cast an `IamDataFrame` with both a `variable` and a `value` arg passed to the "fill-by-value/rename/concat" part of the `format_data()` function,


coveralls: 
[![Coverage Status](https://coveralls.io/builds/21881897/badge)](https://coveralls.io/builds/21881897)

Coverage increased (+0.02%) to 84.623% when pulling **183a294c4ef7d1f9386c181b2e13ce0042f08cfe on danielhuppmann:quickfix_format_data** into **bc5fcbc9fa9c4ea70970f40ff992f2d3290c470e on IAMconsortium:master**.


Patch:
diff --git a/pyam/utils.py b/pyam/utils.py
index 7351fcd..755ff24 100644
--- a/pyam/utils.py
+++ b/pyam/utils.py
@@ -132,18 +132,12 @@ def format_data(df, **kwargs):
     if isinstance(df, pd.Series):
         df = df.to_frame()
 
-    # ensure that only either `value` or `variable` custom setting is used
-    _cols = ['value', 'variable']
-    if any([i in kwargs for i in _cols]) and \
-            all([i in kwargs or i in df.columns for i in _cols]):
-        raise ValueError('using both `value` and `variable` is not valid!')
-
-    # if `value` arg is given, melt columns and use column name as `variable`
-    if 'value' in kwargs:
+    # if `value` is given but not `variable`,
+    # melt value columns and use column name as `variable`
+    if 'value' in kwargs and 'variable' not in kwargs:
         value = kwargs.pop('value')
         idx = set(df.columns) & (set(IAMC_IDX) | set(['year', 'time']))
         _df = df.set_index(list(idx))
-        print(_df)
         dfs = []
         for v in value if islistable(value) else [value]:
             if v not in df.columns:
@@ -153,10 +147,11 @@ def format_data(df, **kwargs):
             dfs.append(vdf.reset_index())
         df = pd.concat(dfs).reset_index(drop=True)
 
-    # for other columns, do a rename or concat multiple columns to IAMC-style
+    # otherwise, do a fill-by-value or rename columns or concat to IAMC-style
     for col, value in kwargs.items():
         if col in df:
-            raise ValueError('conflict of kwarg with column in dataframe!')
+            raise ValueError('conflict of kwarg with column `{}` in dataframe!'
+                             .format(col))
 
         if isstr(value) and value in df:
             df.rename(columns={value: col}, inplace=True)


Test patch:
diff --git a/tests/test_cast_to_iamc.py b/tests/test_cast_to_iamc.py
index a53b915..1d6da9a 100644
--- a/tests/test_cast_to_iamc.py
+++ b/tests/test_cast_to_iamc.py
@@ -51,3 +51,13 @@ def test_cast_by_column_concat(meta_df):
     df = IamDataFrame(df, model='model_a', variable=['var_1', 'var_2'])
     assert compare(meta_df, df).empty
     pd.testing.assert_frame_equal(df.data, meta_df.data)
+
+
+def test_cast_with_variable_and_value(meta_df):
+    pe_df = meta_df.filter(variable='Primary Energy')
+    df = pe_df.data.rename(columns={'value': 'lvl'}).drop('variable', axis=1)
+
+    df = IamDataFrame(df, variable='Primary Energy', value='lvl')
+
+    assert compare(pe_df, df).empty
+    pd.testing.assert_frame_equal(df.data, pe_df.data.reset_index(drop=True))