Repository: natcap/invest (python)

Issue:
Improve CBC error reporting when fields don't match
An issue came up on the forums (https://community.naturalcapitalproject.org/t/soil-disturb-in-coastal-blue-carbon/2321/7?u=jdouglass) where the poster was using a field value (`soil-disturb`) in the transition matrix when `{low,med,high}-impact-disturb` was expected.  All this user got in terms of an error was an unhelpful `KeyError`.

We should be able to identify that the key is invalid and provide a more helpful error message ... particularly considering that CBC has such intricately related tables, that kind of error reporting would help us all out a ton when these issues arise.

Marking as a bug because it's kind a bug in error reporting for table values.

Pull request description:
Add valuation for transition table
## Description
Adds up front valuation for the transition table values. Since we know the expected values we can pre-emptively let users know something is amiss with their table.

~On hold until #1345 which will effect how csv's are read in as pandas dataframes.~

![image](https://github.com/natcap/invest/assets/2659980/53b07f9e-c988-4592-b5b7-e6f1ef684890)

Fixes #729 

## Checklist
- [x] Updated HISTORY.rst and link to any relevant issue (if these changes are user-facing)
- ~[ ] Updated the user's guide (if needed)~
- [x] Tested the affected models' UIs (if relevant)


Patch:
diff --git a/HISTORY.rst b/HISTORY.rst
index 772b0de21..b34b61148 100644
--- a/HISTORY.rst
+++ b/HISTORY.rst
@@ -48,6 +48,10 @@ Unreleased Changes
       has been merged into ``utils.read_csv_to_dataframe``
       (`#1319 <https://github.com/natcap/invest/issues/1319>`_),
       (`#1327 <https://github.com/natcap/invest/issues/1327>`_)
+* Coastal Blue Carbon
+    * Added validation for the transition table, raising a validation error if
+      unexpected values are encountered.
+      (`#729 <https://github.com/natcap/invest/issues/729>`_)
 * Workbench
     * Fixed a bug where sampledata downloads failed silently (and progress bar
       became innacurate) if the Workbench did not have write permission to
diff --git a/src/natcap/invest/coastal_blue_carbon/coastal_blue_carbon.py b/src/natcap/invest/coastal_blue_carbon/coastal_blue_carbon.py
index 96334d917..031144ee2 100644
--- a/src/natcap/invest/coastal_blue_carbon/coastal_blue_carbon.py
+++ b/src/natcap/invest/coastal_blue_carbon/coastal_blue_carbon.py
@@ -118,6 +118,9 @@ INVALID_ANALYSIS_YEAR_MSG = gettext(
     "({latest_year})")
 INVALID_SNAPSHOT_RASTER_MSG = gettext(
     "Raster for snapshot {snapshot_year} could not be validated.")
+INVALID_TRANSITION_VALUES_MSG = gettext(
+    "The transition table expects values of {model_transitions} but found "
+    "values of {transition_values}.")
 
 POOL_SOIL = 'soil'
 POOL_BIOMASS = 'biomass'
@@ -2259,4 +2262,26 @@ def validate(args, limit_to=None):
                         analysis_year=args['analysis_year'],
                         latest_year=max(snapshots.keys()))))
 
+    # check for invalid options in the translation table
+    if ("landcover_transitions_table" not in invalid_keys and
+            "landcover_transitions_table" in sufficient_keys):
+        transitions_spec = MODEL_SPEC['args']['landcover_transitions_table']
+        transition_options = list(
+            transitions_spec['columns']['[LULC CODE]']['options'].keys())
+        # lowercase options since utils call will lowercase table values
+        transition_options = [x.lower() for x in transition_options]
+        transitions_df = utils.read_csv_to_dataframe(
+            args['landcover_transitions_table'], transitions_spec)
+        transitions_mask = ~transitions_df.isin(transition_options) & ~transitions_df.isna()
+        if transitions_mask.any(axis=None):
+            transition_numpy_mask = transitions_mask.values
+            transition_numpy_values = transitions_df.to_numpy()
+            bad_transition_values = list(
+                numpy.unique(transition_numpy_values[transition_numpy_mask]))
+            validation_warnings.append((
+                ['landcover_transitions_table'],
+                INVALID_TRANSITION_VALUES_MSG.format(
+                    model_transitions=(transition_options),
+                    transition_values=bad_transition_values)))
+
     return validation_warnings


Test patch:
diff --git a/tests/test_coastal_blue_carbon.py b/tests/test_coastal_blue_carbon.py
index f06ef17ad..3bec14303 100644
--- a/tests/test_coastal_blue_carbon.py
+++ b/tests/test_coastal_blue_carbon.py
@@ -877,8 +877,18 @@ class TestCBC2(unittest.TestCase):
         # analysis year must be >= the last transition year.
         args['analysis_year'] = baseline_year
 
+        # Write invalid entries to landcover transition table
+        with open(args['landcover_transitions_table'], 'w') as transition_table:
+            transition_table.write('lulc-class,Developed,Forest,Water\n')
+            transition_table.write('Developed,NCC,,invalid\n')
+            transition_table.write('Forest,accum,disturb,low-impact-disturb\n')
+            transition_table.write('Water,disturb,med-impact-disturb,high-impact-disturb\n')
+        transition_options = [
+                'accum', 'high-impact-disturb', 'med-impact-disturb',
+                'low-impact-disturb', 'ncc']
+
         validation_warnings = coastal_blue_carbon.validate(args)
-        self.assertEqual(len(validation_warnings), 2)
+        self.assertEqual(len(validation_warnings), 3)
         self.assertIn(
             coastal_blue_carbon.INVALID_SNAPSHOT_RASTER_MSG.format(
                 snapshot_year=baseline_year + 10),
@@ -887,6 +897,11 @@ class TestCBC2(unittest.TestCase):
             coastal_blue_carbon.INVALID_ANALYSIS_YEAR_MSG.format(
                 analysis_year=2000, latest_year=2010),
             validation_warnings[1][1])
+        self.assertIn(
+            coastal_blue_carbon.INVALID_TRANSITION_VALUES_MSG.format(
+                model_transitions=transition_options,
+                transition_values=['disturb', 'invalid']),
+            validation_warnings[2][1])
 
     def test_track_first_disturbance(self):
         """CBC: Track disturbances over time."""