Repository: ioos/compliance-checker

Issue:
Classify variables defined by `grid_mapping` attribute as aux or ancillary variables
https://github.com/ioos/compliance-checker/blob/master/compliance_checker/cf/cf.py#L161

For each variable in the dataset find any and all attributes `grid_mapping` and see if there is a variable matching that name. If there is, it should be classified as an ancillary variable and not be checked for units or standard name.



Discussion:
This will fix https://github.com/ioos/compliance-checker/issues/223#issuecomment-190780420

Uh looks done already https://github.com/ioos/compliance-checker/blob/master/compliance_checker/cf/cf.py#L186-L189

in that case, we need to figure out why it's not working.

It's working fine.  It's the `check_units` test that doesn't know it is supposed to skip them, see https://github.com/ioos/compliance-checker/blob/master/compliance_checker/cf/cf.py#L530-L575

Or doesn't play by the rules (the dimensionless text?)

Yeah but it should pick it up here:
https://github.com/ioos/compliance-checker/blob/master/compliance_checker/cf/cf.py#L551

which checks ancillary here:
https://github.com/ioos/compliance-checker/blob/master/compliance_checker/cf/cf.py#L204

Unless I'm missing something

It checks ancillary and continues, aka doesn't add it to the list of metadata vars returned.  Either that's an omission or coded specifically this way, I don't remember.

I think that Dave has it on the head.  We aren't adding the mapping variables to the metadata var list, so we are acting as if they need to go through the full 3.1 check.  We should be adding it to the metadata var list and allowing them to skip this section.

Agreed?

to resolve adding ancillary here: https://github.com/ioos/compliance-checker/blob/master/compliance_checker/cf/cf.py#L551

aka here https://github.com/ioos/compliance-checker/blob/master/compliance_checker/cf/cf.py#L204-L205 (why `continue`?)

@daf Because ancillary variables are not technically metadata variables.  Or at least that was the argument laid out by Luke.  

I think it is fine because of 2 things:  1) Maybe there is a check where ancillary shouldn't be grouped with other metadata variables.  2) by making the change i suggested we fix the issue without the fear of disrupting other checks down the line.  

Ok, I see what you mean now - you want to pull ancillary variables as a line after https://github.com/ioos/compliance-checker/blob/master/compliance_checker/cf/cf.py#L551 and skip those in the `check_units`, right?

Correct.

(this new github reactions thing is weird)


Resolving patch:
diff --git a/compliance_checker/cf/cf.py b/compliance_checker/cf/cf.py
--- a/compliance_checker/cf/cf.py
+++ b/compliance_checker/cf/cf.py
@@ -549,6 +549,7 @@ def check_units(self, ds):
         boundary_vars = iter(self._find_boundary_vars(ds).values())
         container_vars = self._find_container_variables(ds)
         metadata_vars = self._find_metadata_vars(ds)
+        ancillary_vars = self._find_ancillary_vars(ds)
 
         for k, v in ds.variables.items():
 
@@ -556,6 +557,7 @@ def check_units(self, ds):
             if v in clim_vars or \
                v in boundary_vars or \
                v in metadata_vars or \
+               v in ancillary_vars or \
                k in container_vars:
                 continue
 
@@ -723,9 +725,9 @@ def check_ancillary_data(self, ds):
             anc_result = Result(BaseCheck.HIGH, name=('§3.4 Ancillary Variables', k))
             msgs = []
 
-            if not isinstance(anc, str):
+            if not isinstance(anc, basestring):
                 anc_result.value = False
-                anc_result.msgs = ["ancillary_variables is not a string"]
+                anc_result.msgs = ["ancillary_variable(s) {} is not a string or unicode".format(anc)]
                 ret_val.append(anc_result)
                 continue
 
@@ -736,7 +738,7 @@ def check_ancillary_data(self, ds):
                 if a in ds.variables:
                     existing += 1
                 else:
-                    msgs.append("ancillary var %s does not exist" % a)
+                    msgs.append("ancillary var {} does not exist".format(a))
 
             anc_result.value = (existing, len(ancs))
             anc_result.msgs = msgs
@@ -2715,31 +2717,29 @@ def check_coordinates_and_metadata(self, ds):
                 if var.dimensions != (name,):
                     name_list.append(name)
 
+        non_data_list = [name for name, var in ds.variables.items() if var in self._find_coord_vars(ds) or var in self._find_ancillary_vars(ds)]
+
         for name, var in ds.variables.items():
             if hasattr(var, 'coordinates'):
-                for each in getattr(var, 'coordinates', '').split(' '):
-                    if each in name_list:
-                        non_data_list.append(each)
-
-            if hasattr(var, 'ancillary_variables'):
-                for each in getattr(var, 'ancillary_variables', '').split(' '):
-                    non_data_list.append(each)
+                for coord in getattr(var, 'coordinates', '').split(' '):
+                    if coord in name_list:
+                        non_data_list.append(coord)
 
-        data_list = [each for each in name_list if each not in non_data_list]
+        data_list = [data for data in name_list if data not in non_data_list]
 
-        for each in data_list:
-            if getattr(ds.variables[each], 'coordinates', ''):
+        for data_entry in data_list:
+            if getattr(ds.variables[data_entry], 'coordinates', ''):
                 result = Result(BaseCheck.MEDIUM,
                                 True,
-                                ('§9.5 Discrete Geometry', each, 'check_coordinates'),
+                                ('§9.5 Discrete Geometry', data_entry, 'check_coordinates'),
                                 reasoning)
                 ret_val.append(result)
                 reasoning = []
             else:
-                reasoning.append('The variable %s does not have associated coordinates' % each)
+                reasoning.append('The variable {} does not have associated coordinates'.format(data_entry))
                 result = Result(BaseCheck.MEDIUM,
                                 False,
-                                ('§9.5 Discrete Geometry', each, 'check_coordinates'),
+                                ('§9.5 Discrete Geometry', data_entry, 'check_coordinates'),
                                 reasoning)
                 ret_val.append(result)
                 reasoning = []


Test patch: