Repository: ioos/compliance-checker

Issue:
CF: Greedy exception catching masks check_missing_data failure
The test for `valid_coordinate_attribute` will always fail as a result of a numpy syntax error causing a ValueError to be thrown and interpreted as a test failure in [cf.py](https://github.com/ioos/compliance-checker/blob/a758e165ba4a08a336a523a8da73dc68264897ee/compliance_checker/cf/cf.py#L2963). Without the try-except block, we get the following traceback:

``` bash
Running Compliance Checker on the dataset from: ../datasets/netcdf/sss_rc201401.v3.0cap.nc
Traceback (most recent call last):
  File "cchecker.py", line 24, in <module>
    main()
  File "cchecker.py", line 21, in main
    args.criteria)
  File "/Users/ochang/podaac/compliance-checker/compliance_checker/runner.py", line 39, in run_checker
    score_groups = cs.run(ds, *checker_names)
  File "/Users/ochang/podaac/compliance-checker/compliance_checker/suite.py", line 87, in run
    vals = list(itertools.chain.from_iterable(map(lambda c: self._run_check(c, dsp), checks)))
  File "/Users/ochang/podaac/compliance-checker/compliance_checker/suite.py", line 87, in <lambda>
    vals = list(itertools.chain.from_iterable(map(lambda c: self._run_check(c, dsp), checks)))
  File "/Users/ochang/podaac/compliance-checker/compliance_checker/suite.py", line 31, in _run_check
    val = check_method(ds)
  File "/Users/ochang/podaac/compliance-checker/compliance_checker/cf/cf.py", line 2955, in check_missing_data
    indices = [i for i, x in enumerate(var[:]) if (var._FillValue == x or '--' == x or 'nan' == x or 'NaN' == x)]
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
```
# Versions

Python 2.7.5, numpy 1.8.1, Mac OSX 10.9.3



Discussion:
Maybe this is dependent on your file, I can't reproduce - the `valid_coordinate_attribute` always passes for me.  I tried it on numpy 1.8.0 and 1.8.1.  I'm also on Python 2.7.6, let me see if I can downgrade and reproduce there.

Can't reproduce on 2.7.5 either.  Can you send/strip out a dataset that exhibits this behavior?  Do you see it on everything (try with one of the files in test-data)?

I sent `sss_rc201401.v3.0cap.nc` to DFoster at asascience.com. I should mention that this is not the original dataset, but one that has had the coordinates attribute manually added in with netCDF4-python. I'm still getting a hang of the cf standards in regards to this, so this very well might be the source of the problem.

The only dataset in `test-data` that triggered the same for loop for me was `ru07...`. But it did not trigger the same exception. Breaking apart the list comprehension, on line 2955, it looks like `type(x) == numpy.ma.core.MaskedConstant` for `ru07`, but `type(x) == numpy.ma.core.MaskedArray` for `sss_rc201401.v3.0cap.nc` 

Running on your dataset (Python 2.7.6, numpy 1.8.1, OSX 10.9.3), I get this, which seems a legimate failure (excuse the python literal output, we need to fix that):

```
var                                    :3:    11/14 :
    sss_cap                            :3:     6/ 9 :
        lat_lon_correct                :3:     0/ 1 :
        valid_coordinate_attibute      :2:     0/ 2 : [u'The coordinate
                                                      attribute is improperly
                                                      defined with the
                                                      coordinate idlat , which
                                                      does not exist', u'The
                                                      coordinate attribute is
                                                      improperly defined with
                                                      the coordinate idlon ,
                                                      which does not exist']
```

I'll try removing that attribute you added and see if that indeed causes the CC to trip in an unexpected way.

That's the message I was getting before removing the try-except block to inspect the Exception. I was trying to understand why idlat and idlon are not valid coordinates since they are variables that satisfy the requirements to be latitude and longitude, respectively.

Ok, poking at what the code is trying to do, I don't think it's handling dimensionality properly.  @DanielJMaher, can you poke?

I'll poke.


Resolving patch:
diff --git a/compliance_checker/cf/cf.py b/compliance_checker/cf/cf.py
--- a/compliance_checker/cf/cf.py
+++ b/compliance_checker/cf/cf.py
@@ -2915,62 +2915,48 @@ def check_missing_data(self, ds):
         ret_val = []
         
         
-        name_list = []
+        name_list = ds.dataset.variables.keys()
+        dim_list = ds.dataset.dimensions.keys()
+
         aux_index_dict = {}
-        index_dict = {}
-        for name, var in ds.dataset.variables.iteritems():
-            name_list.append(name)
+        dim_index_dict = {}
+
+
         for name, var in ds.dataset.variables.iteritems():
             reasoning = []
             valid = False
             aux_valid = False
             
-            
-            for coordinate in getattr(var, 'coordinates', '').split(" "):
-                indices = []
-                if coordinate in name_list and coordinate not in ds.dataset.dimensions:
-                    if hasattr(ds.dataset.variables[coordinate], '_FillValue'):
+            if hasattr(var, 'coordinates'):
+                for coordinate in getattr(var, 'coordinates', '').split(" "):
+                    indices = []
+                    #Finds Auxillary Coordinate Variable
+                    if coordinate in name_list and coordinate not in dim_list:
+                        #Find if the Auxiliary Coordinate Variable is a single dimension non-scalar 
+                        #TODO: Multi-Dimension Auxiliary Coordinate Variable
                         if len(ds.dataset.variables[coordinate][:].shape) == 1 and ds.dataset.variables[coordinate][:].size > 1:
                             indices = [i for i, x in enumerate(ds.dataset.variables[coordinate][:]) if (x == var._FillValue or x =='--' or x =='NaN')]
                         elif ds.dataset.variables[coordinate][:].size == 1:
                             if np.float(ds.dataset.variables[coordinate][:]) in [np.float(ds.dataset.variables[coordinate]._FillValue), '--','NaN']:
-                                index = [1]
+                                indices = []
 
+                        dim_index_dict[name+'-'+coordinate] = indices
                         aux_index_dict[name+'-'+coordinate] = indices
-                        
-                        
-            for coordinate in getattr(var, 'coordinates', '').split(" "):
-                if coordinate in name_list:
-                    if hasattr(ds.dataset.variables[coordinate], '_FillValue'):
+                            
+                    elif coordinate in name_list and coordinate in dim_list:
+                        #Find if the Coordinate Variable is a single dimension non-scalar 
+                        #TODO: Multi-Dimension Coordinate Variable
                         if len(ds.dataset.variables[coordinate][:].shape) == 1 and ds.dataset.variables[coordinate][:].size > 1:
                             indices = [i for i, x in enumerate(ds.dataset.variables[coordinate][:]) if (x == var._FillValue or x =='--' or x =='NaN')]
                         elif ds.dataset.variables[coordinate][:].size == 1:
                             if np.float(ds.dataset.variables[coordinate][:]) in [np.float(ds.dataset.variables[coordinate]._FillValue), '--','NaN']:
-                                index = [1]
-                        index_dict[name+'-'+coordinate] = indices
-                    
-            for coordinate in getattr(var, 'coordinates', '').split(" "):
-                if hasattr(var, 'coordinates') and hasattr(var, '_FillValue'):
-                    try:
-                        if len(ds.dataset.variables[coordinate][:].shape) == 1 and ds.dataset.variables[coordinate][:].size > 1:
-                            indices = [i for i, x in enumerate(var[:]) if (var._FillValue == x or '--' == x or 'nan' == x or 'NaN' == x)]
-                        elif ds.dataset.variables[coordinate][:].size == 1:
-                            index = [i for i, x in enumerate(var[:]) if np.float(x) in [np.float(var._FillValue), '--','NaN']]
-                        result = Result(BaseCheck.MEDIUM,                            \
-                            True,                                       \
-                            ('var', name, 'valid_coordinate_attibute'), \
-                            [])
-                        ret_val.append(result)
-                    except:
-                        result = Result(BaseCheck.MEDIUM,                            \
-                            False,                                       \
-                            ('var', name, 'valid_coordinate_attibute'), \
-                            ["The coordinate attribute is improperly defined with the coordinate %s , which does not exist"%coordinate])
-                        ret_val.append(result)
+                                indices = []
+                        dim_index_dict[name+'-'+coordinate] = indices
+                    else:
+                        dim_index_dict[name+'-'+coordinate] = []
+    
 
-                index_dict[name+'-'+name] = indices
-                
-                
+            #Check to see that all coordinate variable mising data locations are the same
             aux_index_list = []
             for each in aux_index_dict:
                 aux_index_list.append(aux_index_dict[each])
@@ -2979,24 +2965,38 @@ def check_missing_data(self, ds):
             else: 
                 aux_valid = True
             
-            
-            index_list = []
-            for each in index_dict:
-                index_list.append(index_dict[each])  
-            if index_list != [] and aux_index_list != []:
-                valid = all(x == aux_index_list[0] for x in index_list)
+            #Check to see that all auxilliary coordinate variable missing data appears in the coordinate variables
+            dim_index_list = []
+            for each in dim_index_dict:
+                dim_index_list.append(dim_index_dict[each])  
+            if dim_index_list != []:
+                valid = all(x == dim_index_list[0] for x in dim_index_list)
             else:
-                valid = aux_valid
+                valid = True
 
                 
             if aux_valid == False:
                 reasoning.append('The auxillary coordinates do not have the same missing data locations')
             if valid == False:
-                reasoning.append('The dimensions variables and/or data variables do not have the same missing data locations as the auxillary coordinates')
+                reasoning.append('The coordinate variables do not have the same missing data locations as the auxillary coordinates')
             
+
+            #Check to see that all coordinate variable mising data is reflceted in the dataset
+            valid_missing = True
+            count = 0
+
+            if hasattr(var, '_FillValue'):
+                x_indices = np.where(var==var._FillValue)
+                for coordinate in getattr(var,'coordinates').split(" "):
+                    coordinate_ind_list = dim_index_dict[name+'-'+coordinate]
+                    valid_missing = all(each in x_indices[count] for each in coordinate_ind_list)
+                    count = count+1
+                if valid_missing == False:
+                    reasoning.append('The data does not have the same missing data locations as the coordinates')
             
+
             result = Result(BaseCheck.MEDIUM,                            \
-                valid and aux_valid,                                       \
+                valid and aux_valid and valid_missing,                                       \
                 ('var', name, 'missing_data'), \
                 reasoning)
             ret_val.append(result)


Test patch: