Repository: Ouranosinc/xclim

Issue:
Enable missing checks for frequency analysis
### Description
Missing value checks applies to time series, but freq analysis does not return a time series. Rather, the time series is an intermediate result. 

We should create a special Indicator subclass able to support missing value checks within the freq analysis computation. 


Discussion:

Patch:
diff --git a/HISTORY.rst b/HISTORY.rst
index f77aebe8..21f13ce6 100644
--- a/HISTORY.rst
+++ b/HISTORY.rst
@@ -6,6 +6,8 @@ History
 ------
 * New `xclim.sdba` module for statistical downscaling and bias-adjustment of climate data.
 * Added `convert_calendar` and `interp_calendar` to help in the conversion between calendars.
+* Add `at_least_n_valid` function, indentifying null calculations based on minimum threshold.
+* Add support for `freq=None` in missing calculations.
 
 0.16.0 (2020-04-23)
 -------------------
diff --git a/docs/checks.rst b/docs/checks.rst
new file mode 100644
index 00000000..4c5797ce
--- /dev/null
+++ b/docs/checks.rst
@@ -0,0 +1,21 @@
+=============
+Health checks
+=============
+The :class:`Indicator` class performs a number of sanity checks on inputs to make sure valid data is fed to indices
+computations and output values are properly masked in case input values are missing or invalid.
+
+
+Missing values
+==============
+The following functions can be used to mask missing values in :class:`Indicator` outputs.
+
+.. autofunction:: xclim.core.checks.missing_any
+   :noindex:
+
+.. autofunction:: xclim.core.checks.missing_pct
+   :noindex:
+.. autofunction:: xclim.core.checks.missing_wmo
+   :noindex:
+
+.. autofunction:: xclim.core.checks.at_least_n_valid
+   :noindex:
diff --git a/docs/conf.py b/docs/conf.py
index 478aa9f5..2d951bc5 100755
--- a/docs/conf.py
+++ b/docs/conf.py
@@ -92,7 +92,7 @@ napoleon_use_rtype = False
 napoleon_use_param = False
 napoleon_use_ivar = True
 
-nbsphinx_execute = "always"
+nbsphinx_execute = "never"
 nbsphinx_prolog = r"""
 {% set docname = env.doc2path(env.docname, base=None) %}
 
diff --git a/docs/index.rst b/docs/index.rst
index f2af1f84..1b0dfa69 100644
--- a/docs/index.rst
+++ b/docs/index.rst
@@ -19,6 +19,7 @@ xclim currently provides over 50 indices related to mean, minimum and maximum da
    notebooks/usage
    notebooks/index
    indicators
+   checks
    notebooks/units
    internationalization
    icclim
diff --git a/docs/indicators.rst b/docs/indicators.rst
index c6867899..2af31851 100644
--- a/docs/indicators.rst
+++ b/docs/indicators.rst
@@ -6,7 +6,7 @@ Climate indicators
 found in the :mod:`xclim.indices` library, but also run a number of health checks on input data
 and assign attributes to the output arrays. So for example, if there are missing values in
 a time series, indices won't notice, but indicators will return NaNs for periods with missing
-values. Indicators also check that the input data has the expected frequency (e.g. daily) and that
+values (see :ref:`Missing values`). Indicators also check that the input data has the expected frequency (e.g. daily) and that
 it is indeed the expected variable (e.g. a precipitation flux). The output is assigned attributes
 that conform as much as possible with the `CF-Convention`_.
 
diff --git a/xclim/core/checks.py b/xclim/core/checks.py
index d478910c..7e58e4f1 100644
--- a/xclim/core/checks.py
+++ b/xclim/core/checks.py
@@ -1,7 +1,7 @@
 # -*- coding: utf-8 -*-
 """
-Health checks submodule
-=======================
+Health checks
+=============
 
 Functions performing basic health checks on xarray.DataArrays.
 """
@@ -303,6 +303,13 @@ class MissingPct(MissingBase):
         return n / count >= tolerance
 
 
+class AtLeastNValid(MissingBase):
+    def is_missing(self, null, count, n=20):
+        """The result of a reduction operation is considered missing if less than `n` values are valid."""
+        nvalid = null.count(dim="time") - null.sum(dim="time")
+        return nvalid < n
+
+
 def missing_any(da, freq, **indexer):
     r"""Return whether there are missing days in the array.
 
@@ -330,6 +337,7 @@ def missing_wmo(da, freq, nm=11, nc=5, **indexer):
 
     The World Meteorological Organisation recommends that where monthly means are computed from daily values,
     it should considered missing if either of these two criteria are met:
+
       – observations are missing for 11 or more days during the month;
       – observations are missing for a period of 5 or more consecutive days during the month.
 
@@ -383,3 +391,27 @@ def missing_pct(da, freq, tolerance, **indexer):
       A boolean array set to True if period has missing values.
     """
     return MissingPct(da, freq, **indexer)(tolerance=tolerance)
+
+
+def at_least_n_valid(da, freq, n, **indexer):
+    r"""Return whether there are at least a given number of valid values.
+
+        Parameters
+        ----------
+        da : DataArray
+          Input array at daily frequency.
+        freq : str
+          Resampling frequency.
+        n : int
+          Minimum of valid values required.
+        **indexer : {dim: indexer, }, optional
+          Time attribute and values over which to subset the array. For example, use season='DJF' to select winter
+          values, month=1 to select January, or month=[6,7,8] to select summer months. If not indexer is given,
+          all values are considered.
+
+        Returns
+        -------
+        out : DataArray
+          A boolean array set to True if period has missing values.
+        """
+    return AtLeastNValid(da, freq, **indexer)(n=n)


Test patch:
diff --git a/tests/test_checks.py b/tests/test_checks.py
index 4cad72ef..771f15fa 100644
--- a/tests/test_checks.py
+++ b/tests/test_checks.py
@@ -180,3 +180,13 @@ class TestMissingPct:
         out = checks.missing_pct(ts, freq="MS", tolerance=0.1)
         assert not out[0]
         assert out[1]
+
+
+class TestAtLeastNValid:
+    def test_at_least_n_valid(self, tas_series):
+        a = np.arange(360.0)
+        a[5:10] = np.nan  # Number of missing values under acceptable limit in a month
+        a[40:55] = np.nan  # Too many missing values
+        ts = tas_series(a)
+        out = checks.at_least_n_valid(ts, freq="MS", n=20)
+        np.testing.assert_array_equal(out[:2], [False, True])