Repository: rasterio/rasterio

Issue:
Statistics exception and GDAL stderr output
### Discussed in https://github.com/rasterio/rasterio/discussions/2757

<div type='discussions-op-text'>

<sup>Originally posted by **tschaub** February  6, 2023</sup>
Hello.  I'm making use of the `statistics` method on raster bands that was added in #2441.  I notice that when a band includes all nodata values, a `rasterio._err.CPLE_AppDefinedError` is raised.  In addition, an error message is written to `stderr`.

Using `all-nodata.tif` from the attached [all-nodata.tif.zip](https://github.com/rasterio/rasterio/files/10670116/all-nodata.tif.zip), the following script demonstrates the issue:
```py
# using v1.3.4
import rasterio

with rasterio.open("all-nodata.tif") as src:
    stats = src.statistics(1)
```

And this is the output:
```
ERROR 1: all-nodata.tif, band 1: Failed to compute statistics, no valid pixels found in sampling.
Traceback (most recent call last):
  File "/path/to/file.py", line 4, in <module>
    stats = src.statistics(1)
            ^^^^^^^^^^^^^^^^^
  File "rasterio/_io.pyx", line 1111, in rasterio._io.DatasetReaderBase.statistics
  File "rasterio/_io.pyx", line 1107, in rasterio._io.DatasetReaderBase.statistics
  File "rasterio/_err.pyx", line 195, in rasterio._err.exc_wrap_int
rasterio._err.CPLE_AppDefinedError: all-nodata.tif, band 1: Failed to compute statistics, no valid pixels found in sampling.
```

I'm curious if the recommendation here is to catch all `Exception`s or if I should use a more specific class.  It looks like exceptions based on `CPLE_BaseError` are [not intended for API users](https://github.com/rasterio/rasterio/blob/1.3.4/rasterio/_err.pyx#L21-L22).

When I do catch the exception from GDAL, I still see the `ERROR 1: all-nodata.tif, band 1: Failed to compute statistics, no valid pixels found in sampling.` logged to `stderr`.  Poking around a bit, I thought maybe pushing the `CPLQuietErrorHandler ` was supposed to silence this output.

So my questions are
 * Should I catch `rasterio._err.CPLE_AppDefinedError` or `Exception` in the above situation?
 * Is there a recommended way to silence the error message from GDAL (I've seen `with rasterio.drivers(CPL_LOG="/dev/null")` [suggested](https://github.com/rasterio/rasterio/issues/649#issuecomment-208429078) and have tried `logging.getLogger("GDAL").addHandler(NullHandler())` without luck)?


</div>


Discussion:

Resolving patch:
diff --git a/rasterio/errors.py b/rasterio/errors.py
--- a/rasterio/errors.py
+++ b/rasterio/errors.py
@@ -158,3 +158,7 @@ class DatasetIOShapeError(RasterioError):
 
 class WarpOperationError(RasterioError):
     """Raised when a warp operation fails."""
+
+
+class StatisticsError(RasterioError):
+    """Raised when dataset statistics cannot be computed."""
diff --git a/rasterio/transform.py b/rasterio/transform.py
--- a/rasterio/transform.py
+++ b/rasterio/transform.py
@@ -60,13 +60,13 @@ def xy(
         col : int
             Pixel column.
         z : float, optional
-            Height associated with coordinates. Primarily used for RPC based 
-            coordinate transformations. Ignored for affine based 
+            Height associated with coordinates. Primarily used for RPC based
+            coordinate transformations. Ignored for affine based
             transformations. Default: 0.
         offset : str, optional
             Determines if the returned coordinates are for the center of the
             pixel or for a corner.
-        transform_method: TransformMethod, optional 
+        transform_method: TransformMethod, optional
             The coordinate transformation method. Default: `TransformMethod.affine`.
         rpc_options: dict, optional
             Additional arguments passed to GDALCreateRPCTransformer
@@ -103,13 +103,13 @@ def index(
         y : float
             y value in coordinate reference system
         z : float, optional
-            Height associated with coordinates. Primarily used for RPC based 
-            coordinate transformations. Ignored for affine based 
+            Height associated with coordinates. Primarily used for RPC based
+            coordinate transformations. Ignored for affine based
             transformations. Default: 0.
         op : function, optional (default: math.floor)
             Function to convert fractional pixels to whole numbers (floor,
             ceiling, round)
-        transform_method: TransformMethod, optional 
+        transform_method: TransformMethod, optional
             The coordinate transformation method. Default: `TransformMethod.affine`.
         rpc_options: dict, optional
             Additional arguments passed to GDALCreateRPCTransformer
@@ -210,20 +210,20 @@ def xy(transform, rows, cols, zs=None, offset='center', **rpc_options):
     The pixel's center is returned by default, but a corner can be returned
     by setting `offset` to one of `ul, ur, ll, lr`.
 
-    Supports affine, Ground Control Point (GCP), or Rational Polynomial 
+    Supports affine, Ground Control Point (GCP), or Rational Polynomial
     Coefficients (RPC) based coordinate transformations.
 
     Parameters
     ----------
     transform : Affine or sequence of GroundControlPoint or RPC
-        Transform suitable for input to AffineTransformer, GCPTransformer, or RPCTransformer. 
+        Transform suitable for input to AffineTransformer, GCPTransformer, or RPCTransformer.
     rows : list or int
         Pixel rows.
     cols : int or sequence of ints
         Pixel columns.
     zs : list or float, optional
-        Height associated with coordinates. Primarily used for RPC based 
-        coordinate transformations. Ignored for affine based 
+        Height associated with coordinates. Primarily used for RPC based
+        coordinate transformations. Ignored for affine based
         transformations. Default: 0.
     offset : str, optional
         Determines if the returned coordinates are for the center of the
@@ -256,8 +256,8 @@ def rowcol(transform, xs, ys, zs=None, op=math.floor, precision=None, **rpc_opti
     ys : list or float
         y values in coordinate reference system.
     zs : list or float, optional
-        Height associated with coordinates. Primarily used for RPC based 
-        coordinate transformations. Ignored for affine based 
+        Height associated with coordinates. Primarily used for RPC based
+        coordinate transformations. Ignored for affine based
         transformations. Default: 0.
     op : function
         Function to convert fractional pixels to whole numbers (floor, ceiling,
@@ -322,7 +322,7 @@ def close(self):
     @staticmethod
     def _ensure_arr_input(xs, ys, zs=None):
         """Ensure all input coordinates are mapped to array-like objects
-        
+
         Raises
         ------
         TransformError
@@ -343,7 +343,7 @@ def __enter__(self):
     def __exit__(self, *args):
         self.close()
         self._env.close()
-    
+
     def rowcol(self, xs, ys, zs=None, op=math.floor, precision=None):
         """Get rows and cols coordinates given geographic coordinates.
 
@@ -352,8 +352,8 @@ def rowcol(self, xs, ys, zs=None, op=math.floor, precision=None):
         xs, ys : float or list of float
             Geographic coordinates
         zs : float or list of float, optional
-            Height associated with coordinates. Primarily used for RPC based 
-            coordinate transformations. Ignored for affine based 
+            Height associated with coordinates. Primarily used for RPC based
+            coordinate transformations. Ignored for affine based
             transformations. Default: 0.
         op : function, optional (default: math.floor)
             Function to convert fractional pixels to whole numbers (floor,
@@ -380,7 +380,7 @@ def rowcol(self, xs, ys, zs=None, op=math.floor, precision=None):
 
         AS_ARR = True if hasattr(xs, "__iter__") else False
         xs, ys, zs = self._ensure_arr_input(xs, ys, zs=zs)
-        
+
         try:
             new_cols, new_rows = self._transform(
                 xs, ys, zs, transform_direction=TransformDirection.reverse
@@ -402,8 +402,8 @@ def xy(self, rows, cols, zs=None, offset='center'):
         rows, cols : int or list of int
             Image pixel coordinates
         zs : float or list of float, optional
-            Height associated with coordinates. Primarily used for RPC based 
-            coordinate transformations. Ignored for affine based 
+            Height associated with coordinates. Primarily used for RPC based
+            coordinate transformations. Ignored for affine based
             transformations. Default: 0.
         offset : str, optional
             Determines if the returned coordinates are for the center of the
@@ -420,7 +420,7 @@ def xy(self, rows, cols, zs=None, offset='center'):
         """
         AS_ARR = True if hasattr(rows, "__iter__") else False
         rows, cols, zs = self._ensure_arr_input(rows, cols, zs=zs)
-        
+
         if offset == 'center':
             coff, roff = (0.5, 0.5)
         elif offset == 'ul':
@@ -433,7 +433,7 @@ def xy(self, rows, cols, zs=None, offset='center'):
             coff, roff = (1, 1)
         else:
             raise TransformError("Invalid offset")
-        
+
         # shift input coordinates according to offset
         T = IDENTITY.translation(coff, roff)
         offset_rows = []
@@ -454,7 +454,7 @@ def xy(self, rows, cols, zs=None, offset='center'):
                 return (new_xs, new_ys)
         except TypeError:
             raise TransformError("Invalid inputs")
-    
+
     def _transform(self, xs, ys, zs, transform_direction):
         raise NotImplementedError
 
@@ -469,11 +469,11 @@ def __init__(self, affine_transform):
 
     def close(self):
         pass
-    
+
     def _transform(self, xs, ys, zs, transform_direction):
         resxs = []
         resys = []
-        
+
         if transform_direction is TransformDirection.forward:
             transform = self._transformer
         elif transform_direction is TransformDirection.reverse:
@@ -483,9 +483,9 @@ def _transform(self, xs, ys, zs, transform_direction):
             resx, resy = transform * (x, y)
             resxs.append(resx)
             resys.append(resy)
-        
+
         return (resxs, resys)
-    
+
     def __repr__(self):
         return "<AffineTransformer>"
 
@@ -497,7 +497,7 @@ class RPCTransformer(RPCTransformerBase, TransformerBase):
 
     Uses GDALCreateRPCTransformer and GDALRPCTransform for computations. Options
     for GDALCreateRPCTransformer may be passed using `rpc_options`.
-    Ensure that GDAL transformer objects are destroyed by calling `close()` 
+    Ensure that GDAL transformer objects are destroyed by calling `close()`
     method or using context manager interface.
 
     """
@@ -517,7 +517,7 @@ class GCPTransformer(GCPTransformerBase, TransformerBase):
     coordinate transformations.
 
     Uses GDALCreateGCPTransformer and GDALGCPTransform for computations.
-    Ensure that GDAL transformer objects are destroyed by calling `close()` 
+    Ensure that GDAL transformer objects are destroyed by calling `close()`
     method or using context manager interface.
 
     """


Test patch:
diff --git a/tests/data/all-nodata.tif b/tests/data/all-nodata.tif
new file mode 100644
Binary files /dev/null and b/tests/data/all-nodata.tif differ
diff --git a/tests/test_dataset.py b/tests/test_dataset.py
--- a/tests/test_dataset.py
+++ b/tests/test_dataset.py
@@ -2,12 +2,8 @@
 
 
 from pathlib import Path
-try:
-    from unittest.mock import MagicMock
-except ImportError:
-    from mock import MagicMock
+from unittest.mock import MagicMock
 
-from numpy.testing import assert_almost_equal
 import pytest
 
 import rasterio
@@ -72,16 +68,6 @@ def test_dataset_readonly_attributes(path_rgb_byte_tif):
             dataset.crs = "foo"
 
 
-def test_statistics(path_rgb_byte_tif):
-    """Compute, store, and return basic statistics."""
-    with rasterio.open(path_rgb_byte_tif) as dataset:
-        stats = dataset.statistics(1)
-        assert stats.min == 0
-        assert stats.max == 255
-        assert_almost_equal(stats.mean, 29.947726688477)
-        assert_almost_equal(stats.std, 52.340921626611)
-
-
 @pytest.mark.parametrize("blockysize", [1, 2, 3, 7, 61, 62])
 def test_creation_untiled_blockysize(tmp_path, blockysize):
     """Check for fix of gh-2599"""
diff --git a/tests/test_statistics.py b/tests/test_statistics.py
new file mode 100644
--- /dev/null
+++ b/tests/test_statistics.py
@@ -0,0 +1,27 @@
+"""Test of a dataset's statistics method."""
+
+from numpy.testing import assert_almost_equal
+import pytest
+
+import rasterio
+from rasterio.errors import RasterioError
+
+
+def test_statistics(path_rgb_byte_tif):
+    """Compute, store, and return basic statistics."""
+    with rasterio.open(path_rgb_byte_tif) as dataset:
+        stats = dataset.statistics(1)
+        assert stats.min == 0
+        assert stats.max == 255
+        assert_almost_equal(stats.mean, 29.947726688477)
+        assert_almost_equal(stats.std, 52.340921626611)
+
+
+def test_statistics_all_invalid(capsys):
+    """Raise an exception for stats of an invalid dataset."""
+    with rasterio.open("tests/data/all-nodata.tif") as dataset:
+        with pytest.raises(RasterioError):
+            _ = dataset.statistics(1)
+
+    captured = capsys.readouterr()
+    assert "ERROR 1" not in captured.err