Repository: rasterio/rasterio

Issue:
Fill and write mask when masked array is passed to write()
The first change was proposed in https://rasterio.groups.io/g/main/message/891 and I proposed taking it a little further. Generalizing write() to do the reasonably expected thing when passed a masked array seems like it could simplify documentation and prevent bugs due to users not calling write_mask. This might require the addition of a `masked=True` keyword argument so that users can disable the writing of a mask.


Discussion:
> This might require the addition of a `masked=True` keyword argument so that users can disable the writing of a mask.

👍 
@vincentsarago I'll have to think about this a little more. Filling makes a lot of sense. Writing a mask by default is more complicated. What if the format doesn't allow a mask? What if the dataset has a nodata value? Etc.
@sgillies I totally agree, that's why having the user to set `masked=True` makes sense.

in rio-cogeo we also force the user to specifically says they want to use internal mask https://github.com/cogeotiff/rio-cogeo/blob/master/rio_cogeo/cogeo.py#L295-L298

Question: what happens when a raster has an alpha band? 

It's not a trivial thing to do, I think he we end up with too many `if/else` tests we shouldn't do it! 

Regarding 

> Generalizing write() to do the reasonably expected thing when passed a masked array ...

I'd like to add another reasonable behavior where only non-masked values are written to the destination raster.
This option is useful for cases where one wants to write specific information (e.g. segmentation information) to the raster data. Currently, there is no possibility (afaik) to do this directly with rasterio.

Given a masked array 

```
import numpy.ma as ma
masked_array = ma.masked_array(...)
```

one has to use a workaround like the following (since we can not read and write to a raster object at the same time)
```
import os
from shutil import copyfile
with rasterio.open('path/to/input.tif') as src:
	with rasterio.open('path/to/tmp.tif', 'w', **src.profile) as dst:

	src_data = src.read()
	src_data.data[~masked_array.mask] = masked_array.data[~masked_array.mask]
	dst.write(src_data)

os.remove('path/to/input.tif')
copyfile('path/to/tmp.tif', 'path/to/input.tif')
```

This produces a substantial overhead in terms of memory requirements. Reflecting the masked values while writing the (masked) array would simplify the workaround to something like  
```
with rasterio.open('path/to/tmp.tif', 'w+') as dst:
	dst.write(src_data, write_non_masked=True)
```

Do you think there is a chance to extend rasterio with support for mask values as described above?

Edit: I can create a new issue, if you feel this should be moved to a different place.
@SBCV GDAL's data I/O uses unmasked buffers and can't help us (see https://gdal.org/api/gdalrasterband_cpp.html#_CPPv4N14GDALRasterBand8RasterIOE10GDALRWFlagiiiiPvii12GDALDataType8GSpacing8GSpacingP20GDALRasterIOExtraArg). I don't see a way around the requirement to read the source data and merge it with your masked data. Using windowed reads and writes should help keep the memory needs down.
@sgillies Too bad that GDAL has this limitation. Nevertheless: thank you for having a look and sharing this link!


Resolving patch:
diff --git a/rasterio/__init__.py b/rasterio/__init__.py
--- a/rasterio/__init__.py
+++ b/rasterio/__init__.py
@@ -52,7 +52,7 @@ class FilePath:
         have_vsi_plugin = False
 
 __all__ = ['band', 'open', 'pad', 'Env', 'CRS']
-__version__ = "1.3a4"
+__version__ = "1.3b1"
 __gdal_version__ = gdal_version()
 __proj_version__ = ".".join([str(version) for version in get_proj_version()])
 __geos_version__ = ".".join([str(version) for version in get_geos_version()])


Test patch:
diff --git a/tests/test_write.py b/tests/test_write.py
--- a/tests/test_write.py
+++ b/tests/test_write.py
@@ -439,3 +439,71 @@ def test_write_cog(tmpdir, path_rgb_byte_tif):
         profile.update(driver="COG", extent=src.bounds, resampling=Resampling.bilinear)
         with rasterio.open(str(tmpdir.join("test.tif")), "w", **profile) as cog:
             cog.write(src.read())
+
+
+def test_write_masked(tmp_path):
+    """Verify that masked arrays are filled when written."""
+    data = np.ma.masked_less_equal(np.array([[0, 1, 2]], dtype="uint8"), 1)
+    data.fill_value = 3
+
+    with rasterio.open(
+        tmp_path / "test.tif",
+        "w",
+        driver="GTiff",
+        count=1,
+        width=3,
+        height=1,
+        dtype="uint8",
+    ) as dst:
+        dst.write(data, indexes=1)
+
+    # Expect to see the masked array's fill_value in the first two
+    # pixels of the raster.
+    with rasterio.open(tmp_path / "test.tif") as src:
+        arr = src.read()
+        assert list(arr.flatten()) == [3, 3, 2]
+
+
+def test_write_masked_nodata(tmp_path):
+    """Verify that masked arrays are filled with nodata when written."""
+    data = np.ma.masked_less_equal(np.array([[0, 1, 2]], dtype="uint8"), 1)
+
+    with rasterio.open(
+        tmp_path / "test.tif",
+        "w",
+        driver="GTiff",
+        count=1,
+        width=3,
+        height=1,
+        dtype="uint8",
+        nodata=0,
+    ) as dst:
+        dst.write(data, indexes=1)
+
+    # Expect to see the dataset's nodata value in the first two
+    # pixels of the raster.
+    with rasterio.open(tmp_path / "test.tif") as src:
+        arr = src.read()
+        assert list(arr.flatten()) == [0, 0, 2]
+
+
+def test_write_masked_mask(tmp_path):
+    """Verify that a mask is written when we write a masked array."""
+    data = np.ma.masked_less_equal(np.array([[0, 1, 2]], dtype="uint8"), 1)
+
+    with rasterio.open(
+        tmp_path / "test.tif",
+        "w",
+        driver="GTiff",
+        count=1,
+        width=3,
+        height=1,
+        dtype="uint8",
+    ) as dst:
+        dst.write(data, indexes=1, masked=True)
+
+    # Expect to see the dataset's nodata value in the first two
+    # pixels of the raster.
+    with rasterio.open(tmp_path / "test.tif") as src:
+        arr = src.read(masked=True)
+        assert list(arr.flatten()) == [np.ma.masked, np.ma.masked, 2]