Repository: rasterio/rasterio

Issue:
Boundless reads with fill_value co-geotiff performance
## Expected behavior and actual behavior.

I expect getting a fast response when retrieving data from a raster with boundless=true and fill_value=-9999 and a bbox/window that is much larger than the source cloud optimized geotiff.

Note: "We have a more general fix for the problem of filling the empty regions of boundless reads (#1471)." works, but looks to be slow with large boundless reads in overviews.

## Steps to reproduce the problem.

```
with rasterio.open(cloud_optimized_geotiff_path, 'r') as src:
    data = src.read(window=window, out_shape=(256, 256), boundless=True, fill_value=-9999)
    # where:
    # window = large window, for example the whole world 
    # cloud_optimized_geotiff = cloud optimized geotiff that covers a small city somewhere
```
   
Removing the 'fill_value'  parameter from the read call above restores the (normal) performance, however the default 0 fill_value will be used. 

## Operating system

Ubuntu 18.04

## Rasterio version and provenance

Rasterio 1.0.6 via pip, python 2.7.


Discussion:
Note: I'm using the following workaround now that seems to work:

```
                # Set everything outside raster dimensions to fill_value
                # first lower bounds (< 0)
                if window.col_off < 0:
                    # Calculate pixel offset in x direction
                    x_offset = (-window.col_off / window.width) * out_shape[1]
                    raster_data[:, 0:int(x_offset)] = fill_value

                if window.row_off < 0:
                    # Calculate pixel offset in y direction
                    y_offset = (-window.row_off / window.height) * out_shape[0]
                    raster_data[0:int(y_offset), :] = fill_value

                # Outside upper bounds (width/height)
                window_right = window.col_off + window.width

                # Window right coordinate > raster width
                if window_right > src.width:
                    # Fill everything between raster width - window_right
                    # with fill_value
                    x_offset = (
                        (window_right - src.width) / window.width) *\
                        out_shape[1]
                    raster_data[:, out_shape[1] - int(x_offset):] = fill_value

                window_top = window.row_off + window.height

                # Window top coordinate > raster height
                if window_top > src.height:
                    # Fill everything between raster height - window_top
                    # with fill_value
                    y_offset = (
                        (window_top - src.height) / window.height) *\
                        out_shape[1]
                    raster_data[out_shape[0] - int(y_offset):, :] = fill_value
```

@jpprins1 thanks for the report! I agree that the general solution is slow and am happy to see that you have a work around. I think that the good fixes are upstream in GDAL: making sure that VRT mask performance is as good as it can be, and perhaps enhancing VRTs so that they can provide background fill natively.

Resolving patch:
diff --git a/rasterio/__init__.py b/rasterio/__init__.py
--- a/rasterio/__init__.py
+++ b/rasterio/__init__.py
@@ -43,7 +43,7 @@ def emit(self, record):
 
 
 __all__ = ['band', 'open', 'pad', 'Env']
-__version__ = "1.0.6"
+__version__ = "1.0.7"
 __gdal_version__ = gdal_version()
 
 # Rasterio attaches NullHandler to the 'rasterio' logger and its
diff --git a/rasterio/vrt.py b/rasterio/vrt.py
--- a/rasterio/vrt.py
+++ b/rasterio/vrt.py
@@ -69,8 +69,24 @@ def close(self):
         self.stop()
 
 
-def _boundless_vrt_doc(src_dataset, nodata=None, hidenodata=False, width=None, height=None, transform=None):
-    """Make a VRT XML document."""
+def _boundless_vrt_doc(
+        src_dataset, nodata=None, background=None, hidenodata=False,
+        width=None, height=None, transform=None):
+    """Make a VRT XML document.
+
+    Parameters
+    ----------
+    src_dataset : Dataset
+        The dataset to wrap.
+    background : Dataset, optional
+        A dataset that provides the optional VRT background. NB: this dataset
+        must have the same number of bands as the src_dataset.
+
+    Returns
+    -------
+    bytes
+        An ascii-encoded string (an ElementTree detail)
+    """
 
     nodata = nodata or src_dataset.nodata
     width = width or src_dataset.width
@@ -101,12 +117,34 @@ def _boundless_vrt_doc(src_dataset, nodata=None, hidenodata=False, width=None, h
         colorinterp = ET.SubElement(vrtrasterband, 'ColorInterp')
         colorinterp.text = ci.name.capitalize()
 
-        simplesource = ET.SubElement(vrtrasterband, 'SimpleSource')
+        if background is not None:
+            simplesource = ET.SubElement(vrtrasterband, 'SimpleSource')
+            sourcefilename = ET.SubElement(simplesource, 'SourceFilename')
+            sourcefilename.attrib['relativeToVRT'] = "0"
+            sourcefilename.text = vsi_path(parse_path(background.name))
+            sourceband = ET.SubElement(simplesource, 'SourceBand')
+            sourceband.text = str(bidx)
+            sourceproperties = ET.SubElement(simplesource, 'SourceProperties')
+            sourceproperties.attrib['RasterXSize'] = str(width)
+            sourceproperties.attrib['RasterYSize'] = str(height)
+            sourceproperties.attrib['dataType'] = _gdal_typename(dtype)
+            sourceproperties.attrib['BlockYSize'] = str(block_shape[0])
+            sourceproperties.attrib['BlockXSize'] = str(block_shape[1])
+            srcrect = ET.SubElement(simplesource, 'SrcRect')
+            srcrect.attrib['xOff'] = '0'
+            srcrect.attrib['yOff'] = '0'
+            srcrect.attrib['xSize'] = str(background.width)
+            srcrect.attrib['ySize'] = str(background.height)
+            dstrect = ET.SubElement(simplesource, 'DstRect')
+            dstrect.attrib['xOff'] = '0'
+            dstrect.attrib['yOff'] = '0'
+            dstrect.attrib['xSize'] = str(width)
+            dstrect.attrib['ySize'] = str(height)
 
+        simplesource = ET.SubElement(vrtrasterband, 'SimpleSource')
         sourcefilename = ET.SubElement(simplesource, 'SourceFilename')
         sourcefilename.attrib['relativeToVRT'] = "0"
         sourcefilename.text = vsi_path(parse_path(src_dataset.name))
-
         sourceband = ET.SubElement(simplesource, 'SourceBand')
         sourceband.text = str(bidx)
         sourceproperties = ET.SubElement(simplesource, 'SourceProperties')


Test patch: