Repository: opendatacube/datacube-core

Issue:
Very slow to load data when resampling requested in query
### Expected behaviour
Resampling of Landsat to 10m x 10m pixels for comparison with Sentinel 2 would not have a great impact on the speed of `dc.load`.

### Actual behaviour
`dc.load` slowed to the point where it was essentially unusable. e.g. Began extraction prior to a 1 hour meeting, and it hadn't finished loading in the tiny area (all time, all bands) by the time I came back.

### Steps to reproduce the behaviour
`ds = dc.load(product = ls5_nbar_albers, group_by = 'solar_day', **query)`

```
query = {'output_crs': 'EPSG:3577',
         'resolution': (-10, 10),
         'geopolygon': geom
        }
```
Note that the geopolygon is not the problem - same issue encountered when a small bounding box was used instead. Area for extraction is very small (~50 by ~70 pixels in Sentinel2)

### Environment information

* Which ``datacube --version`` are you using?
* What datacube deployment/enviornment are you running against?
DEA on the VDI, loaded with 
```
module use /g/data/v10/public/modules/modulefiles/
module load agdc-py3-prod
```



Discussion:
This TODO: clearly needs fixing:

https://github.com/opendatacube/datacube-core/blob/36b580f1babfee203aec129e8a4651225ae82d75/datacube/storage/storage.py#L350-#L361
Some versions of rasterio had bugs when reprojecting from multi-band file (stacked netcdf in this case). It was fixed in the version currently installed in the `agdc-py3-prod` but we still use slower work around

https://github.com/opendatacube/datacube-core/blob/36b580f1babfee203aec129e8a4651225ae82d75/datacube/storage/storage.py#L402
Looks like 1.0a9 if the version where they fixed it:

https://github.com/mapbox/rasterio/commit/dbd389c483b65459bd6114e5db4ab172836336cd
Oh yeah, that looks like a tragic confluence of bugs, and would explain the behaviour I saw on Claire's computer. There was significantly more network traffic than necessary to load the tiny amount of data requested.

Who wants to work on fixing this?

The immediate fix would be to not fall back to `OverrideBandDataSource` if using a new enough `rasterio`.

Resolving patch:
diff --git a/datacube/storage/storage.py b/datacube/storage/storage.py
--- a/datacube/storage/storage.py
+++ b/datacube/storage/storage.py
@@ -55,6 +55,11 @@
                    if str(rasterio.__gdal_version__) >= '1.10.0' else
                    'NETCDF_DIMENSION_')
 
+# rasterio 1.0aX series had warp bug until 1.0a9 So we use workaround code for
+# these versions only, because of silly numbering scheme we can't use simple
+# comparison operators, since '1.0a10' is less than '1.0a9'
+RASTERIO_WARP_BUG = (str(rasterio.__version__) in set('1.0a%d' % i for i in range(9)))
+
 
 def _rasterio_resampling_method(resampling):
     return RESAMPLING_METHODS[resampling.lower()]
@@ -399,12 +404,12 @@ def open(self):
                     override = True
                     crs = self.get_crs()
 
-                # The 1.0 onwards release of rasterio has a bug that means it
+                # The [1.0a1-1.0a8] releases of rasterio had a bug that means it
                 # cannot read multiband data into a numpy array during reprojection
                 # We override it here to force the reading and reprojection into separate steps
-                # TODO: Remove when rasterio bug fixed
+                # TODO: Remove when we no longer care about those versions of rasterio
                 bandnumber = self.get_bandnumber(src)
-                if bandnumber > 1 and str(rasterio.__version__) >= '1.0':
+                if bandnumber > 1 and RASTERIO_WARP_BUG:
                     override = True
 
                 band = rasterio.band(src, bandnumber)


Test patch: