Repository: rasterio/rasterio

Issue:
Getting datasets values at a point
Here's how you get raster values at a point `x, y` today:

``` python
>>> x = (src.bounds.left + src.bounds.right)/2.0
>>> y = (src.bounds.bottom + src.bounds.top)/2.0
>>> src.index(x, y)
(359, 396)
>>> def tiny_window(dataset, x, y):
...     r, c = dataset.index(x, y)
...     return ((r, r+1), (c, c+1))
...
>>> data = src.read(window=tiny_window(src, x, y))
>>> list(data[:,0,0])
[28, 29, 27]
```

That's not so arduous, but there's no handling of edge/corner cases (`x, y` outside the raster bounds, etc). It might be nice to offer programmers something like

``` python
>>> data = src.read(xy=(x, y))
>>> list(data[:,0,0])
[28, 29, 27]
```

or 

``` python
>>> src.point(x, y)
[28, 29, 27]
```

Thoughts?

/cc @jlivni



Discussion:
Either seem useful abstractions to me -- I can think of a few use cases where you'd want to query at a point (elevation is the common example).

I'm not familiar enough w/rasterio to know which is more in line with the api (for example not sure if operations other than point() return lists vs numpy arrays). The raster I'm looking at has O(hundreds) of bands; I assume lists are always better for this case)

This seems like it would have applicability beyond a single point to allow sampling the raster at a given series of locations, but I think we may want to separate calculation of pixel index from coordinate, and just pass in pixel index to `read()` or similar custom function.  I think coordinate to index could be a separate utility function (in keeping with `window` parameter needing to be in indices rather than coordinates), and would have usefulness elsewhere.

What about:

```
sample_coords = [(x1, y1) ... (xn, yn)]
sample_indices = some_util_func_to_calculate_index_from_coord_for_raster(sample_coords)
# sample_indices: [[row1, col1] ... [rown, colln]]
# could optionally add a 3rd index to indicate band, if absent will sample across all bands

src.sample(sample_indices)
# would return (as numpy ndarrays):
[
    [val1_bnd1 .. val1_bndn],    # value at [row1, col1]
    ...
    [valn_bnd1 .. valn_bndn]     # value at [rown, coln]
]
```

If indices are outside the domain of the raster, we could return empty arrays, entirely masked arrays, throw an exception, etc.

For us, we're far more likely to sample > 1 locations.

Thing is, GDAL supports fast read of rectangular regions (including 1x1 regions) but isn't optimized for reads along, eg, curves. For more than one point, you'd want to read a larger array and then analyze that. But maybe a `sample` method (feels like the perfect name) would abstract over that.

I was thinking of discrete locations (e.g., weather stations), not points related along some geometry, so I think the 1x1 region internally is what we'd want.  I was thinking `sample` or the like would loop over each 1x1 region to build up the returned array, so that the caller didn't have to do that.

[`gdalallocationinfo`](http://www.gdal.org/gdallocationinfo.html) looks similar but not as focused and awesome as what you are proposing here.

@brendan-ward I've looked for prior art in and around Numpy. Consider [numpy.random.choice](http://docs.scipy.org/doc/numpy/reference/generated/numpy.random.choice.html#numpy.random.choice): it returns a 1-D array of samples. I think I'd rather our sample() was a generator: this would let you conditionally break out of sampling and enable some advanced async use cases. The base use case would be:

```
for value in dataset.sample(index_iterable):
    print value
```

I think a helper method to get the index iterable would be nice, too, it would just map the dataset's index() method of a sequence of x, y pairs.

@sgillies the generator approach sounds like a great idea.

How are you going to deal with bands?  Are those handled in `index_iterable`?  Or would you be running `sample` on a single band at a time?

Optional `bidx` argument as in `dataset.read()`?

so it would be this?

```
for value in dataset.sample(index_iterable, bidx=0):
    print value
```

Sounds good.

> Thing is, GDAL supports fast read of rectangular regions (including 1x1 regions) but isn't optimized for reads along, eg, curves. For more than one point, you'd want to read a larger array and then analyze that.

Just a note: I do a lot of reads and writes along curves using a rasterizer which yields for every pixel (or scanline for polygons). In my experience GDAL's block cache does a really good job of handling single pixel reads efficiently.

So far I have not found any reason to optimize beyond adjusting `GDAL_CACHEMAX` and processing the features in windows which match the capacity of the block cache.


Resolving patch:
diff --git a/rasterio/rio/main.py b/rasterio/rio/main.py
--- a/rasterio/rio/main.py
+++ b/rasterio/rio/main.py
@@ -6,3 +6,4 @@
 from rasterio.rio.info import env, info
 from rasterio.rio.merge import merge
 from rasterio.rio.rio import bounds, insp, transform
+from rasterio.rio.sample import sample
diff --git a/rasterio/rio/sample.py b/rasterio/rio/sample.py
new file mode 100644
--- /dev/null
+++ b/rasterio/rio/sample.py
@@ -0,0 +1,94 @@
+import json
+import logging
+import sys
+import warnings
+
+import click
+
+import rasterio
+from rasterio.rio.cli import cli
+
+
+warnings.simplefilter('default')
+
+
+@cli.command(short_help="Sample a dataset.")
+@click.argument('files', nargs=-1, required=True, metavar='FILE "[x, y]"')
+@click.option('--bidx', default=None, help="Indexes of input file bands.")
+@click.pass_context
+def sample(ctx, files, bidx):
+    """Sample a dataset at one or more points
+
+    Sampling points (x, y) encoded as JSON arrays, in the coordinate
+    reference system of the dataset, are read from the second
+    positional argument or stdin. Values of the dataset's bands
+    are also encoded as JSON arrays and are written to stdout.
+
+    Example:
+
+    \b
+        $ cat << EOF | rio sample tests/data/RGB.byte.tif
+        > [220650, 2719200]
+        > [219650, 2718200]
+        > EOF
+        [28, 29, 27]
+        [25, 29, 19]
+
+    By default, rio-sample will sample all bands. Optionally, bands
+    may be specified using a simple syntax:
+
+      --bidx N samples the Nth band (first band is 1).
+
+      --bidx M,N,0 samples bands M, N, and O.
+
+      --bidx M..O samples bands M-O, inclusive.
+
+      --bidx ..N samples all bands up to and including N.
+
+      --bidx N.. samples all bands from N to the end.
+
+    Example:
+
+    \b
+        $ cat << EOF | rio sample tests/data/RGB.byte.tif --bidx ..2
+        > [220650, 2719200]
+        > [219650, 2718200]
+        > EOF
+        [28, 29]
+        [25, 29]
+
+    """
+    verbosity = (ctx.obj and ctx.obj.get('verbosity')) or 1
+    logger = logging.getLogger('rio')
+
+    files = list(files)
+    source = files.pop(0)
+    input = files.pop(0) if files else '-'
+
+    # Handle the case of file, stream, or string input.
+    try:
+        points = click.open_file(input).readlines()
+    except IOError:
+        points = [input]
+
+    try:
+        with rasterio.drivers(CPL_DEBUG=verbosity>2):
+            with rasterio.open(source) as src:
+                if bidx is None:
+                    indexes = src.indexes
+                elif '..' in bidx:
+                    start, stop = map(
+                        lambda x: int(x) if x else None, bidx.split('..'))
+                    if start is None:
+                        start = 1
+                    indexes = src.indexes[slice(start-1, stop)]
+                else:
+                    indexes = list(map(int, bidx.split(',')))
+                for vals in src.sample(
+                            (json.loads(line) for line in points),
+                            indexes=indexes):
+                    click.echo(json.dumps(vals.tolist()))
+        sys.exit(0)
+    except Exception:
+        logger.exception("Failed. Exception caught")
+        sys.exit(1)


Test patch:
diff --git a/tests/test_rio_sample.py b/tests/test_rio_sample.py
new file mode 100644
--- /dev/null
+++ b/tests/test_rio_sample.py
@@ -0,0 +1,81 @@
+import logging
+import sys
+
+import click
+from click.testing import CliRunner
+
+import rasterio
+from rasterio.rio import sample
+
+
+logging.basicConfig(stream=sys.stderr, level=logging.DEBUG)
+
+
+def test_sample_err():
+    runner = CliRunner()
+    result = runner.invoke(
+        sample.sample,
+        ['bogus.tif'],
+        "[220650.0, 2719200.0]")
+    assert result.exit_code == 1
+
+
+def test_sample_stdin():
+    runner = CliRunner()
+    result = runner.invoke(
+        sample.sample,
+        ['tests/data/RGB.byte.tif'],
+        "[220650.0, 2719200.0]\n[220650.0, 2719200.0]",
+        catch_exceptions=False)
+    assert result.exit_code == 0
+    assert result.output.strip() == '[28, 29, 27]\n[28, 29, 27]'
+
+
+def test_sample_arg():
+    runner = CliRunner()
+    result = runner.invoke(
+        sample.sample,
+        ['tests/data/RGB.byte.tif', "[220650.0, 2719200.0]"],
+        catch_exceptions=False)
+    assert result.exit_code == 0
+    assert result.output.strip() == '[28, 29, 27]'
+
+
+def test_sample_bidx():
+    runner = CliRunner()
+    result = runner.invoke(
+        sample.sample,
+        ['tests/data/RGB.byte.tif', '--bidx', '1,2', "[220650.0, 2719200.0]"],
+        catch_exceptions=False)
+    assert result.exit_code == 0
+    assert result.output.strip() == '[28, 29]'
+
+
+def test_sample_bidx2():
+    runner = CliRunner()
+    result = runner.invoke(
+        sample.sample,
+        ['tests/data/RGB.byte.tif', '--bidx', '1..2', "[220650.0, 2719200.0]"],
+        catch_exceptions=False)
+    assert result.exit_code == 0
+    assert result.output.strip() == '[28, 29]'
+
+
+def test_sample_bidx3():
+    runner = CliRunner()
+    result = runner.invoke(
+        sample.sample,
+        ['tests/data/RGB.byte.tif', '--bidx', '..2', "[220650.0, 2719200.0]"],
+        catch_exceptions=False)
+    assert result.exit_code == 0
+    assert result.output.strip() == '[28, 29]'
+
+
+def test_sample_bidx4():
+    runner = CliRunner()
+    result = runner.invoke(
+        sample.sample,
+        ['tests/data/RGB.byte.tif', '--bidx', '3', "[220650.0, 2719200.0]"],
+        catch_exceptions=False)
+    assert result.exit_code == 0
+    assert result.output.strip() == '[27]'
diff --git a/tests/test_sampling.py b/tests/test_sampling.py
new file mode 100644
--- /dev/null
+++ b/tests/test_sampling.py
@@ -0,0 +1,17 @@
+import rasterio
+
+
+def test_sampling():
+    with rasterio.open('tests/data/RGB.byte.tif') as src:
+        data = next(src.sample([(220650.0, 2719200.0)]))
+        assert list(data) == [28, 29, 27]
+
+def test_sampling_beyond_bounds():
+    with rasterio.open('tests/data/RGB.byte.tif') as src:
+        data = next(src.sample([(-10, 2719200.0)]))
+        assert list(data) == [0, 0, 0]
+
+def test_sampling_indexes():
+    with rasterio.open('tests/data/RGB.byte.tif') as src:
+        data = next(src.sample([(220650.0, 2719200.0)], indexes=[2]))
+        assert list(data) == [29]