Repository: rasterio/rasterio

Issue:
Path handling inconsistency
There is inconsistency amongst the commands with regards to the handling of files using the http, https and s3 protocols. For example:
`rio info s3://landsat-pds.s3.amazonaws.com/L8/139/045/LC81390452014295LGN00/LC81390452014295LGN00_B1.TIF`
works completely fine while;
`rio mask s3://landsat-pds.s3.amazonaws.com/L8/139/045/LC81390452014295LGN00/LC81390452014295LGN00_B1.TIF out.tif`
gives a `FileNotFoundError` with a path of '/home/ubuntu/s3:/landsat-pds.s3.amazonaws.com/L8/139/045/LC81390452014295LGN00/LC81390452014295LGN00_B1.TIF`

The difference is in the click file handling. For single input files the handling in done in the rasterio code 
`file_in_arg = click.argument('INPUT', callback=file_in_handler)`
while the `file_inout_arg` is dealt with in the cligj codebase which doesn't have the file_in_handler callback. 


Discussion:
@sgillies Looks like there are two options on this one, either change `@cligj.files_inout_arg` at https://github.com/mapbox/rasterio/blob/master/rasterio/rio/mask.py#L18 to `@options.file_in_arg` or update cligj to handle s3 files. 

Based on the comment here: https://github.com/mapbox/rasterio/blob/master/rasterio/rio/options.py#L47 looks like the preferred options is to get them moved to cligj?
@eseglem in the run up to 1.0, I think it will be best to take the 1st of the two options. After 1.0 we could refactor and update cligj.
@sgillies Actually, might be an issue. Looks like `options` is for a single file, and `cligj` is for multiple files. I am also looking at several other ones: Stack, Calc, Rasterize, and Merge likely all have the same issue as Mask. Might take a bit more reworking than just switching the decorator. 
Oh, I missed Warp. And looks like some of them are defining it inline. Like Clip, Convert, and Sample rather than importing one. Transform is as well, but looks like single file input. I assume all of these should be consistent, rather than distributed out?
@eseglem maybe this is not a "good first issue" after all! I'm going to remove that tag and assign this one to myself since it's more involved than it looks.


I have some code cleanup / standardization in my branch as a start. And I don't mind continuing to look at it. But ended up more of a rabbit hole than I had time for this afternoon. Just need to read some click
documentation. And didn't want to go too far down a road without verifying it was the right one.

https://github.com/mapbox/rasterio/compare/master...eseglem:issue999 is what I have done so far. Which is mostly standardization so far. No real substance towards the actual issue at hand.
Also, looks like there is basically two styles of commands. Ones that take single input, and ones that take any number of inputs. The ones that take multiple inputs seem to allow the final one to be the output, if there is no output specified using `resolve_inout`. To me this seems a bit counter intuitive, I would think output should be explicit and not allow the last input to be an output. It also means that the input and output file handler has to be the same. Right now it looks like all of them do a single local file as output. Which seems reasonable. But input can be from outside sources like s3. So handling them differently is likely necessary.

So I would propose getting rid of `resolve_inout` as a function and requiring output to be explicit across the board. Unless there is a definite need for that use case, but it seems inconsistent. 
@sgillies , I think this still does not work in `rio convert`.
I cannot do `rio convert <options> <network dataset> <local file>`
My interest is in using this with s3://, but I gave an example using https://

```bash
$ rio --version
1.0.6

# can use rio info
rio info https://s3-us-west-2.amazonaws.com/landsat-pds/L8/004/047/LC80040472015120LGN00/LC80040472015120LGN00_BQA.TIF
{"blockxsize": 512, "blockysize": 512, "bounds": [157485.0, 1960785.0, 387615.0, 2195715.0], "colorinterp": ["grey"], "compress": "deflate", "count": 1, "crs": "EPSG:32620", "descriptions": [null], "driver": "GTiff", "dtype": "uint16", "height": 7831, "indexes": [1], "interleave": "band", "lnglat": [-65.15786398200451, 18.783481293567107], "mask_flags": [["all_valid"]], "nodata": null, "res": [30.0, 30.0], "shape": [7831, 7671], "tiled": true, "transform": [30.0, 0.0, 157485.0, 0.0, -30.0, 2195715.0, 0.0, 0.0, 1.0], "units": [null], "width": 7671}

# rio convert can't access network dataset:
$ rio convert --co compress=LZW https://s3-us-west-2.amazonaws.com/landsat-pds/L8/004/047/LC80040472015120LGN00/LC80040472015120LGN00_BQA.TIF compressed.tif
Traceback (most recent call last):
  File "rasterio/_base.pyx", line 199, in rasterio._base.DatasetBase.__init__
  File "rasterio/_shim.pyx", line 64, in rasterio._shim.open_dataset
  File "rasterio/_err.pyx", line 188, in rasterio._err.exc_wrap_pointer
rasterio._err.CPLE_OpenFailedError: /Users/bu_ra3y/Downloads/https:/s3-us-west-2.amazonaws.com/landsat-pds/L8/004/047/LC80040472015120LGN00/LC80040472015120LGN00_BQA.TIF: No such file or directory


# You fixed this, or at least it does not error on the network file name
$ rio mask https://s3-us-west-2.amazonaws.com/landsat-pds/L8/004/047/LC80040472015120LGN00/LC80040472015120LGN00_BQA.TIF out.tif
No GeoJSON provided, INPUT will be copied to OUTPUT
Traceback (most recent call last):
  File "/Users/bu_ra3y/miniconda3/envs/thoron_sandbox/bin/rio", line 11, in <module>
    sys.exit(main_group())
  File "/Users/bu_ra3y/miniconda3/envs/thoron_sandbox/lib/python3.6/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/Users/bu_ra3y/miniconda3/envs/thoron_sandbox/lib/python3.6/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/Users/bu_ra3y/miniconda3/envs/thoron_sandbox/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/bu_ra3y/miniconda3/envs/thoron_sandbox/lib/python3.6/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/bu_ra3y/miniconda3/envs/thoron_sandbox/lib/python3.6/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/Users/bu_ra3y/miniconda3/envs/thoron_sandbox/lib/python3.6/site-packages/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/Users/bu_ra3y/miniconda3/envs/thoron_sandbox/lib/python3.6/site-packages/rasterio/rio/mask.py", line 77, in mask
    rasterio.shutil.copyfiles(input, output)
  File "/Users/bu_ra3y/miniconda3/envs/thoron_sandbox/lib/python3.6/site-packages/rasterio/env.py", line 402, in wrapper
    return f(*args, **kwds)
  File "rasterio/shutil.pyx", line 156, in rasterio.shutil.copyfiles
  File "rasterio/_err.pyx", line 176, in rasterio._err.exc_wrap_int
rasterio._err.CPLE_AppDefinedError: Unable to rename fileset due irregular basenames.
```

Resolving patch:
diff --git a/rasterio/rio/convert.py b/rasterio/rio/convert.py
--- a/rasterio/rio/convert.py
+++ b/rasterio/rio/convert.py
@@ -11,12 +11,7 @@
 
 
 @click.command(short_help="Copy and convert raster dataset.")
-@click.argument(
-    'files',
-    nargs=-1,
-    type=click.Path(resolve_path=True),
-    required=True,
-    metavar="INPUT OUTPUT")
+@options.files_inout_arg
 @options.output_opt
 @format_opt
 @options.dtype_opt
diff --git a/rasterio/rio/rasterize.py b/rasterio/rio/rasterize.py
--- a/rasterio/rio/rasterize.py
+++ b/rasterio/rio/rasterize.py
@@ -30,7 +30,7 @@ def files_handler(ctx, param, value):
 files_inout_arg = click.argument(
     'files',
     nargs=-1,
-    type=click.Path(resolve_path=True),
+    type=click.Path(),
     metavar="INPUTS... OUTPUT",
     callback=files_handler)
 
diff --git a/rasterio/rio/shapes.py b/rasterio/rio/shapes.py
--- a/rasterio/rio/shapes.py
+++ b/rasterio/rio/shapes.py
@@ -16,25 +16,6 @@
 logger = logging.getLogger(__name__)
 
 
-# Common options used below
-
-# Unlike the version in cligj, this one doesn't require values.
-files_inout_arg = click.argument(
-    'files',
-    nargs=-1,
-    type=click.Path(resolve_path=True),
-    metavar="INPUTS... OUTPUT",
-    callback=options.files_inout_handler)
-
-all_touched_opt = click.option(
-    '-a', '--all', '--all_touched', 'all_touched',
-    is_flag=True,
-    default=False,
-    help='Use all pixels touched by features, otherwise (default) use only '
-         'pixels whose center is within the polygon or that are selected by '
-         'Bresenhams line algorithm')
-
-
 @click.command(short_help="Write shapes extracted from bands or masks.")
 @options.file_in_arg
 @options.output_opt


Test patch: