Repository: pallets/click

Issue:
click.Path returns str, not pathlib.Path
It’s surprising that `INT` returns an `int`, `File` a `file`, `STRING` a `str`, but `Path` a …`str`?

A `click.Path` parameter should return a `pathlib.Path` object.

Allow path_type of the Path type to accept a callable (`#405`_)
This would allow to convert path names to pathlib.Path instances
for example.

closes #405


Discussion:
pathlib is Python 3 only afaict, Click isn't.

FWIW I'm certain that an option to return a Path object is useful, but I can't see it becoming the default.

[it’s backported](https://pypi.python.org/pypi/pathlib/)

It's still not possible to make this a default since this would mean that the behavior of Click would depend on whether that backport is installed. And depending on that backport is not worth the added value.

> depending on that backport is not worth the added value.

what cost/value function evaluates to this result? :smile: 

In this case, new functionality. Pathlib is merely a new API for the same
thing.

Sorry, but I don't think this is going to happen. I'll gladly accept PRs for a
non-default option though.

On Mon, Aug 10, 2015 at 06:28:08AM -0700, Philipp A. wrote:

> > depending on that backport is not worth the added value.
> 
> what cost/value function evaluates to this result? :smile: 
> 
> ---
> 
> Reply to this email directly or view it on GitHub:
> https://github.com/mitsuhiko/click/issues/405#issuecomment-129451586

What do you mean with “non-default option”?

Opt-in for returning a pathlib.Path object instead of a string. The user has to
depend on the pathlib library themselves unless it's in the stdlib. I think
that's doable.

On Mon, Aug 10, 2015 at 06:37:04AM -0700, Philipp A. wrote:

> What do you mean with “non-default option”?
> 
> ---
> 
> Reply to this email directly or view it on GitHub:
> https://github.com/mitsuhiko/click/issues/405#issuecomment-129457794

got it, makes sense. best would be to do it on arbitrary levels, right?

e.g. you could create a group that is configured to do it,  passing it down to all subcommands and parameters, or on single parameters or even options.

the interaction of this with `click.CommandCollection` is obvious, but bad: merging a non-pathlib group with a pathlib group would simply behave as if each individual command would have been configured, and result in a mess, but that’s not really a problem i guess, and there’s no alternative anyway

> got it, makes sense. best would be to do it on arbitrary levels, right?

Honestly I don't think that's a good idea, it makes it hard to trace why a particular option is returning this particular type. If you want to avoid repetition/verbosity, you can still bind the created decorator to a new name for reuse, or create a new decorator that has it as default. 

hmm, yeah, using `partial`. only commands are defined on the group, so this would be easy.

Not going to happen.  A custom path implementation might be something for addon packages.

Welp-

worth reevaluating in the light of [PEP 519](https://www.python.org/dev/peps/pep-0519/)?

also are you going to sign the [python 3 statement](http://www.python3statement.org/)? if so, we can tackle this in 2020.
That PEP makes a lot of things easier, but AFAICT it doesn't even apply to all
versions of Python 3.

I'd personally love to drop Python 2 at some point, but even for me 2020 is way
too soon. It's not (only) my decision to make.

On Mon, Jan 23, 2017 at 02:26:37AM -0800, Philipp A. wrote:
> worth reevaluating in the light of [PEP 519](https://www.python.org/dev/peps/pep-0519/)?
> 
> also are you going to sign the [python statement](http://www.python3statement.org/)? if so, we can tackle this in 2020.
> 
> -- 
> You are receiving this because you modified the open/close state.
> Reply to this email directly or view it on GitHub:
> https://github.com/pallets/click/issues/405#issuecomment-274450716

i didn’t want to propose that, just ask 😉

the PEP partly applies to everything, as it describes a protocol, not just an API.

checking for the existence of `__fspath__` works in any version!
It would be really confusing to get a `pathlib.Path` object in 3.6+, but not in 3.5 or lower.

Is gating this on a (non-default) flag on `click.Path` still out per https://github.com/pallets/click/issues/405#issuecomment-140582568?
Since `PATH` does not exist anyways we can do two things. One is we set up a default mapping for `pathlib.Path` as input type (similar to how we have a default type for `str`) being passed to `click.Path(convert=pathlib.Path)` and then add support for `convert` being passed to `click.Path`.

That way someone can also pass it through something else that does something interesting with the path.
I understand finding time for reviews is hard but I wanted to make sure I didn't need to be doing something else for ~~#405~~ #1148.  I think it fits within the suggested approach of having a generic converter which a developer could pass `pathlib.Path` to if they so chose.
err, #1148 
I'd love to see this. I expected `click.Path(path_type=pathlib.PurePath)` would cast my result to a `PurePath`, but @altendky's `converter` is even more clear. Would love to see this pulled ASAP so I can start using it in my app.
I 'd agree with @NotTheEconomist version without new arguments.
Please consider #1234 .
For the impatient, I defined my own like this. It accepts all the same options as `click.Path`:

```python
class PathPath(click.Path):
    """A Click path argument that returns a pathlib Path, not a string"""
    def convert(self, value, param, ctx):
        return Path(super().convert(value, param, ctx))
```

```python
@click.argument('input', type=PathPath(dir_okay=False))
def ...
```

(this usage style also feels cleaner to me than `click.Path(...., converter=pathlib.Path)`, as well as more easily remembered. Path is built into Python's standard library and widely used, so I think it's worth a dedicated type.)
Thanks @jeremyh !

I have packaged this solution into https://pypi.org/project/click-pathlib, so you can use:

```
$ pip install click-pathlib
```

```python
import click
import click_pathlib

@click.command('delete')
@click.argument(
    'existing_file',
    type=click_pathlib.Path(exists=True),
)
def delete(existing_file):
    existing_file.unlink()
```
Seeing as Python 2 is now dead and all, does it maybe make sense to now transition to using pathlib.Path as a default instead of a str value?
Pretty sure that would require a major release, and while pathlib is awesome, I think it's the kind of change that would be a pain in the ass for any downstream user who doesn't already use pathlib.
Also, even if Path is available on Python 3.x it's not mandatory to use it, a developer can still choose to use plain string or a Path instance
Any request for making this the default basically asks for two things:

* New major version of click, break all existing code
* Drop Python 2 *or* pull in dependency on backport

Each of those points in isolation is an extreme measure to make this happen, combined even more.
The second one isn’t, Python 2 isn’t supported anymore so you’re free to move on and get rid of all hideous compatibility hacks :tada:

About the first … click has a lot of major version releases, and derive from your reluctance that they’re all less intrusive than this one?
Click supports Python 2.
You know I mean that Python 2 is unmaintained as of 2 weeks ago. Supporting something that’s unmaintained is pretty futile.
I'm saying that this is a completely separate decision to make. But I forgot that we already announced dropped support for Python 2 so this would probably be fine to land in Click 8.

https://palletsproject.com/blog/ending-python2-support/
Hmm, just spitballing here, but maybe a transitional release could be done (after #1148 is merged). In Click 8, the following would raise a `FutureWarning` like this:

```py
@click.argument('filename', type=click.Path())
```

> You specified no Path converter. `click.Path(converter=None)` will result in a `pathlib.Path` object in Click 9. To avoid this warning, specify `converter=str` or `converter=pathlib.Path`.

And consequently, in Click 9 (or 10), the warning would be gone and the default changed.
An alternative (also depending on #1148) would be to follow @mitsuhiko’s idea in https://github.com/pallets/click/issues/405#issuecomment-324987608:

We just add `click.PATH` and use it in examples instead of `click.Path`, therefore encouraging `click.PATH` without breaking changes. This would be less disruptive.
Ran `tox` on this locally
```
  py37: commands succeeded
SKIPPED:  py36: InterpreterNotFound: python3.6
  py35: commands succeeded
SKIPPED:  py34: InterpreterNotFound: python3.4
  py27: commands succeeded
SKIPPED:  pypy3: InterpreterNotFound: pypy3
SKIPPED:  pypy: InterpreterNotFound: pypy
```

I also tested this with an application I'm developing by adding `git+https://github.com/termim/click.git@I405#egg=click` to my `requirements.txt` and it works as advertised.

However, using `pathlib.Path` as the `path_type` with [Click7.0](https://github.com/pallets/click/commit/a936b99212c69a36a2cc2828252b2c4280c0c88c)
```
@click.option('--config', '-c', type=click.Path(exists=True,dir_okay=False,path_type=pathlib.Path), required=False, help='Config file')
```
sets `config` to type `bytes` though backwards compatibility can be maintained as needed with
```
config = pathlib.Path(config.decode()) if isinstance(config,bytes) else config
```
as `pathlib.Path(None)` isn't accepted.

Also, I'm not sure if there would be an advantage to setting [this](https://github.com/pallets/click/pull/1234/files#diff-9de36928a31dc2292c5f42ea1a3afbb4R141) to `isinstance(input,os.PathLike)` but it may be worth considering.

Aside from the above it looks good to me, I would like to see this merged.
> I looks like [this assertion](https://github.com/pallets/click/pull/1234/files#diff-9de36928a31dc2292c5f42ea1a3afbb4R157) needs to be uncommented.

Done.
> However, using `pathlib.Path` as the `path_type` with [Click7.0](https://github.com/pallets/click/commit/a936b99212c69a36a2cc2828252b2c4280c0c88c)
> 
> ```
> @click.option('--config', '-c', type=click.Path(exists=True,dir_okay=False,path_type=pathlib.Path), required=False, help='Config file')
> ```
> 
> sets `config` to type `bytes` though backwards compatibility can be maintained as needed with

I can't reproduce this:
```
click>$ python3 --version
Python 3.5.2
click>$ grep __version__ ./click/click/__init__.py 
__version__ = '7.1.dev'
click>$ cat t.py
import click
import pathlib

@click.command()
@click.option('--config', '-c', type=click.Path(exists=True,dir_okay=False,path_type=pathlib.Path), required=False, help='Config file')
def hello(config):
    click.echo('Hello %s!' % type(config))

if __name__ == '__main__':
    hello()
click>$ 
click>$ PYTHONPATH=./click python3 ./t.py -c ./t.py
Hello <class 'pathlib.PosixPath'>!
click>$ 
```
> 
> ```
> config = pathlib.Path(config.decode()) if isinstance(config,bytes) else config
> ```
> 
> as `pathlib.Path(None)` isn't accepted.
> 
> Also, I'm not sure if there would be an advantage to setting [this](https://github.com/pallets/click/pull/1234/files#diff-9de36928a31dc2292c5f42ea1a3afbb4R141) to `isinstance(input,os.PathLike)` but it may be worth considering.
> 

That would make click require pathlib2 for python2 ...



> > However, using `pathlib.Path` as the `path_type` with [Click7.0](https://github.com/pallets/click/commit/a936b99212c69a36a2cc2828252b2c4280c0c88c)
> > ```
> > @click.option('--config', '-c', type=click.Path(exists=True,dir_okay=False,path_type=pathlib.Path), required=False, help='Config file')
> > ```
> > sets `config` to type `bytes` though backwards compatibility can be maintained as needed with
> 
> I can't reproduce this:
> 
> ```
> click>$ python3 --version
> Python 3.5.2
> click>$ grep __version__ ./click/click/__init__.py 
> __version__ = '7.1.dev'
> click>$ cat t.py
> import click
> import pathlib
> 
> @click.command()
> @click.option('--config', '-c', type=click.Path(exists=True,dir_okay=False,path_type=pathlib.Path), required=False, help='Config file')
> def hello(config):
>     click.echo('Hello %s!' % type(config))
> 
> if __name__ == '__main__':
>     hello()
> click>$ 
> click>$ PYTHONPATH=./click python3 ./t.py -c ./t.py
> Hello <class 'pathlib.PosixPath'>!
> click>$ 
> ```

I was getting `bytes` using `path_type=pathlib.Path` without your PR on version 7.0.

```
$ python t.py -c t.py
Hello <class 'bytes'>!
```
> I was getting `bytes` using `path_type=pathlib.Path` without your PR on version 7.0.

Ah, yes, without this #1234 it always returns bytes if path_type is not a string ...
`config = pathlib.Path(config.decode()) if isinstance(config,bytes) else config`
provides backwards compatibility, so application developers can move to `pathlib` while waiting for your PR to make it downstream.

Resolving patch:
diff --git a/src/click/types.py b/src/click/types.py
--- a/src/click/types.py
+++ b/src/click/types.py
@@ -649,9 +649,6 @@ class Path(ParamType):
     handle it returns just the filename.  Secondly, it can perform various
     basic checks about what the file or directory should be.
 
-    .. versionchanged:: 6.0
-       `allow_dash` was added.
-
     :param exists: if set to true, the file or directory needs to exist for
                    this value to be valid.  If this is not required and a
                    file does indeed not exist, then all further checks are
@@ -667,11 +664,15 @@ class Path(ParamType):
                          supposed to be done by the shell only.
     :param allow_dash: If this is set to `True`, a single dash to indicate
                        standard streams is permitted.
-    :param path_type: optionally a string type that should be used to
-                      represent the path.  The default is `None` which
-                      means the return value will be either bytes or
-                      unicode depending on what makes most sense given the
-                      input data Click deals with.
+    :param path_type: Convert the incoming path value to this type. If
+        ``None``, keep Python's default, which is ``str``. Useful to
+        convert to :class:`pathlib.Path`.
+
+    .. versionchanged:: 8.0
+        Allow passing ``type=pathlib.Path``.
+
+    .. versionchanged:: 6.0
+        Added the ``allow_dash`` parameter.
     """
 
     envvar_list_splitter = os.path.pathsep
@@ -698,13 +699,10 @@ def __init__(
 
         if self.file_okay and not self.dir_okay:
             self.name = "file"
-            self.path_type = "File"
         elif self.dir_okay and not self.file_okay:
             self.name = "directory"
-            self.path_type = "Directory"
         else:
             self.name = "path"
-            self.path_type = "Path"
 
     def to_info_dict(self):
         info_dict = super().to_info_dict()
@@ -721,9 +719,12 @@ def to_info_dict(self):
     def coerce_path_result(self, rv):
         if self.type is not None and not isinstance(rv, self.type):
             if self.type is str:
-                rv = rv.decode(get_filesystem_encoding())
+                rv = os.fsdecode(rv)
+            elif self.type is bytes:
+                rv = os.fsencode(rv)
             else:
-                rv = rv.encode(get_filesystem_encoding())
+                rv = self.type(rv)
+
         return rv
 
     def convert(self, value, param, ctx):
@@ -741,32 +742,32 @@ def convert(self, value, param, ctx):
                 if not self.exists:
                     return self.coerce_path_result(rv)
                 self.fail(
-                    f"{self.path_type} {filename_to_ui(value)!r} does not exist.",
+                    f"{self.name.title()} {filename_to_ui(value)!r} does not exist.",
                     param,
                     ctx,
                 )
 
             if not self.file_okay and stat.S_ISREG(st.st_mode):
                 self.fail(
-                    f"{self.path_type} {filename_to_ui(value)!r} is a file.",
+                    f"{self.name.title()} {filename_to_ui(value)!r} is a file.",
                     param,
                     ctx,
                 )
             if not self.dir_okay and stat.S_ISDIR(st.st_mode):
                 self.fail(
-                    f"{self.path_type} {filename_to_ui(value)!r} is a directory.",
+                    f"{self.name.title()} {filename_to_ui(value)!r} is a directory.",
                     param,
                     ctx,
                 )
             if self.writable and not os.access(value, os.W_OK):
                 self.fail(
-                    f"{self.path_type} {filename_to_ui(value)!r} is not writable.",
+                    f"{self.name.title()} {filename_to_ui(value)!r} is not writable.",
                     param,
                     ctx,
                 )
             if self.readable and not os.access(value, os.R_OK):
                 self.fail(
-                    f"{self.path_type} {filename_to_ui(value)!r} is not readable.",
+                    f"{self.name.title()} {filename_to_ui(value)!r} is not readable.",
                     param,
                     ctx,
                 )


Test patch:
diff --git a/tests/test_types.py b/tests/test_types.py
--- a/tests/test_types.py
+++ b/tests/test_types.py
@@ -1,3 +1,5 @@
+import pathlib
+
 import pytest
 
 import click
@@ -78,3 +80,23 @@ def test_cast_multi_default(runner, nargs, multiple, default, expect):
     result = runner.invoke(cli, standalone_mode=False)
     assert result.exception is None
     assert result.return_value == expect
+
+
+@pytest.mark.parametrize(
+    ("cls", "expect"),
+    [
+        (None, "a/b/c.txt"),
+        (str, "a/b/c.txt"),
+        (bytes, b"a/b/c.txt"),
+        (pathlib.Path, pathlib.Path("a", "b", "c.txt")),
+    ],
+)
+def test_path_type(runner, cls, expect):
+    cli = click.Command(
+        "cli",
+        params=[click.Argument(["p"], type=click.Path(path_type=cls))],
+        callback=lambda p: p,
+    )
+    result = runner.invoke(cli, ["a/b/c.txt"], standalone_mode=False)
+    assert result.exception is None
+    assert result.return_value == expect