Repository: dask/dask

Issue:
Double work when passing a fastparquet.ParquetFile to read_parquet()
When we pass a fastparquet.ParquetFile to `read_parquet()` I found that the metadata will be read again in `_read_fastparquet`, because in `read_parquet()` , when we pass a fastparquet instance, dask is getting the path from fastparquet metadata and passing it to `_read_fastparquet`. I think if we are passing a fastparquet instance, we should make leverage of that and avoid to read it again to avoid some unnecessary computations. This can be a burden if metadata is stupidly large.

I'm probably missing something here and there is a reason for the metadada to be read again. 

Anyway I'm making comments in the following snippets to mark what I'm saying

```python
def read_parquet(path, columns=None, filters=None, categories=None, index=None,
                 storage_options=None, engine='auto', infer_divisions=None):
    is_ParquetFile = False
    try:
        import fastparquet
        if isinstance(path, fastparquet.api.ParquetFile):
            if path.open != fastparquet.util.default_open:
                assert (re.match('.*://', path.fn)), \
                       ("ParquetFile: Path must contain protocol" +
                        " (e.g., s3://...) when using other than the default" +
                        " LocalFileSystem. Path given: " + path.fn)

            assert (engine in ['auto', 'fastparquet']), \
                   ("'engine' should be set to 'auto' or 'fastparquet' " +
                    'when reading from fastparquet.ParquetFile')
            is_ParquetFile = True
    except ImportError:
            pass

    if is_ParquetFile:
        read = get_engine('fastparquet')['read']
        if path.fn.endswith('_metadata'):
            # remove '_metadata' from path
            urlpath = path.fn[:-len('_metadata')]
        else:
            urlpath = path.fn

        fs, fs_token, paths = get_fs_token_paths(
            urlpath,
            mode='rb',
            storage_options=storage_options
        )
    else:
        read = get_engine(engine)['read']
        fs, fs_token, paths = get_fs_token_paths(
            path, mode='rb',
            storage_options=storage_options
        )

        if isinstance(path, string_types) and len(paths) > 1:
            # Sort paths naturally if multiple paths resulted from a single
            # specification (by '*' globbing)
            paths = sorted(paths, key=natural_sort_key)

   #paths will always be a path and never a fastparquet instance <---- HERE!
    return read(fs, fs_token, paths, columns=columns, filters=filters,
                categories=categories, index=index, infer_divisions=infer_divisions)
```

```python
def _read_fastparquet(fs, fs_token, paths, columns=None, filters=None,
                      categories=None, index=None, infer_divisions=None):
    import fastparquet

    #this line is useless because paths will never be an instance of fastparquet <---- HERE!
    if isinstance(paths, fastparquet.api.ParquetFile): 
        pf = paths
    elif len(paths) > 1:
        if infer_divisions is not False:
            # this scans all the files, allowing index/divisions and filtering
            pf = fastparquet.ParquetFile(paths, open_with=fs.open, sep=fs.sep)
        else:
            return _read_fp_multifile(fs, fs_token, paths, columns=columns,
                                      categories=categories, index=index)
    else:
```


Discussion:
Can you check by adding the following line at the end of the `if is_ParquetFile:` block:
```
paths = path
```


Agree with that, but unfortunately in `_read_fastparquet()` function than len(paths) would never be > 1 and it can happen that you are passing a ParquetFile from a list of files. So we probably need to think a smarter way to do it. Don't you think?
> it can happen that you are passing a ParquetFile from a list of files

Sorry, didn't quite follow that - in this case, we have already used fastparquet to scan the set of files beforehand, no? 
This is really a corner case, but imagine that I read a list of files with fastparquet and now I use that object to make a `read_parquet(parquetObject, infer_divisions=False)` then we would never return `_read_fp_multifile()` because of `len(parquetObject) = 1`. 

```python
elif len(paths) > 1:
        if infer_divisions is not False:
            # this scans all the files, allowing index/divisions and filtering
            pf = fastparquet.ParquetFile(paths, open_with=fs.open, sep=fs.sep)
        else:
            return _read_fp_multifile(fs, fs_token, paths, columns=columns,
                                      categories=categories, index=index)
```
You do not want to reach `_read_fp_multifile`, that function is specialised for the case where you are reading from a list of files, but do *not* want to read their metadata beforehand and construct a ParquetFile object. That applies specifically to `read_parquet('path/*/*.parquet`)` without metadata files, where you can guarantee the schema is the same everywhere, as is the case for pyarrow data-sets and some spark/hive ones.
Yeah, I see, and I was being stupid, because inside `_read_fp_multifile` metadata is read again, so this would make no sense. So I agree with you that it must be enough to only add this line after `is_Parquetfile`
```
paths = path
```

Resolving patch:
diff --git a/dask/dataframe/io/parquet.py b/dask/dataframe/io/parquet.py
--- a/dask/dataframe/io/parquet.py
+++ b/dask/dataframe/io/parquet.py
@@ -1137,6 +1137,7 @@ def read_parquet(path, columns=None, filters=None, categories=None, index=None,
             mode='rb',
             storage_options=storage_options
         )
+        paths = path
     else:
         read = get_engine(engine)['read']
         fs, fs_token, paths = get_fs_token_paths(


Test patch:
diff --git a/dask/dataframe/io/tests/test_parquet.py b/dask/dataframe/io/tests/test_parquet.py
--- a/dask/dataframe/io/tests/test_parquet.py
+++ b/dask/dataframe/io/tests/test_parquet.py
@@ -1361,3 +1361,18 @@ def test_append_cat_fp(tmpdir):
     dd.to_parquet(ddf, path, append=True, ignore_divisions=True)
     d = dd.read_parquet(path).compute()
     assert d['x'].tolist() == ["a", "a", "b", "a", "b"] * 2
+
+
+def test_passing_parquetfile(tmpdir):
+    import shutil
+    fp = pytest.importorskip('fastparquet')
+    path = str(tmpdir)
+    df = pd.DataFrame({"x": [1, 3, 2, 4]})
+    ddf = dd.from_pandas(df, npartitions=1)
+
+    dd.to_parquet(ddf, path)
+    pf = fp.ParquetFile(path)
+    shutil.rmtree(path)
+
+    # should pass, because no need to re-read metadata
+    dd.read_parquet(pf)