Repository: dask/dask

Issue:
Failed wildcard matching with dd.read_csv("s3://[...]/*.csv", compression='infer') raises unintuitive error message
<!-- Please include a self-contained copy-pastable example that generates the issue if possible.

Please be concise with code posted. See guidelines below on how to provide a good bug report:

- Craft Minimal Bug Reports http://matthewrocklin.com/blog/work/2018/02/28/minimal-bug-reports
- Minimal Complete Verifiable Examples https://stackoverflow.com/help/mcve

Bug reports that follow these guidelines are easier to diagnose, and so are often handled much more quickly.
-->



**What happened**:

A wildcard match that matches no S3 objects in `dd.read_csv(compression='infer')` will return an unintuitive IndexError.

```
IndexError: list index out of range
```

**What you expected to happen**:

That wildcard match should likely return the following error message instead:

```
OSError: s3://private_bucket/path/to/files/TYPO/data_*.csv resolved to no files
```

**Minimal Complete Verifiable Example**:

Paths redacted as this is a private S3 bucket, if someone has a public bucket I could try and provide a fully reproducible example.

I tried a wildcard match on a bunch of files in S3, but misspecified the path/key for a wildcard match:

```
import dask.dataframe as dd
data = dd.read_csv("s3://private_bucket/path/to/files/TYPO/data_*.csv", include_path_column=False, assume_missing=True)
```
I got this error:

```
---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
Input In [11], in <cell line: 1>()
----> 1 data = dd.read_csv("s3://private_bucket/path/to/files/TYPO/data_*.csv",include_path_column=False,assume_missing=True)

File ~/miniconda3/lib/python3.9/site-packages/dask/dataframe/io/csv.py:742, in make_reader.<locals>.read(urlpath, blocksize, lineterminator, compression, sample, sample_rows, enforce, assume_missing, storage_options, include_path_column, **kwargs)
    729 def read(
    730     urlpath,
    731     blocksize="default",
   (...)
    740     **kwargs,
    741 ):
--> 742     return read_pandas(
    743         reader,
    744         urlpath,
    745         blocksize=blocksize,
    746         lineterminator=lineterminator,
    747         compression=compression,
    748         sample=sample,
    749         sample_rows=sample_rows,
    750         enforce=enforce,
    751         assume_missing=assume_missing,
    752         storage_options=storage_options,
    753         include_path_column=include_path_column,
    754         **kwargs,
    755     )

File ~/miniconda3/lib/python3.9/site-packages/dask/dataframe/io/csv.py:521, in read_pandas(reader, urlpath, blocksize, lineterminator, compression, sample, sample_rows, enforce, assume_missing, storage_options, include_path_column, **kwargs)
    516     paths = get_fs_token_paths(urlpath, mode="rb", storage_options=storage_options)[
    517         2
    518     ]
    520     # Infer compression from first path
--> 521     compression = infer_compression(paths[0])
    523 if blocksize == "default":
    524     blocksize = AUTO_BLOCKSIZE

IndexError: list index out of range
```

`paths` likely returns an empty list, and Python raises an error.

But if I add `compression=None` to stop compression inference:

```
data = dd.read_csv("s3://private_bucket/path/to/files/TYPO/data_*.csv",include_path_column=False,assume_missing=True,compression=None)
```

I instead get this much more intuitive error:

```
---------------------------------------------------------------------------
OSError                                   Traceback (most recent call last)
Input In [12], in <cell line: 1>()
----> 1 data = dd.read_csv("s3://private_bucket/path/to/files/TYPO/data_*.csv",include_path_column=False,assume_missing=True,compression=None)

File ~/miniconda3/lib/python3.9/site-packages/dask/dataframe/io/csv.py:742, in make_reader.<locals>.read(urlpath, blocksize, lineterminator, compression, sample, sample_rows, enforce, assume_missing, storage_options, include_path_column, **kwargs)
    729 def read(
    730     urlpath,
    731     blocksize="default",
   (...)
    740     **kwargs,
    741 ):
--> 742     return read_pandas(
    743         reader,
    744         urlpath,
    745         blocksize=blocksize,
    746         lineterminator=lineterminator,
    747         compression=compression,
    748         sample=sample,
    749         sample_rows=sample_rows,
    750         enforce=enforce,
    751         assume_missing=assume_missing,
    752         storage_options=storage_options,
    753         include_path_column=include_path_column,
    754         **kwargs,
    755     )

File ~/miniconda3/lib/python3.9/site-packages/dask/dataframe/io/csv.py:546, in read_pandas(reader, urlpath, blocksize, lineterminator, compression, sample, sample_rows, enforce, assume_missing, storage_options, include_path_column, **kwargs)
    544     sample = blocksize
    545 b_lineterminator = lineterminator.encode()
--> 546 b_out = read_bytes(
    547     urlpath,
    548     delimiter=b_lineterminator,
    549     blocksize=blocksize,
    550     sample=sample,
    551     compression=compression,
    552     include_path=include_path_column,
    553     **(storage_options or {}),
    554 )
    556 if include_path_column:
    557     b_sample, values, paths = b_out

File ~/miniconda3/lib/python3.9/site-packages/dask/bytes/core.py:84, in read_bytes(urlpath, delimiter, not_zero, blocksize, sample, compression, include_path, **kwargs)
     81 fs, fs_token, paths = get_fs_token_paths(urlpath, mode="rb", storage_options=kwargs)
     83 if len(paths) == 0:
---> 84     raise OSError("%s resolved to no files" % urlpath)
     86 if blocksize is not None:
     87     if isinstance(blocksize, str):

OSError: s3://private_bucket/path/to/files/TYPO/data_*.csv resolved to no files
```

**Anything else we need to know?**:

**Environment**:

- Dask version: 2022.02.1
- Python version: 3.9
- Operating System: Ubuntu 20.04.3 LTS
- Install method (conda, pip, source): conda

Failed wildcard matching with dd.read_csv("s3://[...]/*.csv", compression='infer') raises unintuitive error message
<!-- Please include a self-contained copy-pastable example that generates the issue if possible.

Please be concise with code posted. See guidelines below on how to provide a good bug report:

- Craft Minimal Bug Reports http://matthewrocklin.com/blog/work/2018/02/28/minimal-bug-reports
- Minimal Complete Verifiable Examples https://stackoverflow.com/help/mcve

Bug reports that follow these guidelines are easier to diagnose, and so are often handled much more quickly.
-->



**What happened**:

A wildcard match that matches no S3 objects in `dd.read_csv(compression='infer')` will return an unintuitive IndexError.

```
IndexError: list index out of range
```

**What you expected to happen**:

That wildcard match should likely return the following error message instead:

```
OSError: s3://private_bucket/path/to/files/TYPO/data_*.csv resolved to no files
```

**Minimal Complete Verifiable Example**:

Paths redacted as this is a private S3 bucket, if someone has a public bucket I could try and provide a fully reproducible example.

I tried a wildcard match on a bunch of files in S3, but misspecified the path/key for a wildcard match:

```
import dask.dataframe as dd
data = dd.read_csv("s3://private_bucket/path/to/files/TYPO/data_*.csv", include_path_column=False, assume_missing=True)
```
I got this error:

```
---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
Input In [11], in <cell line: 1>()
----> 1 data = dd.read_csv("s3://private_bucket/path/to/files/TYPO/data_*.csv",include_path_column=False,assume_missing=True)

File ~/miniconda3/lib/python3.9/site-packages/dask/dataframe/io/csv.py:742, in make_reader.<locals>.read(urlpath, blocksize, lineterminator, compression, sample, sample_rows, enforce, assume_missing, storage_options, include_path_column, **kwargs)
    729 def read(
    730     urlpath,
    731     blocksize="default",
   (...)
    740     **kwargs,
    741 ):
--> 742     return read_pandas(
    743         reader,
    744         urlpath,
    745         blocksize=blocksize,
    746         lineterminator=lineterminator,
    747         compression=compression,
    748         sample=sample,
    749         sample_rows=sample_rows,
    750         enforce=enforce,
    751         assume_missing=assume_missing,
    752         storage_options=storage_options,
    753         include_path_column=include_path_column,
    754         **kwargs,
    755     )

File ~/miniconda3/lib/python3.9/site-packages/dask/dataframe/io/csv.py:521, in read_pandas(reader, urlpath, blocksize, lineterminator, compression, sample, sample_rows, enforce, assume_missing, storage_options, include_path_column, **kwargs)
    516     paths = get_fs_token_paths(urlpath, mode="rb", storage_options=storage_options)[
    517         2
    518     ]
    520     # Infer compression from first path
--> 521     compression = infer_compression(paths[0])
    523 if blocksize == "default":
    524     blocksize = AUTO_BLOCKSIZE

IndexError: list index out of range
```

`paths` likely returns an empty list, and Python raises an error.

But if I add `compression=None` to stop compression inference:

```
data = dd.read_csv("s3://private_bucket/path/to/files/TYPO/data_*.csv",include_path_column=False,assume_missing=True,compression=None)
```

I instead get this much more intuitive error:

```
---------------------------------------------------------------------------
OSError                                   Traceback (most recent call last)
Input In [12], in <cell line: 1>()
----> 1 data = dd.read_csv("s3://private_bucket/path/to/files/TYPO/data_*.csv",include_path_column=False,assume_missing=True,compression=None)

File ~/miniconda3/lib/python3.9/site-packages/dask/dataframe/io/csv.py:742, in make_reader.<locals>.read(urlpath, blocksize, lineterminator, compression, sample, sample_rows, enforce, assume_missing, storage_options, include_path_column, **kwargs)
    729 def read(
    730     urlpath,
    731     blocksize="default",
   (...)
    740     **kwargs,
    741 ):
--> 742     return read_pandas(
    743         reader,
    744         urlpath,
    745         blocksize=blocksize,
    746         lineterminator=lineterminator,
    747         compression=compression,
    748         sample=sample,
    749         sample_rows=sample_rows,
    750         enforce=enforce,
    751         assume_missing=assume_missing,
    752         storage_options=storage_options,
    753         include_path_column=include_path_column,
    754         **kwargs,
    755     )

File ~/miniconda3/lib/python3.9/site-packages/dask/dataframe/io/csv.py:546, in read_pandas(reader, urlpath, blocksize, lineterminator, compression, sample, sample_rows, enforce, assume_missing, storage_options, include_path_column, **kwargs)
    544     sample = blocksize
    545 b_lineterminator = lineterminator.encode()
--> 546 b_out = read_bytes(
    547     urlpath,
    548     delimiter=b_lineterminator,
    549     blocksize=blocksize,
    550     sample=sample,
    551     compression=compression,
    552     include_path=include_path_column,
    553     **(storage_options or {}),
    554 )
    556 if include_path_column:
    557     b_sample, values, paths = b_out

File ~/miniconda3/lib/python3.9/site-packages/dask/bytes/core.py:84, in read_bytes(urlpath, delimiter, not_zero, blocksize, sample, compression, include_path, **kwargs)
     81 fs, fs_token, paths = get_fs_token_paths(urlpath, mode="rb", storage_options=kwargs)
     83 if len(paths) == 0:
---> 84     raise OSError("%s resolved to no files" % urlpath)
     86 if blocksize is not None:
     87     if isinstance(blocksize, str):

OSError: s3://private_bucket/path/to/files/TYPO/data_*.csv resolved to no files
```

**Anything else we need to know?**:

**Environment**:

- Dask version: 2022.02.1
- Python version: 3.9
- Operating System: Ubuntu 20.04.3 LTS
- Install method (conda, pip, source): conda



Discussion:
Thanks for opening this issue! @ncclementi I think this is a reasonable request, what do you think?

#### Reproducer

```python
import dask.dataframe as dd
data = dd.read_csv("s3://nyc-tlc/trip data/test-*.csv") # IndexError: list index out of range
data = dd.read_csv("s3://nyc-tlc/trip data/test-*.csv", compression=None) # OSError: s3://nyc-tlc/trip data/test-*.csv resolved to no files
```

#### Possible solution

Add:
```python
if len(paths) == 0:
    raise OSError("%s resolved to no files" % urlpath)
```
to:
https://github.com/dask/dask/blob/0b36d7fcaf54ee9a78fff4b07f124cb0c8741cdf/dask/dataframe/io/csv.py#L514-L521

@rfilmyer Would you be interested in opening a PR for this?
That seems like a very reasonable solution @pavithraes 
Sounds reasonable, I can do the PR for this.
Thanks for opening this issue! @ncclementi I think this is a reasonable request, what do you think?

#### Reproducer

```python
import dask.dataframe as dd
data = dd.read_csv("s3://nyc-tlc/trip data/test-*.csv") # IndexError: list index out of range
data = dd.read_csv("s3://nyc-tlc/trip data/test-*.csv", compression=None) # OSError: s3://nyc-tlc/trip data/test-*.csv resolved to no files
```

#### Possible solution

Add:
```python
if len(paths) == 0:
    raise OSError("%s resolved to no files" % urlpath)
```
to:
https://github.com/dask/dask/blob/0b36d7fcaf54ee9a78fff4b07f124cb0c8741cdf/dask/dataframe/io/csv.py#L514-L521

@rfilmyer Would you be interested in opening a PR for this?
That seems like a very reasonable solution @pavithraes 
Sounds reasonable, I can do the PR for this.

Resolving patch:
diff --git a/dask/dataframe/io/csv.py b/dask/dataframe/io/csv.py
--- a/dask/dataframe/io/csv.py
+++ b/dask/dataframe/io/csv.py
@@ -517,6 +517,10 @@ def read_pandas(
             2
         ]
 
+        # Check for at least one valid path
+        if len(paths) == 0:
+            raise OSError(f"{urlpath} resolved to no files")
+
         # Infer compression from first path
         compression = infer_compression(paths[0])
 


Test patch: