Repository: pandas-dev/pandas (Python)

Issue:
BUG: read_csv raises an error when both `prefix` and `names` are set to `None`
- [x] I have checked that this issue has not already been reported.

- [x] I have confirmed this bug exists on the latest version of pandas.

- [x] (optional) I have confirmed this bug exists on the master branch of pandas.

---

Hi everyone, I'm running into this issue since pandas 1.3.0:

#### Code Sample, a copy-pastable example

```python
import pandas as pd

pd.read_csv("path/to/any/csv", names=None, prefix=None)
```

raises
```python
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-6-681798f605ab> in <module>()
----> 1 pd.read_csv("/content/sample_data/mnist_test.csv", names=None, prefix=None)

2 frames
/usr/local/lib/python3.7/dist-packages/pandas/io/parsers/readers.py in _refine_defaults_read(dialect, delimiter, delim_whitespace, engine, sep, error_bad_lines, warn_bad_lines, on_bad_lines, names, prefix, defaults)
   1304 
   1305     if names is not lib.no_default and prefix is not lib.no_default:
-> 1306         raise ValueError("Specified named and prefix; you can only specify one.")
   1307 
   1308     kwds["names"] = None if names is lib.no_default else names

ValueError: Specified named and prefix; you can only specify one.
```

#### Problem description

With `names=None` and `prefix=None`  those parameters shouldn't be considered as specified, and the code should run as if they were not passed as keyword arguments.

This is due to the changes in this PR https://github.com/pandas-dev/pandas/pull/41446 that changed the default values of those two parameters from `None` to `no_default`

#### Expected Output

The code should load the csv using the default behavior as if `names` and `prefix` were not passed as keyword arguments

#### Output of ``pd.show_versions()``

<details>
INSTALLED VERSIONS
------------------
commit           : f00ed8f47020034e752baf0250483053340971b0
python           : 3.7.10.final.0
python-bits      : 64
OS               : Linux
OS-release       : 5.4.104+
Version          : #1 SMP Sat Jun 5 09:50:34 PDT 2021
machine          : x86_64
processor        : x86_64
byteorder        : little
LC_ALL           : None
LANG             : en_US.UTF-8
LOCALE           : en_US.UTF-8

pandas           : 1.3.0
numpy            : 1.19.5
pytz             : 2018.9
dateutil         : 2.8.1
pip              : 19.3.1
setuptools       : 57.0.0
Cython           : 0.29.23
pytest           : 3.6.4
hypothesis       : None
sphinx           : 1.8.5
blosc            : None
feather          : 0.4.1
xlsxwriter       : None
lxml.etree       : 4.2.6
html5lib         : 1.0.1
pymysql          : None
psycopg2         : 2.7.6.1 (dt dec pq3 ext lo64)
jinja2           : 2.11.3
IPython          : 5.5.0
pandas_datareader: 0.9.0
bs4              : 4.6.3
bottleneck       : 1.3.2
fsspec           : None
fastparquet      : None
gcsfs            : None
matplotlib       : 3.2.2
numexpr          : 2.7.3
odfpy            : None
openpyxl         : 2.5.9
pandas_gbq       : 0.13.3
pyarrow          : 3.0.0
pyxlsb           : None
s3fs             : None
scipy            : 1.4.1
sqlalchemy       : 1.4.18
tables           : 3.4.4
tabulate         : 0.8.9
xarray           : 0.18.2
xlrd             : 1.1.0
xlwt             : 1.3.0
numba            : 0.51.2

</details>



Discussion:
> With `names=None` and `prefix=None` those parameters shouldn't be considered as specified

in Python, passing None is not the same as not passing optional arguments.

the api docs for 1.2 (https://pandas.pydata.org/pandas-docs/version/1.2.0/reference/api/pandas.read_csv.html) clearly state the accepted types for both the `names` and `prefix` parameters as `names : array-like, optional` and `prefix : str, optional`
we would take a PR to fix this.
Even  [Eland](https://github.com/elastic/eland/)  has  a similar issue, where we are passing both as None. 
At the moment, we are handling it by not passing one of them if both are `None`.
@simonjayhawkins Are you fine with patching this? I can send a PR if so, but it would probably miss the 1.3.1 cut.
sure. we discussed this on the dev call and patching this is fine. 1.3.1 probably Sunday.
Thanks !



Patch:
diff --git a/doc/source/whatsnew/v1.3.2.rst b/doc/source/whatsnew/v1.3.2.rst
index 3350d9469a9ee..116bdd6e1d98f 100644
--- a/doc/source/whatsnew/v1.3.2.rst
+++ b/doc/source/whatsnew/v1.3.2.rst
@@ -19,6 +19,7 @@ Fixed regressions
 - Regression in :meth:`DataFrame.from_records` with empty records (:issue:`42456`)
 - Fixed regression in :meth:`DataFrame.shift` where TypeError occurred when shifting DataFrame created by concatenation of slices and fills with values (:issue:`42719`)
 - Regression in :meth:`DataFrame.agg` when the ``func`` argument returned lists and ``axis=1`` (:issue:`42727`)
+- Fixed regression where :meth:`pandas.read_csv` raised a ``ValueError`` when parameters ``names`` and ``prefix`` were both set to None (:issue:`42387`)
 - Fixed regression in comparisons between :class:`Timestamp` object and ``datetime64`` objects outside the implementation bounds for nanosecond ``datetime64`` (:issue:`42794`)
 -
 
diff --git a/pandas/io/parsers/readers.py b/pandas/io/parsers/readers.py
index 06bdbe3054a15..c639a4a9d494e 100644
--- a/pandas/io/parsers/readers.py
+++ b/pandas/io/parsers/readers.py
@@ -1302,7 +1302,12 @@ def _refine_defaults_read(
     if delimiter and (sep is not lib.no_default):
         raise ValueError("Specified a sep and a delimiter; you can only specify one.")
 
-    if names is not lib.no_default and prefix is not lib.no_default:
+    if (
+        names is not None
+        and names is not lib.no_default
+        and prefix is not None
+        and prefix is not lib.no_default
+    ):
         raise ValueError("Specified named and prefix; you can only specify one.")
 
     kwds["names"] = None if names is lib.no_default else names


Test patch:
diff --git a/pandas/tests/io/parser/common/test_common_basic.py b/pandas/tests/io/parser/common/test_common_basic.py
index a1c76e2740dbe..b2e528aa5f8d5 100644
--- a/pandas/tests/io/parser/common/test_common_basic.py
+++ b/pandas/tests/io/parser/common/test_common_basic.py
@@ -764,15 +764,24 @@ def test_read_table_delim_whitespace_non_default_sep(all_parsers, delimiter):
 
 
 @pytest.mark.parametrize("func", ["read_csv", "read_table"])
-@pytest.mark.parametrize("prefix", [None, "x"])
-@pytest.mark.parametrize("names", [None, ["a"]])
-def test_names_and_prefix_not_lib_no_default(all_parsers, names, prefix, func):
+def test_names_and_prefix_not_None_raises(all_parsers, func):
     # GH#39123
     f = StringIO("a,b\n1,2")
     parser = all_parsers
     msg = "Specified named and prefix; you can only specify one."
     with pytest.raises(ValueError, match=msg):
-        getattr(parser, func)(f, names=names, prefix=prefix)
+        getattr(parser, func)(f, names=["a", "b"], prefix="x")
+
+
+@pytest.mark.parametrize("func", ["read_csv", "read_table"])
+@pytest.mark.parametrize("prefix, names", [(None, ["x0", "x1"]), ("x", None)])
+def test_names_and_prefix_explicit_None(all_parsers, names, prefix, func):
+    # GH42387
+    f = StringIO("a,b\n1,2")
+    expected = DataFrame({"x0": ["a", "1"], "x1": ["b", "2"]})
+    parser = all_parsers
+    result = getattr(parser, func)(f, names=names, sep=",", prefix=prefix, header=None)
+    tm.assert_frame_equal(result, expected)
 
 
 def test_dict_keys_as_names(all_parsers):