Repository: pandas-dev/pandas
Pull request: BUG: #31464 Fix error when parsing JSON list of bool into Series
Add a missing exception type to the except clause, to cover
the TypeError that is thrown by Cythonized array_to_datetime
function when trying to convert bool to nonseconds.

- [x] closes #31464
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry


Problem statement:
read_json with typ="series" of json list of bools results in timestamps/Exception
#### Code Sample, a copy-pastable example if possible

```python
import pandas as pd
pd.read_json('[true, true, false]', typ="series")
```
results in the following Pandas Series object in older Pandas versions:
```
0   1970-01-01 00:00:01
1   1970-01-01 00:00:01
2   1970-01-01 00:00:00
dtype: datetime64[ns]
```
Since 1.0.0 it raises TypeError: <class 'bool'> is not convertible to datetime


#### Problem description
The expected output would be a Pandas Series of bools. Note that
* with typ="frame" it works and the result is a dataframe with one column with bool values
* with convert_dates set to False correctly outputs a Series of boolean values

This is a problem because
* users would expect a Series of bools (and neither an exception nor a series of timestamps)
* it is inconsistent with the "frame" case

#### Expected Output

#### Output of ``pd.show_versions()``

<details>

[paste the output of ``pd.show_versions()`` here below this line]

INSTALLED VERSIONS
------------------
commit           : None
python           : 3.8.1.final.0
python-bits      : 64
OS               : Linux
OS-release       : 5.4.13-arch1-1
machine          : x86_64
processor        : 
byteorder        : little
LC_ALL           : None
LANG             : de_DE.UTF-8
LOCALE           : de_DE.UTF-8

pandas           : 1.0.0
numpy            : 1.18.1
pytz             : 2019.3
dateutil         : 2.8.1
pip              : 20.0.2
setuptools       : 44.0.0
Cython           : 0.29.14
pytest           : 5.2.4
hypothesis       : None
sphinx           : None
blosc            : None
feather          : None
xlsxwriter       : None
lxml.etree       : 4.4.2
html5lib         : 1.0.1
pymysql          : None
psycopg2         : None
jinja2           : 2.10.3
IPython          : 7.11.1
pandas_datareader: None
bs4              : None
bottleneck       : None
fastparquet      : None
gcsfs            : None
lxml.etree       : 4.4.2
matplotlib       : 3.1.2
numexpr          : None
odfpy            : None
openpyxl         : None
pandas_gbq       : None
pyarrow          : None
pytables         : None
pytest           : 5.2.4
pyxlsb           : None
s3fs             : None
scipy            : 1.3.2
sqlalchemy       : 1.3.11
tables           : None
tabulate         : None
xarray           : None
xlrd             : 1.2.0
xlwt             : None
xlsxwriter       : None
numba            : None
</details>

Hints from issue discussion:
take

Commit message: Code review feedback: don't even try to convert bool to date

Patch to review:
diff --git a/doc/source/whatsnew/v1.1.0.rst b/doc/source/whatsnew/v1.1.0.rst
index 6f2b9b4f946c7..840571c21da50 100644
--- a/doc/source/whatsnew/v1.1.0.rst
+++ b/doc/source/whatsnew/v1.1.0.rst
@@ -429,6 +429,7 @@ I/O
 - Bug in :meth:`read_sas` was raising an ``AttributeError`` when reading files from Google Cloud Storage (issue:`33069`)
 - Bug in :meth:`DataFrame.to_sql` where an ``AttributeError`` was raised when saving an out of bounds date (:issue:`26761`)
 - Bug in :meth:`read_excel` did not correctly handle multiple embedded spaces in OpenDocument text cells. (:issue:`32207`)
+- Bug in :meth:`read_json` was raising ``TypeError`` when reading a list of booleans into a Series. (:issue:`31464`)
 
 Plotting
 ^^^^^^^^
diff --git a/pandas/io/json/_json.py b/pandas/io/json/_json.py
index 886387a7a9fe6..4b819606d5bc9 100644
--- a/pandas/io/json/_json.py
+++ b/pandas/io/json/_json.py
@@ -978,6 +978,10 @@ def _try_convert_to_date(self, data):
             if not in_range.all():
                 return data, False
 
+        # ignore bool
+        if new_data.dtype == "bool":
+            return data, False
+
         date_units = (self.date_unit,) if self.date_unit else self._STAMP_UNITS
         for date_unit in date_units:
             try:
diff --git a/pandas/tests/io/json/test_readlines.py b/pandas/tests/io/json/test_readlines.py
index e531457627342..ebc28a8351d7f 100644
--- a/pandas/tests/io/json/test_readlines.py
+++ b/pandas/tests/io/json/test_readlines.py
@@ -179,3 +179,9 @@ def test_readjson_unicode(monkeypatch):
         result = read_json(path)
         expected = pd.DataFrame({"£©µÀÆÖÞßéöÿ": ["АБВГДабвгд가"]})
         tm.assert_frame_equal(result, expected)
+
+
+def test_readjson_bool_series():
+    result = read_json("[true, true, false]", typ="series")
+    expected = pd.Series([True, True, False])
+    tm.assert_series_equal(result, expected)


Code review comments:
diff_hunk: @@ -978,6 +978,10 @@ def _try_convert_to_date(self, data):
             if not in_range.all():
                 return data, False
 
+        # ignore bool
+        if new_data.dtype == "bool":
+            return data, False
path: pandas/io/json/_json.py
text: @user1:
can you add a comment inside the if `# GH#33373 ignore bool`

@author:
Done. Test failures appear to be unrelated; the FDIC site that the html tests use is returning intermittent 404s.

@user2:
Hmm actually not sure about doing it this way; we've done this type of checking in groupby and its not a great precedent. We do already catch the three errors a few lines above this - shouldn't we just do the same here?

@author:
The first approach seems a bit simpler to me—this seems to be a case where whoever put the TypeError into the other except clauses accidentally omitted it from the one that I originally added it to. Let me know what you and @user1 decide and I’ll be happy to update accordingly.

@user1:
im happy to defer to @user2 in this part of the code.  catching TypeError is definitely more idiomatic python, but we've definitely seen how catching too much has let bugs remain un-surfaced for a long time elsewhere

@user2:
Yea I think should catch the TypeError as done originally; would match what we have on L968 anyway

@author:
Sounds good -- pushed the revert back to the original change. Thanks for the quick turnaround -- let me know if anything else needed.