Repository: pandas-dev/pandas
Pull request: BUG: AttributeError: 'BooleanArray' object has no attribute 'sum' while infer types #44079
- [x] closes #44079
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [x] whatsnew entry


Problem statement:
BUG: AttributeError: 'BooleanArray' object has no attribute 'sum' while infer types
Line 694 in the pandas/io/parsers/base_parser.py file, 
```
        if issubclass(values.dtype.type, (np.number, np.bool_)):
            # error: Argument 2 to "isin" has incompatible type "List[Any]"; expected
            # "Union[Union[ExtensionArray, ndarray], Index, Series]"
            mask = algorithms.isin(values, list(na_values))  # type: ignore[arg-type]
            na_count =mask.sum()
            ^^^^^^^^^^^^^THIS
            if na_count > 0:
                if is_integer_dtype(values):
                    values = values.astype(np.float64)
                np.putmask(values, mask, np.nan)
            return values, na_count
```

Caused an "AttributeError: 'BooleanArray' object has no attribute 'sum'" following some global ananconda update. Changed the line to 
`            na_count =mask.astype('uint8').sum()`

fixed the error in my case.

Edit by rhshadrach:

Minimal example

```
from io import StringIO

import pandas as pd

df = pd.read_csv(
     StringIO("0,1"),
     names=['a', 'b'],
     index_col=['a'],
     dtype={'a': 'UInt8'},
)
```

Hints from issue discussion:
Can you post a reproducible example along with the version of pandas you're using.
I'm having the same problem but from another place, it happens when I use `read_csv` to read a CSV file that has multiple indexes whose columns use the `UInt8` dtype, although it also happens with `UInt32`; I don't know if the same will happen with the other numeric dtypes.

This is the code that I'm using:

```Python console
>>> import pandas as pd
>>> df = pd.read_csv(
...     'geo20210715_pp.txt',
...     names=[
...         'cod_est', 'cod_mun', 'cod_par',
...         'nombre_est', 'nombre_mun', 'nombre_par'
...     ],
...     index_col=['cod_est', 'cod_mun', 'cod_par'],
...     dtype={
...         'cod_est': 'UInt8', 'cod_mun': 'UInt8', 'cod_par': 'UInt8',
...         'nombre_est': 'string', 'nombre_mun': 'string',
...         'nombre_par': 'string'
...     },
...     engine='c',
...     encoding='cp1252'
... )
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.9/site-packages/pandas/util/_decorators.py", line 311, in wrapper
    return func(*args, **kwargs)
  File "/usr/lib/python3.9/site-packages/pandas/io/parsers/readers.py", line 586, in read_csv
    return _read(filepath_or_buffer, kwds)
  File "/usr/lib/python3.9/site-packages/pandas/io/parsers/readers.py", line 488, in _read
    return parser.read(nrows)
  File "/usr/lib/python3.9/site-packages/pandas/io/parsers/readers.py", line 1047, in read
    index, columns, col_dict = self._engine.read(nrows)
  File "/usr/lib/python3.9/site-packages/pandas/io/parsers/c_parser_wrapper.py", line 309, in read
    index, names = self._make_index(data, alldata, names)
  File "/usr/lib/python3.9/site-packages/pandas/io/parsers/base_parser.py", line 416, in _make_index
    index = self._agg_index(index)
  File "/usr/lib/python3.9/site-packages/pandas/io/parsers/base_parser.py", line 512, in _agg_index
    arr, _ = self._infer_types(arr, col_na_values | col_na_fvalues)
  File "/usr/lib/python3.9/site-packages/pandas/io/parsers/base_parser.py", line 695, in _infer_types
    na_count = mask.sum()
AttributeError: 'BooleanArray' object has no attribute 'sum'
``` 

And this is the CSV file (don't worry, is public information): [geo20210715_pp.txt](https://github.com/pandas-dev/pandas/files/7524090/geo20210715_pp.txt).

My way for circumventing it is this:

```Python console
>>> import pandas as pd
>>> df = pd.read_csv(
...     '/home/galaxyljgd/Documentos/otros/registro_electoral_2021/geo20210715_pp.txt',
...     names=[
...         'cod_est', 'cod_mun', 'cod_par',
...         'nombre_est', 'nombre_mun', 'nombre_par'
...     ],
...     dtype={
...         'cod_est': 'UInt8', 'cod_mun': 'UInt8', 'cod_par': 'UInt8',
...         'nombre_est': 'string', 'nombre_mun': 'string',
...         'nombre_par': 'string'
...     },
...     engine='c',
...     encoding='cp1252'
... )
>>> df.set_index(['cod_est', 'cod_mun', 'cod_par'], inplace=True)
>>> df
                           nombre_est      nombre_mun                 nombre_par
cod_est cod_mun cod_par                                                         
21      1       3          EDO. ZULIA      MP. BARALT   PQ. MANUEL GUANIPA MATOS
                4          EDO. ZULIA      MP. BARALT      PQ. MARCELINO BRICEÑO
                5          EDO. ZULIA      MP. BARALT            PQ. SAN TIMOTEO
                6          EDO. ZULIA      MP. BARALT           PQ. PUEBLO NUEVO
        2       1          EDO. ZULIA  MP. SANTA RITA  PQ. PEDRO LUCAS URRIBARRI
...                               ...             ...                        ...
6       1       11       EDO. BOLIVAR      MP. CARONI             PQ. 5 DE JULIO
99      13      4            EMBAJADA          CANADA                  VANCOUVER
        97      1            EMBAJADA        JORDANIA                      AMMAN
        98      1            EMBAJADA          CHIPRE                    NICOSIA
        99      1            EMBAJADA          SERBIA                   BELGRADO

[1394 rows x 3 columns]
```
Thanks @LawrenceJGD, I've updated the OP with a minimal example and confirmed this exists on master. Investigations and PRs to fix are most welcome!
take

Commit message: BUG GH44079 added whatsnew note

Patch to review:
diff --git a/doc/source/whatsnew/v1.4.0.rst b/doc/source/whatsnew/v1.4.0.rst
index a593a03de5c25..aee28faad0dfe 100644
--- a/doc/source/whatsnew/v1.4.0.rst
+++ b/doc/source/whatsnew/v1.4.0.rst
@@ -639,6 +639,7 @@ I/O
 - Bug in :func:`read_csv` with :code:`float_precision="round_trip"` which did not skip initial/trailing whitespace (:issue:`43713`)
 - Bug in dumping/loading a :class:`DataFrame` with ``yaml.dump(frame)`` (:issue:`42748`)
 - Bug in :func:`read_csv` raising ``ValueError`` when ``parse_dates`` was used with ``MultiIndex`` columns (:issue:`8991`)
+- Bug in :func:`read_csv` raising ``AttributeError`` when attempting to read .csv files and infer index column dtype (:issue:`44079`)
 -
 
 Period
diff --git a/pandas/io/parsers/base_parser.py b/pandas/io/parsers/base_parser.py
index 42b9c8c9f10fe..44156d9aef054 100644
--- a/pandas/io/parsers/base_parser.py
+++ b/pandas/io/parsers/base_parser.py
@@ -700,7 +700,7 @@ def _infer_types(self, values, na_values, try_num_bool=True):
             # error: Argument 2 to "isin" has incompatible type "List[Any]"; expected
             # "Union[Union[ExtensionArray, ndarray], Index, Series]"
             mask = algorithms.isin(values, list(na_values))  # type: ignore[arg-type]
-            na_count = mask.sum()
+            na_count = mask.astype("uint8", copy=False).sum()
             if na_count > 0:
                 if is_integer_dtype(values):
                     values = values.astype(np.float64)
diff --git a/pandas/tests/io/parser/test_index_col.py b/pandas/tests/io/parser/test_index_col.py
index 646cb2029919d..26e3e9c182b42 100644
--- a/pandas/tests/io/parser/test_index_col.py
+++ b/pandas/tests/io/parser/test_index_col.py
@@ -297,3 +297,24 @@ def test_multiindex_columns_index_col_with_data(all_parsers):
         index=Index(["data"]),
     )
     tm.assert_frame_equal(result, expected)
+
+
+@skip_pyarrow
+def test_infer_types_boolean_sum(all_parsers):
+    # GH#44079
+    parser = all_parsers
+    result = parser.read_csv(
+        StringIO("0,1"),
+        names=["a", "b"],
+        index_col=["a"],
+        dtype={"a": "UInt8"},
+    )
+    expected = DataFrame(
+        data={
+            "a": [
+                0,
+            ],
+            "b": [1],
+        }
+    ).set_index("a")
+    tm.assert_frame_equal(result, expected, check_index_type=False)


Code review comments:
diff_hunk: @@ -639,6 +639,7 @@ I/O
 - Bug in :func:`read_csv` with :code:`float_precision="round_trip"` which did not skip initial/trailing whitespace (:issue:`43713`)
 - Bug in dumping/loading a :class:`DataFrame` with ``yaml.dump(frame)`` (:issue:`42748`)
 - Bug in :func:`read_csv` raising ``ValueError`` when ``parse_dates`` was used with ``MultiIndex`` columns (:issue:`8991`)
+- Bug in :func:`read_csv` raising ``AttributeError`` when attempting to read .csv files and infer index column dtype (:issue:`44079`)
path: doc/source/whatsnew/v1.4.0.rst
text: @user1:
this needs to specifically mention using an nullable integer type (as this is not generic rather a specific thing)

@author:
done