Repository: pandas-dev/pandas
Pull request: BUG: IntegerArray.astype(boolean)
Closes https://github.com/pandas-dev/pandas/issues/31102

Problem statement:
BUG: Cannot astype from IntegerArray to BooleanArray with missing values
For the boolean -> integer array conversion, I added a special path for that. But `astype`ing integer to boolean is currently not working:

```
In [23]: a = pd.array([1, 0, pd.NA])  

In [24]: a  
Out[24]: 
<IntegerArray>
[1, 0, <NA>]
Length: 3, dtype: Int64

In [25]: a.astype("boolean")  
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-25-41973ed53ee3> in <module>
----> 1 a.astype("boolean")

~/scipy/pandas/pandas/core/arrays/integer.py in astype(self, dtype, copy)
    454             kwargs = {}
    455 
--> 456         data = self.to_numpy(dtype=dtype, **kwargs)
    457         return astype_nansafe(data, dtype, copy=False)
    458 

~/scipy/pandas/pandas/core/arrays/masked.py in to_numpy(self, dtype, copy, na_value)
    124             ):
    125                 raise ValueError(
--> 126                     f"cannot convert to '{dtype}'-dtype NumPy array "
    127                     "with missing values. Specify an appropriate 'na_value' "
    128                     "for this dtype."

ValueError: cannot convert to 'boolean'-dtype NumPy array with missing values. Specify an appropriate 'na_value' for this dtype.

In [26]: a.astype(pd.BooleanDtype()) 
...
ValueError: cannot convert to 'boolean'-dtype NumPy array with missing values. Specify an appropriate 'na_value' for this dtype.
```

while for conversions to other nullable dtypes, this should be possible.

Hints from issue discussion:
Will we treat all non-zero values (other than NA) as True? We should match the behavior in the constructor.
Good question. I suppose yes? That's what numpy also does. In the constructor, we are actually a bit more conservative, and only allow 0/1/NA-like:

```
In [30]: pd.array([0, 1], dtype="boolean")
Out[30]: 
<BooleanArray>
[False, True]
Length: 2, dtype: boolean

In [31]: pd.array([0, 1, 2], dtype="boolean")
...
TypeError: Need to pass bool-like values
```

Commit message: BUG: IntegerArray.astype(boolean)

Closes https://github.com/pandas-dev/pandas/issues/31102

Patch to review:
diff --git a/pandas/core/arrays/integer.py b/pandas/core/arrays/integer.py
index 67036761bc62a..022e6a7322872 100644
--- a/pandas/core/arrays/integer.py
+++ b/pandas/core/arrays/integer.py
@@ -19,6 +19,7 @@
     is_list_like,
     is_object_dtype,
     is_scalar,
+    pandas_dtype,
 )
 from pandas.core.dtypes.dtypes import register_extension_dtype
 from pandas.core.dtypes.missing import isna
@@ -440,11 +441,17 @@ def astype(self, dtype, copy=True):
             if incompatible type with an IntegerDtype, equivalent of same_kind
             casting
         """
+        from pandas.core.arrays.boolean import BooleanArray, BooleanDtype
+
+        dtype = pandas_dtype(dtype)
 
         # if we are astyping to an existing IntegerDtype we can fastpath
         if isinstance(dtype, _IntegerDtype):
             result = self._data.astype(dtype.numpy_dtype, copy=False)
             return type(self)(result, mask=self._mask, copy=False)
+        elif isinstance(dtype, BooleanDtype):
+            result = self._data.astype("bool", copy=False)
+            return BooleanArray(result, mask=self._mask, copy=False)
 
         # coerce
         if is_float_dtype(dtype):
diff --git a/pandas/tests/arrays/test_integer.py b/pandas/tests/arrays/test_integer.py
index 9cb1e4176df96..ec462e7a63a80 100644
--- a/pandas/tests/arrays/test_integer.py
+++ b/pandas/tests/arrays/test_integer.py
@@ -680,6 +680,13 @@ def test_astype_str(self):
         tm.assert_numpy_array_equal(a.astype(str), expected)
         tm.assert_numpy_array_equal(a.astype("str"), expected)
 
+    def test_astype_boolean(self):
+        # https://github.com/pandas-dev/pandas/issues/31102
+        a = pd.array([1, 0, None], dtype="Int64")
+        result = a.astype("boolean")
+        expected = pd.array([True, False, None], dtype="boolean")
+        tm.assert_extension_array_equal(result, expected)
+
 
 def test_frame_repr(data_missing):
 


Code review comments:
diff_hunk: @@ -680,6 +680,13 @@ def test_astype_str(self):
         tm.assert_numpy_array_equal(a.astype(str), expected)
         tm.assert_numpy_array_equal(a.astype("str"), expected)
 
+    def test_astype_boolean(self):
+        # https://github.com/pandas-dev/pandas/issues/31102
+        a = pd.array([1, 0, None], dtype="Int64")
path: pandas/tests/arrays/test_integer.py
text: @author:
I've deliberately tested only 0 and 1 here. We should decide on a behavior for other values. Are all non-zero, non-NA values allowed and considered True?

Right now, BooleanArray._from_sequence doesn't allow other values

```python
In [2]: pd.array([1, 2], dtype='boolean')
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-2-2a7683b5ab7f> in <module>
----> 1 pd.array([1, 2], dtype='boolean')

~/sandbox/pandas/pandas/core/construction.py in array(data, dtype, copy)
    282     if is_extension_array_dtype(dtype):
    283         cls = cast(ExtensionDtype, dtype).construct_array_type()
--> 284         return cls._from_sequence(data, dtype=dtype, copy=copy)
    285
    286     if dtype is None:

~/sandbox/pandas/pandas/core/arrays/boolean.py in _from_sequence(cls, scalars, dtype, copy)
    284         if dtype:
    285             assert dtype == "boolean"
--> 286         values, mask = coerce_to_array(scalars, copy=copy)
    287         return BooleanArray(values, mask)
    288

~/sandbox/pandas/pandas/core/arrays/boolean.py in coerce_to_array(values, mask, copy)
    169                 == values_object[~mask_values].astype(float)
    170             ):
--> 171                 raise TypeError("Need to pass bool-like values")
    172
    173     if mask is None and mask_values is None:

TypeError: Need to pass bool-like values
```

@user1:
I would think want to follow Python semantics (?) but probably solved separate from this

@user2:
I think there should be *some* way to go from "potentially not only 0's and 1's" to boolean, so I am fine here to follow Python and Numpy behavior for conversion (`bool(..)`, numpy's `astype`).

The question then is: should we follow this in the constructor, or are we fine with being more strict there? (I am personally fine with being more strict)

@author:
OK, added a test for non 0 / 1 values.