BUG: Series construction with EA dtype and index but no data fails
State: closed

- [ ] closes #26469
- [ ] closes #33559
- [ ] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Bug
name: ExtensionArray
description: Extending pandas with custom dtypes or arrays.
name: Constructors
description: Series/DataFrame/Index/pd.array Constructors

Comments:
Maybe a test in tests.extension?
> Maybe a test in tests.extension?

Series construction passes through `_from_sequence` but this PR is not changing the EAs directly since the EA interface doesn't require accepting a scalar for `_from_sequence`, see @jorisvandenbossche comment https://github.com/pandas-dev/pandas/issues/26469#issue-446047785

> The problem is that the None / np.nan is eventually passed to the _from_sequence method, which expects a list-like. I don't think we should let people fix their _from_sequence, but we should rather make sure we never pass that to it.

so not sure what tests could be added to extension tests since we have a code check rule:

Check for the following code in the extension array base tests: `tm.assert_frame_equal` and `tm.assert_series_equal`

and this PR is about Series construction.

could maybe add more parameterisation to the Series test? Do we have a fixture for internal EA dtypes?
> so not sure what tests could be added to extension tests since we have a code check rule:
> 
> Check for the following code in the extension array base tests: `tm.assert_frame_equal` and `tm.assert_series_equal`

ah, using tm instead of self.

will add tests to extension tests and remove this IntegerArray test from pandas/tests/series/test_constructors.py
The errors seem to indicate that we pass the scalar to the ExtensionArray constructor. But it should maybe the responsibility of the Series constructor to convert a scalar to something like `[scalar] * len` before passing it to the EA constructor?
> something like `[scalar] * len` before passing it to the EA constructor

Thats what i was thinking, yah
> > something like `[scalar] * len` before passing it to the EA constructor
> 
> Thats what i was thinking, yah

That would be inconsistent with how non-EA series are constructed. Are we OK with that?

IIUC `sanitize_array` broadcasts the single element array to match the index. 

the traceback in the issue OP is no longer relevant. on master..

```
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-3-bed90de02f57> in <module>
----> 1 pd.Series(pd.NA, index=[1, 2, 3], dtype="Int64")

c:\users\simon\pandas\pandas\core\series.py in __init__(self, data, index, dtype, name, copy, fastpath)
    327                     data = data.copy()
    328             else:
--> 329                 data = sanitize_array(data, index, dtype, copy, raise_cast_failure=True)
    330 
    331                 data = SingleBlockManager.from_array(data, index)

c:\users\simon\pandas\pandas\core\construction.py in sanitize_array(data, index, dtype, copy, raise_cast_failure)
    452         raise TypeError("Set type is unordered")
    453     else:
--> 454         subarr = _try_cast(data, dtype, copy, raise_cast_failure)
    455 
    456     # scalar like, GH

c:\users\simon\pandas\pandas\core\construction.py in _try_cast(arr, dtype, copy, raise_cast_failure)
    535         # DatetimeTZ case needs to go through maybe_cast_to_datetime
    536         array_type = dtype.construct_array_type()._from_sequence
--> 537         subarr = array_type(arr, dtype=dtype, copy=copy)
    538         return subarr
    539 

c:\users\simon\pandas\pandas\core\arrays\integer.py in _from_sequence(cls, scalars, dtype, copy)
    353     @classmethod
    354     def _from_sequence(cls, scalars, dtype=None, copy: bool = False) -> "IntegerArray":
--> 355         return integer_array(scalars, dtype=dtype, copy=copy)
    356 
    357     @classmethod

c:\users\simon\pandas\pandas\core\arrays\integer.py in integer_array(values, dtype, copy)
    144     TypeError if incompatible types
    145     """
--> 146     values, mask = coerce_to_array(values, dtype=dtype, copy=copy)
    147     return IntegerArray(values, mask)
    148 

c:\users\simon\pandas\pandas\core\arrays\integer.py in coerce_to_array(values, dtype, mask, copy)
    219         inferred_type = lib.infer_dtype(values, skipna=True)
    220         if inferred_type == "empty":
--> 221             values = np.empty(len(values))
    222             values.fill(np.nan)
    223         elif inferred_type not in [

TypeError: len() of unsized object
```
Not directly related to this PR, but may offer chance of avoiding `_from_sequence` and the constraints of the EA api is to investigate making pd.array constructor consistent with numpy. Not looked into this, so not sure at this stage whether the same constraints would apply.

```
>>> import numpy as np
>>> import pandas as pd
>>>
>>> pd.__version__
'1.1.0.dev0+1422.gb8c895429'
>>>
>>> np.array(1, dtype="int64")
array(1, dtype=int64)
>>>
>>> pd.array(1, dtype="Int64")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\simon\pandas\pandas\core\construction.py", line 270, in array
    raise ValueError(msg)
ValueError: Cannot pass scalar '1' to 'pandas.array'.
>>>
```
> That would be inconsistent with how non-EA series are constructed. Are we OK with that?

actually atm this 'fix' is not 100% consistent

for `pd.Series(5, index=[1, 2, 3], dtype="int")` the scalar value 5 is passed to _try_cast and then on L563 construct_1d_ndarray_preserving_na is called with the scalar value. The return value is actually a 0d array contrary to the function name. 

```
>>> pd.core.dtypes.cast.construct_1d_ndarray_preserving_na(5, np.dtype("int"))
array(5)
```
in the current 'fix' a 1d EA is returned from _try_cast, so not 100% consistent with non-EA types. so maybe https://github.com/pandas-dev/pandas/pull/33846#issuecomment-621368303 may need to be fixed first.

L538 could be changed to continue and a construct_1d_pdarray_preserving_na created to duplicate
 the behaviour of construct_1d_ndarray_preserving_na for EA types. wdyt? 
> `>>> pd.array(1, dtype="Int64")`

pd.array is strictly 1D only, while your numpy example (`np.array(1, dtype="int64")`) is creating a 0D array. So for this case we don't need consistency with numpy IMO

> > something like `[scalar] * len` before passing it to the EA constructor
> 
> Thats what i was thinking, yah

try_cast returns an array. with non-EA dtypes it's a 0-d array and then in sanitize_array, the `if getattr(subarr, "ndim", 0) == 0:` broadcasts the scalar to the same length as the index if passed.

whereas atm, this 'fix' returns a 1-d array (we don't want to implement 0-d arrays for EA, https://github.com/pandas-dev/pandas/pull/33846#issuecomment-621447971) and then in

```
    elif subarr.ndim == 1:
        if index is not None:

            # a 1-element ndarray
            if len(subarr) != len(index) and len(subarr) == 1:
                subarr = construct_1d_arraylike_from_scalar(
                    subarr[0], len(index), subarr.dtype
                )
```
we create the correct length array from the single element EA array returned from _try_cast.

so if this fix is not suitable, I see a few other options.

1. return a scalar from try_cast and then additional checks on the return type in sanitize_array
2. pass the index through to try_cast so that the suggestion of  doing `[scalar] * len` before passing it to the EA constructor
3. avoid the call to try_cast for a scalar and an EA type.

I think that union return types (1) should be avoided, passing additional keywords (2) violates the single responsibility principle and additional special casing (3) should be avoided.

any other suggestions?

another constructor case fixed by this branch for consistency with numpy types.

```
>>> pd.Series(1, index=['foo'], dtype="int")
foo    1
dtype: int32
>>>
>>> pd.Series(1, index=['foo'], dtype="Int64")
foo    1
dtype: Int64
>>>
```

on master
```
>>> pd.Series(1, index=['foo'], dtype="int")
foo    1
dtype: int32
>>>
>>> pd.Series(1, index=['foo'], dtype="Int64")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\simon\pandas\pandas\core\series.py", line 329, in __init__
    data = sanitize_array(data, index, dtype, copy, raise_cast_failure=True)
  File "C:\Users\simon\pandas\pandas\core\construction.py", line 454, in sanitize_array
    subarr = _try_cast(data, dtype, copy, raise_cast_failure)
  File "C:\Users\simon\pandas\pandas\core\construction.py", line 537, in _try_cast
    subarr = array_type(arr, dtype=dtype, copy=copy)
  File "C:\Users\simon\pandas\pandas\core\arrays\integer.py", line 355, in _from_sequence
    return integer_array(scalars, dtype=dtype, copy=copy)
  File "C:\Users\simon\pandas\pandas\core\arrays\integer.py", line 146, in integer_array
    values, mask = coerce_to_array(values, dtype=dtype, copy=copy)
  File "C:\Users\simon\pandas\pandas\core\arrays\integer.py", line 244, in coerce_to_array
    raise TypeError("values must be a 1D list-like")
TypeError: values must be a 1D list-like
>>>
```

so could add some more test cases.
hmm, as an aside, lib.is_scalar does not consider numpy 0-d array as a scalar

on master.
```
>>> pd.Series(np.array(1, dtype="int64"))
0    1
dtype: object
>>>
>>> pd.Series(np.array(1, dtype="int64"), index=[1, 2, 3])
1    1
2    1
3    1
dtype: int64
>>>
>>> pd.Series(np.array(1, dtype="int64"), index=[1, 2, 3], dtype="Int64")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\simon\pandas\pandas\core\series.py", line 329, in __init__
    data = sanitize_array(data, index, dtype, copy, raise_cast_failure=True)
  File "C:\Users\simon\pandas\pandas\core\construction.py", line 427, in sanitize_array
    subarr = _try_cast(data, dtype, copy, raise_cast_failure)
  File "C:\Users\simon\pandas\pandas\core\construction.py", line 537, in _try_cast
    subarr = array_type(arr, dtype=dtype, copy=copy)
  File "C:\Users\simon\pandas\pandas\core\arrays\integer.py", line 355, in _from_sequence
    return integer_array(scalars, dtype=dtype, copy=copy)
  File "C:\Users\simon\pandas\pandas\core\arrays\integer.py", line 146, in integer_array
    values, mask = coerce_to_array(values, dtype=dtype, copy=copy)
  File "C:\Users\simon\pandas\pandas\core\arrays\integer.py", line 244, in coerce_to_array
    raise TypeError("values must be a 1D list-like")
TypeError: values must be a 1D list-like
>>>
```
the first case results in an object dtype (bug in master?). this PR dosn't fix the last case to make it consistent with case 2
> hmm, as an aside, lib.is_scalar does not consider numpy 0-d array as a scalar
> 
> on master.
> 
> ```
> >>> pd.Series(np.array(1, dtype="int64"))
> 0    1
> dtype: object
> >>>
> >>> pd.Series(np.array(1, dtype="int64"), index=[1, 2, 3])
> 1    1
> 2    1
> 3    1
> dtype: int64
> >>>
> >>> pd.Series(np.array(1, dtype="int64"), index=[1, 2, 3], dtype="Int64")
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
>   File "C:\Users\simon\pandas\pandas\core\series.py", line 329, in __init__
>     data = sanitize_array(data, index, dtype, copy, raise_cast_failure=True)
>   File "C:\Users\simon\pandas\pandas\core\construction.py", line 427, in sanitize_array
>     subarr = _try_cast(data, dtype, copy, raise_cast_failure)
>   File "C:\Users\simon\pandas\pandas\core\construction.py", line 537, in _try_cast
>     subarr = array_type(arr, dtype=dtype, copy=copy)
>   File "C:\Users\simon\pandas\pandas\core\arrays\integer.py", line 355, in _from_sequence
>     return integer_array(scalars, dtype=dtype, copy=copy)
>   File "C:\Users\simon\pandas\pandas\core\arrays\integer.py", line 146, in integer_array
>     values, mask = coerce_to_array(values, dtype=dtype, copy=copy)
>   File "C:\Users\simon\pandas\pandas\core\arrays\integer.py", line 244, in coerce_to_array
>     raise TypeError("values must be a 1D list-like")
> TypeError: values must be a 1D list-like
> >>>
> ```
> 
> the first case results in an object dtype (bug in master?). this PR dosn't fix the last case to make it consistent with case 2

correct, these need to be unwrapped (easiest), e.g. your check I think will work like this
```
if lib.is_scalar(data) or hasattr(data, 'ndim') and getattr(data, 'ndim') == 0:
```
lgtm. can merge on green.