Repository: pandas-dev/pandas

Issue:
ExtensionArray constructor issue
I don't think that pandas should make any assumptions on how subclasses `__init__` method. Currently we assume that the first positional argument accepts an instance of the class or a sequence of the class's scalar type.

I'd rather break that into two distinct methods.

```python
@classmedthod
def from_extension_array(cls, extension_array: ExtensionArray) -> ExtensionArray:
    """Construct a new ExtensionArray from an instance of the same type

    Parameters
    ----------
    extension_array : ExtensionArray
        An instance of 'cls'

    Returns
    -------
    ExtensionArray
    """


ScalarType = ExtensionArray.dtype.type

@classmethod
def from_scalars(cls, scalars: Sequence[ScalarType]) -> ExtensionArray:
    """Construct a new ExtensionArray from a sequence of the scalar type

    Parameters
    ----------
    scalars : Sequence[ScalarType]
        A sequence of cls.dtype.type, the scalar type for this array

    Returns
    -------
    ExtensionArray
    """
```

I think these should be abstract. For many subclasses, a simple `cls(arg)` should suffice.


Discussion:
what is the reason for this?
In case a subclass has a different `__init__` signature.
\+1 

I only would make them private `_from_scalars` and `_from_extension_array`, to give the extension author to give them more meaningful public names if they want to add something like that (eg in geopandas we currently call 'from_scalars' `from_shapely`). 

Also we could here make clear expectations about data being copied or not (and for that, maybe add a `copy` keyword.
> Also we could here make clear expectations about data being copied or not

Yes, I thought about making that part of the `_from_extension_array` one, but I wasn't sure if all subclasses would be able to honor it, and what implications that would have for pandas using it, since we couldn't really *rely* on it. But if it's just a performance thing, subclasses maybe having to copy, even with `copy=Falsse`, isn't such a big deal.


I think the fact we cannot rely upon is if it always can be done *without* copy, but we can rely on *requiring* a copy? (similar to the `copy` keyword in `np.array`)
> similar to the copy keyword in np.array

Yes, that sounds right.

Also +1 to making them private

Are there any other cases we should consider? In cyberpandas I have two others

- `_from_ndarray`: https://github.com/ContinuumIO/cyberpandas/blob/e9e8f195601e188e58e3d6a2038f7e0e28c1737e/cyberpandas/ip_array.py#L220 That's a zero-copy constructor from a NumPy structured array with the correct fields. Too specific for pandas
- `from_bytes`: from a buffer of bytes. Way too specific to IP Addresses.
and we *already* have and use ``_construtor`` in pandas proper. Having alternative constructors is fine, except we have no way to actively select when / how one is called. So this is a *huge* break with current pracitce. I am aa huge -1 on doing this w/o actually changing pandas to do this. I don't think this is possible.
This is as much (or more so) for external code as it is for pandas. We can't impose the same restrictions on 3rd party libraries as we do internally.

The use of these constructors will be limited. The only place returning `ExtensionArrays` will be in `ExtensionArray` itself. It's not something that's going to pollute all of our internals.
We are *adding* functionality here. There is no place in pandas where we currently call this for ExtensionArrays, so each call we add we can evaluate to see which one we should use. 

For example, `Categorical` (currenlty the only `ExtensionArray` in pandas) has a `_constructor`, but it is used in a very specific way (it is always used with passing `codes` (with `fastpath=True`)), so that is not something we could rely upon in general anyhow. I think it would be clearer if we had an internal `_from_codes` or `_constructor_from_codes` (with less checks than the public `from_codes`) instead of a `_constructor(..., fastpath=True)`
> The use of these constructors will be limited. The only place returning ExtensionArrays will be in ExtensionArray itself. It's not something that's going to pollute all of our internals.

Indeed, eg the `_from_scalars` would currently mainly be used in the tests (apart from the 'fallback' unique implementation)
I agree with this change.

Resolving patch:
diff --git a/pandas/core/arrays/base.py b/pandas/core/arrays/base.py
--- a/pandas/core/arrays/base.py
+++ b/pandas/core/arrays/base.py
@@ -18,6 +18,7 @@ class ExtensionArray(object):
     The interface includes the following abstract methods that must be
     implemented by subclasses:
 
+    * _constructor_from_sequence
     * __getitem__
     * __len__
     * dtype
@@ -56,6 +57,25 @@ class ExtensionArray(object):
     # '_typ' is for pandas.core.dtypes.generic.ABCExtensionArray.
     # Don't override this.
     _typ = 'extension'
+
+    # ------------------------------------------------------------------------
+    # Constructors
+    # ------------------------------------------------------------------------
+    @classmethod
+    def _constructor_from_sequence(cls, scalars):
+        """Construct a new ExtensionArray from a sequence of scalars.
+
+        Parameters
+        ----------
+        scalars : Sequence
+            Each element will be an instance of the scalar type for this
+            array, ``cls.dtype.type``.
+        Returns
+        -------
+        ExtensionArray
+        """
+        raise AbstractMethodError(cls)
+
     # ------------------------------------------------------------------------
     # Must be a Sequence
     # ------------------------------------------------------------------------
diff --git a/pandas/core/arrays/categorical.py b/pandas/core/arrays/categorical.py
--- a/pandas/core/arrays/categorical.py
+++ b/pandas/core/arrays/categorical.py
@@ -364,6 +364,10 @@ def __init__(self, values, categories=None, ordered=None, dtype=None,
         self._dtype = self._dtype.update_dtype(dtype)
         self._codes = coerce_indexer_dtype(codes, dtype.categories)
 
+    @classmethod
+    def _constructor_from_sequence(cls, scalars):
+        return cls(scalars)
+
     @property
     def categories(self):
         """The categories of this categorical.


Test patch:
diff --git a/pandas/tests/extension/base/constructors.py b/pandas/tests/extension/base/constructors.py
--- a/pandas/tests/extension/base/constructors.py
+++ b/pandas/tests/extension/base/constructors.py
@@ -9,6 +9,11 @@
 
 class BaseConstructorsTests(BaseExtensionTests):
 
+    def test_array_from_scalars(self, data):
+        scalars = [data[0], data[1], data[2]]
+        result = data._constructor_from_sequence(scalars)
+        assert isinstance(result, type(data))
+
     def test_series_constructor(self, data):
         result = pd.Series(data)
         assert result.dtype == data.dtype
diff --git a/pandas/tests/extension/decimal/array.py b/pandas/tests/extension/decimal/array.py
--- a/pandas/tests/extension/decimal/array.py
+++ b/pandas/tests/extension/decimal/array.py
@@ -32,6 +32,10 @@ def __init__(self, values):
 
         self.values = values
 
+    @classmethod
+    def _constructor_from_sequence(cls, scalars):
+        return cls(scalars)
+
     def __getitem__(self, item):
         if isinstance(item, numbers.Integral):
             return self.values[item]
diff --git a/pandas/tests/extension/json/array.py b/pandas/tests/extension/json/array.py
--- a/pandas/tests/extension/json/array.py
+++ b/pandas/tests/extension/json/array.py
@@ -33,11 +33,17 @@ def __init__(self, values):
                 raise TypeError
         self.data = values
 
+    @classmethod
+    def _constructor_from_sequence(cls, scalars):
+        return cls(scalars)
+
     def __getitem__(self, item):
         if isinstance(item, numbers.Integral):
             return self.data[item]
         elif isinstance(item, np.ndarray) and item.dtype == 'bool':
-            return type(self)([x for x, m in zip(self, item) if m])
+            return self._constructor_from_sequence([
+                x for x, m in zip(self, item) if m
+            ])
         else:
             return type(self)(self.data[item])
 
@@ -77,7 +83,7 @@ def isna(self):
     def take(self, indexer, allow_fill=True, fill_value=None):
         output = [self.data[loc] if loc != -1 else self._na_value
                   for loc in indexer]
-        return type(self)(output)
+        return self._constructor_from_sequence(output)
 
     def copy(self, deep=False):
         return type(self)(self.data[:])