NumPyBackedExtensionArray
State: closed

Adds a NumPyBckedExtensionArray, a thin wrapper around ndarray implementing
the EA interface.

We use this to ensure that `Series.array -> ExtensionArray`, rather than
a `Union[ndarray, ExtensionArray]`.

xref #23995 

---

cc @jreback @jorisvandenbossche @shoyer.

Some idle thoughts:

1. I'm not sure whether or not we should do this. In theory, I like the idea of `Series.array` always being an ExtensionArray. It gives us so much more freedom going forward.
2. This is probably bad, but I think unavoidable (for now)
```python
In [4]: ser = pd.Series([1, 2, 3])

In [5]: ser.array is ser.array
Out[5]: False
```
3. Should add a method for going from a NumPyBackedExtensionArray to an ndarray.
4. temporarily patching the Series /frame constructors to box all ndarrays in an ExtensionArray will be a good way to ensure that all of pandas supports EAs 
5. Should the Series / Index constructors unbox NumPyBackedExtensionArrays, so that you can't "accidentally" put one in? (I'm going to explore this quick).


TODO:

- [ ] name (NumPyBackedExtensionArray is a bit long)
- [ ] more docs
- [ ] add public API for the ndarray? Maybe `NumPyBackedExtensionArray.to_numpy()`?

Labels:
name: ExtensionArray
description: Extending pandas with custom dtypes or arrays.

Comments:
Hello @TomAugspurger! Thanks for updating the PR.







































Cheers ! There are no PEP8 issues in this Pull Request. :beers: 

##### Comment last updated on December 28, 2018 at 17:40 Hours UTC
this is ultimately going to be pretty non-performant, meaning now columns are separated. but it is where we are going. let's not do this for 0.24 :<
This is about all we need to ensure that a numpy EA never ends up in a Series / index under normal conditions. Doing frame now.

<details>

```diff
diff --git a/pandas/core/arrays/numpy_.py b/pandas/core/arrays/numpy_.py
index f9b88b325..67314826e 100644
--- a/pandas/core/arrays/numpy_.py
+++ b/pandas/core/arrays/numpy_.py
@@ -56,6 +56,7 @@ class NumPyExtensionDtype(ExtensionDtype):
 
 
 class NumPyExtensionArray(ExtensionArray, ExtensionOpsMixin):
+    _typ = "npy_extension"
     __array_priority__ = 1000
 
     def __init__(self, values):
diff --git a/pandas/core/dtypes/generic.py b/pandas/core/dtypes/generic.py
index 7a3ff5d29..7bdcdabef 100644
--- a/pandas/core/dtypes/generic.py
+++ b/pandas/core/dtypes/generic.py
@@ -67,7 +67,11 @@ ABCExtensionArray = create_pandas_abc_type("ABCExtensionArray", "_typ",
                                            ("extension",
                                             "categorical",
                                             "periodarray",
+                                            "npy_extension",
                                             ))
+ABCNumPyExtensionArray = create_pandas_abc_type("ABCNumPyExtensionArray",
+                                                "_typ",
+                                                ("npy_extension",))
 
 
 class _ABCGeneric(type):
diff --git a/pandas/core/indexes/base.py b/pandas/core/indexes/base.py
index 811d66c74..352c23189 100644
--- a/pandas/core/indexes/base.py
+++ b/pandas/core/indexes/base.py
@@ -27,7 +27,7 @@ import pandas.core.dtypes.concat as _concat
 from pandas.core.dtypes.generic import (
     ABCDataFrame, ABCDateOffset, ABCDatetimeIndex, ABCIndexClass,
     ABCMultiIndex, ABCPeriodIndex, ABCSeries, ABCTimedeltaArray,
-    ABCTimedeltaIndex)
+    ABCTimedeltaIndex, ABCNumPyExtensionArray)
 from pandas.core.dtypes.missing import array_equivalent, isna
 
 from pandas.core import ops
@@ -261,6 +261,9 @@ class Index(IndexOpsMixin, PandasObject):
                 return cls._simple_new(data, name)
 
         from .range import RangeIndex
+        if isinstance(data, ABCNumPyExtensionArray):
+            # ensure users don't accidentally put a NumPyEA in an index.
+            data = data._ndarray
 
         # range
         if isinstance(data, RangeIndex):
diff --git a/pandas/core/internals/construction.py b/pandas/core/internals/construction.py
index c43745679..f08fbe2e7 100644
--- a/pandas/core/internals/construction.py
+++ b/pandas/core/internals/construction.py
@@ -24,11 +24,13 @@ from pandas.core.dtypes.common import (
     is_integer_dtype, is_iterator, is_list_like, is_object_dtype, pandas_dtype)
 from pandas.core.dtypes.generic import (
     ABCDataFrame, ABCDatetimeIndex, ABCIndexClass, ABCPeriodIndex, ABCSeries,
-    ABCTimedeltaIndex)
+    ABCTimedeltaIndex, ABCNumPyExtensionArray)
 from pandas.core.dtypes.missing import isna
 
 from pandas.core import algorithms, common as com
-from pandas.core.arrays import Categorical, ExtensionArray, period_array
+from pandas.core.arrays import (
+    Categorical, ExtensionArray, period_array,
+)
 from pandas.core.index import (
     Index, _get_objs_combined_axis, _union_indexes, ensure_index)
 from pandas.core.indexes import base as ibase
@@ -577,6 +579,9 @@ def sanitize_array(data, index, dtype=None, copy=False,
             # we will try to copy be-definition here
             subarr = _try_cast(data, True, dtype, copy, raise_cast_failure)
 
+    elif isinstance(data, ABCNumPyExtensionArray):
+        # don't let people put NumPy EAs into Series.
+        subarr = data._ndarray
     elif isinstance(data, ExtensionArray):
         subarr = data
 
diff --git a/pandas/tests/indexes/test_base.py b/pandas/tests/indexes/test_base.py
index 2580a47e8..7c52a8a3e 100644
--- a/pandas/tests/indexes/test_base.py
+++ b/pandas/tests/indexes/test_base.py
@@ -260,6 +260,12 @@ class TestIndex(Base):
         with pytest.raises(ValueError, match=msg):
             Index(data, dtype=dtype)
 
+    def test_constructor_no_numpy_backed_ea(self):
+        ser = pd.Series([1, 2, 3])
+        result = pd.Index(ser.array)
+        expected = pd.Index([1, 2, 3])
+        tm.assert_index_equal(result, expected)
+
     @pytest.mark.parametrize("klass,dtype,na_val", [
         (pd.Float64Index, np.float64, np.nan),
         (pd.DatetimeIndex, 'datetime64[ns]', pd.NaT)
diff --git a/pandas/tests/series/test_constructors.py b/pandas/tests/series/test_constructors.py
index f5a445e2c..6b0f0b02e 100644
--- a/pandas/tests/series/test_constructors.py
+++ b/pandas/tests/series/test_constructors.py
@@ -21,6 +21,7 @@ from pandas import (
     Categorical, DataFrame, Index, IntervalIndex, MultiIndex, NaT, Series,
     Timestamp, date_range, isna, period_range, timedelta_range)
 from pandas.api.types import CategoricalDtype
+from pandas.core.internals.blocks import IntBlock
 from pandas.core.arrays import period_array
 import pandas.util.testing as tm
 from pandas.util.testing import assert_series_equal
@@ -1238,3 +1239,9 @@ class TestSeriesConstructors():
         result = Series(dt_list)
         expected = Series(dt_list, dtype=object)
         tm.assert_series_equal(result, expected)
+
+    def test_constructor_no_numpy_backed_ea(self):
+        ser = pd.Series([1, 2, 3])
+        result = pd.Series(ser.array)
+        tm.assert_series_equal(ser, result)
+        assert isinstance(result._data.blocks[0], IntBlock)
```

</details>

> but it is where we are going. let's not do this for 0.24 :<

Do you think it's likely to delay things? Since pandas isn't using `.array` in many places it's not touching that much.

# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/24227?src=pr&el=h1) Report
> Merging [#24227](https://codecov.io/gh/pandas-dev/pandas/pull/24227?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/c994e98c0e216622ad025c0c9e28421a4a3ee6af?src=pr&el=desc) will **decrease** coverage by `49.22%`.
> The diff coverage is `38.17%`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/24227/graphs/tree.svg?width=650&token=eZ4WkYLtcO&height=150&src=pr)](https://codecov.io/gh/pandas-dev/pandas/pull/24227?src=pr&el=tree)

```diff
@@             Coverage Diff             @@
##           master   #24227       +/-   ##
===========================================
- Coverage   92.21%   42.99%   -49.23%     
===========================================
  Files         162      163        +1     
  Lines       51763    51938      +175     
===========================================
- Hits        47733    22330    -25403     
- Misses       4030    29608    +25578
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `?` | |
| #single | `42.99% <38.17%> (-0.02%)` | :arrow_down: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/24227?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/core/internals/blocks.py](https://codecov.io/gh/pandas-dev/pandas/pull/24227/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW50ZXJuYWxzL2Jsb2Nrcy5weQ==) | `52.07% <0%> (-41.74%)` | :arrow_down: |
| [pandas/core/reshape/reshape.py](https://codecov.io/gh/pandas-dev/pandas/pull/24227/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvcmVzaGFwZS9yZXNoYXBlLnB5) | `13.62% <0%> (-85.94%)` | :arrow_down: |
| [pandas/core/arrays/\_\_init\_\_.py](https://codecov.io/gh/pandas-dev/pandas/pull/24227/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvYXJyYXlzL19faW5pdF9fLnB5) | `100% <100%> (ø)` | :arrow_up: |
| [pandas/core/arrays/categorical.py](https://codecov.io/gh/pandas-dev/pandas/pull/24227/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvYXJyYXlzL2NhdGVnb3JpY2FsLnB5) | `41.8% <100%> (-53.51%)` | :arrow_down: |
| [pandas/core/arrays/numpy\_.py](https://codecov.io/gh/pandas-dev/pandas/pull/24227/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvYXJyYXlzL251bXB5Xy5weQ==) | `36.74% <36.74%> (ø)` | |
| [pandas/core/base.py](https://codecov.io/gh/pandas-dev/pandas/pull/24227/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvYmFzZS5weQ==) | `31.77% <60%> (-65.87%)` | :arrow_down: |
| [pandas/core/dtypes/common.py](https://codecov.io/gh/pandas-dev/pandas/pull/24227/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZHR5cGVzL2NvbW1vbi5weQ==) | `72.72% <71.42%> (-22.91%)` | :arrow_down: |
| [pandas/io/formats/latex.py](https://codecov.io/gh/pandas-dev/pandas/pull/24227/diff?src=pr&el=tree#diff-cGFuZGFzL2lvL2Zvcm1hdHMvbGF0ZXgucHk=) | `0% <0%> (-100%)` | :arrow_down: |
| [pandas/core/categorical.py](https://codecov.io/gh/pandas-dev/pandas/pull/24227/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvY2F0ZWdvcmljYWwucHk=) | `0% <0%> (-100%)` | :arrow_down: |
| [pandas/io/sas/sas\_constants.py](https://codecov.io/gh/pandas-dev/pandas/pull/24227/diff?src=pr&el=tree#diff-cGFuZGFzL2lvL3Nhcy9zYXNfY29uc3RhbnRzLnB5) | `0% <0%> (-100%)` | :arrow_down: |
| ... and [127 more](https://codecov.io/gh/pandas-dev/pandas/pull/24227/diff?src=pr&el=tree-more) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/24227?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/24227?src=pr&el=footer). Last update [c994e98...a1fecf4](https://codecov.io/gh/pandas-dev/pandas/pull/24227?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/24227?src=pr&el=h1) Report
> Merging [#24227](https://codecov.io/gh/pandas-dev/pandas/pull/24227?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/ab55d05e7a12ebbccfd71511d36efa075fcfa0dc?src=pr&el=desc) will **increase** coverage by `<.01%`.
> The diff coverage is `93.87%`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/24227/graphs/tree.svg?width=650&token=eZ4WkYLtcO&height=150&src=pr)](https://codecov.io/gh/pandas-dev/pandas/pull/24227?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #24227      +/-   ##
==========================================
+ Coverage   92.29%    92.3%   +<.01%     
==========================================
  Files         163      165       +2     
  Lines       51948    52181     +233     
==========================================
+ Hits        47945    48165     +220     
- Misses       4003     4016      +13
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `90.72% <93.87%> (+0.01%)` | :arrow_up: |
| #single | `42.96% <35.91%> (-0.03%)` | :arrow_down: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/24227?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/core/base.py](https://codecov.io/gh/pandas-dev/pandas/pull/24227/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvYmFzZS5weQ==) | `97.7% <100%> (+0.02%)` | :arrow_up: |
| [pandas/core/internals/blocks.py](https://codecov.io/gh/pandas-dev/pandas/pull/24227/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW50ZXJuYWxzL2Jsb2Nrcy5weQ==) | `93.81% <100%> (-0.01%)` | :arrow_down: |
| [pandas/core/arrays/categorical.py](https://codecov.io/gh/pandas-dev/pandas/pull/24227/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvYXJyYXlzL2NhdGVnb3JpY2FsLnB5) | `95.47% <100%> (+0.12%)` | :arrow_up: |
| [pandas/core/reshape/reshape.py](https://codecov.io/gh/pandas-dev/pandas/pull/24227/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvcmVzaGFwZS9yZXNoYXBlLnB5) | `99.56% <100%> (ø)` | :arrow_up: |
| [pandas/core/internals/construction.py](https://codecov.io/gh/pandas-dev/pandas/pull/24227/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW50ZXJuYWxzL2NvbnN0cnVjdGlvbi5weQ==) | `96.66% <100%> (+0.01%)` | :arrow_up: |
| [pandas/core/arrays/\_\_init\_\_.py](https://codecov.io/gh/pandas-dev/pandas/pull/24227/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvYXJyYXlzL19faW5pdF9fLnB5) | `100% <100%> (ø)` | :arrow_up: |
| [pandas/core/dtypes/generic.py](https://codecov.io/gh/pandas-dev/pandas/pull/24227/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZHR5cGVzL2dlbmVyaWMucHk=) | `100% <100%> (ø)` | :arrow_up: |
| [pandas/core/indexes/base.py](https://codecov.io/gh/pandas-dev/pandas/pull/24227/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW5kZXhlcy9iYXNlLnB5) | `96.28% <100%> (ø)` | :arrow_up: |
| [pandas/core/internals/arrays.py](https://codecov.io/gh/pandas-dev/pandas/pull/24227/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW50ZXJuYWxzL2FycmF5cy5weQ==) | `100% <100%> (ø)` | |
| [pandas/core/arrays/numpy\_.py](https://codecov.io/gh/pandas-dev/pandas/pull/24227/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvYXJyYXlzL251bXB5Xy5weQ==) | `93.05% <93.05%> (ø)` | |
| ... and [3 more](https://codecov.io/gh/pandas-dev/pandas/pull/24227/diff?src=pr&el=tree-more) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/24227?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/24227?src=pr&el=footer). Last update [ab55d05...35f50a5](https://codecov.io/gh/pandas-dev/pandas/pull/24227?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

a1fecf4e912b601b56948397381ae4edbcc75752 has the changes that ensure these don't enter pandas by normal means (you can still create the block directly, or monekypatch as we do in the tests).
On the name, what do people think about just `PandasArray`? That gives us more flexibility into the future, since we don't tie the name to the implementation.
A few questions

1. What downsides are we missing here? What can this break that we aren't expecting? On the one hand this feels like a large change for this late in the release cycle, but not much is using `Seires/Index.array` yet so very few changes were needed to pass the test suite.
2. Should we intentionally limit the scope of `PandasArray` for now? (remove ops, reductions) in the spirit of keeping things small?
3. Do we want to do this?
I like the separate PandasArray. That makes it easier to understand that it
gets unboxed as a special case, and we can save NumpyArray for the boxed
version. I don't think there are good reasons to limit its scope -- there
are probably some use cases for wrapping the ExtensionArray interface
directly.
On Tue, Dec 11, 2018 at 8:00 PM Tom Augspurger <notifications@github.com>
wrote:

> A few questions
>
>    1. What downsides are we missing here? What can this break that we
>    aren't expecting? On the one hand this feels like a large change for this
>    late in the release cycle, but not much is using Seires/Index.array
>    yet so very few changes were needed to pass the test suite.
>    2. Should we intentionally limit the scope of PandasArray for now?
>    (remove ops, reductions) in the spirit of keeping things small?
>    3. Do we want to do this?
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> <https://github.com/pandas-dev/pandas/pull/24227#issuecomment-446453957>,
> or mute the thread
> <https://github.com/notifications/unsubscribe-auth/ABKS1u9qm-kzqBhP0dpS2igb0i7_jwd5ks5u4H9pgaJpZM4ZNoQZ>
> .
>

``PandasArray`` sgtm. But I will say this needs much more integration into pandas proper. I don't think introducing this in 0.24.0 at this late date is a good idea. Rather finish DTA and release.
> I don't think introducing this in 0.24.0 at this late date is a good idea.

I share some of that sentiment, but I still thing this is good for 0.24. We have a new API in `.array`, and this really seems like something that we'll wish we had done.

Plus... I think it's done :)

> Rather finish DTA and release.

I needed a break from staring out our JSON serializer :) But I'll get back to that now.
> I needed a break from staring out our JSON serializer :) But I'll get back to that now.

lol
The light integration is intentional.

> We store ExtensionArrays separately. So this is yet another break with the implementation. This makes understanding what is going on very confusing.

The catching of `PandasArray` specifically and unboxing it is indeed different, and confusing. No argument there. But, I think we also agree that these shouldn't actually end up inside a Series / DataFrame (IIRC @shoyer proposed that, not sure if @jorisvandenbossche weighed in).

To me, it comes down to balancing some additional maintenance burden (the checks for `ABCNumPyExtensionArray`) in order to not break user-code if / when we decide that re-writing the block manager is a good idea. I think that it's likely enough to happen that we should consider this.

So, my vote is for either

1. Not doing this at all (and maybe having to break code in 2.x if that's when we re-do the block manager)
2. Taking this PR roughly as is (`.array` returns a PandasArray, but it isn't allowed inside our containers under normal usage).

The 3rd option is adding `PandasArray` and allowing it in pandas (no `ABCNumPyExtensionArray` checks) but I don't think we should do that.
> The catching of PandasArray specifically and unboxing it is indeed different, and confusing. No argument there. But, I think we also agree that these shouldn't actually end up inside a Series / DataFrame (IIRC @shoyer proposed that, not sure if @jorisvandenbossche weighed in).

Yes, that's what I more or less was commenting here: https://github.com/pandas-dev/pandas/pull/23581#issuecomment-445982660. If doing this, I was assuming that we would not actually store then as ExtensionBlocks, but still as consolidated blocks.

It's certainly an added complexity that `.array` is sometimes the actual stored data, or sometimes a PandasArray with a view on (a part of) the stored numpy array, and indeed breaks the model. But the current situation where `.array` returns an ExtensionArray or a numpy ndarray depending on the dtype is *also* not that clean and consistent.

Regarding 0.24/0.25: if we want to do this, I think it is the most logical to do it now for 0.24, since it is now that we are adding the `Series.array` and `pd.array`. 
And I think the argument of it needing more time in master is equally valid for DatetimeArray (for which IMO we will also need at least a couple of weeks in master).

> (for which IMO we will also need at least a couple of weeks in master)

Just to verify, we should do a release candidate with DatetimeArray ASAP, right? And then 1-2 weeks on master while the RC is out?
Now that we've had some time to think on it, what are people's thoughts?

@shoyer and @jorisvandenbossche are I think +1 (in broad terms. May disagree with certain parts of the implementation).

I just think that

1. Having `.array -> ExtensionArray` rather than `-> Union[ndarray, ExtensionArray]` is going to be more useful for people working with this.
2. The basics of principal of "inplace modification on `Series.array` will be reflected back in the Series" is still true (and still probably a bad idea).
3. We can swap out the underlying data backing the `.array` in the future without breaking user code (at least for users doing "normal" things).

I think those benefits outweigh the costs of this PR (the checking for a `PandasArray` in the constructors).

It'd be nice to get a go / no go on this, so that I can update https://github.com/pandas-dev/pandas/pull/23581 accordingly. I've actually already done the work for `pd.array` to return a PandasArray, just haven't pushed it :)
I am not not against this idea generally at all. Rather this needs quite some integration. I worry that this will be implemented, then we just don't change anything else, leaving the internals just to rot. 
> Rather this needs quite some integration. I worry that this will be implemented, then we just don't change anything else, leaving the internals just to rot.

I guess I'm not sure how PandasArray changes that, other than us implementing PandasArray later, after internals starts using `.array`, causing us to have to rewrite the parts of pandas' internals that were expecting `.array` to be an ndarray. If anything, we should be doing PandasArray *before* we start using `.array` all over the place.
Just because I was curious, I added support for `__array_ufunc__` to PandasArray. Not sure if we should go forward with it for 0.24, since we only require NumPy 1.12.

The main thing this required was changing the signature of methods that numpy dispatches to like `sum` to have the signature expected by NumPy.

I'm really excited about the combination of `__array_ufunc__` and `ExtensionArray`. We should encourage EA authors to implement it.
Where do we stand here? I'll try to summarize some responses here.

1. Why is there a `pandas.arrays` instead of `pandas.core.arrays`.

We shouldn't expose parts of `pandas.core` in the public API. We've claimed that `pandas.core` is private, and adding references to it from the public API docs confuses people. Most people won't be creating a `PandasArray` directly. Rather, they'll get them back from `pd.array` or from `Series/Index.array`. But the fact that they're getting this object means we need to document it.

2. Why this ABCNumpyExtensionArray thing?

(aside on the name: it's a leftover from when a called the new wrapper array NumPyExtensionArray. I'd be happy to change it to something that indicates "this our thin wrapper around numpy arrays).

At this point, I don't think we want to users to be able to put a `PandasArray` inside a Series. IOW, the block for a `pd.Series(pd.Series([1, 2, 3]).array)`  should be an IntBlock, not an extension block. In theory, the two should be identical in results, if not performance. In practice there are differences (upcasting on setitem, for example).

3. Implementing ufuncs via `__array_ufunc__`.

I did it via `NDArrayOperatorMixin` because it was relatively easy. Don't really have a preference for whether or not we should do it. Since not all EAs will implement `__array_ufunc__`, users can't be assured that ufuncs will work correctly on EAs. But I think we should be a model for other EA authors here, by showing that `__array_ufunc__` should be implemented, and a (simple) example of how to do that.

Anything else?
> 1. Why is there a pandas.arrays instead of pandas.core.arrays.

I think this is the wrong logic. why are you exposing ``PandasArray`` in any different way than other extension arrays? I dont' think this is any different, and as a consequence you should simply live in ``pandas.core.arrays``

> 2. Why this ABCNumpyExtensionArray thing?

Well then let's call it ``NumpyExtensionArray`` then, rather than ``PandasArray``. these should be consistent


> I think this is the wrong logic. why are you exposing PandasArray in any different way than other extension arrays?

To be clear, this is collecting them in the same place as https://github.com/pandas-dev/pandas/pull/23581. We already have `Categorical` and `SparseArray` in the top-level, but in the issue for #23581 there was some pushback against putting these all in the top-level. We also didn't want to put them in `pandas.api.extensions` because that's reserved for EA authors.

In 
https://github.com/pandas-dev/pandas/pull/24227#issuecomment-446573170 and 
https://github.com/pandas-dev/pandas/pull/24227#issuecomment-446465276 @shoyer and @jreback agreed that PandasArray was better. That gives us more freedom to change the internals. We could easily have an Apache Arrow backed Series return a `PandasArray`. Having it return a thing called `NumPyExtensionArray` would be strange.
If we use ``PandasArray.`` then call it ``ABCPandasArray``, the inconsistency is the problem.

> To be clear, this is collecting them in the same place as #23581. We already have Categorical and SparseArray in the top-level, but in the issue for #23581 there was some pushback against putting these all in the top-level. We also didn't want to put them in pandas.api.extensions because that's reserved for EA authors.

why is ``pandas.core.array`` a problem? creating *another* top-level, ``pandas.array`` is a huge -1. 
> why is pandas.core.array a problem? creating another top-level, pandas.array is a huge -1.

`pandas.core` is private. I don't think we should be adding references to it from our public API docs.
> pandas.core is private. 

sure. so what exactly is the problem?
@TomAugspurger ok so you are doing the same thing as you did in #23581. ok I am -1 on that too then. why aren't you putting them in ``pandas.core.arrays`` and then simply exposing them in ``pandas.api.arrays``. new top-level api is -1.
> sure. so what exactly is the problem?

We don't want people doing things like `if isinstance(thing, pd.core.arrays.PandasArray)`, so we need to expose it somewhere public.

w.r.t. `pandas.arrays` vs. `pandas.api.arrays`, what does `.api` add? Python has a well-established convention of "things that don't start with underscore are part of the public API".
> w.r.t. pandas.arrays vs. pandas.api.arrays, what does .api add? Python has a well-established convention of "things that don't start with underscore are part of the public API".

we already have a well-established pattern. don't reinvent the wheel.
We also have `pandas.errors`, `pandas.plotting`, `pandas.io`, `pandas.tseries`, and `pandas.testing`. I think that `pandas.arrays` fits in nicely with those.
all of these should not be public, this is a historical problem. don't exacerbate it.