Repository: pandas-dev/pandas

Issue:
ENH: add in extension dtype registry
precursor to #21160 


Discussion:
# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/21185?src=pr&el=h1) Report
> Merging [#21185](https://codecov.io/gh/pandas-dev/pandas/pull/21185?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/ad76ffcca0d92c3885c279c80701c2f4a3f3f177?src=pr&el=desc) will **increase** coverage by `<.01%`.
> The diff coverage is `92.98%`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/21185/graphs/tree.svg?token=eZ4WkYLtcO&src=pr&width=650&height=150)](https://codecov.io/gh/pandas-dev/pandas/pull/21185?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #21185      +/-   ##
==========================================
+ Coverage    91.9%   91.91%   +<.01%     
==========================================
  Files         154      154              
  Lines       49659    49673      +14     
==========================================
+ Hits        45640    45657      +17     
+ Misses       4019     4016       -3
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `90.29% <92.98%> (ø)` | :arrow_up: |
| #single | `41.95% <75.43%> (+0.05%)` | :arrow_up: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/21185?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/io/formats/format.py](https://codecov.io/gh/pandas-dev/pandas/pull/21185/diff?src=pr&el=tree#diff-cGFuZGFzL2lvL2Zvcm1hdHMvZm9ybWF0LnB5) | `98.25% <ø> (ø)` | :arrow_up: |
| [pandas/core/algorithms.py](https://codecov.io/gh/pandas-dev/pandas/pull/21185/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvYWxnb3JpdGhtcy5weQ==) | `94.85% <100%> (ø)` | :arrow_up: |
| [pandas/core/dtypes/base.py](https://codecov.io/gh/pandas-dev/pandas/pull/21185/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZHR5cGVzL2Jhc2UucHk=) | `92.3% <100%> (+0.2%)` | :arrow_up: |
| [pandas/core/series.py](https://codecov.io/gh/pandas-dev/pandas/pull/21185/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvc2VyaWVzLnB5) | `94.19% <100%> (ø)` | :arrow_up: |
| [pandas/core/indexes/interval.py](https://codecov.io/gh/pandas-dev/pandas/pull/21185/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW5kZXhlcy9pbnRlcnZhbC5weQ==) | `93.13% <100%> (ø)` | :arrow_up: |
| [pandas/core/dtypes/common.py](https://codecov.io/gh/pandas-dev/pandas/pull/21185/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZHR5cGVzL2NvbW1vbi5weQ==) | `95.07% <100%> (+0.44%)` | :arrow_up: |
| [pandas/core/arrays/base.py](https://codecov.io/gh/pandas-dev/pandas/pull/21185/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvYXJyYXlzL2Jhc2UucHk=) | `87.85% <100%> (+0.26%)` | :arrow_up: |
| [pandas/core/dtypes/cast.py](https://codecov.io/gh/pandas-dev/pandas/pull/21185/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZHR5cGVzL2Nhc3QucHk=) | `88.36% <50%> (-0.13%)` | :arrow_down: |
| [pandas/core/internals.py](https://codecov.io/gh/pandas-dev/pandas/pull/21185/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW50ZXJuYWxzLnB5) | `95.52% <81.81%> (-0.07%)` | :arrow_down: |
| [pandas/core/dtypes/dtypes.py](https://codecov.io/gh/pandas-dev/pandas/pull/21185/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZHR5cGVzL2R0eXBlcy5weQ==) | `95.98% <96.77%> (+0.03%)` | :arrow_up: |
| ... and [2 more](https://codecov.io/gh/pandas-dev/pandas/pull/21185/diff?src=pr&el=tree-more) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/21185?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/21185?src=pr&el=footer). Last update [ad76ffc...d2c91d7](https://codecov.io/gh/pandas-dev/pandas/pull/21185?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

split out ops to a separate PR #21191 
all fixed up @jorisvandenbossche  (except for docs on Registry in Extension section)
see if can make this dispatch to create_array_type as a function
updated docs

@jorisvandenbossche @TomAugspurger if any final comments.
ok all fixed up. @jorisvandenbossche @TomAugspurger if you want to look.
@jreback I'm wondering if there is a meta issue with the registry that should be thought about.  Let's say that pandas registers 10 types - they are always in the registry for all pandas users.  Now let's say you have 2 different people who independently extend pandas with the EA capabilities creating libraries ABC and DEF.  But they happen to both call their extension array type `MyEAType` .  Now if someone wants to use both libraries ABC and DEF, there will be a collision in the registry of `MyEAType`.  So does there need to be some "master" registry in the documentation containing the names of all EA types that people have registered?

I realize this is a highly unlikely occurrence.  

Put another way, shouldn't there be code for the registry that says "this type has already been registered"?

Also, the implementation of the registry is using a list.  Shouldn't you use a dict or set?
@Dr-Irv you don;t *need* to register the EA types, this is *only* for the purpose of translating a string dtype name -> EA dtype (e.g. 'categorical' -> Categorical), basically for convenience.

so sure you *could* have a collision, but then user would have to deal with that (this wouldn't show up on pandas because we won't have colliding names).
@Dr-Irv I had this an OrderdedDict but its actually not necessary, and is actually not correct. If we have a string, we have to ask the dtypes can you construct a type from it. The issue is that multiple strings can map to a dtype when its parameterized, e.g. ``datetime64[ns, US/Eastern]`` is a valid string dtype as is ``datetime64[ns, Asia/Tokyo]`` and w/o imbuing the knowledge to say its 'good' inside the Register (bad idea), we can't know something works unless we duck type it.
@jreback Thanks for the clarification of the *need* versus *require* of registration of EA types.

On the implementation, wouldn't OrderedDict be faster once there are a lot of types?
I had removed append FYI (it should not longer be in this PR)
I see it at https://github.com/pandas-dev/pandas/pull/21185/files#diff-99633b1041c2ea6283aa3d5bcf79e8cdR584

Maybe need to push?
pushed, yeah had it on the wrong branch.
any final comments. going to rebase.
@jorisvandenbossche @TomAugspurger 
merging.

Resolving patch:
diff --git a/pandas/core/arrays/integer.py b/pandas/core/arrays/integer.py
--- a/pandas/core/arrays/integer.py
+++ b/pandas/core/arrays/integer.py
@@ -8,6 +8,7 @@
 from pandas.compat import u, range
 from pandas.compat import set_function_name
 
+from pandas.core.dtypes.cast import astype_nansafe
 from pandas.core.dtypes.generic import ABCSeries, ABCIndexClass
 from pandas.core.dtypes.common import (
     is_integer, is_scalar, is_float,
@@ -391,7 +392,7 @@ def astype(self, dtype, copy=True):
 
         # coerce
         data = self._coerce_to_ndarray()
-        return data.astype(dtype=dtype, copy=False)
+        return astype_nansafe(data, dtype, copy=None)
 
     @property
     def _ndarray_values(self):
diff --git a/pandas/core/dtypes/cast.py b/pandas/core/dtypes/cast.py
--- a/pandas/core/dtypes/cast.py
+++ b/pandas/core/dtypes/cast.py
@@ -647,7 +647,16 @@ def conv(r, dtype):
 
 def astype_nansafe(arr, dtype, copy=True):
     """ return a view if copy is False, but
-        need to be very careful as the result shape could change! """
+        need to be very careful as the result shape could change!
+
+    Parameters
+    ----------
+    arr : ndarray
+    dtype : np.dtype
+    copy : bool, default True
+        If False, a view will be attempted but may fail, if
+        e.g. the itemsizes don't align.
+    """
 
     # dispatch on extension dtype if needed
     if is_extension_array_dtype(dtype):
@@ -733,8 +742,10 @@ def astype_nansafe(arr, dtype, copy=True):
                       FutureWarning, stacklevel=5)
         dtype = np.dtype(dtype.name + "[ns]")
 
-    if copy:
+    if copy or is_object_dtype(arr) or is_object_dtype(dtype):
+        # Explicit copy, or required since NumPy can't view from / to object.
         return arr.astype(dtype, copy=True)
+
     return arr.view(dtype)
 
 
diff --git a/pandas/core/internals/blocks.py b/pandas/core/internals/blocks.py
--- a/pandas/core/internals/blocks.py
+++ b/pandas/core/internals/blocks.py
@@ -637,22 +637,25 @@ def _astype(self, dtype, copy=False, errors='raise', values=None,
             # force the copy here
             if values is None:
 
-                if issubclass(dtype.type,
-                              (compat.text_type, compat.string_types)):
+                if self.is_extension:
+                    values = self.values.astype(dtype)
+                else:
+                    if issubclass(dtype.type,
+                                  (compat.text_type, compat.string_types)):
 
-                    # use native type formatting for datetime/tz/timedelta
-                    if self.is_datelike:
-                        values = self.to_native_types()
+                        # use native type formatting for datetime/tz/timedelta
+                        if self.is_datelike:
+                            values = self.to_native_types()
 
-                    # astype formatting
-                    else:
-                        values = self.get_values()
+                        # astype formatting
+                        else:
+                            values = self.get_values()
 
-                else:
-                    values = self.get_values(dtype=dtype)
+                    else:
+                        values = self.get_values(dtype=dtype)
 
-                # _astype_nansafe works fine with 1-d only
-                values = astype_nansafe(values.ravel(), dtype, copy=True)
+                    # _astype_nansafe works fine with 1-d only
+                    values = astype_nansafe(values.ravel(), dtype, copy=True)
 
                 # TODO(extension)
                 # should we make this attribute?


Test patch:
diff --git a/pandas/tests/extension/decimal/array.py b/pandas/tests/extension/decimal/array.py
--- a/pandas/tests/extension/decimal/array.py
+++ b/pandas/tests/extension/decimal/array.py
@@ -15,6 +15,17 @@ class DecimalDtype(ExtensionDtype):
     name = 'decimal'
     na_value = decimal.Decimal('NaN')
 
+    def __init__(self, context=None):
+        self.context = context or decimal.getcontext()
+
+    def __eq__(self, other):
+        if isinstance(other, type(self)):
+            return self.context == other.context
+        return super(DecimalDtype, self).__eq__(other)
+
+    def __repr__(self):
+        return 'DecimalDtype(context={})'.format(self.context)
+
     @classmethod
     def construct_array_type(cls):
         """Return the array type associated with this dtype
@@ -35,13 +46,12 @@ def construct_from_string(cls, string):
 
 
 class DecimalArray(ExtensionArray, ExtensionScalarOpsMixin):
-    dtype = DecimalDtype()
 
-    def __init__(self, values, dtype=None, copy=False):
+    def __init__(self, values, dtype=None, copy=False, context=None):
         for val in values:
-            if not isinstance(val, self.dtype.type):
+            if not isinstance(val, decimal.Decimal):
                 raise TypeError("All values must be of type " +
-                                str(self.dtype.type))
+                                str(decimal.Decimal))
         values = np.asarray(values, dtype=object)
 
         self._data = values
@@ -51,6 +61,11 @@ def __init__(self, values, dtype=None, copy=False):
         # those aliases are currently not working due to assumptions
         # in internal code (GH-20735)
         # self._values = self.values = self.data
+        self._dtype = DecimalDtype(context)
+
+    @property
+    def dtype(self):
+        return self._dtype
 
     @classmethod
     def _from_sequence(cls, scalars, dtype=None, copy=False):
@@ -82,6 +97,11 @@ def copy(self, deep=False):
             return type(self)(self._data.copy())
         return type(self)(self)
 
+    def astype(self, dtype, copy=True):
+        if isinstance(dtype, type(self.dtype)):
+            return type(self)(self._data, context=dtype.context)
+        return super(DecimalArray, self).astype(dtype, copy)
+
     def __setitem__(self, key, value):
         if pd.api.types.is_list_like(value):
             value = [decimal.Decimal(v) for v in value]
diff --git a/pandas/tests/extension/decimal/test_decimal.py b/pandas/tests/extension/decimal/test_decimal.py
--- a/pandas/tests/extension/decimal/test_decimal.py
+++ b/pandas/tests/extension/decimal/test_decimal.py
@@ -205,6 +205,27 @@ def test_dataframe_constructor_with_dtype():
     tm.assert_frame_equal(result, expected)
 
 
+@pytest.mark.parametrize("frame", [True, False])
+def test_astype_dispatches(frame):
+    # This is a dtype-specific test that ensures Series[decimal].astype
+    # gets all the way through to ExtensionArray.astype
+    # Designing a reliable smoke test that works for arbitrary data types
+    # is difficult.
+    data = pd.Series(DecimalArray([decimal.Decimal(2)]), name='a')
+    ctx = decimal.Context()
+    ctx.prec = 5
+
+    if frame:
+        data = data.to_frame()
+
+    result = data.astype(DecimalDtype(ctx))
+
+    if frame:
+        result = result['a']
+
+    assert result.dtype.context.prec == ctx.prec
+
+
 class TestArithmeticOps(BaseDecimal, base.BaseArithmeticOpsTests):
 
     def check_opname(self, s, op_name, other, exc=None):
diff --git a/pandas/tests/extension/integer/test_integer.py b/pandas/tests/extension/integer/test_integer.py
--- a/pandas/tests/extension/integer/test_integer.py
+++ b/pandas/tests/extension/integer/test_integer.py
@@ -697,6 +697,15 @@ def test_cross_type_arithmetic():
     tm.assert_series_equal(result, expected)
 
 
+def test_astype_nansafe():
+    # https://github.com/pandas-dev/pandas/pull/22343
+    arr = IntegerArray([np.nan, 1, 2], dtype="Int8")
+
+    with tm.assert_raises_regex(
+            ValueError, 'cannot convert float NaN to integer'):
+        arr.astype('uint32')
+
+
 # TODO(jreback) - these need testing / are broken
 
 # shift