ENH: add in extension dtype registry
State: closed

precursor to #21160 

Labels:
name: ExtensionArray
description: Extending pandas with custom dtypes or arrays.

Comments:
# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/21185?src=pr&el=h1) Report
> Merging [#21185](https://codecov.io/gh/pandas-dev/pandas/pull/21185?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/ad76ffcca0d92c3885c279c80701c2f4a3f3f177?src=pr&el=desc) will **increase** coverage by `<.01%`.
> The diff coverage is `92.98%`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/21185/graphs/tree.svg?token=eZ4WkYLtcO&src=pr&width=650&height=150)](https://codecov.io/gh/pandas-dev/pandas/pull/21185?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #21185      +/-   ##
==========================================
+ Coverage    91.9%   91.91%   +<.01%     
==========================================
  Files         154      154              
  Lines       49659    49673      +14     
==========================================
+ Hits        45640    45657      +17     
+ Misses       4019     4016       -3
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `90.29% <92.98%> (ø)` | :arrow_up: |
| #single | `41.95% <75.43%> (+0.05%)` | :arrow_up: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/21185?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/io/formats/format.py](https://codecov.io/gh/pandas-dev/pandas/pull/21185/diff?src=pr&el=tree#diff-cGFuZGFzL2lvL2Zvcm1hdHMvZm9ybWF0LnB5) | `98.25% <ø> (ø)` | :arrow_up: |
| [pandas/core/algorithms.py](https://codecov.io/gh/pandas-dev/pandas/pull/21185/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvYWxnb3JpdGhtcy5weQ==) | `94.85% <100%> (ø)` | :arrow_up: |
| [pandas/core/dtypes/base.py](https://codecov.io/gh/pandas-dev/pandas/pull/21185/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZHR5cGVzL2Jhc2UucHk=) | `92.3% <100%> (+0.2%)` | :arrow_up: |
| [pandas/core/series.py](https://codecov.io/gh/pandas-dev/pandas/pull/21185/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvc2VyaWVzLnB5) | `94.19% <100%> (ø)` | :arrow_up: |
| [pandas/core/indexes/interval.py](https://codecov.io/gh/pandas-dev/pandas/pull/21185/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW5kZXhlcy9pbnRlcnZhbC5weQ==) | `93.13% <100%> (ø)` | :arrow_up: |
| [pandas/core/dtypes/common.py](https://codecov.io/gh/pandas-dev/pandas/pull/21185/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZHR5cGVzL2NvbW1vbi5weQ==) | `95.07% <100%> (+0.44%)` | :arrow_up: |
| [pandas/core/arrays/base.py](https://codecov.io/gh/pandas-dev/pandas/pull/21185/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvYXJyYXlzL2Jhc2UucHk=) | `87.85% <100%> (+0.26%)` | :arrow_up: |
| [pandas/core/dtypes/cast.py](https://codecov.io/gh/pandas-dev/pandas/pull/21185/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZHR5cGVzL2Nhc3QucHk=) | `88.36% <50%> (-0.13%)` | :arrow_down: |
| [pandas/core/internals.py](https://codecov.io/gh/pandas-dev/pandas/pull/21185/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW50ZXJuYWxzLnB5) | `95.52% <81.81%> (-0.07%)` | :arrow_down: |
| [pandas/core/dtypes/dtypes.py](https://codecov.io/gh/pandas-dev/pandas/pull/21185/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZHR5cGVzL2R0eXBlcy5weQ==) | `95.98% <96.77%> (+0.03%)` | :arrow_up: |
| ... and [2 more](https://codecov.io/gh/pandas-dev/pandas/pull/21185/diff?src=pr&el=tree-more) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/21185?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/21185?src=pr&el=footer). Last update [ad76ffc...d2c91d7](https://codecov.io/gh/pandas-dev/pandas/pull/21185?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

split out ops to a separate PR #21191 
all fixed up @jorisvandenbossche  (except for docs on Registry in Extension section)
see if can make this dispatch to create_array_type as a function
updated docs

@jorisvandenbossche @TomAugspurger if any final comments.
ok all fixed up. @jorisvandenbossche @TomAugspurger if you want to look.
@jreback I'm wondering if there is a meta issue with the registry that should be thought about.  Let's say that pandas registers 10 types - they are always in the registry for all pandas users.  Now let's say you have 2 different people who independently extend pandas with the EA capabilities creating libraries ABC and DEF.  But they happen to both call their extension array type `MyEAType` .  Now if someone wants to use both libraries ABC and DEF, there will be a collision in the registry of `MyEAType`.  So does there need to be some "master" registry in the documentation containing the names of all EA types that people have registered?

I realize this is a highly unlikely occurrence.  

Put another way, shouldn't there be code for the registry that says "this type has already been registered"?

Also, the implementation of the registry is using a list.  Shouldn't you use a dict or set?
@Dr-Irv you don;t *need* to register the EA types, this is *only* for the purpose of translating a string dtype name -> EA dtype (e.g. 'categorical' -> Categorical), basically for convenience.

so sure you *could* have a collision, but then user would have to deal with that (this wouldn't show up on pandas because we won't have colliding names).
@Dr-Irv I had this an OrderdedDict but its actually not necessary, and is actually not correct. If we have a string, we have to ask the dtypes can you construct a type from it. The issue is that multiple strings can map to a dtype when its parameterized, e.g. ``datetime64[ns, US/Eastern]`` is a valid string dtype as is ``datetime64[ns, Asia/Tokyo]`` and w/o imbuing the knowledge to say its 'good' inside the Register (bad idea), we can't know something works unless we duck type it.
@jreback Thanks for the clarification of the *need* versus *require* of registration of EA types.

On the implementation, wouldn't OrderedDict be faster once there are a lot of types?
I had removed append FYI (it should not longer be in this PR)
I see it at https://github.com/pandas-dev/pandas/pull/21185/files#diff-99633b1041c2ea6283aa3d5bcf79e8cdR584

Maybe need to push?
pushed, yeah had it on the wrong branch.
any final comments. going to rebase.
@jorisvandenbossche @TomAugspurger 
merging.