BUG: merging an Integer EA rasises
State: closed

- [ ] closes #23020 
- [ ] 1 test added
- [ ] whatsnew entry


Labels:
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode
name: ExtensionArray
description: Extending pandas with custom dtypes or arrays.

Comments:
Hello @makbigc! Thanks for submitting the PR.

 - There are no PEP8 issues in the file [`pandas/core/reshape/merge.py`](https://github.com/makbigc/pandas/blob/a4696244117f1011545e2d7b72e1459d0475a6b6/pandas/core/reshape/merge.py) !

 - There are no PEP8 issues in the file [`pandas/tests/reshape/merge/test_merge.py`](https://github.com/makbigc/pandas/blob/a4696244117f1011545e2d7b72e1459d0475a6b6/pandas/tests/reshape/merge/test_merge.py) !


can you merge master and update
@jreback before the push, I already rebased my code. But all checks have failed. Did my change break the checks? Or rebasing hasn't get the code to the latest, has it? Please help.
@makbigc can you clarify

> But all checks have failed?

It'd be easier to help if you push your changes so that we can see what you have. 
Could you also address @jreback's comments?
Please give me some more time. I am working on it.
Thanks. No rush for now, if you think you'll have some time by next week
that'd be great.

On Thu, Nov 8, 2018 at 8:04 AM Mak Sze Chun <notifications@github.com>
wrote:

> Reopened #23262 <https://github.com/pandas-dev/pandas/pull/23262>.
>
> —
> You are receiving this because you commented.
> Reply to this email directly, view it on GitHub
> <https://github.com/pandas-dev/pandas/pull/23262#event-1954354112>, or mute
> the thread
> <https://github.com/notifications/unsubscribe-auth/ABQHItIaWxb7_FAxylHQu4eD9wslHz0Yks5utDnYgaJpZM4XyfK9>
> .
>

@makbigc can you merge master. 
# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/23262?src=pr&el=h1) Report
> Merging [#23262](https://codecov.io/gh/pandas-dev/pandas/pull/23262?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/9f2c7164ec581a484ee167c04478bf9e4f81426b?src=pr&el=desc) will **decrease** coverage by `<.01%`.
> The diff coverage is `100%`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/23262/graphs/tree.svg?width=650&token=eZ4WkYLtcO&height=150&src=pr)](https://codecov.io/gh/pandas-dev/pandas/pull/23262?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #23262      +/-   ##
==========================================
- Coverage    92.2%    92.2%   -0.01%     
==========================================
  Files         162      162              
  Lines       51714    51722       +8     
==========================================
+ Hits        47682    47689       +7     
- Misses       4032     4033       +1
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `90.6% <100%> (ø)` | :arrow_up: |
| #single | `43.01% <0%> (-0.02%)` | :arrow_down: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/23262?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/core/reshape/merge.py](https://codecov.io/gh/pandas-dev/pandas/pull/23262/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvcmVzaGFwZS9tZXJnZS5weQ==) | `94.35% <100%> (+0.06%)` | :arrow_up: |
| [pandas/io/json/json.py](https://codecov.io/gh/pandas-dev/pandas/pull/23262/diff?src=pr&el=tree#diff-cGFuZGFzL2lvL2pzb24vanNvbi5weQ==) | `92.61% <0%> (-0.48%)` | :arrow_down: |
| [pandas/util/testing.py](https://codecov.io/gh/pandas-dev/pandas/pull/23262/diff?src=pr&el=tree#diff-cGFuZGFzL3V0aWwvdGVzdGluZy5weQ==) | `87.51% <0%> (+0.09%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/23262?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/23262?src=pr&el=footer). Last update [9f2c716...b8ed3e9](https://codecov.io/gh/pandas-dev/pandas/pull/23262?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/23262?src=pr&el=h1) Report
> Merging [#23262](https://codecov.io/gh/pandas-dev/pandas/pull/23262?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/216986d4691297d5cfec33b5c62be7890b9a54d7?src=pr&el=desc) will **increase** coverage by `<.01%`.
> The diff coverage is `100%`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/23262/graphs/tree.svg?width=650&token=eZ4WkYLtcO&height=150&src=pr)](https://codecov.io/gh/pandas-dev/pandas/pull/23262?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #23262      +/-   ##
==========================================
+ Coverage   92.28%   92.28%   +<.01%     
==========================================
  Files         162      162              
  Lines       51831    51837       +6     
==========================================
+ Hits        47830    47837       +7     
+ Misses       4001     4000       -1
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `90.68% <100%> (ø)` | :arrow_up: |
| #single | `43% <0%> (-0.01%)` | :arrow_down: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/23262?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/core/reshape/merge.py](https://codecov.io/gh/pandas-dev/pandas/pull/23262/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvcmVzaGFwZS9tZXJnZS5weQ==) | `94.3% <100%> (+0.01%)` | :arrow_up: |
| [pandas/core/indexes/period.py](https://codecov.io/gh/pandas-dev/pandas/pull/23262/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW5kZXhlcy9wZXJpb2QucHk=) | `93.09% <0%> (+0.03%)` | :arrow_up: |
| [pandas/core/reshape/tile.py](https://codecov.io/gh/pandas-dev/pandas/pull/23262/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvcmVzaGFwZS90aWxlLnB5) | `94.82% <0%> (+0.06%)` | :arrow_up: |
| [pandas/util/testing.py](https://codecov.io/gh/pandas-dev/pandas/pull/23262/diff?src=pr&el=tree#diff-cGFuZGFzL3V0aWwvdGVzdGluZy5weQ==) | `87.58% <0%> (+0.09%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/23262?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/23262?src=pr&el=footer). Last update [216986d...6bae6ca](https://codecov.io/gh/pandas-dev/pandas/pull/23262?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

@jreback rebased. Please tell me if anything needs to be done.
There's a linting error: https://dev.azure.com/pandas-dev/pandas/_build/results?buildId=4750

```
2018-12-05T15:26:49.5779150Z Check import format using isort
2018-12-05T15:26:55.4954327Z ERROR: /home/vsts/work/1/s/pandas/core/reshape/merge.py Imports are incorrectly sorted.
2018-12-05T15:26:55.5184554Z Check import format using isort DONE
```
The test is rewritten. So the EA is the merging key, not like the previous test in which integer array is the merging key for every cases.
The old test is placed as a special case in the tests/reshape/merge/test_merge.py.
Would you give me some hints to pass azure test? It seems the new test doesn't work with python 2.7.15.
Thanks.
I'm not too sure about the azure failure, but notice that the data generated includes a duplicate in the first three values

```python
data = [I, a, a, ...
```

That data is randomly generated. It looks like your `expected` may not handle cases where the `key` is not unique.

I believe the only requirement on `data` are that it's

* length 100
* the first two values are distinct
* the first two values are not NA.
@makbigc I'm going to push a few changes here if that's OK.
I started to write a test that would include NA values in the merge key, but I think the expected behavior of that is up in the air right now: https://github.com/pandas-dev/pandas/issues/22491.
Updated the coercion of lk / rk to do

1. all the pre-processing on the values (datetimetz, categorical, extension)
2. Hashtable / dtype validate

in two distinct blocks.
@TomAugspurger Thanks for your reply and help.
thanks @makbigc  and @TomAugspurger 

maybe we should create some asv's for EA's generally