BUG: alignment of CategoricalIndex
State: closed

- [x] closes #13365
- [x] tests added / passed
- [x] passes `git diff upstream/master | flake8 --diff`
- [ ] whatsnew entry

@jreback take this just as a wild guess of how to fix #13365 . If the approach makes sense, I will add tests, whatsnew and some more docs.


Labels:
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: Categorical
description: Categorical Data Type

Comments:
## [Current coverage](https://codecov.io/gh/pydata/pandas/pull/13440?src=pr) is **84.34%**

> Merging [#13440](https://codecov.io/gh/pydata/pandas/pull/13440?src=pr) into [master](https://codecov.io/gh/pydata/pandas/branch/master?src=pr) will increase coverage by **<.01%**

``` diff
@@             master     #13440   diff @@
==========================================
  Files           138        138          
  Lines         51107      51124    +17   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
+ Hits          43103      43119    +16   
- Misses         8004       8005     +1   
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last updated by [bf4786a...7f4a72a](https://codecov.io/gh/pydata/pandas/compare/bf4786a15f51d14ccf1318e44ae080868d8595ee...7f4a72a2b8f3733129c017fe7adcfa1a563fff75)

you are adding a crazy amount of code here.

Thanks for the `.get_indexer()` suggestion, fixed. Apart from that, do you have in mind a simpler approach? What I _could_ do would be to override the entire `_join_non_unique()`, duplicating some lines of code but removing the need for the two new methods currently used in the `CategoricalIndex`\- specific parts.

@jreback : what do you want to do?

1) I complete the current approach with tests and docs
2) I override the entire `_join_non_unique()` instead that adding two new methods
3) you suggest another approach
4) we close this

The approach is not bad, but the actual impl seems not very elegant. Further your impl of `_array_putmask` is not good. 

Any _constructive_ criticism? :-) My mood doesn't depend on them, but maybe this PR/issue does.

maybe @sinhrks has some

I may misunderstand, but the point is how pandas handle extended dtype? We should decide whether to make them more numpy compat, or use separated logic using if-else branch. 

I think most of other impl takes latter (if-else) option currently.

@sinhrks : yes, the bug is due to a numpy method (`putmask()`) not working (and it's not a mere problem of type checks, there's really missing logic, which this PR introduces) on `Categorical`.

Certainly the cleanest option (for code organization, not that it would spare us from including the missing logic) would be that `Categorical` inherits from np.ndarray. But that's obviously a huge work... and might  (I don't know enough to judge) even be plain impossible due to the in-memory structure of `Categorical` being different from a typical ndarray.

Then sure, we can solve this also with "if-else"s without overriding... I have hard time considering such approach more "elegant" than mine, but if it is the path of least resistance to closing the bug I'll update the PR.

@toobaz I wasn't looking for a general 'why are we doing things this way', that is obvious. The issue is you are jumping thru hoops and not using the _existing_ `Categorical` machinery

OK, now I'm (extending and) using `union_categorical()` (thanks for the tip - although I don't have clear what you mean by "violates the guarantee"), and overriding directly `_join_non_unique()` so that the change is much less invasive.

@jreback ping

Feel free to close, I won't be offended.

yep, don't really have time to look, but needs some refactoring. if you wan to submit again pls feel free.

Submit again?! Well... what?!

I guess the bug(s) will be fixed by somebody who understands how contributing to this project works. I tried in vain.