WIP: Bitarray backed Int EAs
State: closed

First pass at #21839 and most likely a precursor to #22226.

Very hacky at the moment but was able to get tests to pass locally (will fail CI given new Bitarray depdencendy). Going to work through a refactor of this but sharing as POC in case anyone has feedback.

Generally the hard part here was working around places in the code base that expect an ndarray (ex: via `sum` method calls) and figuring out how / when to best manage the conversions between bit arrays and ndarrays

Labels:
name: ExtensionArray
description: Extending pandas with custom dtypes or arrays.

Comments:
Hello @WillAyd! Thanks for updating the PR.





 - In the file [`pandas/util/testing.py`](https://github.com/pandas-dev/pandas/blob/d2bff821d46e433dc0d8178f5428d232f683c06a/pandas/util/testing.py), following are the PEP8 issues :

> [Line 1295:1](https://github.com/pandas-dev/pandas/blob/d2bff821d46e433dc0d8178f5428d232f683c06a/pandas/util/testing.py#L1295): [W293](https://duckduckgo.com/?q=pep8%20W293) blank line contains whitespace



##### Comment last updated on December 28, 2018 at 21:50 Hours UTC
How active is bitarray development? Would it make sense to just vendor it since it won’t be exposed to users?
Based on it's pulse it has arguably been more active as of late, though at the same time it does mostly look backed by one person:

<img width="989" alt="image" src="https://user-images.githubusercontent.com/609873/43848692-5dc02900-9ae8-11e8-8d9a-e1fd83674527.png">


I'm not opposed to vendoring if we feel like it's easier to support that way
nice idea, this would have to be an optional dep i think, so makes supporting this a bit more tricky. closing as stale.
@WillAyd I was watching ur talk about this PR. Any chance this come to life soon?
Haven't put much into it lately but if you'd like to pick up where I left off @eyadsibai would certainly be welcome!
Reopening as I should be able to put more focus in on this one
Just merged master though not quite ready for review (providing in case someone else is interested).

@jreback was thinking about the best way to make this optional and was looking at what we do with bottleneck as inspiration. IIUC we have a wrapper for that which intercepts method calls and replaces them with the bottleneck equivalent where applicable.

Was thinking we could do relatively the same thing here, though not sure if you had other thoughts on maybe creating some kind of mask mixin since this could be utilized by perhaps any EA that needs a NA mask
yeah you could make a wrapper class that abstracts this (meaning it would use bitarray if available or a numpy uint_8 if not)( eventually maybe using a pyarrow as another alternative)

you could put in pandas/core/arrays/_mask.py i think

it wouldn’t be a full extension array itself but just expose a way to create a suitable array and work with it

then this can just be a drop in replacement for it in integer_na

i would opt for composition here rather than a mixin
Nowhere near ready for merge and causes tons of failures. Updated title to show as WIP.

Latest commit provided just for any feedback on overall structure
Closing again for now; will re-open once I can get some more time to actively invest, though certainly if anyone else wants to pick this up open to that!