[ArrowStringArray] Use `utf8_is_*` functions from Apache Arrow if available
State: closed

xref https://github.com/xhochy/fletcher/pull/203

marked as draft since AFAICT there is performance issues with BooleanDtype().__from_arrow__

Labels:
name: Performance
description: Memory or execution speed performance
name: Strings
description: String extension data type and string data

Comments:
Cool!

> marked as draft since AFAICT there is performance issues with BooleanDtype().from_arrow

I opened https://github.com/pandas-dev/pandas/pull/41051. In the end it's only a modest speed-up (3x), but I don't think we can improve further ourselves. Also compared to the actual string operation, I assume this conversion will be fast enough. 
with the changes in  #41051 now actually see an improvement.

```
s = pd.Series(["a", None, "1"] * 100_000, dtype="string")
s2 = pd.Series(["a", None, "1"] * 100_000, dtype="arrow_string")

%timeit s.str.isalnum()
# 19.6 ms ± 235 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)  <-- PR/master


%timeit s2.str.isalnum()
# 22.9 ms ± 229 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)  <-- master
# 1.9 ms ± 6.89 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)  <-- PR
```
@jorisvandenbossche i'll leave this as draft till #41051 is merged.
I merged #41051, so you can update this now
> I merged #41051, so you can update this now

Thanks @jorisvandenbossche 

Have also paramaterised some more tests for the `is_*` functions. side effect is that, to avoid xfailing or breaking up a test, have also "fixed" extract/extractall xfailed tests. could break-off the test changes into a precursor if needed.
```
[  5.56%] ··· strings.Methods.time_isalnum                                                                                                                ok
[  5.56%] ··· ============== ==========
                  dtype                
              -------------- ----------
                   str        19.3±0ms 
                  string      12.2±0ms 
               arrow_string   2.55±0ms 
              ============== ==========

[ 11.11%] ··· strings.Methods.time_isalpha                                                                                                                ok
[ 11.11%] ··· ============== ==========
                  dtype                
              -------------- ----------
                   str        15.9±0ms 
                  string      10.9±0ms 
               arrow_string   3.00±0ms 
              ============== ==========

[ 16.67%] ··· strings.Methods.time_isdecimal                                                                                                              ok
[ 16.67%] ··· ============== ==========
                  dtype                
              -------------- ----------
                   str        15.3±0ms 
                  string      8.88±0ms 
               arrow_string   1.76±0ms 
              ============== ==========

[ 22.22%] ··· strings.Methods.time_isdigit                                                                                                                ok
[ 22.22%] ··· ============== ==========
                  dtype                
              -------------- ----------
                   str        15.3±0ms 
                  string      8.84±0ms 
               arrow_string   1.82±0ms 
              ============== ==========

[ 27.78%] ··· strings.Methods.time_islower                                                                                                                ok
[ 27.78%] ··· ============== ==========
                  dtype                
              -------------- ----------
                   str        17.6±0ms 
                  string      10.8±0ms 
               arrow_string   3.67±0ms 
              ============== ==========

[ 33.33%] ··· strings.Methods.time_isnumeric                                                                                                              ok
[ 33.33%] ··· ============== ==========
                  dtype                
              -------------- ----------
                   str        15.2±0ms 
                  string      8.84±0ms 
               arrow_string   1.78±0ms 
              ============== ==========

[ 38.89%] ··· strings.Methods.time_isspace                                                                                                                ok
[ 38.89%] ··· ============== ==========
                  dtype                
              -------------- ----------
                   str        14.1±0ms 
                  string      8.41±0ms 
               arrow_string   1.74±0ms 
              ============== ==========

[ 44.44%] ··· strings.Methods.time_istitle                                                                                                                ok
[ 44.44%] ··· ============== ==========
                  dtype                
              -------------- ----------
                   str        17.3±0ms 
                  string      10.8±0ms 
               arrow_string   3.38±0ms 
              ============== ==========

[ 50.00%] ··· strings.Methods.time_isupper                                                                                                                ok
[ 50.00%] ··· ============== ==========
                  dtype                
              -------------- ----------
                   str        16.0±0ms 
                  string      10.2±0ms 
               arrow_string   3.55±0ms 
              ============== ==========
```
Looking good, thanks for the benchmarks!
I don't expect any problem (since you check the presence of the attribute), but can you merge latest master (I just merged a PR to fix CI build with pyarrow 0.15-
Thanks!