ENH: Add Arrow CSV Reader
State: closed

- [ ] closes #23697
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [ ] whatsnew entry

Picking up from #38370. 
Perf is OK. (StringIO/Text buffer reading gets like 20% overhead b/c of BytesIOWrapper, we should probably look into more efficient ways of doing it/cythonize it. Bytes IO performance looks good though).

Benchmarks 100k rows, 5 col
```
[  0.00%] ··· io.csv.ReadCSVEngine.time_read_bytescsv                                                                                                                             ok
[  0.00%] ··· ========= ============
                engine              
              --------- ------------
                  c       22.9±2ms  
                python    259±20ms  
               pyarrow   5.79±0.3ms 
              ========= ============

[  0.00%] ··· io.csv.ReadCSVEngine.time_read_stringcsv                                                                                                                            ok
[  0.00%] ··· ========= ============
                engine              
              --------- ------------
                  c       20.6±1ms  
                python    257±1ms   
               pyarrow   7.06±0.3ms 
              ========= ============
```

Tests will still fail tho, working on that rn.

My machine is a 2019 MBP(Intel) with 6 cores. YMMV based on the number of CPUs you have. Surprised that it doesn't quite scale linearly, maybe the file is too small to feel the benefit of all 6 cores.

Labels:
name: Enhancement
name: Performance
description: Memory or execution speed performance
name: IO CSV
description: read_csv, to_csv
name: Arrow
description: pyarrow functionality

Comments:
> Picking up from #38370.
> Perf is OK. (StringIO gets like 20% overhead b/c of BytesIOWrapper, we should probably look into more efficient ways of doing it/cythonize it).

Isn't the `BytesIOWrapper` only used when the user provides a text-handle? If the user provides something path-like, `get_handle` should (hopefully) directly open it in binary mode, or?
Yup, that's why the StringIO benchmark is slower(``bytescsv`` is unaffected as you can see above).. Hopefully the bytes benchmark is closer to the true speed of pyarrow, but haven't profiled. Perf there does look reasonable to me.

Sorry for being a little ambiguous in the description.

I think I've addressed all the comments on the previous PR. LMK otherwise. This is ready for a preliminary review now, will ping everyone involved once I get CI to green.(22 tests to go left, but some tests are getting stuck and timing out).
as green as it gets.
also cc @arw2019 @xhochy 
Raised exceptions tested here.
https://github.com/pandas-dev/pandas/pull/43072/files#diff-2a0f93df26fa0ee1d10f216142db735ff227fcfb02f161ad165fdfcb3f767cd1R125-R146. 
@pandas-dev/pandas-core if any comments. Ideally we could do these in a followup.
Ok, I think we should try to merge this now, if no other comments(it has been 1 week). Planning on enabling more features and doing some parser cleanup after this.
thanks @lithomas1 very nice!


pls create a follow up issue with checkboxes
thanks @arw2019 for a lot of the initial work here!
#38889 
#38872

FWIW, is it possible to create an Arrow label for pyarrow things? Arrow issues are still filed under IO Parquet which is wrong. I think this might be relevant for things like the ArrowStringArray too.
i created a label just now 
im seeing pyarrow io tests stalling out locally (and not KeyboardInterrupt-able), can anyone else on mac confirm?
Maybe try setting the env var ``OMP_NUM_THREADS`` to 1? We should be running these tests single threaded right now.
I'll try to reproduce when I get back on my mac, but its interesting how this is not failing on CI.
Ok, looking into this right now. I think I can reproduce locally, but interested in how this doesn't reproduce on CI.
I fixed it locally by upgrading pyarrow