Speed up tokenizing of a row in csv and xstrtod parsing
State: closed

- [x] closes: N/A
- [x] tests passed
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry

I will update the PR when CI finishes running, as I locally tested `io.parser.test_common` only.


Labels:
name: Performance
description: Memory or execution speed performance
name: IO CSV
description: read_csv, to_csv

Comments:
`asv continuous -f 1.05 origin/master HEAD -b io.csv` results:

before           |after         |ratio|test name
-|-|-|-
[e8d951d2]|       [a4f6dcdb]||
**master**|       **speed-up-tokenizer**||
1.58±0.02ms|      1.49±0.02ms|     0.94|  io.csv.ReadCSVFloatPrecision.time_read_csv(',', '.', 'high')
5.23±0.1ms|      4.87±0.03ms|     0.93|  io.csv.ReadUint64Integers.time_read_uint64_na_values
10.1±0.2ms|       9.15±0.3ms|     0.91|  io.csv.ReadCSVSkipRows.time_skipprows(10000)
13.0±0.3ms|       11.6±0.2ms|     0.89|  io.csv.ReadCSVThousands.time_thousands('\|', None)

So this basically gives from 5% to 10% increase on mini-benchmarks. Note that this scales quite well, and on cases where csv files are million lines long it still gives same 5-10%.
The results are even more promising if I allow more warmup and more sampling time so Turboboost and frequency scaling don't impact the performance too much.

Running `asv continuous -f 1.05 origin/master HEAD -b io.csv -a sample_time=2 -a warmup_time=2` yields:

before           |after         |ratio|test name
-|-|-|-
[e8d951d2]       |[a4f6dcdb]||
**master**|       **speed-up-tokenizer**||
34.9±0.2ms|         32.9±1ms|     0.94|  io.csv.ReadCSVCategorical.time_convert_direct
13.5±0.02ms|      12.6±0.08ms|     0.93|  io.csv.ReadCSVThousands.time_thousands(',', ',')
5.10±0.07ms|      4.71±0.09ms|     0.92|  io.csv.ReadUint64Integers.time_read_uint64_neg_values
14.6±0.06ms|      13.0±0.04ms|     0.89|  io.csv.ReadCSVThousands.time_thousands('\|', ',')
16.0±0.3ms|      13.8±0.09ms|     0.86|  io.csv.ReadCSVSkipRows.time_skipprows(None)
10.3±0.1ms|       8.81±0.1ms|     0.86|  io.csv.ReadCSVSkipRows.time_skipprows(10000)
12.9±0.1ms|      10.7±0.09ms|     0.84|  io.csv.ReadCSVThousands.time_thousands('\|', None)
13.0±0.05ms|      10.8±0.08ms|     0.83|  io.csv.ReadCSVThousands.time_thousands(',', None)

@jreback I've fixed whatsnew entry per your comment and rebased to latest master for clean history.
# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/25784?src=pr&el=h1) Report
> Merging [#25784](https://codecov.io/gh/pandas-dev/pandas/pull/25784?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/46639512c06300a9844ea27f90167d5648c9b93a?src=pr&el=desc) will **decrease** coverage by `<.01%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/25784/graphs/tree.svg?width=650&token=eZ4WkYLtcO&height=150&src=pr)](https://codecov.io/gh/pandas-dev/pandas/pull/25784?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #25784      +/-   ##
==========================================
- Coverage   91.26%   91.26%   -0.01%     
==========================================
  Files         173      173              
  Lines       52982    52982              
==========================================
- Hits        48356    48355       -1     
- Misses       4626     4627       +1
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `89.83% <ø> (ø)` | :arrow_up: |
| #single | `41.76% <ø> (ø)` | :arrow_up: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/25784?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/util/testing.py](https://codecov.io/gh/pandas-dev/pandas/pull/25784/diff?src=pr&el=tree#diff-cGFuZGFzL3V0aWwvdGVzdGluZy5weQ==) | `89.3% <0%> (-0.11%)` | :arrow_down: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/25784?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/25784?src=pr&el=footer). Last update [4663951...12a6df9](https://codecov.io/gh/pandas-dev/pandas/pull/25784?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/25784?src=pr&el=h1) Report
> Merging [#25784](https://codecov.io/gh/pandas-dev/pandas/pull/25784?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/46639512c06300a9844ea27f90167d5648c9b93a?src=pr&el=desc) will **increase** coverage by `<.01%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/25784/graphs/tree.svg?width=650&token=eZ4WkYLtcO&height=150&src=pr)](https://codecov.io/gh/pandas-dev/pandas/pull/25784?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #25784      +/-   ##
==========================================
+ Coverage   91.26%   91.27%   +<.01%     
==========================================
  Files         173      173              
  Lines       52982    53002      +20     
==========================================
+ Hits        48356    48375      +19     
- Misses       4626     4627       +1
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `89.83% <ø> (ø)` | :arrow_up: |
| #single | `41.77% <ø> (+0.01%)` | :arrow_up: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/25784?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/util/testing.py](https://codecov.io/gh/pandas-dev/pandas/pull/25784/diff?src=pr&el=tree#diff-cGFuZGFzL3V0aWwvdGVzdGluZy5weQ==) | `89.3% <0%> (-0.11%)` | :arrow_down: |
| [pandas/core/series.py](https://codecov.io/gh/pandas-dev/pandas/pull/25784/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvc2VyaWVzLnB5) | `93.67% <0%> (-0.01%)` | :arrow_down: |
| [pandas/core/ops.py](https://codecov.io/gh/pandas-dev/pandas/pull/25784/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvb3BzLnB5) | `91.74% <0%> (ø)` | :arrow_up: |
| [pandas/core/frame.py](https://codecov.io/gh/pandas-dev/pandas/pull/25784/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZnJhbWUucHk=) | `96.79% <0%> (ø)` | :arrow_up: |
| [pandas/core/generic.py](https://codecov.io/gh/pandas-dev/pandas/pull/25784/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZ2VuZXJpYy5weQ==) | `93.52% <0%> (ø)` | :arrow_up: |
| [pandas/core/computation/expr.py](https://codecov.io/gh/pandas-dev/pandas/pull/25784/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvY29tcHV0YXRpb24vZXhwci5weQ==) | `88.52% <0%> (+0.35%)` | :arrow_up: |
| [pandas/core/computation/common.py](https://codecov.io/gh/pandas-dev/pandas/pull/25784/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvY29tcHV0YXRpb24vY29tbW9uLnB5) | `89.47% <0%> (+3.75%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/25784?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/25784?src=pr&el=footer). Last update [4663951...72c7570](https://codecov.io/gh/pandas-dev/pandas/pull/25784?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

lgtm.
> ping on green

@jreback all tests are green
thanks @vnlitvin nice patch!