move io.c from using unbuffered fread()s to read()s.
State: closed

pandas already buffers reads coming from io.c itself, so it previously
used setbuf() to disable buffering inside fread(). however, certain
implementations of unbuffered stdio reads are sub-optimal. for example,
fread() in solaris ends up doing a read() for each individual byte of
the underlying filedescriptor, which turns out to be very slow.

instead, this code now open()s a file descritor and read()s directly
into the buffer that pandas has already allocated. this is effectively
what other libcs (eg, glibc) do underneath an unbuffered fread() anyway,
but this is more explicit.

while here, this tweaks the mmap backend to use open() too, and also
properly checks for mmap failure by comparing its result to MAP_FAILED
instead of NULL.

 - [ ] closes #xxxx
 - [ ] tests added / passed
 - [ ] passes ``git diff upstream/master --name-only -- '*.py' | flake8 --diff``
 - [ ] whatsnew entry


Labels:
name: IO CSV
description: read_csv, to_csv
name: Compat
description: pandas objects compatability with Numpy or Python functions

Comments:
https://travis-ci.org/pandas-dev/pandas/jobs/222988790
couple of cpplint errors.

Is there any kind of test which can detect this? (or just perf on *some* systems)?
As you implied, it isn't a hard failure. The problem is performance on Solaris libc based systems is bad. A test of the bad performance is simply to use `read_csv()` on such a system and compare the runtime before and after the change.
# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/16039?src=pr&el=h1) Report
> Merging [#16039](https://codecov.io/gh/pandas-dev/pandas/pull/16039?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/cd1031f28664928c7e258dbb04d1f5861bad6b37?src=pr&el=desc) will **increase** coverage by `<.01%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/16039/graphs/tree.svg?token=eZ4WkYLtcO&width=650&height=150&src=pr)](https://codecov.io/gh/pandas-dev/pandas/pull/16039?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #16039      +/-   ##
==========================================
+ Coverage   90.99%      91%   +<.01%     
==========================================
  Files         153      153              
  Lines       50481    50481              
==========================================
+ Hits        45937    45938       +1     
+ Misses       4544     4543       -1
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `88.76% <ø> (ø)` | :arrow_up: |
| #single | `40.43% <ø> (ø)` | :arrow_up: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/16039?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/core/common.py](https://codecov.io/gh/pandas-dev/pandas/pull/16039?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvY29tbW9uLnB5) | `90.68% <0%> (-0.35%)` | :arrow_down: |
| [pandas/util/testing.py](https://codecov.io/gh/pandas-dev/pandas/pull/16039?src=pr&el=tree#diff-cGFuZGFzL3V0aWwvdGVzdGluZy5weQ==) | `80.73% <0%> (+0.18%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/16039?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/16039?src=pr&el=footer). Last update [cd1031f...b36c1ff](https://codecov.io/gh/pandas-dev/pandas/pull/16039?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/16039?src=pr&el=h1) Report
> Merging [#16039](https://codecov.io/gh/pandas-dev/pandas/pull/16039?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/b17e28638995bbfc32bc3dd7c53cd8a343191e58?src=pr&el=desc) will **decrease** coverage by `0.02%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/16039/graphs/tree.svg?width=650&height=150&src=pr&token=eZ4WkYLtcO)](https://codecov.io/gh/pandas-dev/pandas/pull/16039?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #16039      +/-   ##
==========================================
- Coverage   90.84%   90.82%   -0.03%     
==========================================
  Files         159      159              
  Lines       50791    50791              
==========================================
- Hits        46143    46132      -11     
- Misses       4648     4659      +11
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `88.6% <ø> (-0.03%)` | :arrow_down: |
| #single | `40.36% <ø> (ø)` | :arrow_up: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/16039?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/plotting/\_converter.py](https://codecov.io/gh/pandas-dev/pandas/pull/16039?src=pr&el=tree#diff-cGFuZGFzL3Bsb3R0aW5nL19jb252ZXJ0ZXIucHk=) | `63.54% <0%> (-1.82%)` | :arrow_down: |
| [pandas/core/common.py](https://codecov.io/gh/pandas-dev/pandas/pull/16039?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvY29tbW9uLnB5) | `91.03% <0%> (+0.34%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/16039?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/16039?src=pr&el=footer). Last update [b17e286...b29b032](https://codecov.io/gh/pandas-dev/pandas/pull/16039?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

I'm not sure, but I don't think the new build failures are the result of b36c1ff, which was basically whitespace tweaks. Alternatively fixing lint has made a later check run which did hit a problem. However, I don't think that is the case here.
can you show running an asv (perf testing) on csv benchmarks (just to assert that nothing has changed).
ill try and figure out asv. in the meantime, here are some runs before and after the change.

the py code i tested with:
```
import pandas as pd

data = pd.read_csv('yellow_tripdata_2016-12.csv')
print('Length: {} rows.'.format(len(data)))
```
this is a zone (container) on a solaris derivative box (https://www.joyent.com/smartos):

simple wc (to warm the caches?):
```
[admin@ade240c5-d3bf-40b3-8b24-1940e686775a ~]$ time wc -l yellow_tripdata_2016-12.csv 
10449409 yellow_tripdata_2016-12.csv

real    0m0.973s
user    0m0.600s
sys     0m0.371s
```
before:
```
[admin@ade240c5-d3bf-40b3-8b24-1940e686775a ~]$ time python pd.py 
Length: 10449408 rows.

real    35m8.845s
user    3m24.646s
sys     31m42.769s
```
after:
```
[admin@ade240c5-d3bf-40b3-8b24-1940e686775a ~]$ time python pd.py 
Length: 10449408 rows.

real    0m36.593s
user    0m30.888s
sys     0m4.038s
```
pretty brutal.

on a linux box:

wc:
```
uqdgwynn@mango ~$ time wc -l yellow_tripdata_2016-12.csv 
10449409 yellow_tripdata_2016-12.csv

real    0m0.945s
user    0m0.409s
sys     0m0.517s
```
before:
```
uqdgwynn@mango ~$ time python pd.py 
Length: 10449408 rows.

real    1m25.830s
user    0m56.329s
sys     0m26.963s
uqdgwynn@mango ~$ time python pd.py 
Length: 10449408 rows.

real    1m17.969s
user    1m2.792s
sys     0m14.248s
```
after:
```
uqdgwynn@mango ~$ time python pd.py 
Length: 10449408 rows.

real    0m47.950s
user    0m37.004s
sys     0m8.825s
```

im surprised the linux test shows a difference.
i ran `asv continuous -f 1.1 -E virtualenv origin/parser-no-stdio HEAD` in the asv_bench directory, and after each individual benchmark it produced "BENCHMARKS NOT SIGNIFICANTLY CHANGED." in green.
this looks good perf wise (neutral on my tests), so if its better on other platforms +1.
thanks @dgwynne