revert #16663, which was a revert of #16039
State: closed

i believe the issue with #16039 was the loss of the semantics provided by the "b" part of the mode in `fopen()`. this is a nop on unix-like systems, but appears to be significant on windows.

the equivalent functionality on windows in `open()` is the `O_BINARY` flag. this adds that to the `open()` call, and provides compat for it on platforms without the flag by defining it to 0.

while here, fix an fd leak in `new_file_source()` if the buffer allocation fails.

 - [ ] closes #xxxx
 - [ ] tests added / passed
 - [ ] passes ``git diff upstream/master --name-only -- '*.py' | flake8 --diff``
 - [ ] whatsnew entry


Labels:
name: IO CSV
description: read_csv, to_csv

Comments:
You need to re-add the test that I wrote to ensure that your change is correct.
i did think it was going too well...
# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/16675?src=pr&el=h1) Report
> Merging [#16675](https://codecov.io/gh/pandas-dev/pandas/pull/16675?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/0281886fbdf0d1837c2af08af15949c1d98bf612?src=pr&el=desc) will **decrease** coverage by `<.01%`.
> The diff coverage is `100%`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/16675/graphs/tree.svg?token=eZ4WkYLtcO&width=650&src=pr&height=150)](https://codecov.io/gh/pandas-dev/pandas/pull/16675?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #16675      +/-   ##
==========================================
- Coverage   90.92%   90.92%   -0.01%     
==========================================
  Files         161      161              
  Lines       49272    49271       -1     
==========================================
- Hits        44803    44802       -1     
  Misses       4469     4469
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `88.69% <100%> (-0.01%)` | :arrow_down: |
| #single | `40.22% <0%> (ø)` | :arrow_up: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/16675?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/io/parsers.py](https://codecov.io/gh/pandas-dev/pandas/pull/16675?src=pr&el=tree#diff-cGFuZGFzL2lvL3BhcnNlcnMucHk=) | `95.43% <100%> (-0.01%)` | :arrow_down: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/16675?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/16675?src=pr&el=footer). Last update [0281886...0a831b6](https://codecov.io/gh/pandas-dev/pandas/pull/16675?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/16675?src=pr&el=h1) Report
> Merging [#16675](https://codecov.io/gh/pandas-dev/pandas/pull/16675?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/0281886fbdf0d1837c2af08af15949c1d98bf612?src=pr&el=desc) will **decrease** coverage by `<.01%`.
> The diff coverage is `100%`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/16675/graphs/tree.svg?src=pr&token=eZ4WkYLtcO&height=150&width=650)](https://codecov.io/gh/pandas-dev/pandas/pull/16675?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #16675      +/-   ##
==========================================
- Coverage   90.92%   90.92%   -0.01%     
==========================================
  Files         161      161              
  Lines       49272    49271       -1     
==========================================
- Hits        44803    44802       -1     
  Misses       4469     4469
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `88.69% <100%> (-0.01%)` | :arrow_down: |
| #single | `40.22% <0%> (ø)` | :arrow_up: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/16675?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/io/parsers.py](https://codecov.io/gh/pandas-dev/pandas/pull/16675?src=pr&el=tree#diff-cGFuZGFzL2lvL3BhcnNlcnMucHk=) | `95.43% <100%> (-0.01%)` | :arrow_down: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/16675?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/16675?src=pr&el=footer). Last update [0281886...0a831b6](https://codecov.io/gh/pandas-dev/pandas/pull/16675?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/16675?src=pr&el=h1) Report
> Merging [#16675](https://codecov.io/gh/pandas-dev/pandas/pull/16675?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/0281886fbdf0d1837c2af08af15949c1d98bf612?src=pr&el=desc) will **decrease** coverage by `<.01%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/16675/graphs/tree.svg?height=150&width=650&token=eZ4WkYLtcO&src=pr)](https://codecov.io/gh/pandas-dev/pandas/pull/16675?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #16675      +/-   ##
==========================================
- Coverage   90.92%   90.92%   -0.01%     
==========================================
  Files         161      161              
  Lines       49272    49276       +4     
==========================================
+ Hits        44803    44804       +1     
- Misses       4469     4472       +3
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `88.68% <ø> (-0.01%)` | :arrow_down: |
| #single | `40.18% <ø> (-0.04%)` | :arrow_down: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/16675?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/core/indexes/datetimes.py](https://codecov.io/gh/pandas-dev/pandas/pull/16675?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW5kZXhlcy9kYXRldGltZXMucHk=) | `95.23% <0%> (-0.1%)` | :arrow_down: |
| [pandas/plotting/\_core.py](https://codecov.io/gh/pandas-dev/pandas/pull/16675?src=pr&el=tree#diff-cGFuZGFzL3Bsb3R0aW5nL19jb3JlLnB5) | `81.83% <0%> (-0.1%)` | :arrow_down: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/16675?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/16675?src=pr&el=footer). Last update [0281886...28ec409](https://codecov.io/gh/pandas-dev/pandas/pull/16675?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

@dgwynne : LGTM, well done re-patching your patch :smile:
@gfyoung : thanks. sorry for the breakage though. it never even occurred to me that there'd be encoding issues like this.

it's also encouraging to see how quickly such issues are handled. reverting the diff was obviously right at the time.
@dgwynne : No worries!  We can't see everything that breaks if we don't test for it. ;)
thanks @dgwynne !