DOC, ENH: Support memory_map for Python engine
State: closed

Title is self-explanatory.


Labels:
name: IO CSV
description: read_csv, to_csv

Comments:
## [Current coverage](https://codecov.io/gh/pydata/pandas/pull/13381?src=pr) is **84.24%**

> Merging [#13381](https://codecov.io/gh/pydata/pandas/pull/13381?src=pr) into [master](https://codecov.io/gh/pydata/pandas/branch/master?src=pr) will increase coverage by **0.01%**

``` diff
@@             master     #13381   diff @@
==========================================
  Files           138        138          
  Lines         50763      50775    +12   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
+ Hits          42756      42773    +17   
+ Misses         8007       8002     -5   
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last updated by [158ae5b...1f6aae9](https://codecov.io/gh/pydata/pandas/compare/158ae5bf1e2c15945590f8938bc97d3512ce162b...1f6aae9c68a88565b55f029e16a052493b516998)

Do you have any benchmarks showing positive performance gains?

If this is an unambiguous win, we should enable it by default rather than adding yet another option.

I can run benchmarks but remember this is AFAICT a performance vs memory tradeoff given what memory_map does.  I am inclined to leave this still as an option, though how many people use this option I am not sure.

Ran basic `%timeit` with CSV file sizes of 10000 rows and 1000000 rows, and the performance flip-flops between the two.  I think the I/O overhead starts to count as the file sizes get larger and larger, but I don't know if there is a specific cutoff.  I think this would indicate leaving it as an option until there are more conclusive results.

@shoyer this is an _already_ existing option, @gfyoung is just documenting. Though point taken that it doesn't exist for the python engine, but I think the consistency outweight having to special case options and think about which engine I am using.

I would also add that this PR does provide support for `memory_map` in the Python engine via Python's `mmap` library.

Just to clarify, this is already supported by the C engine via `**kwargs`, but is currently undocumented?

I'm OK with documenting it and adding it as another argument then -- we already have quite a few arguments for `read_csv`, adding another one won't make things much worse.

@shoyer : If you look at the signature <a href="http://pandas.pydata.org/pandas-docs/stable/generated/pandas.read_csv.html">here</a>, you will see that it is already in the signature **explicitly**.  However, no explanation of it is given.  Also, read @jreback comment <a href="https://github.com/pydata/pandas/pull/13381#issuecomment-224262023">above</a>.

Travis was having build issues, so I'm rerunning tests.  @jreback could you cancel this old build <a href="https://travis-ci.org/pydata/pandas/builds/135889496">here</a>?

@jreback : Added the `except Exception` block as @shoyer requested, and Travis is giving the green light.  Ready to merge if there are no other concerns.

thanks!