WIP - Additional documentation for Performance Improvements using Pandas
State: closed

Building off https://github.com/pandas-dev/pandas/issues/16310, to start discussing ways to be more performant with pandas.


Labels:
name: Docs
name: Performance
description: Memory or execution speed performance

Comments:
# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/16439?src=pr&el=h1) Report
> Merging [#16439](https://codecov.io/gh/pandas-dev/pandas/pull/16439?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/49ec31bbaeca81a6f58fc1be26fe80f3ac188cdd?src=pr&el=desc) will **decrease** coverage by `<.01%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/16439/graphs/tree.svg?token=eZ4WkYLtcO&src=pr&width=650&height=150)](https://codecov.io/gh/pandas-dev/pandas/pull/16439?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #16439      +/-   ##
==========================================
- Coverage   90.42%   90.42%   -0.01%     
==========================================
  Files         161      161              
  Lines       51023    51023              
==========================================
- Hits        46138    46137       -1     
- Misses       4885     4886       +1
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `88.26% <ø> (-0.01%)` | :arrow_down: |
| #single | `40.17% <ø> (ø)` | :arrow_up: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/16439?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/core/common.py](https://codecov.io/gh/pandas-dev/pandas/pull/16439?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvY29tbW9uLnB5) | `91.05% <0%> (-0.34%)` | :arrow_down: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/16439?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/16439?src=pr&el=footer). Last update [49ec31b...e2ac61a](https://codecov.io/gh/pandas-dev/pandas/pull/16439?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/16439?src=pr&el=h1) Report
> Merging [#16439](https://codecov.io/gh/pandas-dev/pandas/pull/16439?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/49ec31bbaeca81a6f58fc1be26fe80f3ac188cdd?src=pr&el=desc) will **increase** coverage by `<.01%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/16439/graphs/tree.svg?token=eZ4WkYLtcO&src=pr&width=650&height=150)](https://codecov.io/gh/pandas-dev/pandas/pull/16439?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #16439      +/-   ##
==========================================
+ Coverage   90.42%   90.43%   +<.01%     
==========================================
  Files         161      161              
  Lines       51023    51048      +25     
==========================================
+ Hits        46138    46164      +26     
+ Misses       4885     4884       -1
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `88.27% <ø> (ø)` | :arrow_up: |
| #single | `40.16% <ø> (-0.01%)` | :arrow_down: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/16439?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/io/feather\_format.py](https://codecov.io/gh/pandas-dev/pandas/pull/16439?src=pr&el=tree#diff-cGFuZGFzL2lvL2ZlYXRoZXJfZm9ybWF0LnB5) | `85.71% <0%> (-2.17%)` | :arrow_down: |
| [pandas/core/frame.py](https://codecov.io/gh/pandas-dev/pandas/pull/16439?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZnJhbWUucHk=) | `97.66% <0%> (-0.03%)` | :arrow_down: |
| [pandas/core/reshape/merge.py](https://codecov.io/gh/pandas-dev/pandas/pull/16439?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvcmVzaGFwZS9tZXJnZS5weQ==) | `94.18% <0%> (-0.01%)` | :arrow_down: |
| [pandas/errors/\_\_init\_\_.py](https://codecov.io/gh/pandas-dev/pandas/pull/16439?src=pr&el=tree#diff-cGFuZGFzL2Vycm9ycy9fX2luaXRfXy5weQ==) | `100% <0%> (ø)` | :arrow_up: |
| [pandas/core/missing.py](https://codecov.io/gh/pandas-dev/pandas/pull/16439?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvbWlzc2luZy5weQ==) | `84.27% <0%> (ø)` | :arrow_up: |
| [pandas/core/reshape/reshape.py](https://codecov.io/gh/pandas-dev/pandas/pull/16439?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvcmVzaGFwZS9yZXNoYXBlLnB5) | `99.28% <0%> (ø)` | :arrow_up: |
| [pandas/io/formats/style.py](https://codecov.io/gh/pandas-dev/pandas/pull/16439?src=pr&el=tree#diff-cGFuZGFzL2lvL2Zvcm1hdHMvc3R5bGUucHk=) | `95.92% <0%> (+0.01%)` | :arrow_up: |
| [pandas/io/parsers.py](https://codecov.io/gh/pandas-dev/pandas/pull/16439?src=pr&el=tree#diff-cGFuZGFzL2lvL3BhcnNlcnMucHk=) | `95.33% <0%> (+0.01%)` | :arrow_up: |
| [pandas/core/ops.py](https://codecov.io/gh/pandas-dev/pandas/pull/16439?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvb3BzLnB5) | `91.69% <0%> (+0.01%)` | :arrow_up: |
| [pandas/plotting/\_core.py](https://codecov.io/gh/pandas-dev/pandas/pull/16439?src=pr&el=tree#diff-cGFuZGFzL3Bsb3R0aW5nL19jb3JlLnB5) | `81.92% <0%> (+0.02%)` | :arrow_up: |
| ... and [3 more](https://codecov.io/gh/pandas-dev/pandas/pull/16439?src=pr&el=tree-more) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/16439?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/16439?src=pr&el=footer). Last update [49ec31b...16467cb](https://codecov.io/gh/pandas-dev/pandas/pull/16439?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

Thank you for the feedback, will incorporate it now.

Sent from my iPhone

> On May 23, 2017, at 3:51 AM, Jeff Reback <notifications@github.com> wrote:
> 
> @jreback commented on this pull request.
> 
> In doc/source/index.rst.template:
> 
> > @@ -146,6 +146,7 @@ See the package overview for more detail about what's in the library.
>      comparison_with_r
>      comparison_with_sql
>      comparison_with_sas
> +    performance
> yes putting this at the top of enhancing perf is just fine
> 
> —
> You are receiving this because you authored the thread.
> Reply to this email directly, view it on GitHub, or mute the thread.
> 

Still need to add notes from the pycon 2017 lecture, and bullets 3-5. 
Thanks will fix today!
if you can update and respond to comments.
closing as stale if you'd like to cointinue, pls ping