PERF: Speedup in printing DataFrames of long strings
State: closed

Currently, when printing DataFrames, each element of a table to be printed is first formatted in its entirety by `GenericArrayFormatter._format_strings` and then truncated by `_make_fixed_width`.  When processing DataFrames that contain very long strings, there is a noticeable computation delay when calling `DataFrame.__repr__`.

The bottleneck is in `_format_strings`, where each string runs several `str.replace` to format it for printing.  We can eliminate wasted time by slicing any any strings longer than the maximum column width **before** pretty formatting the string, leaving enough buffer space such that formatting the string won't bring it below the maximum column width.

### Performance Demonstration
Setup:
```python
>>> data = pd.DataFrame([str(list(range(5000000)))]*50)
>>> len(data.values[0][0]) #length of each string
3888890
>>> _ = data.__repr__()
```

Final steps goes from 0.97386 seconds to 0.0089 seconds to execute.

Labels:
name: Performance
description: Memory or execution speed performance
name: Output-Formatting
description: __repr__ of pandas objects, to_string

Comments:
Hello @david-liu-brattle-1! Thanks for updating the PR.



Cheers ! There are no PEP8 issues in this Pull Request. :beers: 

##### Comment last updated on June 18, 2018 at 19:57 Hours UTC
# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/21110?src=pr&el=h1) Report
> Merging [#21110](https://codecov.io/gh/pandas-dev/pandas/pull/21110?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/9e982e18678c47c83ce2bb5932f844e2d9ee21f3?src=pr&el=desc) will **increase** coverage by `<.01%`.
> The diff coverage is `99.14%`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/21110/graphs/tree.svg?token=eZ4WkYLtcO&src=pr&height=150&width=650)](https://codecov.io/gh/pandas-dev/pandas/pull/21110?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #21110      +/-   ##
==========================================
+ Coverage   91.92%   91.92%   +<.01%     
==========================================
  Files         153      153              
  Lines       49587    49590       +3     
==========================================
+ Hits        45583    45586       +3     
  Misses       4004     4004
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `90.32% <99.14%> (ø)` | :arrow_up: |
| #single | `41.89% <51.7%> (ø)` | :arrow_up: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/21110?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/core/base.py](https://codecov.io/gh/pandas-dev/pandas/pull/21110/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvYmFzZS5weQ==) | `96.83% <ø> (ø)` | :arrow_up: |
| [pandas/core/indexing.py](https://codecov.io/gh/pandas-dev/pandas/pull/21110/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW5kZXhpbmcucHk=) | `93.55% <ø> (ø)` | :arrow_up: |
| [pandas/io/gbq.py](https://codecov.io/gh/pandas-dev/pandas/pull/21110/diff?src=pr&el=tree#diff-cGFuZGFzL2lvL2dicS5weQ==) | `25% <ø> (ø)` | :arrow_up: |
| [pandas/core/reshape/reshape.py](https://codecov.io/gh/pandas-dev/pandas/pull/21110/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvcmVzaGFwZS9yZXNoYXBlLnB5) | `99.78% <ø> (ø)` | :arrow_up: |
| [pandas/core/groupby/groupby.py](https://codecov.io/gh/pandas-dev/pandas/pull/21110/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZ3JvdXBieS9ncm91cGJ5LnB5) | `92.66% <ø> (ø)` | :arrow_up: |
| [pandas/core/resample.py](https://codecov.io/gh/pandas-dev/pandas/pull/21110/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvcmVzYW1wbGUucHk=) | `96.07% <ø> (ø)` | :arrow_up: |
| [pandas/core/strings.py](https://codecov.io/gh/pandas-dev/pandas/pull/21110/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvc3RyaW5ncy5weQ==) | `98.62% <ø> (ø)` | :arrow_up: |
| [pandas/io/json/normalize.py](https://codecov.io/gh/pandas-dev/pandas/pull/21110/diff?src=pr&el=tree#diff-cGFuZGFzL2lvL2pzb24vbm9ybWFsaXplLnB5) | `96.87% <ø> (ø)` | :arrow_up: |
| [pandas/core/generic.py](https://codecov.io/gh/pandas-dev/pandas/pull/21110/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZ2VuZXJpYy5weQ==) | `96.12% <ø> (ø)` | :arrow_up: |
| [pandas/io/pytables.py](https://codecov.io/gh/pandas-dev/pandas/pull/21110/diff?src=pr&el=tree#diff-cGFuZGFzL2lvL3B5dGFibGVzLnB5) | `92.43% <ø> (ø)` | :arrow_up: |
| ... and [63 more](https://codecov.io/gh/pandas-dev/pandas/pull/21110/diff?src=pr&el=tree-more) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/21110?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/21110?src=pr&el=footer). Last update [9e982e1...eb1d348](https://codecov.io/gh/pandas-dev/pandas/pull/21110?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).