REF: repr - allow block to override values that get formatted
State: closed

This change allows an external defined Block to override which values are used by the SeriesFormatter and DataFrameFormatter. 
By default, the 'internal values' are used (currently it uses `Series._values` which calles Block.internal_values()), so for pandas itself nothing should change.

#xref https://github.com/pandas-dev/pandas/pull/17144

Labels:
name: Internals
description: Related to non-user accessible pandas implementation

Comments:
# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/17143?src=pr&el=h1) Report
> Merging [#17143](https://codecov.io/gh/pandas-dev/pandas/pull/17143?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/f2b0bdc9bc4e57e101e306db7555eb7db28172e9?src=pr&el=desc) will **decrease** coverage by `0.07%`.
> The diff coverage is `100%`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/17143/graphs/tree.svg?width=650&height=150&src=pr&token=eZ4WkYLtcO)](https://codecov.io/gh/pandas-dev/pandas/pull/17143?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #17143      +/-   ##
==========================================
- Coverage   91.03%   90.95%   -0.08%     
==========================================
  Files         161      161              
  Lines       49405    49411       +6     
==========================================
- Hits        44974    44944      -30     
- Misses       4431     4467      +36
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `88.74% <100%> (-0.04%)` | :arrow_down: |
| #single | `40.21% <100%> (-0.13%)` | :arrow_down: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/17143?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/core/internals.py](https://codecov.io/gh/pandas-dev/pandas/pull/17143?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW50ZXJuYWxzLnB5) | `94.03% <100%> (ø)` | :arrow_up: |
| [pandas/io/formats/format.py](https://codecov.io/gh/pandas-dev/pandas/pull/17143?src=pr&el=tree#diff-cGFuZGFzL2lvL2Zvcm1hdHMvZm9ybWF0LnB5) | `96.04% <100%> (ø)` | :arrow_up: |
| [pandas/core/series.py](https://codecov.io/gh/pandas-dev/pandas/pull/17143?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvc2VyaWVzLnB5) | `94.86% <100%> (-0.09%)` | :arrow_down: |
| [pandas/io/feather\_format.py](https://codecov.io/gh/pandas-dev/pandas/pull/17143?src=pr&el=tree#diff-cGFuZGFzL2lvL2ZlYXRoZXJfZm9ybWF0LnB5) | `20% <0%> (-65.72%)` | :arrow_down: |
| [pandas/io/gbq.py](https://codecov.io/gh/pandas-dev/pandas/pull/17143?src=pr&el=tree#diff-cGFuZGFzL2lvL2dicS5weQ==) | `25% <0%> (-58.34%)` | :arrow_down: |
| [pandas/core/frame.py](https://codecov.io/gh/pandas-dev/pandas/pull/17143?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZnJhbWUucHk=) | `97.56% <0%> (-0.2%)` | :arrow_down: |
| [pandas/core/generic.py](https://codecov.io/gh/pandas-dev/pandas/pull/17143?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZ2VuZXJpYy5weQ==) | `91.96% <0%> (-0.06%)` | :arrow_down: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/17143?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/17143?src=pr&el=footer). Last update [f2b0bdc...ec7ff5d](https://codecov.io/gh/pandas-dev/pandas/pull/17143?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/17143?src=pr&el=h1) Report
> Merging [#17143](https://codecov.io/gh/pandas-dev/pandas/pull/17143?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/8e6b09ff3a09de58e82da6dcabbfddba61a743d6?src=pr&el=desc) will **decrease** coverage by `0.03%`.
> The diff coverage is `100%`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/17143/graphs/tree.svg?token=eZ4WkYLtcO&width=650&src=pr&height=150)](https://codecov.io/gh/pandas-dev/pandas/pull/17143?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #17143      +/-   ##
==========================================
- Coverage   91.01%   90.98%   -0.04%     
==========================================
  Files         162      162              
  Lines       49497    49508      +11     
==========================================
- Hits        45051    45043       -8     
- Misses       4446     4465      +19
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `88.75% <100%> (-0.02%)` | :arrow_down: |
| #single | `40.27% <100%> (-0.06%)` | :arrow_down: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/17143?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/core/series.py](https://codecov.io/gh/pandas-dev/pandas/pull/17143?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvc2VyaWVzLnB5) | `95.04% <100%> (ø)` | :arrow_up: |
| [pandas/core/internals.py](https://codecov.io/gh/pandas-dev/pandas/pull/17143?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW50ZXJuYWxzLnB5) | `94.04% <100%> (+0.01%)` | :arrow_up: |
| [pandas/io/formats/format.py](https://codecov.io/gh/pandas-dev/pandas/pull/17143?src=pr&el=tree#diff-cGFuZGFzL2lvL2Zvcm1hdHMvZm9ybWF0LnB5) | `96.04% <100%> (ø)` | :arrow_up: |
| [pandas/io/gbq.py](https://codecov.io/gh/pandas-dev/pandas/pull/17143?src=pr&el=tree#diff-cGFuZGFzL2lvL2dicS5weQ==) | `25% <0%> (-58.34%)` | :arrow_down: |
| [pandas/plotting/\_converter.py](https://codecov.io/gh/pandas-dev/pandas/pull/17143?src=pr&el=tree#diff-cGFuZGFzL3Bsb3R0aW5nL19jb252ZXJ0ZXIucHk=) | `63.23% <0%> (-1.82%)` | :arrow_down: |
| [pandas/core/frame.py](https://codecov.io/gh/pandas-dev/pandas/pull/17143?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZnJhbWUucHk=) | `97.66% <0%> (-0.11%)` | :arrow_down: |
| [pandas/core/nanops.py](https://codecov.io/gh/pandas-dev/pandas/pull/17143?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvbmFub3BzLnB5) | `98.03% <0%> (ø)` | :arrow_up: |
| [pandas/core/generic.py](https://codecov.io/gh/pandas-dev/pandas/pull/17143?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZ2VuZXJpYy5weQ==) | `92.02% <0%> (ø)` | :arrow_up: |
| [pandas/io/parsers.py](https://codecov.io/gh/pandas-dev/pandas/pull/17143?src=pr&el=tree#diff-cGFuZGFzL2lvL3BhcnNlcnMucHk=) | `95.45% <0%> (ø)` | :arrow_up: |
| ... and [4 more](https://codecov.io/gh/pandas-dev/pandas/pull/17143?src=pr&el=tree-more) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/17143?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/17143?src=pr&el=footer). Last update [8e6b09f...b41b713](https://codecov.io/gh/pandas-dev/pandas/pull/17143?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

@jreback updated according to your feedback

There is actually also another possible, more invasive option, which is to actually move the actual formatting logic (ar at least which ArrayFormatter class to use for a specific block, not necessarily the full formatter) to the block level. 
So in practice the minimal could be to move the if/else logic in `format_array` to the block level (https://github.com/pandas-dev/pandas/blob/f4330611ff5ac1cbb4a89c4a7dab3d0900f9e64a/pandas/io/formats/format.py#L1729-L1749). But not sure if that would be better (it would seem a bit cleaner as it doesn't involve such tedious if/else based on the type in format.py, but not sure how it would interact with index formatting)
> But not sure if that would be better (it would seem a bit cleaner as it doesn't involve such tedious if/else based on the type in format.py, but not sure how it would interact with index formatting)

yes this is exactly the reason. Since we don't have a full fledged 1d array type, with instead it being implemented mostly in Index/Series and then *again* in some of the Block logic, this avoids duplicated logic (the Index/Series also call this for formatting).
@jreback any more comments?
lgtm. (minor comment). merge away. not sure if you had other changes, but can of course always follow up.
Hello @jorisvandenbossche! Thanks for updating the PR.











Cheers ! There are no PEP8 issues in this Pull Request. :beers: 

##### Comment last updated on August 03, 2017 at 21:53 Hours UTC
@jorisvandenbossche I can't see `_formatting_values` values used in geopandas.

should `_formatting_values` also be deprecated on Series/DataFrame and this PR reverted?
> I can't see _formatting_values values used in geopandas.

It's used in formats.py

> should _formatting_values also be deprecated on Series/DataFrame and this PR reverted?

It's not public, so if we would not need it anymore after some refactor, we can just remove it (no need to deprecate it). But at the moment we are still using it.

> But at the moment we are still using it.

yes..

https://github.com/pandas-dev/pandas/blob/a4a18a9a694ba2641ec3ba98afc20615b2d39ad7/pandas/io/formats/format.py#L250-L253

but not consistent with use in ExtensonArrayFormatter where `_values` is used for Series.

https://github.com/pandas-dev/pandas/blob/a4a18a9a694ba2641ec3ba98afc20615b2d39ad7/pandas/io/formats/format.py#L1191-L1192


> This change allows an external defined Block to override which values are used by the SeriesFormatter and DataFrameFormatter.
> By default, the 'internal values' are used (currently it uses `Series._values` which calles Block.internal_values()), so for pandas itself nothing should change.

so for pandas, `_formatting values` is not needed. and we can just use `_values` instead to be consistent with the underlying EA?


> so for pandas, _formatting values is not needed. and we can just use _values instead to be consistent with the underlying EA?

Should look in more detail to be sure, but, it might be that it is *only* still needed to handle the deprecation of EA._formatting_values