BUG-17280 to_html follows display.precision for column numbers in notebooks
State: closed

- [X] closes #17280
- [X] tests added / passed
- [X] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [X] whatsnew entry

When printing column labels, check if they are floats and if they are, then round according to display.precision preferences

Labels:
name: Output-Formatting
description: __repr__ of pandas objects, to_string

Comments:
# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/25914?src=pr&el=h1) Report
> Merging [#25914](https://codecov.io/gh/pandas-dev/pandas/pull/25914?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/8b9f933b221f7a1e69c2c1559cfeef5e4dca7ef8?src=pr&el=desc) will **decrease** coverage by `<.01%`.
> The diff coverage is `100%`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/25914/graphs/tree.svg?width=650&token=eZ4WkYLtcO&height=150&src=pr)](https://codecov.io/gh/pandas-dev/pandas/pull/25914?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #25914      +/-   ##
==========================================
- Coverage   91.77%   91.77%   -0.01%     
==========================================
  Files         175      175              
  Lines       52607    52610       +3     
==========================================
- Hits        48282    48281       -1     
- Misses       4325     4329       +4
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `90.32% <100%> (ø)` | :arrow_up: |
| #single | `41.9% <0%> (-0.08%)` | :arrow_down: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/25914?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/io/formats/html.py](https://codecov.io/gh/pandas-dev/pandas/pull/25914/diff?src=pr&el=tree#diff-cGFuZGFzL2lvL2Zvcm1hdHMvaHRtbC5weQ==) | `99.36% <100%> (ø)` | :arrow_up: |
| [pandas/io/gbq.py](https://codecov.io/gh/pandas-dev/pandas/pull/25914/diff?src=pr&el=tree#diff-cGFuZGFzL2lvL2dicS5weQ==) | `75% <0%> (-12.5%)` | :arrow_down: |
| [pandas/core/frame.py](https://codecov.io/gh/pandas-dev/pandas/pull/25914/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZnJhbWUucHk=) | `96.79% <0%> (-0.12%)` | :arrow_down: |
| [pandas/io/formats/css.py](https://codecov.io/gh/pandas-dev/pandas/pull/25914/diff?src=pr&el=tree#diff-cGFuZGFzL2lvL2Zvcm1hdHMvY3NzLnB5) | `100% <0%> (ø)` | :arrow_up: |
| [pandas/io/formats/excel.py](https://codecov.io/gh/pandas-dev/pandas/pull/25914/diff?src=pr&el=tree#diff-cGFuZGFzL2lvL2Zvcm1hdHMvZXhjZWwucHk=) | `97.4% <0%> (ø)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/25914?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/25914?src=pr&el=footer). Last update [8b9f933...9040b98](https://codecov.io/gh/pandas-dev/pandas/pull/25914?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/25914?src=pr&el=h1) Report
> Merging [#25914](https://codecov.io/gh/pandas-dev/pandas/pull/25914?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/4814a2875d078d6e9e9300bd1f77af15ef4f0f0a?src=pr&el=desc) will **decrease** coverage by `<.01%`.
> The diff coverage is `100%`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/25914/graphs/tree.svg?width=650&token=eZ4WkYLtcO&height=150&src=pr)](https://codecov.io/gh/pandas-dev/pandas/pull/25914?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #25914      +/-   ##
==========================================
- Coverage   91.84%   91.83%   -0.01%     
==========================================
  Files         175      175              
  Lines       52550    52554       +4     
==========================================
- Hits        48266    48265       -1     
- Misses       4284     4289       +5
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `90.39% <100%> (ø)` | :arrow_up: |
| #single | `41.89% <0%> (-0.07%)` | :arrow_down: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/25914?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/io/formats/html.py](https://codecov.io/gh/pandas-dev/pandas/pull/25914/diff?src=pr&el=tree#diff-cGFuZGFzL2lvL2Zvcm1hdHMvaHRtbC5weQ==) | `99.36% <100%> (ø)` | :arrow_up: |
| [pandas/io/gbq.py](https://codecov.io/gh/pandas-dev/pandas/pull/25914/diff?src=pr&el=tree#diff-cGFuZGFzL2lvL2dicS5weQ==) | `75% <0%> (-12.5%)` | :arrow_down: |
| [pandas/core/frame.py](https://codecov.io/gh/pandas-dev/pandas/pull/25914/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZnJhbWUucHk=) | `96.79% <0%> (-0.12%)` | :arrow_down: |
| [pandas/util/testing.py](https://codecov.io/gh/pandas-dev/pandas/pull/25914/diff?src=pr&el=tree#diff-cGFuZGFzL3V0aWwvdGVzdGluZy5weQ==) | `90.61% <0%> (-0.11%)` | :arrow_down: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/25914?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/25914?src=pr&el=footer). Last update [4814a28...ebcc8f8](https://codecov.io/gh/pandas-dev/pandas/pull/25914?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

As this is a display option, shouldn't it only affect the notebook display and leave the `to_html` untouched?
I have tested this fix in a notebook and it works as expected. As for whether this is the appropriate location for a fix, `NotebookFormatter` is the class responsible for displaying data in Jupyter notebooks, and it inherits from `HTMLFormatter` (and the bug was present in `to_html(notebook=False)` as well).
> and the bug was present in `to_html(notebook=False)` as well

I would argue it is not a bug here. since the display options should not affect the to_html() output. compare with the display options, `max_rows`, `max_columns`, `show_dimensions`, `max_colwidth`, etc. which only apply to the notebook display.
Hello @JustinZhengBC! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:





There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2019-04-03 15:23:23 UTC
I agree. I have altered the fix so it only applies to `NotebookFormatter` and not `HTMLFormatter`, since `to_html` is used for display purposes in a notebook.
> As for whether this is the appropriate location for a fix

i've opened separate issues for float values in index names #25917, object indexes #25919 and float-like values #25920. So this is probably not the appropriate location but i guess as a bug-fix that would be OK.
I think if, for now, you just change

https://github.com/pandas-dev/pandas/blob/96a128eaa0a7425dd4285d219780ef29c2727e46/pandas/io/formats/html.py#L295

to 

```python
row.extend(self.columns.format())
```
it'll fix the open issue by making the behavior for the single level column labels consistent with the index labels and the `multiIndex` column labels.

if the behavior should be different in `to_html()` output than the notebook display then this would probably need to include changing the index labels and the `multiIndex` column labels, so outside the scope of this PR.

then in the future we probably need a `_get_formatted_column_labels` method for code parity with `DataFrameFormatter`
@simonjayhawkins your fix works. It also changes columns with `None` to `NaN` as a side effect. Is that okay? If so we can go with that.
> It also changes columns with `None` to `NaN` as a side effect. Is that okay? If so we can go with that.

i'm getting `nan`  changed to `NaN` for `None`, which is consistent with the index and the multiIndex columns cases.

```
import pandas as pd
import numpy as np
pd.options.display.precision = 3
labels = np.random.rand(10)
labels[3] = None
p = pd.DataFrame(np.random.randn(10,10),index=labels,columns=labels)
p
```
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>0.04785803831057456</th>
      <th>0.6401016774531172</th>
      <th>0.9164407229270924</th>
      <th>nan</th>
      <th>0.6926779069059682</th>
      <th>0.4610792740713505</th>
      <th>0.9600160671031376</th>
      <th>0.35015957890745153</th>
      <th>0.14431674279365037</th>
      <th>0.6268220237028823</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>0.048</th>
      <td>-1.836</td>
      <td>-0.269</td>
      <td>1.081</td>
      <td>0.267</td>
      <td>-0.195</td>
      <td>-0.586</td>
      <td>0.916</td>
      <td>2.162</td>
      <td>0.572</td>
      <td>0.198</td>
    </tr>
    <tr>
      <th>0.640</th>
      <td>0.757</td>
      <td>0.656</td>
      <td>1.502</td>
      <td>-0.705</td>
      <td>0.379</td>
      <td>1.254</td>
      <td>-0.393</td>
      <td>-0.478</td>
      <td>0.940</td>
      <td>1.735</td>
    </tr>
    <tr>
      <th>0.916</th>
      <td>0.298</td>
      <td>0.816</td>
      <td>0.239</td>
      <td>-0.166</td>
      <td>-0.503</td>
      <td>-0.261</td>
      <td>2.781</td>
      <td>-1.195</td>
      <td>-0.613</td>
      <td>1.763</td>
    </tr>
    <tr>
      <th>NaN</th>
      <td>-0.792</td>
      <td>0.622</td>
      <td>-0.738</td>
      <td>0.489</td>
      <td>-1.947</td>
      <td>-0.804</td>
      <td>0.578</td>
      <td>-0.086</td>
      <td>-1.157</td>
      <td>0.471</td>
    </tr>
    <tr>
      <th>0.693</th>
      <td>-0.414</td>
      <td>-0.122</td>
      <td>-1.248</td>
      <td>-0.410</td>
      <td>-1.132</td>
      <td>0.315</td>
      <td>-1.116</td>
      <td>-0.746</td>
      <td>-0.977</td>
      <td>2.093</td>
    </tr>
    <tr>
      <th>0.461</th>
      <td>2.213</td>
      <td>-0.615</td>
      <td>0.594</td>
      <td>-0.262</td>
      <td>1.749</td>
      <td>0.994</td>
      <td>-0.539</td>
      <td>1.187</td>
      <td>0.156</td>
      <td>0.673</td>
    </tr>
    <tr>
      <th>0.960</th>
      <td>-1.578</td>
      <td>0.199</td>
      <td>0.903</td>
      <td>-0.578</td>
      <td>0.859</td>
      <td>0.969</td>
      <td>-0.030</td>
      <td>-1.371</td>
      <td>-0.330</td>
      <td>-0.141</td>
    </tr>
    <tr>
      <th>0.350</th>
      <td>0.023</td>
      <td>1.053</td>
      <td>0.902</td>
      <td>1.489</td>
      <td>1.278</td>
      <td>-1.473</td>
      <td>0.910</td>
      <td>0.197</td>
      <td>-0.461</td>
      <td>0.714</td>
    </tr>
    <tr>
      <th>0.144</th>
      <td>1.797</td>
      <td>0.233</td>
      <td>-0.672</td>
      <td>-0.288</td>
      <td>-0.455</td>
      <td>-0.872</td>
      <td>-1.535</td>
      <td>-2.370</td>
      <td>1.545</td>
      <td>-0.962</td>
    </tr>
    <tr>
      <th>0.627</th>
      <td>-2.079</td>
      <td>-0.473</td>
      <td>-2.075</td>
      <td>0.391</td>
      <td>-0.228</td>
      <td>0.751</td>
      <td>0.155</td>
      <td>0.648</td>
      <td>-0.679</td>
      <td>-1.043</td>
    </tr>
  </tbody>
</table>

```
p.columns.format()
['0.249',
 '0.833',
 '0.730',
 'NaN  ',
 '0.520',
 '0.545',
 '0.960',
 '0.233',
 '0.247',
 '0.990']
```

```
multi = pd.MultiIndex.from_arrays([labels,labels])
p = pd.DataFrame(np.random.randn(10,10),index=multi,columns=multi)
p
```

<table border="1" class="dataframe">
  <thead>
    <tr>
      <th></th>
      <th></th>
      <th>0.249</th>
      <th>0.833</th>
      <th>0.730</th>
      <th>NaN</th>
      <th>0.520</th>
      <th>0.545</th>
      <th>0.960</th>
      <th>0.233</th>
      <th>0.247</th>
      <th>0.990</th>
    </tr>
    <tr>
      <th></th>
      <th></th>
      <th>0.249</th>
      <th>0.833</th>
      <th>0.730</th>
      <th>NaN</th>
      <th>0.520</th>
      <th>0.545</th>
      <th>0.960</th>
      <th>0.233</th>
      <th>0.247</th>
      <th>0.990</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>0.249</th>
      <th>0.249</th>
      <td>0.419</td>
      <td>0.043</td>
      <td>-0.141</td>
      <td>0.408</td>
      <td>-0.040</td>
      <td>1.376</td>
      <td>-0.041</td>
      <td>-1.439</td>
      <td>0.192</td>
      <td>-1.223</td>
    </tr>
    <tr>
      <th>0.833</th>
      <th>0.833</th>
      <td>0.880</td>
      <td>-1.105</td>
      <td>-0.302</td>
      <td>-0.652</td>
      <td>-1.237</td>
      <td>0.851</td>
      <td>-0.481</td>
      <td>1.156</td>
      <td>-0.787</td>
      <td>1.032</td>
    </tr>
    <tr>
      <th>0.730</th>
      <th>0.730</th>
      <td>-0.939</td>
      <td>-0.504</td>
      <td>-0.397</td>
      <td>0.647</td>
      <td>-0.898</td>
      <td>-0.219</td>
      <td>-0.285</td>
      <td>-1.120</td>
      <td>-1.268</td>
      <td>-0.277</td>
    </tr>
    <tr>
      <th>NaN</th>
      <th>NaN</th>
      <td>-0.072</td>
      <td>0.020</td>
      <td>1.558</td>
      <td>0.463</td>
      <td>-1.322</td>
      <td>0.388</td>
      <td>0.373</td>
      <td>0.107</td>
      <td>-1.913</td>
      <td>0.370</td>
    </tr>
    <tr>
      <th>0.520</th>
      <th>0.520</th>
      <td>-0.343</td>
      <td>-0.121</td>
      <td>2.011</td>
      <td>0.068</td>
      <td>0.409</td>
      <td>-0.326</td>
      <td>-0.485</td>
      <td>1.271</td>
      <td>1.244</td>
      <td>-0.313</td>
    </tr>
    <tr>
      <th>0.545</th>
      <th>0.545</th>
      <td>-0.977</td>
      <td>-0.755</td>
      <td>0.145</td>
      <td>0.154</td>
      <td>0.293</td>
      <td>1.243</td>
      <td>1.441</td>
      <td>-0.198</td>
      <td>-0.318</td>
      <td>0.339</td>
    </tr>
    <tr>
      <th>0.960</th>
      <th>0.960</th>
      <td>-1.041</td>
      <td>1.479</td>
      <td>-1.120</td>
      <td>0.101</td>
      <td>-1.302</td>
      <td>1.436</td>
      <td>0.572</td>
      <td>-0.806</td>
      <td>1.306</td>
      <td>-1.249</td>
    </tr>
    <tr>
      <th>0.233</th>
      <th>0.233</th>
      <td>-0.021</td>
      <td>-0.742</td>
      <td>-0.394</td>
      <td>0.185</td>
      <td>0.028</td>
      <td>-0.999</td>
      <td>1.373</td>
      <td>-1.598</td>
      <td>1.140</td>
      <td>-0.386</td>
    </tr>
    <tr>
      <th>0.247</th>
      <th>0.247</th>
      <td>0.280</td>
      <td>-0.246</td>
      <td>0.222</td>
      <td>-0.767</td>
      <td>-1.433</td>
      <td>0.144</td>
      <td>-0.466</td>
      <td>0.920</td>
      <td>0.941</td>
      <td>1.195</td>
    </tr>
    <tr>
      <th>0.990</th>
      <th>0.990</th>
      <td>0.633</td>
      <td>-1.737</td>
      <td>-0.649</td>
      <td>0.396</td>
      <td>-0.007</td>
      <td>-0.897</td>
      <td>0.635</td>
      <td>1.555</td>
      <td>-2.001</td>
      <td>0.047</td>
    </tr>
  </tbody>
</table>

@JustinZhengBC : lgtm. https://github.com/pandas-dev/pandas/pull/25914#discussion_r270387375 still to do?
@simonjayhawkins sorry about that, thanks for catching it.
@simonjayhawkins this is orthogonal to your other PR? #25983 
> @simonjayhawkins this is orthogonal to your other PR? #25983

yes. this is ok. probably a merge conflict on the test, but just an accept both.
thanks @JustinZhengBC