Repository: pandas-dev/pandas
Pull request: BUG-17280 to_html follows display.precision for column numbers in notebooks
- [X] closes #17280
- [X] tests added / passed
- [X] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [X] whatsnew entry

When printing column labels, check if they are floats and if they are, then round according to display.precision preferences

Problem statement:
display.precision not honored for column headers
#### Code Sample, a copy-pastable example if possible

```python
import pandas as pd
import numpy as np
pd.options.display.precision = 3
labels = np.random.rand(10)
p = pd.DataFrame(np.random.randn(10,10),index=labels,columns=labels)
p

```
#### Problem description

The above code works just fine in a standard console, but in a jupyter notebook the float valued column headers are not displayed with a precision of 3.  This appears to be due to the notebook_repr_html option, since setting that to False again honors the precision.  Note that the same applies to other formatting options, and if I switch to a MultiIndex it looks fine. I posted this as a question on the pydata discussion site and Joris Van den Bossche confirmed that it was an issue and suggested that I report it here.

#### Expected Output

#### Output of ``pd.show_versions()``

<details>

[paste the output of ``pd.show_versions()`` here below this line]

</details>

Hints from issue discussion:
Thanks for reporting!

>  Note that the same applies to other formatting options

What do you mean exactly with this? That also other formatting options don't work with the notebook repr?

Yes. I couldn't remember them off the top of my head, but I think that there's a float_format option and an eng_float_format, or something like that. All behave the same way.

Thanks, Paul

Sent from my iPhone

On Aug 18, 2017, at 7:01 AM, Joris Van den Bossche <notifications@github.com<mailto:notifications@github.com>> wrote:


Thanks for reporting!

Note that the same applies to other formatting options

What do you mean exactly with this? That also other formatting options don't work with the notebook repr?

-
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub<https://github.com/pandas-dev/pandas/issues/17280#issuecomment-323361774>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AEG2KrjA9m1I944PDF6vhXglCny7ADzRks5sZZkQgaJpZM4O7jej>.

I believe this is not fixed in pandas 0.23.0 at least. Unless it should be seen as a Jupyter notebook issue after all. If we `print(p)` rather than simply `p`, the issue does not manifest (but in that case the display is more primitive than without the `print`).

Commit message: fix whitespace

Patch to review:
diff --git a/doc/source/whatsnew/v0.25.0.rst b/doc/source/whatsnew/v0.25.0.rst
index c7554a70bea91..3a0063a0aca09 100644
--- a/doc/source/whatsnew/v0.25.0.rst
+++ b/doc/source/whatsnew/v0.25.0.rst
@@ -353,6 +353,7 @@ I/O
 - Bug in :meth:`DataFrame.to_string` and :meth:`DataFrame.to_latex` that would lead to incorrect output when the ``header`` keyword is used (:issue:`16718`)
 - Bug in :func:`read_csv` not properly interpreting the UTF8 encoded filenames on Windows on Python 3.6+ (:issue:`15086`)
 - Improved performance in :meth:`pandas.read_stata` and :class:`pandas.io.stata.StataReader` when converting columns that have missing values (:issue:`25772`)
+- Bug in :meth:`DataFrame.to_html` where header numbers would ignore display options when rounding (:issue:`17280`)
 
 
 Plotting
diff --git a/pandas/io/formats/html.py b/pandas/io/formats/html.py
index 6ba24a95032f8..2b399377f9fea 100644
--- a/pandas/io/formats/html.py
+++ b/pandas/io/formats/html.py
@@ -491,6 +491,15 @@ class NotebookFormatter(HTMLFormatter):
     DataFrame._repr_html_() and DataFrame.to_html(notebook=True)
     """
 
+    def __init__(self, formatter, classes=None, border=None):
+        super(NotebookFormatter, self).__init__(formatter,
+                                                classes=classes,
+                                                border=border)
+        precision = get_option('display.precision')
+        fmt = lambda f: '{float:.{p}f}'.format(float=f, p=precision)
+        fmt_floats = lambda f: fmt(f) if isinstance(f, float) else f
+        self.columns = self.columns.map(fmt_floats)
+
     def _get_formatted_values(self):
         return {i: self.fmt._format_col(i) for i in range(self.ncols)}
 
diff --git a/pandas/tests/io/formats/test_to_html.py b/pandas/tests/io/formats/test_to_html.py
index 148a42a332661..6b0487d239d4a 100644
--- a/pandas/tests/io/formats/test_to_html.py
+++ b/pandas/tests/io/formats/test_to_html.py
@@ -633,3 +633,12 @@ def test_to_html_invalid_classes_type(classes):
 
     with pytest.raises(TypeError, match=msg):
         df.to_html(classes=classes)
+
+
+def test_to_html_round_column_headers():
+    df = DataFrame([1], columns=[0.55555])
+    with pd.option_context('display.precision', 3):
+        html = df.to_html(notebook=False)
+        notebook = df.to_html(notebook=True)
+    assert "0.55555" in html
+    assert "0.556" in notebook


Code review comments:
diff_hunk: @@ -491,6 +491,15 @@ class NotebookFormatter(HTMLFormatter):
     DataFrame._repr_html_() and DataFrame.to_html(notebook=True)
     """
 
+    def __init__(self, formatter, classes=None, border=None):
path: pandas/io/formats/html.py
text: @user1:
does this really need to be in ``__init__``, rather have this call a method in the base class which is overriden here, e.g.

``self.columns = self._get_columns_formatted_values()``

@user2:
> rather have this call a method in the base class which is overriden here

in #24651, i separated out the notebook functionality from `HTMLFormatter` into `NotebookFormatter` using `HTMLFormatter` as the base class. the base class of `HTMLFormatter` is `TableFormatter` which is also the base class of `DataFrameFormatter` (used for `to_string` and `to _latex`) and `LatexFormatter`.

I envisaged that at some point in the development cycle it may become desirable to also create a `ToHTMLFormatter` using `HTMLFormatter` as the base.

IMO `HTMLFormatter` and `NotebookFormatter` are not the appropriate location for non-markup related formatting issues that are common across output-formatting methods. I think this issue is in that category and should ideally be in `DataFrameFormatter` or `TableFormatter`.

However, if to close the open issue the fix is somewhere in `io/formats/html.py`, then i think a  TODO to remove the code at a later date should be added. 

@author:
>> and the bug was present in `to_html(notebook=False)` as well
>
> I would argue it is not a bug here. since the display options should not affect the to_html() output. compare with the display options, `max_rows`, `max_columns`, `show_dimensions`, `max_colwidth`, etc. which only apply to the notebook display.

I think I'm getting a bit confused here. Doesn't this mean that this is not an issue that is common across formatters? Because `to_html` should only check display preferences when used for display purposes in a notebook and not when generating HTML for a web page?

@user2:
yeah. i can understand why your confused, there is a slight problem with the current class hierarchy. the workaround for `max_colwidth` was to use `with option_context` to ignore the display option for `to_html`, see #24841


@user2:
i think the ideal solution would be to have say a `use_display_options` attribute in `DataFrameFormatter` and then in `HTMLFormatter` we just have `self.fmt.use_display_options = False` and in `NoteBookFormatter` we have `self.fmt.use_display_options = True` and then all the display formatting could be handled in `io/formats/format.py`.

But this is way outside the scope of this PR. So a `with option_context` work-around would be fine for now IMO.