Repository: pandas-dev/pandas
Pull request: PERF: only output an html id if a style is applied
- [x] closes #20695
- [x] tests added / passed
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`

Greatly reduces the amount of `id` tag in the cells by only assigning one when a style is applied to that cell. If that is the not correct approach for solving #20695, I am open to suggestions.

Also I was confused by the `%-` tags to handle the whitespaces. Is there a reason the default options to handle whitespace are not used?

Problem statement:
Huge html with df.style.render to due css duplications
When creating html with `df.style` e.g.
```python
df.style.apply(color_f, axis=1).render()
```
pandas assigns to each html cell of the table a unique css class. This means huge html are created, while it would be possible to use the same class for cell with the same style.

<details>

------------------
commit: None
python: 2.7.14.final.0
python-bits: 64
OS: Linux
OS-release: 4.15.14-300.fc27.x86_64
machine: x86_64
processor: x86_64
byteorder: little
LC_ALL: None
LANG: it_IT.UTF-8
LOCALE: None.None

pandas: 0.22.0
pytest: 3.1.3
pip: 9.0.1
setuptools: 36.0.1
Cython: 0.27.3
numpy: 1.14.2
scipy: 1.0.0
pyarrow: None
xarray: None
IPython: 5.4.1
sphinx: None
patsy: 0.4.1
dateutil: 2.7.2
pytz: 2018.4
blosc: None
bottleneck: 1.2.1
tables: 3.4.2
numexpr: 2.6.2
feather: None
matplotlib: 2.0.2
openpyxl: None
xlrd: 1.0.0
xlwt: 1.1.2
xlsxwriter: None
lxml: None
bs4: 4.6.0
html5lib: 0.9999999
sqlalchemy: 1.1.11
pymysql: None
psycopg2: None
jinja2: 2.9.6
s3fs: None
fastparquet: None
pandas_gbq: None
pandas_datareader: None
</details>

Hints from issue discussion:
Can you make a small copy-pastable example, and note which CSS classes you would want to exclude?

We could offer a couple optimizations

1. disable ID per cell
2. Disable row / col classes for cells that aren't referenced by a style

This will be a bit of work though.
Sure, here a very minimal example
```
import pandas as pd
import numpy as np
df = pd.DataFrame(np.random.random(size=(2, 2)), columns=list('AB'))
print df.style.render()
```

as you can see I am not applying any special style. Even in this trivial case I get one css class and id for each cell.

```
<style  type="text/css" >
</style>  
<table id="T_af65670a_40b3_11e8_9422_ac220bbe67c8" > 
<thead>    <tr> 
        <th class="blank level0" ></th> 
        <th class="col_heading level0 col0" >A</th> 
        <th class="col_heading level0 col1" >B</th> 
    </tr></thead> 
<tbody>    <tr> 
        <th id="T_af65670a_40b3_11e8_9422_ac220bbe67c8level0_row0" class="row_heading level0 row0" >0</th> 
        <td id="T_af65670a_40b3_11e8_9422_ac220bbe67c8row0_col0" class="data row0 col0" >0.628483</td> 
        <td id="T_af65670a_40b3_11e8_9422_ac220bbe67c8row0_col1" class="data row0 col1" >0.961722</td> 
    </tr>    <tr> 
        <th id="T_af65670a_40b3_11e8_9422_ac220bbe67c8level0_row1" class="row_heading level0 row1" >1</th> 
        <td id="T_af65670a_40b3_11e8_9422_ac220bbe67c8row1_col0" class="data row1 col0" >0.0814626</td> 
        <td id="T_af65670a_40b3_11e8_9422_ac220bbe67c8row1_col1" class="data row1 col1" >0.723978</td> 
    </tr></tbody> 
</table> 
```

In my real case I have thousands of row and hundres of lines, so I and end with html ~10Mb.

What you propose seems ok (don't know how the id is used), but as final solution I guess you have to group cell css class using the same classid if the cells have the same style.
You're welcome to take a look at https://github.com/pandas-dev/pandas/blob/d5d5a718254f45b4bdc386c360c830df395ec02a/pandas/io/formats/style.py#L407 and 
https://github.com/pandas-dev/pandas/blob/d5d5a718254f45b4bdc386c360c830df395ec02a/pandas/io/formats/style.py#L177 to explore how things can be (optionally) optimized.

Commit message: Add a parameter to Styler constructor

Patch to review:
diff --git a/pandas/io/formats/style.py b/pandas/io/formats/style.py
index f4bb53ba4f218..6474a2617ceb0 100644
--- a/pandas/io/formats/style.py
+++ b/pandas/io/formats/style.py
@@ -65,6 +65,8 @@ class Styler(object):
         a unique identifier to avoid CSS collisions; generated automatically
     caption: str, default None
         caption to attach to the table
+    all_ids: bool, default False
+        if True, each cell will have an ``id`` attribute in their HTML tag.
 
     Attributes
     ----------
@@ -113,7 +115,7 @@ class Styler(object):
     template = env.get_template("html.tpl")
 
     def __init__(self, data, precision=None, table_styles=None, uuid=None,
-                 caption=None, table_attributes=None):
+                 caption=None, table_attributes=None, all_ids=False):
         self.ctx = defaultdict(list)
         self._todo = []
 
@@ -137,6 +139,7 @@ def __init__(self, data, precision=None, table_styles=None, uuid=None,
         self.table_attributes = table_attributes
         self.hidden_index = False
         self.hidden_columns = []
+        self.all_ids = all_ids
 
         # display_funcs maps (row, col) -> formatting function
 
@@ -307,14 +310,16 @@ def format_attr(pair):
                 cs.extend(cell_context.get("data", {}).get(r, {}).get(c, []))
                 formatter = self._display_funcs[(r, c)]
                 value = self.data.iloc[r, c]
-                row_es.append({
-                    "type": "td",
-                    "value": value,
-                    "class": " ".join(cs),
-                    "id": "_".join(cs[1:]),
-                    "display_value": formatter(value),
-                    "is_visible": (c not in hidden_columns)
-                })
+                row_dict = {"type": "td",
+                            "value": value,
+                            "class": " ".join(cs),
+                            "display_value": formatter(value),
+                            "is_visible": (c not in hidden_columns)}
+                # only add an id if the cell has a style
+                if self.all_ids or \
+                        not(len(ctx[r, c]) == 1 and ctx[r, c][0] == ''):
+                    row_dict["id"] = "_".join(cs[1:])
+                row_es.append(row_dict)
                 props = []
                 for x in ctx[r, c]:
                     # have to handle empty styles like ['']
diff --git a/pandas/io/formats/templates/html.tpl b/pandas/io/formats/templates/html.tpl
index 706db1ecdd961..36e65e25496fe 100644
--- a/pandas/io/formats/templates/html.tpl
+++ b/pandas/io/formats/templates/html.tpl
@@ -50,17 +50,21 @@
 {%- endblock thead %} 
 {%- block tbody %} 
 <tbody> 
-    {%- block before_rows %}{%- endblock before_rows %}
-    {%- for r in body %} 
-    {%- block tr scoped %}
-    <tr> 
-        {%- for c in r %} 
-        {%- if c.is_visible != False %} 
+    {% block before_rows %}{% endblock before_rows %}
+    {% for r in body %}
+    {% block tr scoped %}
+    <tr>
+        {%- for c in r %}
+        {%- if c.is_visible != False %}
+        {%- if c.id: %}
         <{{ c.type }} id="T_{{ uuid }}{{ c.id }}" class="{{ c.class }}" {{ c.attributes|join(" ") }}>{{ c.display_value }}</{{ c.type }}>
-        {%- endif %} 
-        {%- endfor %} 
-    </tr> 
-    {%- endblock tr %}
+        {% else: -%}
+        <{{ c.type }} class="{{ c.class }}" {{ c.attributes|join(" ") }}>{{ c.display_value }}</{{ c.type }}>
+        {%- endif %}
+        {%- endif %}
+        {%- endfor %}
+    </tr>
+    {% endblock tr %}
     {%- endfor %} 
     {%- block after_rows %}{%- endblock after_rows %}
 </tbody> 


Code review comments:
diff_hunk: @@ -65,6 +65,8 @@ class Styler(object):
         a unique identifier to avoid CSS collisions; generated automatically
     caption: str, default None
         caption to attach to the table
+    all_ids: bool, default False
path: pandas/io/formats/style.py
text: @user1:
Perhaps ``cell_ids`` is a bit more descriptive?

@user1:
And the default should be True (backwards compatible)

diff_hunk: @@ -65,6 +65,8 @@ class Styler(object):
         a unique identifier to avoid CSS collisions; generated automatically
     caption: str, default None
         caption to attach to the table
+    all_ids: bool, default False
+        if True, each cell will have an ``id`` attribute in their HTML tag.
path: pandas/io/formats/style.py
text: @user1:
Capitalize `if`.

Could you include the form these take? I believe it's `T_<uuid>_row<i>_col<j>` for values in the table.

diff_hunk: @@ -50,17 +50,21 @@
 {%- endblock thead %} 
 {%- block tbody %} 
 <tbody> 
-    {%- block before_rows %}{%- endblock before_rows %}
-    {%- for r in body %} 
-    {%- block tr scoped %}
-    <tr> 
-        {%- for c in r %} 
-        {%- if c.is_visible != False %} 
+    {% block before_rows %}{% endblock before_rows %}
+    {% for r in body %}
+    {% block tr scoped %}
+    <tr>
+        {%- for c in r %}
+        {%- if c.is_visible != False %}
+        {%- if c.id: %}
path: pandas/io/formats/templates/html.tpl
text: @user1:
I'm not that familiar with jinja, but presumably there's a way to deduplicate this a bit? I worry about these two cases drifting apart in the future.

diff_hunk: @@ -307,14 +310,16 @@ def format_attr(pair):
                 cs.extend(cell_context.get("data", {}).get(r, {}).get(c, []))
                 formatter = self._display_funcs[(r, c)]
                 value = self.data.iloc[r, c]
-                row_es.append({
-                    "type": "td",
-                    "value": value,
-                    "class": " ".join(cs),
-                    "id": "_".join(cs[1:]),
-                    "display_value": formatter(value),
-                    "is_visible": (c not in hidden_columns)
-                })
+                row_dict = {"type": "td",
+                            "value": value,
+                            "class": " ".join(cs),
+                            "display_value": formatter(value),
+                            "is_visible": (c not in hidden_columns)}
+                # only add an id if the cell has a style
+                if self.all_ids or \
path: pandas/io/formats/style.py
text: @user1:
This should just be `if self.cell_ids` I think. We don't want a special case for the number of styles.