Repository: pandas-dev/pandas
Pull request: BUG: render dataframe as html do not produce duplicate element id's (#16780)
closes #16780 
tests added / passed
passes ``git diff upstream/master --name-only -- '*.py' | flake8 --diff``
whatsnew entry


Problem statement:
Pandas style render set_uuid sets Multiple ids
#### Code Sample, a copy-pastable example if possible

```python
df = pd.DataFrame({'a': [1, 3, 5, 6], 'b': [2, 4, 12, 21]})
df.style.render(uuid='test')
## or same with df.style.set_uuid('test').render()

<style  type="text/css" >\n</style>  \n<table id="T_test" > \n<thead>    <tr> \n        <th class="blank level0" ></th> \n        <th class="col_heading level0 col0" >a</th> \n        <th class="col_heading level0 col1" >b</th> \n    </tr></thead> \n<tbody>    <tr> \n        <th id="T_test" class="row_heading level0 row0" >0</th> \n        <td id="T_testrow0_col0" class="data row0 col0" >1</td> \n        <td id="T_testrow0_col1" class="data row0 col1" >2</td> \n    </tr>    <tr> \n        <th id="T_test" class="row_heading level0 row1" >1</th> \n        <td id="T_testrow1_col0" class="data row1 col0" >3</td> \n        <td id="T_testrow1_col1" class="data row1 col1" >4</td> \n    </tr>    <tr> \n        <th id="T_test" class="row_heading level0 row2" >2</th> \n        <td id="T_testrow2_col0" class="data row2 col0" >5</td> \n        <td id="T_testrow2_col1" class="data row2 col1" >12</td> \n    </tr>    <tr> \n        <th id="T_test" class="row_heading level0 row3" >3</th> \n        <td id="T_testrow3_col0" class="data row3 col0" >6</td> \n        <td id="T_testrow3_col1" class="data row3 col1" >21</td> \n    </tr></tbody> \n</table> 

```
#### Problem description

I am told that a css id should be unique and therefore, when `<table id="T_test" >` and `<th id="T_test"...>` in the above HTML output have the same ids, this breaks my css and causes the settings on my ids to be ignored.

#### Output of ``pd.show_versions()``

<details>
INSTALLED VERSIONS
------------------
commit: None
python: 3.5.2.final.0
python-bits: 64
OS: Linux
OS-release: 3.16.0-4-amd64
machine: x86_64
processor: 
byteorder: little
LC_ALL: None
LANG: en_GB.UTF-8
LOCALE: en_GB.UTF-8

pandas: 0.20.2
pytest: 2.9.2
pip: 9.0.1
setuptools: 27.2.0
Cython: 0.24.1
numpy: 1.12.1
scipy: 0.19.0
xarray: None
IPython: 5.1.0
sphinx: 1.4.6
patsy: 0.4.1
dateutil: 2.6.0
pytz: 2017.2
blosc: None
bottleneck: 1.2.0
tables: 3.4.2
numexpr: 2.6.2
feather: None
matplotlib: 2.0.2
openpyxl: 2.3.2
xlrd: 1.0.0
xlwt: 1.1.2
xlsxwriter: 0.9.3
lxml: 3.6.4
bs4: 4.5.1
html5lib: 0.999999999
sqlalchemy: 1.0.13
pymysql: 0.7.9.None
psycopg2: None
jinja2: 2.8
s3fs: None
pandas_gbq: None
pandas_datareader: None

</details>

Hints from issue discussion:
Thanks for the report. For reference the problem IDs are generated at

- https://github.com/pandas-dev/pandas/blob/65a0e642e1270791e6586c967758b362b865b6a8/pandas/io/formats/templates/html.tpl#L28
- https://github.com/pandas-dev/pandas/blob/65a0e642e1270791e6586c967758b362b865b6a8/pandas/io/formats/templates/html.tpl#L59

The second ones are supposed to be getting a `c.id` attribute, but apparently that isn't defined.
I see this is tagged as Novice and Effort Low. I would be happy to give this a try but I am not sure of the process to do this. Any tips/links would be great.
start here: http://pandas.pydata.org/pandas-docs/stable/contributing.html

create a test (e.g. the above), and see it fail. provide a fix, see your test pass.
I think the bug is somewhere around https://github.com/pandas-dev/pandas/blob/664348c440ccb4dab3d4c420d6aaee7c688c9b0e/pandas/io/formats/style.py#L235
I have setup my dev environment and written a the test,

```
def test_unique_ids():
    df = pd.DataFrame({'a': [1, 3, 5, 6], 'b': [2, 4, 12, 21]})
    test_str = df.style.render(uuid='test')
    a = re.findall('.*id=("T_test *\w*").*', test_str)
    assert len(set(a)) == len(a)
```

This successfully breaks when running pytest from my local pandas dir.

Since then I struggled to build and reinstall pandas with debugging breakpoints and `print()`. I assume this is the wrong way to go about it. Is there some further guidance on how to do this? Whatever I am doing has been hit and miss this far.

@TomAugspurger The problem areas were very helpful to get me started. It seems to me that [L284](https://github.com/pandas-dev/pandas/blob/664348c440ccb4dab3d4c420d6aaee7c688c9b0e/pandas/io/formats/style.py#L284) should have the an `id` attribute like [L308](https://github.com/pandas-dev/pandas/blob/664348c440ccb4dab3d4c420d6aaee7c688c9b0e/pandas/io/formats/style.py#L308). This is why the `<th id='T_test'>` I believe. Will continue this tomorrow.

**Edit**: Looks like I am too slow on this one. Still glad I found the right bits to fix.
Submitted pull request for fix for this issue.

Commit message: CLN: removing spaces in code causes pylint check to fail

Patch to review:
diff --git a/doc/source/whatsnew/v0.21.0.txt b/doc/source/whatsnew/v0.21.0.txt
index de2516d75040b..86952375f6068 100644
--- a/doc/source/whatsnew/v0.21.0.txt
+++ b/doc/source/whatsnew/v0.21.0.txt
@@ -92,6 +92,8 @@ Performance Improvements
 Bug Fixes
 ~~~~~~~~~
 
+- fix issue when rendering dataframe to html where element id's were not unique (:issue:`16780`)
+
 Conversion
 ^^^^^^^^^^
 
diff --git a/pandas/io/formats/style.py b/pandas/io/formats/style.py
index 3d7e0fcdc69b3..b08d3877f3b03 100644
--- a/pandas/io/formats/style.py
+++ b/pandas/io/formats/style.py
@@ -281,13 +281,14 @@ def format_attr(pair):
         for r, idx in enumerate(self.data.index):
             row_es = []
             for c, value in enumerate(rlabels[r]):
+                rid = [ROW_HEADING_CLASS, "level%s" % c, "row%s" % r]
                 es = {
                     "type": "th",
                     "is_visible": _is_visible(r, c, idx_lengths),
                     "value": value,
                     "display_value": value,
-                    "class": " ".join([ROW_HEADING_CLASS, "level%s" % c,
-                                       "row%s" % r]),
+                    "id": "_".join(rid[1:]),
+                    "class": " ".join(rid)
                 }
                 rowspan = idx_lengths.get((c, r), 0)
                 if rowspan > 1:
diff --git a/pandas/tests/io/formats/test_style.py b/pandas/tests/io/formats/test_style.py
index 9911888f758fb..59d9f938734ab 100644
--- a/pandas/tests/io/formats/test_style.py
+++ b/pandas/tests/io/formats/test_style.py
@@ -1,5 +1,6 @@
 import copy
 import textwrap
+import re
 
 import pytest
 import numpy as np
@@ -505,6 +506,14 @@ def test_uuid(self):
         assert result is styler
         assert result.uuid == 'aaa'
 
+    def test_unique_id(self):
+        # See https://github.com/pandas-dev/pandas/issues/16780
+        df = pd.DataFrame({'a': [1, 3, 5, 6], 'b': [2, 4, 12, 21]})
+        result = df.style.render(uuid='test')
+        assert 'test' in result
+        ids = re.findall('id="(.*?)"', result)
+        assert np.unique(ids).size == len(ids)
+
     def test_table_styles(self):
         style = [{'selector': 'th', 'props': [('foo', 'bar')]}]
         styler = Styler(self.df, table_styles=style)
@@ -719,12 +728,13 @@ def test_mi_sparse(self):
         df = pd.DataFrame({'A': [1, 2]},
                           index=pd.MultiIndex.from_arrays([['a', 'a'],
                                                            [0, 1]]))
+
         result = df.style._translate()
         body_0 = result['body'][0][0]
         expected_0 = {
             "value": "a", "display_value": "a", "is_visible": True,
             "type": "th", "attributes": ["rowspan=2"],
-            "class": "row_heading level0 row0",
+            "class": "row_heading level0 row0", "id": "level0_row0"
         }
         tm.assert_dict_equal(body_0, expected_0)
 
@@ -732,6 +742,7 @@ def test_mi_sparse(self):
         expected_1 = {
             "value": 0, "display_value": 0, "is_visible": True,
             "type": "th", "class": "row_heading level1 row0",
+            "id": "level1_row0"
         }
         tm.assert_dict_equal(body_1, expected_1)
 
@@ -739,6 +750,7 @@ def test_mi_sparse(self):
         expected_10 = {
             "value": 'a', "display_value": 'a', "is_visible": False,
             "type": "th", "class": "row_heading level0 row1",
+            "id": "level0_row1"
         }
         tm.assert_dict_equal(body_10, expected_10)
 


Code review comments:
diff_hunk: @@ -92,6 +92,8 @@ Performance Improvements
 Bug Fixes
 ~~~~~~~~~
 
+- fix issue when rendering dataframe to html where element id's were not unique (:issue:`16780`)
path: doc/source/whatsnew/v0.21.0.txt
text: @user1:
This can be moved to 0.20.3.txt (releasing next week)