Repository: pandas-dev/pandas

Issue:
BUG: to_latex() output broken when the index has a name
Hey folks,

I posted this on [SO](https://stackoverflow.com/questions/31582114/groupby-to-latex-output-broken) and was asked to file a report here as well. 

I'm trying to export `pandas.DataFrame.describe()` to `LaTex` using the `to_latex()`-method. This works all fine as long as I don't apply the `groupby()`-method beforehand. With a grouped DataFrame, the first row has no values, even though its label is `count`. Note that the first row of a grouped dataframe is used to mark down the variable used for grouping in iPython notebook.

I'm using pandas 0.16.2, python 3.
Is this a bug or am I doing something wrong? 

Cheers,
Jakob

Here some examples:

Without `groupby`:

```
\begin{tabular}{lr}
\toprule
{} &    IS\_FEMALE \\
\midrule
count &  2267.000000 \\
mean  &     0.384649 \\
...
...
75\%   &     1.000000 \\
max   &     1.000000 \\
\bottomrule
\end{tabular}
```

[![enter image description here](http://i.stack.imgur.com/gK10V.png)](http://i.stack.imgur.com/gK10V.png)

With `groupby`:

```
\begin{tabular}{llr}
\toprule
  &       &    IS\_FEMALE \\
\midrule
0 & count &              \\     % <-- note missing value here
  & mean &  1134.000000 \\
  & std &     0.554674 \\
...
...
  & 75\% &     0.000000 \\
  & max &     0.000000 \\
\bottomrule
\end{tabular}
```

[![enter image description here](http://i.stack.imgur.com/lt9z9.png)](http://i.stack.imgur.com/lt9z9.png)

Output in the notebook:

[![enter image description here](http://i.stack.imgur.com/wT8Tp.png)](http://i.stack.imgur.com/wT8Tp.png)



Discussion:
Thanks for the report! Can you:
- try to provide a small reproducible example? (so some code we can run that makes up a dummy dataframe and that reproduces the error)
- check if it is an issue with groupby, or just with `to_latex`. For example, if you create a similar dataframe comparable to the output of the groupby by hand, and then export it to latex, do you experience the same error?

Sure. This snippet re-creates the issue. Sorry for the messy DataFrame-construction. First time I create one with numpy.

```
import pandas as pd
import numpy as np

cols = ['Group','Value']
group = np.random.randint(2, size=10)
values = np.random.random_sample(10)
df = pd.DataFrame([group, values]).T
df.columns = cols

print(df.groupby('Group').describe().to_latex())
```

I don't really know how to test your second point, to be honest. The first 'blank' row of a groupby is just visualization, I reckon?

Thanks for the reproducible example! That indeeds triggers the error for me as well.

Here is an example of just a small dataframe that also shows the error (as it has as such nothing to do with the `groupby`, it is just that it creates a multi-index that `to_latex` handles incorrectly):

```

In [22]: df = pd.DataFrame({'a':[0,0,1,1], 'b':list('abab'), 'c':[1,2,3,4]})

In [23]: df = df.set_index(['a', 'b'])

In [24]: df
Out[24]:
     c
a b
0 a  1
  b  2
1 a  3
  b  4

In [25]: print(df.to_latex())
\begin{tabular}{llr}
\toprule
  &   &  c \\
\midrule
0 & a &    \\
  & b &  1 \\
1 & a &  2 \\
  & b &  3 \\
\bottomrule
\end{tabular}

```

It seems that all values are shifted one line below.

It seems this has something to do with the index level names:

```
In [35]: df.index.names = [None, None]

In [36]: df
Out[36]:
     c
0 a  1
  b  2
1 a  3
  b  4

In [37]: print df.to_latex()
\begin{tabular}{llr}
\toprule
  &   &  c \\
\midrule
0 & a &  1 \\
  & b &  2 \\
1 & a &  3 \\
  & b &  4 \\
\bottomrule
\end{tabular}

```

And possibly related: #9908

dupe if #2942 ?

No, I don't think so, as this one not only applies to multi-index:

```
In [45]: df = pd.DataFrame({'a':list('abc'), 'b':[1,2,3]})

In [46]: df = df.set_index(['a'])

In [47]: df
Out[47]:
   b
a
a  1
b  2
c  3

In [49]: print df.to_latex()
\begin{tabular}{lr}
\toprule
{} &  b \\
\midrule
a &    \\
a &  1 \\
b &  2 \\
c &  3 \\
\bottomrule
\end{tabular}
```

So it is something with the index name.

Is the proposed fix of [#9908](https://github.com/pydata/pandas/pull/9908) implemented in 0.16.2?

@jakbaum yes, it is already in 0.16.1. But it does not fix this one, it possibly fixed a related issue, but should look into more detail into that.

And very welcome to look into the problem if you want! It shouldn't be that hard I think.

also #8336

@jorisvandenbossche Your belief in my coding qualities honor me, but quite honestly: I don't think I'm capable of fixing this. I wouldn't even know how to start and I don't want to mess things up. Actually, I'm more of a copy-paste coder than anything else. :)

@jakbaum no problem, thanks for reporting it anyway!


Resolving patch:
diff --git a/pandas/core/format.py b/pandas/core/format.py
--- a/pandas/core/format.py
+++ b/pandas/core/format.py
@@ -636,11 +636,13 @@ def get_col_type(dtype):
         if self.index and isinstance(self.frame.index, MultiIndex):
             clevels = self.frame.columns.nlevels
             strcols.pop(0)
-            name = any(self.frame.columns.names)
+            name = any(self.frame.index.names)
             for i, lev in enumerate(self.frame.index.levels):
-                lev2 = lev.format(name=name)
+                lev2 = lev.format()
                 blank = ' ' * len(lev2[0])
                 lev3 = [blank] * clevels
+                if name:
+                    lev3.append(lev.name)
                 for level_idx, group in itertools.groupby(
                         self.frame.index.labels[i]):
                     count = len(list(group))
@@ -667,6 +669,8 @@ def write(buf, frame, column_format, strcols, longtable=False):
                 buf.write('\\toprule\n')
 
             nlevels = frame.columns.nlevels
+            if any(frame.index.names):
+                nlevels += 1
             for i, row in enumerate(zip(*strcols)):
                 if i == nlevels:
                     buf.write('\\midrule\n')  # End of header


Test patch:
diff --git a/pandas/tests/test_format.py b/pandas/tests/test_format.py
--- a/pandas/tests/test_format.py
+++ b/pandas/tests/test_format.py
@@ -2645,6 +2645,50 @@ def test_to_latex_multiindex(self):
 c3 & 0 &  0 &  1 &  2 &  3 \\
 \bottomrule
 \end{tabular}
+"""
+        self.assertEqual(result, expected)
+
+        # GH 10660
+        df = pd.DataFrame({'a':[0,0,1,1], 'b':list('abab'), 'c':[1,2,3,4]})
+        result = df.set_index(['a', 'b']).to_latex()
+        expected = r"""\begin{tabular}{llr}
+\toprule
+  &   &  c \\
+a & b &    \\
+\midrule
+0 & a &  1 \\
+  & b &  2 \\
+1 & a &  3 \\
+  & b &  4 \\
+\bottomrule
+\end{tabular}
+"""
+        self.assertEqual(result, expected)
+
+        result = df.groupby('a').describe().to_latex()
+        expected = r"""\begin{tabular}{llr}
+\toprule
+  &       &         c \\
+a & {} &           \\
+\midrule
+0 & count &  2.000000 \\
+  & mean &  1.500000 \\
+  & std &  0.707107 \\
+  & min &  1.000000 \\
+  & 25\% &  1.250000 \\
+  & 50\% &  1.500000 \\
+  & 75\% &  1.750000 \\
+  & max &  2.000000 \\
+1 & count &  2.000000 \\
+  & mean &  3.500000 \\
+  & std &  0.707107 \\
+  & min &  3.000000 \\
+  & 25\% &  3.250000 \\
+  & 50\% &  3.500000 \\
+  & 75\% &  3.750000 \\
+  & max &  4.000000 \\
+\bottomrule
+\end{tabular}
 """
         self.assertEqual(result, expected)