Escape special characters in to_latex() output
State: closed

Some characters have special meaning in latex: & % $ # _ { } ~ ^ .  When a dataframe contains a special character (as part of an index label, for example), to_latex() spits out invalid latex. 


Labels:

Comments:
May consider escaping all of these:

```
& % $ # _ { } ~ ^ \
```

Pandas already deals with three of them:

```
% & _
```

Looks fine, need a minimal test case to ensure this behavior comes out
correctly (i.e., a few rows, covering all the characters you mention and
then make sure that everything was escaped appropriately).

can you add some tests for this?

@y-p hows this look? 

Looks ok to me.

@bjornarneson can you squash this down to a single commit?

see this: https://github.com/pydata/pandas/wiki/Using-Git

@bjornarneson ...look ok and passes...ready to merge?

actually...can you add a release note that indicates this change? (in 0.13.1 improvements would be fine, ref this PR as the issue)

pls rebase this on master and squash down to a single commit....see here:https://github.com/pydata/pandas/wiki/Using-Git

@jreback It is probably obvious that I know only enough Git to be dangerous.... I have tried to follow the directions at the link you provided, but have been stymied by failed merges/pushes/rebases/etc. Short of starting over completely, fixing the mess I have made is beyond my current abilities right now. I am sorry about this!

Someone can walk you through it; git can be frustrating at first (and once you're familiar with it, come to think of it). Checkout your `patch-1` branch and do a `git status` to start and report back what you see.

Thanks a lot -- I'd like to learn how to do this properly. I checked out a clean copy of my patch-1 branch:

```
bjorna@9722-HP9470 ~/Documents/GitHub/pandas (patch-1)
$ git status
# On branch patch-1
nothing to commit, working directory clean
```

Then I fetch/rebase:

```
git fetch upstream
git rebase --interactive upstream/master
```

...and use the interactive window to squash all the commits into one commit. Then I get this message:

```
bjorna@9722-HP9470 ~/Documents/GitHub/pandas (patch-1)
$ git rebase --interactive upstream/master
error: could not apply 34d700a... Escape special characters when writing datafra
me to latex

When you have resolved this problem, run "git rebase --continue".
If you prefer to skip this patch, run "git rebase --skip" instead.
To check out the original branch and stop rebasing, run "git rebase --abort".
Could not apply 34d700a... Escape special characters when writing dataframe to l
atex
```

Git status shows this:

```
bjorna@9722-HP9470 ~/Documents/GitHub/pandas (patch-1|REBASE-i 1/6)
$ git status
# rebase in progress; onto d7c6801
# You are currently rebasing branch 'patch-1' on 'd7c6801'.
#   (fix conflicts and then run "git rebase --continue")
#   (use "git rebase --skip" to skip this patch)
#   (use "git rebase --abort" to check out the original branch)
#
# Changes to be committed:
#   (use "git reset HEAD <file>..." to unstage)
#
#       modified:   pandas/core/format.py
#       modified:   pandas/tests/test_format.py
#
# Unmerged paths:
#   (use "git reset HEAD <file>..." to unstage)
#   (use "git add <file>..." to mark resolution)
#
#       both modified:      doc/source/release.rst
#
```

This is the point where I don't know how to proceed. Any teaching would be most appreciated.

That last output is saying that git can't automatically merge your changes to `release.rst` and what's been done on master since you made your changes. If you open up that file, `doc/source/release.rst`, you'll see that git inserted some stuff where the two branches overlap:

```
<<<<<<< HEAD

# some text here from head

=======
  - escape special characters when writing to latex (:issue: `5374`)
>>>>>>> 34d700a... Escape special characters when writing dataframe to latex
```

Delete the `<<<<<<< HEAD`, `=======`, and `>>>>>>> 34d700a... Escape special characters when writing dataframe to latex` and move your `- escape special characters when writing to latex (:issue:`5374`)` line just above the `.. _release.bug_fixes-0.13.1:` line.

Then add that file `git add doc/source/release.rst` to say you've fixed the conflicts, `git rebase --continue` and then `git push -f origin` to force push your changes.

Thanks Tom -- I appreciate the help. Hopefully this last commit is in good shape. I think I was fouling things up with another command somewhere between manually fixing the release.rst file and the force push.

@bjornarneson thanks!

and @TomAugspurger  thanks for the help!

Hi guys. Are there any way to switch off escaping? I.e. if there's a column with the latex-formatted text:

```
>>> tmp = pd.DataFrame({'a':[10,20,30], 'b':['$1\pm0.1$','$2\pm0.2$','$3\pm0.3$']})
>>> print tmp.to_latex()
\begin{tabular}{lrl}
\toprule
{} &   a &          b \\
\midrule
0 &  10 &  \$1\textbackslashpm0.1\$ \\
1 &  20 &  \$2\textbackslashpm0.2\$ \\
2 &  30 &  \$3\textbackslashpm0.3\$ \\
\bottomrule
\end{tabular}
```

`formatters` do not help in this situation:

```
print tmp.to_latex(formatters={'a':lambda x: '%d'%(x), 'b':lambda x: '%s'%(x)})
```

will result in the same output.

I have also been bitten by this, as I use latex formatting and even latex macros inside strings which are then to be rendered with to_latex:
The symbol lambda in the following table will render correctly in an ipython notebook, but not when using to_latex to render it (paste this into an ipython notebook, then use nbconvert --to-latex and try to run pdflatex on the result and you will see what I mean)

import pandas as pd # version 0.13 revno 5096 or later
pd.DataFrame._repr_latex\_ = pd.DataFrame.to_latex
df = pd.DataFrame(index=['value'])
df['$\lambda$'] = '3.14'
df.T

What I wanted to say with my former comment, I also need an option to switch off escaping of latex characters!

easy enough to do this

see #6472 
@dalcolmo @vfilimonov one of you want to do a PR?

simply add a `escape='True'` keyword to maintain compat, and then you can just turn it off....