ENH: Styler.to_latex(): conditional styling with native latex format
State: closed

## Enhancing Styler to allow LaTeX Input 

- [x] closes #21673

also indirectly (by providing an alternative to DataFrame.to_latex):

- [x] closes #38328 (enh: adding highlighting options to to_latex function)
- [x] closes #28291 (enh: add option to format \hline in to_latex)
- [x] closes #26278 (easier digit format in to_latex) (although this seems to have been separately addressed)
- [x] closes #26111 (to_latex adds extra empty header row when index has a name even if index=False)
- [x] closes #20008 (to_latex shifts column names if MultiIndex contains empty label)
- [x] closes #13203 (enhancement: add optional row/col paramater to to_latex formatter) 
- [x] closes #41388 (text truncates in to_latex) 
- [x] closes #6491 (text truncates in to_latex)
- [x] closes #41636 (bug in latex formatting decimal of complex numbers)

This PR leverages the conditional rendering mechanics and unit tests in `Styler` to create a pure LaTeX version, i.e. a Styler that instead of CSS (attribute, value) tuples has LaTeX (command, options) tuples and renders directly to LaTeX with nested cell 

An extension to this is provided in another PR which converts an HTML-CSS Styler to a LaTeX Styler, and then renders in LaTeX.

### How is this achieved?

#### Internally:
 - [x] a new jinja2 template is created for generating `latex` output.
 - [x] a new `Styler._render_latex()` method is added to invoke the new template.
 - [x] a `Styler._translate_latex` method is added to make structural changes to the usual render dict `d` to make it suitable for latex template.
 - [x] parsing functions are added which do simple tasks to facilitate the jinja2 template's operation, including sparsifying multiindexes

#### For Users:
 - [x] a new `Styler.to_latex()` method is introduced to give the user control. 

## Comparison with DataFrame.to_latex()

### Input Arguments

 - Replicating the **io** arguments: ``buf``, ``encoding``, 
 - Replicating the **LaTeX** arguments: ``position``, ``caption``, ``label``, ``sparsify``, ``column_format``
 - Adding additional **LaTeX** arguments: ``position_float``, ``hrules``
 - Removing all of the **formatting** arguments: ``na_rep``, ``formatters``, ``float_format``, ``escape``, ``decimal``
     - The pattern ``styler.format(...).to_latex(...)`` fully replicates the functionality, and can be intermediately viewed in a Notebook.
     - The pattern ``styler.format(..)...format(..).to_latex(...)`` performs better formatting than a single ``to_latex(..)`` with formatting options could replicate.
     - For code maintenance it is better to only maintain the ``.format`` method rather than ``.to_latex`` as downstream.
  - Removing the pseudo-formatting arguments: ``columns``, ``index``, ``index_names``, ``header``, 
     - The pattern ``styler.hide_index().hide_columns()`` can replicate the first three, the last seemed a bit unnecessary.
  - Modifying the multiindex args:
     - Now we have 
``sparsify {bool}``
 ``sparse_index`` and ``sparse_columns`` and ``multicol_align {str}`` and ``multirow_align {str}``.
     - Previously the options were ``sparsify {bool}``, ``multicolumn {bool}``, ``multirow {bool}`` and ``multicolumn_format {str}``
  - Additional functionality not coded (yet?): ``bold_rows`` (maybe ``bold_labels``), and ``longtable``.

### Performance

Suppose a user will never want to create a LaTeX bigger than (200,20) so we test that size:

| Method | Time |
|---------|------|
| df.to_latex() | 151ms |
| df.style.to_latex() | 43ms |
| df.to_latex(float_format="{:.2f}".format) | 65ms |
| 
df.style.format(":.2f".format).to_latex()
 | 
61ms
 |
| df.style.format(":.2f".format).to_latex() | 45ms |

(this was improved by #41269)

Styler is broadly unaffected here by adding formatting methods, although `DataFrame.to_latex` is oddly much faster. Anyway the conclusion seems to be for this size frame and smaller performance is broadly the same, if not a bit better in Styler.

### Docs

Here are the rendered versions of the to_latex documentation

[docs_to_latex.zip](https://github.com/pandas-dev/pandas/files/6292495/docs_to_latex.zip)




Labels:
name: Enhancement
name: IO LaTeX
description: to_latex
name: Styler
description: conditional formatting using DataFrame.style

Comments:
after this is it possible to deprecate `DataFrame.to_latex` in favor of this?
> after this is it possible to deprecate `DataFrame.to_latex` in favor of this?

yes that was the initial goal, as it is for my other `to_html` PR, but it requires your thoughts on my treatment of the **input arguments** above - whether you agree with my reasons.
> this is making Styler quite a bit clunkier. Can you propose a more modular design, e.g. we do something similar for the string formatting. Where we create a rendering engine,e g. you could have Html, Latex, Excel. Mostly could keep the code separate (the rendering vs the ops things).
> 
> Please do this as a pre-cursor. Also would not be averse to making another level here (e.g. pandas/io/formats/style/html.py, latex.py, excel.py, base.py or similar)
> 
> even though extra nesting is usually a negative, here it can provide more clarity.

I will have a think about this and submit a pre-cursor PR, just focused on existing HTML-STyler and see if it fits what you are expecting.
> this is making Styler quite a bit clunkier. Can you propose a more modular design, e.g. we do something similar for the string formatting. Where we create a rendering engine,e g. you could have Html, Latex, Excel. Mostly could keep the code separate (the rendering vs the ops things).

@jreback I am divided in my opinion. I like modularity. But: There are only two template-based methods (Latex and HTML) and one completely different (Excel). Is there much benefit in separating them? If I understand correctly, Latex and HTML share much of their functionality.

@jreback 
The progress made on this following your comments is:
    - split styler into modular components (already merged): `to_latex` is added to `style.py`, and parsing and rending is added to `styler_render.py`.
    - rename `template` as `template_html` (already merged)
    - remove all latex info from the Styler user guide. All documentation is now on `Styler.to_latex`. Zip file in OP.

Let me know how to proceed - what, if any, needs to be improved.
Will look at this in the weekend
> wow, this looks very nice. can you show a rendered screen shot of an example (and the equiv html) more for curiosity.
## HTML
![Screen Shot 2021-04-23 at 13 12 34](https://user-images.githubusercontent.com/24256554/115863398-ad246e80-a435-11eb-811f-c313ae5511ba.png)

## LaTeX
![Screen Shot 2021-04-23 at 13 14 34](https://user-images.githubusercontent.com/24256554/115863538-e1982a80-a435-11eb-8186-c653673d3fd6.png)
![Screen Shot 2021-04-23 at 13 15 45](https://user-images.githubusercontent.com/24256554/115863640-0ab8bb00-a436-11eb-9b72-b560978cfd1e.png)



cc @pandas-dev/pandas-core for any further reviews
Recent commits updated the `sparsify` keyword arg now that separate `sparse_index` and `sparse_columns` are available.

@jreback pinging on green
thanks @attack68 very nice!

so basically the followons are
- add / update docs
- deprecate `DataFrame.to_latex`  / point to styler.to_latex (do we have an issue for this?)
@jreback I would also add that the short caption feature has to be implemented here before deprecating ``DataFrame.to_latex`` (see https://github.com/pandas-dev/pandas/issues/36267)
sure

@attack68 can u add a follow up issue of things that we need to keep before depreciation 
Thanks for the great work, @attack68! I can't wait to use it in my dissertation!