ENH: Add Styler.pipe() method (#23229)
State: closed

Added `Styler.pipe()` method.  This allows users to easily apply and compose functions that operate on Styler objects, just like the `DataFrame.pipe()` method does for dataframes.

- [x] closes #23229
- [x] tests added / passed
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry

Labels:
name: Enhancement
name: Code Style
description: Code style, linting, code_checks

Comments:
Hello @nmusolino! Thanks for updating the PR.

 - There are no PEP8 issues in the file [`pandas/io/formats/style.py`](https://github.com/nmusolino/pandas/blob/28d7098d4a36c8bafe094e903fc57059257d3f59/pandas/io/formats/style.py) !

 - There are no PEP8 issues in the file [`pandas/tests/io/formats/test_style.py`](https://github.com/nmusolino/pandas/blob/28d7098d4a36c8bafe094e903fc57059257d3f59/pandas/tests/io/formats/test_style.py) !



##### Comment last updated on November 18, 2018 at 02:04 Hours UTC
# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/23384?src=pr&el=h1) Report
> Merging [#23384](https://codecov.io/gh/pandas-dev/pandas/pull/23384?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/0e7cf4814ae8f85dc90ed6d8e24f0459be0b75e3?src=pr&el=desc) will **increase** coverage by `<.01%`.
> The diff coverage is `100%`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/23384/graphs/tree.svg?width=650&token=eZ4WkYLtcO&height=150&src=pr)](https://codecov.io/gh/pandas-dev/pandas/pull/23384?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #23384      +/-   ##
==========================================
+ Coverage   92.31%   92.31%   +<.01%     
==========================================
  Files         161      161              
  Lines       51471    51473       +2     
==========================================
+ Hits        47515    47517       +2     
  Misses       3956     3956
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `90.7% <100%> (ø)` | :arrow_up: |
| #single | `42.43% <50%> (ø)` | :arrow_up: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/23384?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/io/formats/style.py](https://codecov.io/gh/pandas-dev/pandas/pull/23384/diff?src=pr&el=tree#diff-cGFuZGFzL2lvL2Zvcm1hdHMvc3R5bGUucHk=) | `96.71% <100%> (+0.01%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/23384?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/23384?src=pr&el=footer). Last update [0e7cf48...726d01d](https://codecov.io/gh/pandas-dev/pandas/pull/23384?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

I have not pushed any changes yet, but this what the `pipe()` docs look like after local changes:

<img width="771" alt="pandas_styler_pipe_docs_1037" src="https://user-images.githubusercontent.com/3150861/47611317-0fed4880-da39-11e8-8b9e-cb64a7068cba.png">

I'll summarize the changes made since the original push:

* Simplified the second part of the unit test
* Added "Notes" and "Examples" section to the docstring.  A rendering of the HTML documentation can be found in an earlier comment.
* Added a "version added" directive to the docstring (caught that one myself).
* Moved the "whatsnew" entry from a one-liner to a short paragraph/example.

I wasn't able to test how the whatsnew will be rendered.

At this point, the main questions are:
* Do the notes/examples for the new method look reasonable?  I'm open to any suggestions there.
* Should I leave this PR as five distinct commits, or manually rebase it into one commit?  I'm reasonably familiar with different approaches, but don't know what the common practice is for pandas.

I'm fine with skipping the example with f and g. That could also be made
into a ..code-block:: python directive.

On Thu, Nov 1, 2018 at 8:03 AM Nicholas Musolino <notifications@github.com>
wrote:

> *@nmusolino* commented on this pull request.
> ------------------------------
>
> In pandas/io/formats/style.py
> <https://github.com/pandas-dev/pandas/pull/23384#discussion_r230063306>:
>
> > @@ -1222,6 +1222,76 @@ class MyStyler(cls):
>
>          return MyStyler
>
> +    def pipe(self, func, *args, **kwargs):
>
> I am traveling at the moment and am not able to copy-paste the output at
> this moment. But my recollection is:
>
>    1. There were no docstring style issues identified by the
>    validate_docstrings.py script, in my most recent version.
>    2. The initial doctests on line 1260 (namely >>>
>    f(g(df.style.set_precision(3), arg1=a), arg2=b, arg3=c)) and the
>    subsequent rewritten version fail because the functions f and g are not
>    defined. I suppose the same thing happens in the analogous DataFrame.pipe
>    doctest
>    <https://github.com/pandas-dev/pandas/blob/0409521665bd436a10aea7e06336066bf07ff057/pandas/core/generic.py#L4230>
>    .
>    3. The later doctest on line 1287 executes as intended, but fails when
>    the actual output (a string like <pandas.io.formats.style.Styler at
>    0xaabcd538364>) does not match the listed output in the doctest (no
>    output included). I could change this to make it formally pass, but it
>    would require a doctest: +ELLIPSIS directive (to accommodate the
>    memory address in the string), and it would be inconsistent with other
>    doctests in this module (which don’t show the styler’s string
>    representation as output).
>
> —
> You are receiving this because your review was requested.
> Reply to this email directly, view it on GitHub
> <https://github.com/pandas-dev/pandas/pull/23384#discussion_r230063306>,
> or mute the thread
> <https://github.com/notifications/unsubscribe-auth/ABQHImhjtI1ukK57kFlbGC6WyVYQWrNxks5uqw0lgaJpZM4X9o-F>
> .
>

Merge conflict in 0.24.0, if you could update.
I can resolve the merge conflict and turn the first example into a `code-block` instead of a doctest.  I’ll also clean up the whatsnew entry as discussed earlier.  I will push some changes Sunday night or Monday night. 


Here is the updated method documentation:

_[Large, outdated image removed for brevity]_

Here is the output from the `validate_docstring.py` script:
```
$ python ./scripts/validate_docstrings.py pandas.io.formats.style.Styler.pipe

################################################################################
############### Docstring (pandas.io.formats.style.Styler.pipe)  ###############
################################################################################

Apply ``func(self, *args, **kwargs)``, and return the result.

.. versionadded:: 0.24.0

Parameters
----------
func : function
    Function to apply to the Styler.
    ``*args``, and ``**kwargs`` are passed into ``func``.
    Alternatively a ``(callable, data_keyword)`` tuple where
    ``data_keyword`` is a string indicating the keyword of
    ``callable`` that expects the Styler.
*args : iterable, optional
    Positional arguments passed into ``func``.
**kwargs : dict, optional
    Dictionary of keyword arguments passed into ``func``.

Returns
-------
result : object
    The value returned by ``func``.

See Also
--------
DataFrame.pipe : Analogous method for DataFrame.
Styler.apply : Apply a function row-wise, column-wise, or table-wise to
    modify the dataframe's styling.

Notes
-----
Like :meth:`DataFrame.pipe`, this method can simplify the
application of several user-defined functions to a styler.  Instead
of writing:

.. code-block:: python

    f(g(df.style.set_precision(3), arg1=a), arg2=b, arg3=c)

users can write:

.. code-block:: python

    (df.style.set_precision(3)
       .pipe(g, arg1=a)
       .pipe(f, arg2=b, arg3=c))

In particular, this allows users to define functions that take a
styler object, along with other parameters, and return the styler after
making styling changes (such as calling :meth:`Styler.apply` or
:meth:`Styler.set_properties`).  Using ``.pipe``, these user-defined
style "transformations" can be interleaved with calls to the built-in
Styler interface.

Examples
--------
>>> def set_standard_formatting(styler):
...     return (styler.set_properties(**{'text-align': 'right'})
...                   .format({'X': '{:.1%}'}))

The user-defined highlight function above can be called within a
sequence of other style modifications:

>>> df = pd.DataFrame({'A': list(range(-1, 4)), 'X': np.arange(0.2, 1.2, 0.2)})
>>> (df.style
...    .set_properties(subset=['X'], **{'background-color': 'yellow'})
...    .pipe(set_standard_formatting)
...    .set_caption("Results with column 'X' highlighted."))

################################################################################
################################## Validation ##################################
################################################################################

Errors found:
	Examples do not pass tests

################################################################################
################################### Doctests ###################################
################################################################################

**********************************************************************
Line 65, in pandas.io.formats.style.Styler.pipe
Failed example:
    (df.style
       .set_properties(subset=['X'], **{'background-color': 'yellow'})
       .pipe(set_standard_formatting)
       .set_caption("Results with column 'X' highlighted."))
Expected nothing
Got:
    <pandas.io.formats.style.Styler object at 0x11c982b00>
```
@nmusolino a few lines are too long: 
https://github.com/pandas-dev/pandas/pull/23384#issuecomment-433654308
@TomAugspurger, I've fixed the line length issues, and made sure the "whatsnew" documentation builds correctly, and looks reasonable.

Any other changes you would suggest?  And would you like me to squash, rebase, and force-push to clean up the commit history?

An updated docstring rendering can be found below.

<img width="761" alt="screen shot 2018-11-17 at 9 02 47 pm" src="https://user-images.githubusercontent.com/3150861/48667800-a1862e00-eaac-11e8-954a-1511400db6f0.png">
@jreback @TomAugspurger , could you please take another look at this?  I have rebased twice in order to keep up with changes in the whatsnew file, and I'd like to get this PR ready for completion before rebasing again.
@TomAugspurger , I resolved the conflict, and CI is passing.  Would you be able to merge?
Looks good, thanks @nmusolino!