ENH Adding categorical parallel_coordinates implementation
State: closed

 - [x] closes #12341
 - [x] tests added / passed
 - [x] passes ``git diff upstream/master --name-only -- '*.py' | flake8 --diff``
 - [ ] whatsnew entry

Initial PR for #12341 

I have a couple of questions, the signature is different from the parallel_coordinates method and I was not sure if we need all of them (for example axvlines)?

I've added some tests but need suggestions on how to test _connect_polyline(). I was thinking of simply using a mock object for the axes objects and verifying their attributes were being accessed.

Some context, I have a new method called parallel_sets() which converts all of its inputs to a string expecting categorical data always, via a string_to_integer dictionary, and plots those integers.

Labels:
name: Enhancement
name: Visualization
description: plotting

Comments:
# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/16219?src=pr&el=h1) Report
> Merging [#16219](https://codecov.io/gh/pandas-dev/pandas/pull/16219?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/f1549668c84b481a67547bb46261d1394c8dd3e7?src=pr&el=desc) will **decrease** coverage by `0.01%`.
> The diff coverage is `94.8%`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/16219/graphs/tree.svg?src=pr&width=650&token=eZ4WkYLtcO&height=150)](https://codecov.io/gh/pandas-dev/pandas/pull/16219?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #16219      +/-   ##
==========================================
- Coverage   90.89%   90.87%   -0.02%     
==========================================
  Files         162      162              
  Lines       50884    50961      +77     
==========================================
+ Hits        46250    46311      +61     
- Misses       4634     4650      +16
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `88.66% <94.8%> (-0.02%)` | :arrow_down: |
| #single | `40.27% <14.28%> (-0.04%)` | :arrow_down: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/16219?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/plotting/\_\_init\_\_.py](https://codecov.io/gh/pandas-dev/pandas/pull/16219?src=pr&el=tree#diff-cGFuZGFzL3Bsb3R0aW5nL19faW5pdF9fLnB5) | `77.77% <ø> (ø)` | :arrow_up: |
| [pandas/plotting/\_misc.py](https://codecov.io/gh/pandas-dev/pandas/pull/16219?src=pr&el=tree#diff-cGFuZGFzL3Bsb3R0aW5nL19taXNjLnB5) | `51.12% <94.8%> (+12.14%)` | :arrow_up: |
| [pandas/plotting/\_converter.py](https://codecov.io/gh/pandas-dev/pandas/pull/16219?src=pr&el=tree#diff-cGFuZGFzL3Bsb3R0aW5nL19jb252ZXJ0ZXIucHk=) | `63.54% <0%> (-1.82%)` | :arrow_down: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/16219?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/16219?src=pr&el=footer). Last update [f154966...9b72cb2](https://codecov.io/gh/pandas-dev/pandas/pull/16219?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/16219?src=pr&el=h1) Report
> Merging [#16219](https://codecov.io/gh/pandas-dev/pandas/pull/16219?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/f1549668c84b481a67547bb46261d1394c8dd3e7?src=pr&el=desc) will **decrease** coverage by `0.62%`.
> The diff coverage is `14.28%`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/16219/graphs/tree.svg?src=pr&token=eZ4WkYLtcO&width=650&height=150)](https://codecov.io/gh/pandas-dev/pandas/pull/16219?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #16219      +/-   ##
==========================================
- Coverage   90.89%   90.26%   -0.63%     
==========================================
  Files         162      161       -1     
  Lines       50884    50940      +56     
==========================================
- Hits        46250    45980     -270     
- Misses       4634     4960     +326
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `88.05% <14.28%> (-0.63%)` | :arrow_down: |
| #single | `40.28% <14.28%> (-0.04%)` | :arrow_down: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/16219?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/plotting/\_misc.py](https://codecov.io/gh/pandas-dev/pandas/pull/16219?src=pr&el=tree#diff-cGFuZGFzL3Bsb3R0aW5nL19taXNjLnB5) | `33.61% <14.28%> (-5.38%)` | :arrow_down: |
| [pandas/util/decorators.py](https://codecov.io/gh/pandas-dev/pandas/pull/16219?src=pr&el=tree#diff-cGFuZGFzL3V0aWwvZGVjb3JhdG9ycy5weQ==) | `0% <0%> (-65.31%)` | :arrow_down: |
| [pandas/io/msgpack/\_version.py](https://codecov.io/gh/pandas-dev/pandas/pull/16219?src=pr&el=tree#diff-cGFuZGFzL2lvL21zZ3BhY2svX3ZlcnNpb24ucHk=) | `44.86% <0%> (-55.14%)` | :arrow_down: |
| [pandas/io/formats/excel.py](https://codecov.io/gh/pandas-dev/pandas/pull/16219?src=pr&el=tree#diff-cGFuZGFzL2lvL2Zvcm1hdHMvZXhjZWwucHk=) | `73.15% <0%> (-23.49%)` | :arrow_down: |
| [pandas/io/excel.py](https://codecov.io/gh/pandas-dev/pandas/pull/16219?src=pr&el=tree#diff-cGFuZGFzL2lvL2V4Y2VsLnB5) | `61.68% <0%> (-18.94%)` | :arrow_down: |
| [pandas/core/tools/datetimes.py](https://codecov.io/gh/pandas-dev/pandas/pull/16219?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvdG9vbHMvZGF0ZXRpbWVzLnB5) | `66.94% <0%> (-18.62%)` | :arrow_down: |
| [pandas/core/computation/expressions.py](https://codecov.io/gh/pandas-dev/pandas/pull/16219?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvY29tcHV0YXRpb24vZXhwcmVzc2lvbnMucHk=) | `90.47% <0%> (-2.21%)` | :arrow_down: |
| [pandas/core/indexes/base.py](https://codecov.io/gh/pandas-dev/pandas/pull/16219?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW5kZXhlcy9iYXNlLnB5) | `95.75% <0%> (-0.46%)` | :arrow_down: |
| [pandas/compat/\_\_init\_\_.py](https://codecov.io/gh/pandas-dev/pandas/pull/16219?src=pr&el=tree#diff-cGFuZGFzL2NvbXBhdC9fX2luaXRfXy5weQ==) | `61.77% <0%> (-0.45%)` | :arrow_down: |
| [pandas/io/parsers.py](https://codecov.io/gh/pandas-dev/pandas/pull/16219?src=pr&el=tree#diff-cGFuZGFzL2lvL3BhcnNlcnMucHk=) | `95.32% <0%> (-0.33%)` | :arrow_down: |
| ... and [55 more](https://codecov.io/gh/pandas-dev/pandas/pull/16219?src=pr&el=tree-more) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/16219?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/16219?src=pr&el=footer). Last update [f154966...18e3835](https://codecov.io/gh/pandas-dev/pandas/pull/16219?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

Not sure why https://travis-ci.org/pandas-dev/pandas/jobs/228503476 is failing on TestClipboard.test_round_trip_frame_sep, I'm not able to see the failure on my local environment.
I restarted it; that test does occasionally fail. Haven't been able to debug it yet.
This looks fine. @TomAugspurger , @jorisvandenbossche 

side issue is we should create an issue to move all of the interesting plots to an external lib, maybe ``pandas-plotting`` (I mean the ones in ``pandas.plotting``)
@jreback Added a few more tests, and one change, I thought the class labels should be in the legend but think it's a better idea to just leave it as an axes in the plot similar to http://kosara.net/papers/2006/Kosara_TVCG_2006.pdf
@stangirala can you please show an example of the visualizations this function produces and the code required?
@shoyer Try this,
~~~~
import pandas as pd
import matplotlib.pyplot as plt
df = pd.DataFrame({'1':['abc', 'b', 'c', 'b'], '2':['d', 'f', 'g', 'd'], \
        '3':['h', 'e', 'j', 'h'], 'four': ['x', 'y', 'x', 'y']})
from pandas.plotting import parallel_sets
parallel_sets(df, "3")
plt.show()
~~~~

You should get this plot,
<img width="998" alt="screen shot 2017-05-07 at 22 39 35" src="https://cloud.githubusercontent.com/assets/1548135/25791549/19920f44-3376-11e7-9c92-e08f47566bfc.png">
,
Sorry for lettings this sit for so long @stangirala 

Quick question: Can you explain the difference between parallel_coordinates and parallel_sets? Or let me know if this explanation is correct:

- Parallel coordinates plots one line per observation across various dimensions of the data
- Parallel sets plots an *aggregation* of the data, across various dimensions, where the width of each polygon is proportional to the frequency of that value in the dataset?
@TomAugspurger (Was on vacation :) Yes, your explanation is correct. The important distinction is the notion of frequency being captured with parallel sets. I think I made a mistake of using the term parallel sets to distinguish from the parallel coordinates function given the ability to handle categorical data (a fixed size set as opposed to a number set). If I fix the title and the method name it should be clearer. Sorry about the confusion.
@TomAugspurger If that makes sense, I was thinking of replacing the existing parallel coordinates function with the new one.
So the advantage of your implementation over the current one is that it handles non-numeric data, correct?

> @TomAugspurger If that makes sense, I was thinking of replacing the existing parallel coordinates function with the new one.

You can give it a shot and see which tests fail. I suspect that they all will. If I had to guess, the output (visual, and actual objects returned) are too different to simply replace it. We'd probably want to deprecate the old way and replace it with yours, assuming there are no downsides (do you see any?)
>  We'd probably want to deprecate the old way and replace it with yours, assuming there are no downsides (do you see any?)

I think the current behaviour when having only numerical columns should (more or less) just stay the same? But adding the ability to also use the function with non-numerical data. 
One question is how to do the y-axis (as this one is now shared for numerical columns). But also for numerical columns, having the option to *not* share the y-axis can be useful I think.
@TomAugspurger @jorisvandenbossche One way to keep the current implementation is we can have as many subplots as there are columns and squish them next to each other (https://matplotlib.org/faq/howto_faq.html#move-the-edge-of-an-axes-to-make-room-for-tick-labels)?
if you can update according to comments
closing as stale, if you'd like to continue, pls ping.