Repository: matplotlib/matplotlib
Pull request: Feature: draw percentiles in violinplot
## PR Summary
Resolve #8532

Added an optional argument, a list for percentiles, to violinplot function. User now can specify the percentiles they want to draw for each violin.

I'm opening this as part of coursework from University of Toronto Scarborough's CSCD01 taught by Dr. Anya Tafliovich @atafliovich. (https://www.utsc.utoronto.ca/~atafliovich/cscd01/index.html)
## PR Checklist

- [X] Has Pytest style unit tests
- [X] Code is [Flake 8](http://flake8.pycqa.org/en/latest/) compliant
- [X] New features are documented, with examples if plot related
- [X] Documentation is sphinx and numpydoc compliant
- [ ] Added an entry to doc/users/next_whats_new/ if major new feature (follow instructions in README.rst there)
- [ ] Documented in doc/api/api_changes.rst if API changed in a backward-incompatible way

<!--
Thank you so much for your PR!  To help us review your contribution, please
consider the following points:

- A development guide is available at https://matplotlib.org/devdocs/devel/index.html.

- Help with git and github is available at
  https://matplotlib.org/devel/gitwash/development_workflow.html.

- Do not create the PR out of master, but out of a separate branch.

- The PR title should summarize the changes, for example "Raise ValueError on
  non-numeric input to set_xlim".  Avoid non-descriptive titles such as
  "Addresses issue #8576".

- The summary should provide at least 1-2 sentences describing the pull request
  in detail (Why is this change required?  What problem does it solve?) and
  link to any relevant issues.

- If you are contributing fixes to docstrings, please pay attention to
  http://matplotlib.org/devel/documenting_mpl.html#formatting.  In particular,
  note the difference between using single backquotes, double backquotes, and
  asterisks in the markup.

We understand that PRs can sometimes be overwhelming, especially as the
reviews start coming in.  Please let us know if the reviews are unclear or
the recommended next step seems overly demanding, if you would like help in
addressing a reviewer's comments, or if you have been waiting too long to hear
back on your PR.
-->


Problem statement:
Feature Request: draw percentiles in violinplot
At the moment violineplot allows to draw the median and the mean into the violin.
Would it be possible to add as an argument e.g. a list for percentiles to be drawn into the violin as well?

Hints from issue discussion:
Hey @tino-michael 

This is definitely possible, it's a matter of finding someone with time to implement, test, and document it. I'd be happy to review a PR if you can take this on.

In the meantime, I'm tempted to simply kick this over to [seaborn](http://seaborn.pydata.org/generated/seaborn.violinplot.html#seaborn.violinplot), which already handles this case nicely

As far as I can see, seaborn also "only" allows to print the quartiles with the `inner` parameter but not user-defined percentiles (some colleagues insist to see the "1sigma" -- i.e. 68th percentile -- point in the distributions).

I will try to find some time (you are right on that regard) to dig into it. I just thought that it would be quickly enough to implement for somebody that is already familiar with the code -- which I am not.
Hello, 
     What color should the percentile lines be? And what size in comparison to the mean/median lines?

Commit message: Run boilerplate

Patch to review:
diff --git a/examples/statistics/violinplot.py b/examples/statistics/violinplot.py
index 19d80b494fba..2d3a4217eee1 100644
--- a/examples/statistics/violinplot.py
+++ b/examples/statistics/violinplot.py
@@ -28,7 +28,7 @@
 pos = [1, 2, 4, 5, 7, 8]
 data = [np.random.normal(0, std, size=100) for std in pos]
 
-fig, axes = plt.subplots(nrows=2, ncols=3, figsize=(6, 6))
+fig, axes = plt.subplots(nrows=2, ncols=5, figsize=(10, 6))
 
 axes[0, 0].violinplot(data, pos, points=20, widths=0.3,
                       showmeans=True, showextrema=True, showmedians=True)
@@ -43,19 +43,40 @@
                       showextrema=True, showmedians=True, bw_method=0.5)
 axes[0, 2].set_title('Custom violinplot 3', fontsize=fs)
 
+axes[0, 3].violinplot(data, pos, points=60, widths=0.7, showmeans=True,
+                      showextrema=True, showmedians=True, bw_method=0.5,
+                      percentiles=[[10], [], [], [17.5, 95.4], [75], [25]])
+axes[0, 3].set_title('Custom violinplot 4', fontsize=fs)
+
+axes[0, 4].violinplot(data[-1:], pos[-1:], points=60, widths=0.7,
+                      showmeans=True, showextrema=True, showmedians=True,
+                      percentiles=[5, 10, 80, 90], bw_method=0.5)
+axes[0, 4].set_title('Custom violinplot 5', fontsize=fs)
+
 axes[1, 0].violinplot(data, pos, points=80, vert=False, widths=0.7,
                       showmeans=True, showextrema=True, showmedians=True)
-axes[1, 0].set_title('Custom violinplot 4', fontsize=fs)
+axes[1, 0].set_title('Custom violinplot 6', fontsize=fs)
 
 axes[1, 1].violinplot(data, pos, points=100, vert=False, widths=0.9,
                       showmeans=True, showextrema=True, showmedians=True,
                       bw_method='silverman')
-axes[1, 1].set_title('Custom violinplot 5', fontsize=fs)
+axes[1, 1].set_title('Custom violinplot 7', fontsize=fs)
 
 axes[1, 2].violinplot(data, pos, points=200, vert=False, widths=1.1,
                       showmeans=True, showextrema=True, showmedians=True,
                       bw_method=0.5)
-axes[1, 2].set_title('Custom violinplot 6', fontsize=fs)
+axes[1, 2].set_title('Custom violinplot 8', fontsize=fs)
+
+axes[1, 3].violinplot(data, pos, points=200, vert=False, widths=1.1,
+                      showmeans=True, showextrema=True, showmedians=True,
+                      percentiles=[[10], [], [], [17.5, 95.4], [75], [25]],
+                      bw_method=0.5)
+axes[1, 3].set_title('Custom violinplot 9', fontsize=fs)
+
+axes[1, 4].violinplot(data[-1:], pos[-1:], points=200, vert=False, widths=1.1,
+                      showmeans=True, showextrema=True, showmedians=True,
+                      percentiles=[5, 10, 80, 90], bw_method=0.5)
+axes[1, 4].set_title('Custom violinplot 10', fontsize=fs)
 
 for ax in axes.flat:
     ax.set_yticklabels([])
diff --git a/lib/matplotlib/axes/_axes.py b/lib/matplotlib/axes/_axes.py
index 685a8d0a04e6..7eab82c847b7 100644
--- a/lib/matplotlib/axes/_axes.py
+++ b/lib/matplotlib/axes/_axes.py
@@ -7890,14 +7890,14 @@ def matshow(self, Z, **kwargs):
     @_preprocess_data(replace_names=["dataset"])
     def violinplot(self, dataset, positions=None, vert=True, widths=0.5,
                    showmeans=False, showextrema=True, showmedians=False,
-                   points=100, bw_method=None):
+                   percentiles=None, points=100, bw_method=None):
         """
         Make a violin plot.
 
         Make a violin plot for each column of *dataset* or each vector in
         sequence *dataset*.  Each filled area extends to represent the
         entire data range, with optional lines at the mean, the median,
-        the minimum, and the maximum.
+        the minimum, the maximum, and user-specified percentiles.
 
         Parameters
         ----------
@@ -7926,6 +7926,11 @@ def violinplot(self, dataset, positions=None, vert=True, widths=0.5,
         showmedians : bool, default = False
           If `True`, will toggle rendering of the medians.
 
+        percentiles : array-like, default=None
+          If not None, set a list of float number in interval [0, 100] for
+          each violin, which stands for the percentiles that will be rendered
+          for that violin.
+
         points : scalar, default = 100
           Defines the number of points to evaluate each of the
           gaussian kernel density estimations at.
@@ -7963,6 +7968,11 @@ def violinplot(self, dataset, positions=None, vert=True, widths=0.5,
           - ``cmedians``: A `~.collections.LineCollection` instance that
             marks the median values of each of the violin's distribution.
 
+          - ``cpercentiles``: A
+            :class:`matplotlib.collections.LineCollection` instance created to
+            identify the percentile values of each of the violin's
+            distribution.
+
         """
 
         def _kde_method(X, coords):
@@ -7972,7 +7982,8 @@ def _kde_method(X, coords):
             kde = mlab.GaussianKDE(X, bw_method)
             return kde.evaluate(coords)
 
-        vpstats = cbook.violin_stats(dataset, _kde_method, points=points)
+        vpstats = cbook.violin_stats(dataset, _kde_method,
+                                     percentiles=percentiles, points=points)
         return self.violin(vpstats, positions=positions, vert=vert,
                            widths=widths, showmeans=showmeans,
                            showextrema=showextrema, showmedians=showmedians)
@@ -7983,7 +7994,7 @@ def violin(self, vpstats, positions=None, vert=True, widths=0.5,
 
         Draw a violin plot for each column of `vpstats`. Each filled area
         extends to represent the entire data range, with optional lines at the
-        mean, the median, the minimum, and the maximum.
+        mean, the median, the minimum, the maximum, and the percentile values.
 
         Parameters
         ----------
@@ -8007,6 +8018,8 @@ def violin(self, vpstats, positions=None, vert=True, widths=0.5,
 
           - ``max``: The maximum value for this violin's dataset.
 
+          - ``percentiles``: The percentile values for this violin's dataset.
+
         positions : array-like, default = [1, 2, ..., n]
           Sets the positions of the violins. The ticks and limits are
           automatically set to match the positions.
@@ -8053,6 +8066,12 @@ def violin(self, vpstats, positions=None, vert=True, widths=0.5,
 
           - ``cmedians``: A `~.collections.LineCollection` instance that
             marks the median values of each of the violin's distribution.
+
+          - ``cpercentiles``: A
+            :class:`matplotlib.collections.LineCollection` instance created to
+            identify the percentile values of each of the violin's
+            distribution.
+
         """
 
         # Statistical quantities to be plotted on the violins
@@ -8060,6 +8079,7 @@ def violin(self, vpstats, positions=None, vert=True, widths=0.5,
         mins = []
         maxes = []
         medians = []
+        percentiles = np.asarray([])
 
         # Collections to be returned
         artists = {}
@@ -8116,6 +8136,7 @@ def violin(self, vpstats, positions=None, vert=True, widths=0.5,
             mins.append(stats['min'])
             maxes.append(stats['max'])
             medians.append(stats['median'])
+            percentiles = np.concatenate((percentiles, stats['percentiles']))
         artists['bodies'] = bodies
 
         # Render means
@@ -8139,6 +8160,20 @@ def violin(self, vpstats, positions=None, vert=True, widths=0.5,
                                              pmaxes,
                                              colors=edgecolor)
 
+        # Render percentile values
+        if percentiles.size > 0:
+            # Recalculate the pmins and pmaxes for percentiles
+            ppmins = np.asarray([])
+            ppmaxs = np.asarray([])
+            for stats, cmin, cmax in zip(vpstats, pmins, pmaxes):
+                ppmins = np.concatenate((ppmins, [cmin] *
+                                         np.size(stats['percentiles'])))
+                ppmaxs = np.concatenate((ppmaxs, [cmax] *
+                                         np.size(stats['percentiles'])))
+            # Start rendering
+            artists['cpercentiles'] = perp_lines(percentiles, ppmins, ppmaxs,
+                                                 colors=edgecolor)
+
         return artists
 
     # Methods that are entirely implemented in other modules.
diff --git a/lib/matplotlib/cbook/__init__.py b/lib/matplotlib/cbook/__init__.py
index bbeb82b18f1a..4df8a9165b06 100644
--- a/lib/matplotlib/cbook/__init__.py
+++ b/lib/matplotlib/cbook/__init__.py
@@ -1431,7 +1431,7 @@ def _reshape_2D(X, name):
         raise ValueError("{} must have 2 or fewer dimensions".format(name))
 
 
-def violin_stats(X, method, points=100):
+def violin_stats(X, method, percentiles=None, points=100):
     """
     Returns a list of dictionaries of data which can be used to draw a series
     of violin plots. See the `Returns` section below to view the required keys
@@ -1451,6 +1451,11 @@ def violin_stats(X, method, points=100):
         return a vector of the values of the KDE evaluated at the values
         specified in coords.
 
+    percentiles : array-like, default = None
+        Defines (if not None) a list of float number in interval [0, 100] for
+        each column of data, which represents the percentiles that will be
+        rendered for that column of data.
+
     points : scalar, default = 100
         Defines the number of points to evaluate each of the gaussian kernel
         density estimates at.
@@ -1469,6 +1474,7 @@ def violin_stats(X, method, points=100):
         - median: The median value for this column of data.
         - min: The minimum value for this column of data.
         - max: The maximum value for this column of data.
+        - percentiles: The percentile values for this column of data.
     """
 
     # List of dictionaries describing each of the violins.
@@ -1477,13 +1483,29 @@ def violin_stats(X, method, points=100):
     # Want X to be a list of data sequences
     X = _reshape_2D(X, "X")
 
-    for x in X:
+    # Want percentiles to be as the same shape as data sequences
+    if percentiles is not None and len(percentiles) != 0:
+        percentiles = _reshape_2D(percentiles, "percentiles")
+    # Else, mock percentiles if is none or empty
+    else:
+        percentiles = [[]] * np.shape(X)[0]
+
+    # percentiles should has the same size as dataset
+    if np.shape(X)[:1] != np.shape(percentiles)[:1]:
+        raise ValueError("List of violinplot statistics and percentile values"
+                         " must have the same length")
+
+    # Zip x and percentiles
+    data = zip(X, percentiles)
+
+    for (x, p) in data:
         # Dictionary of results for this distribution
         stats = {}
 
         # Calculate basic stats for the distribution
         min_val = np.min(x)
         max_val = np.max(x)
+        p_val = np.percentile(x, p)
 
         # Evaluate the kernel density estimate
         coords = np.linspace(min_val, max_val, points)
@@ -1495,6 +1517,7 @@ def violin_stats(X, method, points=100):
         stats['median'] = np.median(x)
         stats['min'] = min_val
         stats['max'] = max_val
+        stats['percentiles'] = np.atleast_1d(p_val)
 
         # Append to output
         vpstats.append(stats)
diff --git a/lib/matplotlib/pyplot.py b/lib/matplotlib/pyplot.py
index ec2224513667..90a5b8c4bfad 100644
--- a/lib/matplotlib/pyplot.py
+++ b/lib/matplotlib/pyplot.py
@@ -2990,12 +2990,13 @@ def triplot(*args, **kwargs):
 def violinplot(
         dataset, positions=None, vert=True, widths=0.5,
         showmeans=False, showextrema=True, showmedians=False,
-        points=100, bw_method=None, *, data=None):
+        percentiles=None, points=100, bw_method=None, *, data=None):
     return gca().violinplot(
         dataset, positions=positions, vert=vert, widths=widths,
         showmeans=showmeans, showextrema=showextrema,
-        showmedians=showmedians, points=points, bw_method=bw_method,
-        **({"data": data} if data is not None else {}))
+        showmedians=showmedians, percentiles=percentiles,
+        points=points, bw_method=bw_method, **({"data": data} if data
+        is not None else {}))
 
 
 # Autogenerated by boilerplate.py.  Do not edit as changes will be lost.
diff --git a/lib/matplotlib/tests/baseline_images/test_axes/violinplot_horiz_showall.png b/lib/matplotlib/tests/baseline_images/test_axes/violinplot_horiz_showall.png
index 95c657bd81d5..b3e807c153d9 100644
Binary files a/lib/matplotlib/tests/baseline_images/test_axes/violinplot_horiz_showall.png and b/lib/matplotlib/tests/baseline_images/test_axes/violinplot_horiz_showall.png differ
diff --git a/lib/matplotlib/tests/baseline_images/test_axes/violinplot_horiz_showpercentiles.png b/lib/matplotlib/tests/baseline_images/test_axes/violinplot_horiz_showpercentiles.png
new file mode 100644
index 000000000000..737bad5e463f
Binary files /dev/null and b/lib/matplotlib/tests/baseline_images/test_axes/violinplot_horiz_showpercentiles.png differ
diff --git a/lib/matplotlib/tests/baseline_images/test_axes/violinplot_vert_showall.png b/lib/matplotlib/tests/baseline_images/test_axes/violinplot_vert_showall.png
index cdda9b7e1c65..49383f98bc3e 100644
Binary files a/lib/matplotlib/tests/baseline_images/test_axes/violinplot_vert_showall.png and b/lib/matplotlib/tests/baseline_images/test_axes/violinplot_vert_showall.png differ
diff --git a/lib/matplotlib/tests/baseline_images/test_axes/violinplot_vert_showpercentiles.png b/lib/matplotlib/tests/baseline_images/test_axes/violinplot_vert_showpercentiles.png
new file mode 100644
index 000000000000..57555727a6af
Binary files /dev/null and b/lib/matplotlib/tests/baseline_images/test_axes/violinplot_vert_showpercentiles.png differ
diff --git a/lib/matplotlib/tests/test_axes.py b/lib/matplotlib/tests/test_axes.py
index d7383ff9ee27..e9e6c12c973c 100644
--- a/lib/matplotlib/tests/test_axes.py
+++ b/lib/matplotlib/tests/test_axes.py
@@ -2629,6 +2629,18 @@ def test_vert_violinplot_showmedians():
                   showmedians=1)
 
 
+@image_comparison(baseline_images=['violinplot_vert_showpercentiles'],
+                  extensions=['png'])
+def test_vert_violinplot_showpercentiles():
+    ax = plt.axes()
+    # First 9 digits of frac(sqrt(67))
+    np.random.seed(185352771)
+    data = [np.random.normal(size=100) for i in range(4)]
+    ax.violinplot(data, positions=range(4), showmeans=0, showextrema=0,
+                  showmedians=0,
+                  percentiles=[[10, 90], [20, 80], [30, 70], [40, 60]])
+
+
 @image_comparison(baseline_images=['violinplot_vert_showall'],
                   extensions=['png'])
 def test_vert_violinplot_showall():
@@ -2637,7 +2649,8 @@ def test_vert_violinplot_showall():
     np.random.seed(316624790)
     data = [np.random.normal(size=100) for i in range(4)]
     ax.violinplot(data, positions=range(4), showmeans=1, showextrema=1,
-                  showmedians=1)
+                  showmedians=1,
+                  percentiles=[[10, 90], [20, 80], [30, 70], [40, 60]])
 
 
 @image_comparison(baseline_images=['violinplot_vert_custompoints_10'],
@@ -2706,6 +2719,18 @@ def test_horiz_violinplot_showextrema():
                   showextrema=1, showmedians=0)
 
 
+@image_comparison(baseline_images=['violinplot_horiz_showpercentiles'],
+                  extensions=['png'])
+def test_horiz_violinplot_showpercentiles():
+    ax = plt.axes()
+    # First 9 digits of frac(sqrt(71))
+    np.random.seed(426149773)
+    data = [np.random.normal(size=100) for i in range(4)]
+    ax.violinplot(data, positions=range(4), vert=False, showmeans=0,
+                  showextrema=0, showmedians=0,
+                  percentiles=[[10, 90], [20, 80], [30, 70], [40, 60]])
+
+
 @image_comparison(baseline_images=['violinplot_horiz_showall'],
                   extensions=['png'])
 def test_horiz_violinplot_showall():
@@ -2714,7 +2739,8 @@ def test_horiz_violinplot_showall():
     np.random.seed(82762530)
     data = [np.random.normal(size=100) for i in range(4)]
     ax.violinplot(data, positions=range(4), vert=False, showmeans=1,
-                  showextrema=1, showmedians=1)
+                  showextrema=1, showmedians=1,
+                  percentiles=[[10, 90], [20, 80], [30, 70], [40, 60]])
 
 
 @image_comparison(baseline_images=['violinplot_horiz_custompoints_10'],
@@ -2757,6 +2783,48 @@ def test_violinplot_bad_widths():
         ax.violinplot(data, positions=range(4), widths=[1, 2, 3])
 
 
+def test_violinplot_bad_percentiles():
+    ax = plt.axes()
+    # First 9 digits of frac(sqrt(73))
+    np.random.seed(544003745)
+    data = [np.random.normal(size=100)]
+
+    # Different size percentile list and plots
+    with pytest.raises(ValueError):
+        ax.violinplot(data, percentiles=[[10, 20], [50]])
+
+
+def test_violinplot_outofrange_percentiles():
+    ax = plt.axes()
+    # First 9 digits of frac(sqrt(79))
+    np.random.seed(888194417)
+    data = [np.random.normal(size=100)]
+
+    # Percentile value above 100
+    with pytest.raises(ValueError):
+        ax.violinplot(data, percentiles=[[10, 20, 30, 105]])
+
+    # Percentile value below 0
+    with pytest.raises(ValueError):
+        ax.violinplot(data, percentiles=[[-5, 20, 30, 75]])
+
+
+@check_figures_equal(extensions=["png"])
+def test_violinplot_single_list_percentiles(fig_test, fig_ref):
+    # Ensures percentile list for 1D can be passed in as single list
+    # First 9 digits of frac(sqrt(83))
+    np.random.seed(110433579)
+    data = [np.random.normal(size=100)]
+
+    # Test image
+    ax = fig_test.subplots()
+    ax.violinplot(data, percentiles=[10, 30, 90])
+
+    # Reference image
+    ax = fig_ref.subplots()
+    ax.violinplot(data, percentiles=[[10, 30, 90]])
+
+
 def test_manage_xticks():
     _, ax = plt.subplots()
     ax.set_xlim(0, 4)


Code review comments:
diff_hunk: @@ -1431,7 +1431,7 @@ def _reshape_2D(X, name):
         raise ValueError("{} must have 2 or fewer dimensions".format(name))
 
 
-def violin_stats(X, method, points=100):
+def violin_stats(X, method, percentiles=None, points=100):
path: lib/matplotlib/cbook/__init__.py
text: @user1:
Could you add the new keyword argument to the end of the signature instead, so people calling it like `(X, method, 100)` with no keywords don't have to change their code.

diff_hunk: @@ -7926,6 +7926,11 @@ def violinplot(self, dataset, positions=None, vert=True, widths=0.5,
         showmedians : bool, default = False
           If `True`, will toggle rendering of the medians.
 
+        percentiles : array-like, default=None
+          If not None, set a list of float number in interval [0, 100] for
path: lib/matplotlib/axes/_axes.py
text: @user1:
```suggestion
          If not None, set a list of floats in interval [0, 100] for
```

diff_hunk: @@ -1451,6 +1451,11 @@ def violin_stats(X, method, points=100):
         return a vector of the values of the KDE evaluated at the values
         specified in coords.
 
+    percentiles : array-like, default = None
path: lib/matplotlib/cbook/__init__.py
text: @user1:
Could you specify the shape this array should be.