Repository: pandas-dev/pandas

Issue:
VIS: errorbar plotting
quoting @jreback 

maybe

s = Series

`s.plot(error_bar = [-1,1])` 

equiv to:

```
df = DataFrame(dict(s = s, top = s.std(), bot = -s.std())).plot()
```

bottom seems the right way to do this....



Discussion:
It would be also great if something like this could work:

``` python
df = pd.DataFrame({'row': list('abc' * 6),
                   'col': list('xxxxxxyyyyyyzzzzzz'),
                   'value': np.random.randn(18)+5})
mean = df.pivot_table(values='value', rows='row',
                      cols='col', aggfunc='mean')
err  = df.pivot_table(values='value', rows='row',
                      cols='col', aggfunc='std')
mean.plot(kind='bar', yerr=err)
```

Or any variation on the subject...

I've been thinking a little about this API. It's a bit annoying because there umpteen parameters to `matplotlib.errorbar`, so it might be worth making a separate method or a function in `pandas.tools.plotting`. Generally speaking though we don't want to pollute the namespace with a method for every kind of plot, but here it might be necessary.

@alefnula Your input on the API is most welcome.

On second thought, the most obvious of having `plot(kind='line', yerr=yerr, xerr=xerr, **kwds)` is probably the way to go. Maybe allowing `yerr` and `xerr` to be column names?

I'm not fully mastering the forking, branching, pull requesting process yet, so I try to contribute in this rather primitive way...

The example of alefnula works perfect if one includes the following few lines at line 1488 of [pandas.tools.plotting](https://github.com/pydata/pandas/blob/master/pandas/tools/plotting.py) (i.e. inside the for loop of the `_make_plot` function):

```
from pandas.core.frame import DataFrame
if 'yerr' in self.kwds and isinstance(self.kwds['yerr'], DataFrame):
    kwds['yerr'] = self.kwds['yerr'][label]
```

With those lines added, also [this matplotlib gallery example](http://matplotlib.org/examples/pylab_examples/barchart_demo.html) becomes straightforward:

```
import pandas as pd
import matplotlib.pyplot as plt

index = tuple('ABDCE')
means_men = (20, 35, 30, 35, 27)
std_men = (2, 3, 4, 1, 2)
means_women = (25, 32, 34, 20, 25)
std_women = (3, 5, 2, 3, 3)

df_means = pd.DataFrame({'Men':means_men, 'Women':means_women}, index=index)
df_yerr = pd.DataFrame({'Men':std_men, 'Women':std_women}, index=index)

fig, ax = plt.subplots()
df_means.plot(kind='bar',
             ax=ax,
             color=['b','r'],
             alpha=0.4,
             yerr=df_yerr,
             error_kw={'ecolor': '0.3'})
ax.set(xlabel='Group', ylabel='Scores', title='Scores by group and gender')
plt.show()
```

The solution of kdebrab is definitely the right way to go. However, by default matplotlib uses the same color for the errorbars if no error_kw is provided, which makes it impossible to see. I would suggest using a different default for the error bar colors.

I have tried this too and could not get this working.
I also agree with the method cited by [alefnula](https://github.com/alefnula). This was my first try, with no previous knowledge that it could not work.
One method that take in consideration the errorbars as a DataFrame or a Serie could be the way.

I've made some headway on implementing this feature (using a lot of the nice suggestions in this thread). Right now my changes allow for errorbars on Line and Bar plots given a list/tuple/ndarray/Series of error values, a DataFrame of label-matched errors, or a column name. There are a couple of things I'm struggling with in terms of code neatness at this point.

Log scale on line plots: the way this currently works is that depending on the values of logx/y keyword arguments, a specific matplotlib function is used (plot, semilogx, semilogy, or loglog). However, the simplest way to get error bars is with the matplotlib's errorbars, which can handle log scales via interacting with the axis directly:

```
ax.set_xscale('log')
ax.set_yscale('log')
```

I think a clean (but less conservative) way to get around this is to rewrite the line plot code to use matplotlib's plot and handle log axes with ax.set_xscale and ax.set_yscale, which as far as I can tell is what semilogx/y are doing under the hood. Any thoughts? I'll also give it a shot, maybe it's not going to be disruptive as I suspect.

You can give it a shot.  You'll need to be careful if you're changing things with the `LinePlot` code. Unless I'm looking in the wrong places, I don't think that any of it is tested.

It's not directly tested. It's only tested through the user-facing API.

## 

Best,
Phillip Cloud

On Mon, Nov 25, 2013 at 9:08 AM, Tom Augspurger notifications@github.comwrote:

> You can give it a shot. You'll need to be careful if you're changing
> things with the LinePlot code. Unless I'm looking in the wrong places, I
> don't think that any of it is tested.
> 
> —
> Reply to this email directly or view it on GitHubhttps://github.com/pydata/pandas/issues/3796#issuecomment-29203701
> .

I'm new to contributing, so I'd welcome any advice on how to make sure this is working properly. I've written some of my own tests, which are basically just running as many different combinations of plot arguments as I can think of. It survives those. Is there anything else I should try?

Have you pushed your code to a branch yet? Try that and then we can look at it.

Here's some notes on [contributing](https://github.com/pydata/pandas/blob/master/CONTRIBUTING.md#pull-requests), just post if you get stuck somewhere.

Thanks for the link! I'll try to get up to speed. In the meantime, I've pushed the changes to my github account... https://github.com/gibbonorbiter/pandas. 


Resolving patch:
diff --git a/pandas/tools/plotting.py b/pandas/tools/plotting.py
--- a/pandas/tools/plotting.py
+++ b/pandas/tools/plotting.py
@@ -831,6 +831,11 @@ def __init__(self, data, kind=None, by=None, subplots=False, sharex=True,
         self.fig = fig
         self.axes = None
 
+        # parse errorbar input if given
+        for err_dim in 'xy':
+            if err_dim+'err' in kwds:
+                kwds[err_dim+'err'] = self._parse_errorbars(error_dim=err_dim, **kwds)
+
         if not isinstance(secondary_y, (bool, tuple, list, np.ndarray)):
             secondary_y = [secondary_y]
         self.secondary_y = secondary_y
@@ -971,6 +976,11 @@ def _setup_subplots(self):
 
             axes = [ax]
 
+        if self.logx:
+            [a.set_xscale('log') for a in axes]
+        if self.logy:
+            [a.set_yscale('log') for a in axes]
+
         self.fig = fig
         self.axes = axes
 
@@ -1090,14 +1100,16 @@ def _is_datetype(self):
                                         'time'))
 
     def _get_plot_function(self):
-        if self.logy:
-            plotf = self.plt.Axes.semilogy
-        elif self.logx:
-            plotf = self.plt.Axes.semilogx
-        elif self.loglog:
-            plotf = self.plt.Axes.loglog
-        else:
+        '''
+        Returns the matplotlib plotting function (plot or errorbar) based on
+        the presence of errorbar keywords.
+        '''
+
+        if ('xerr' not in self.kwds) and \
+        ('yerr' not in self.kwds):
             plotf = self.plt.Axes.plot
+        else:
+            plotf = self.plt.Axes.errorbar
 
         return plotf
 
@@ -1180,6 +1192,78 @@ def _get_marked_label(self, label, col_num):
         else:
             return label
 
+    def _parse_errorbars(self, error_dim='y', **kwds):
+        '''
+        Look for error keyword arguments and return the actual errorbar data
+        or return the error DataFrame/dict
+
+        Error bars can be specified in several ways:
+            Series: the user provides a pandas.Series object of the same
+                    length as the data
+            ndarray: provides a np.ndarray of the same length as the data
+            DataFrame/dict: error values are paired with keys matching the
+                    key in the plotted DataFrame
+            str: the name of the column within the plotted DataFrame
+        '''
+
+        err_kwd = kwds.pop(error_dim+'err', None)
+        if err_kwd is None:
+            return None
+
+        from pandas import DataFrame, Series
+
+        def match_labels(data, err):
+            err = err.reindex_axis(data.index)
+            return err
+
+        # key-matched DataFrame
+        if isinstance(err_kwd, DataFrame):
+            err = err_kwd
+            err = match_labels(self.data, err)
+
+        # key-matched dict
+        elif isinstance(err_kwd, dict):
+            err = err_kwd
+
+        # Series of error values
+        elif isinstance(err_kwd, Series):
+            # broadcast error series across data
+            err = match_labels(self.data, err_kwd)
+            err = np.atleast_2d(err)
+            err = np.tile(err, (self.nseries, 1))
+
+        # errors are a column in the dataframe
+        elif isinstance(err_kwd, str):
+            err = np.atleast_2d(self.data[err_kwd].values)
+            self.data = self.data[self.data.columns.drop(err_kwd)]
+            err = np.tile(err, (self.nseries, 1))
+
+        elif isinstance(err_kwd, (tuple, list, np.ndarray)):
+
+            # raw error values
+            err = np.atleast_2d(err_kwd)
+
+            err_shape = err.shape
+
+            # asymmetrical error bars
+            if err.ndim==3:
+                if (err_shape[0] != self.nseries) or \
+                    (err_shape[1] != 2) or \
+                    (err_shape[2] != len(self.data)):
+                    msg = "Asymmetrical error bars should be provided " + \
+                    "with the shape (%u, 2, %u)" % \
+                        (self.nseries, len(self.data))
+                    raise ValueError(msg)
+
+            # broadcast errors to each data series
+            if len(err)==1:
+                err = np.tile(err, (self.nseries, 1))
+
+        else:
+            msg = "No valid %serr detected" % error_dim
+            raise ValueError(msg)
+
+        return err
 
 class KdePlot(MPLPlot):
     def __init__(self, data, bw_method=None, ind=None, **kwargs):
@@ -1191,7 +1275,7 @@ def _make_plot(self):
         from scipy.stats import gaussian_kde
         from scipy import __version__ as spv
         from distutils.version import LooseVersion
-        plotf = self._get_plot_function()
+        plotf = self.plt.Axes.plot
         colors = self._get_colors()
         for i, (label, y) in enumerate(self._iter_data()):
             ax = self._get_ax(i)
@@ -1376,8 +1460,9 @@ def _make_plot(self):
         # this is slightly deceptive
         if not self.x_compat and self.use_index and self._use_dynamic_x():
             data = self._maybe_convert_index(self.data)
-            self._make_ts_plot(data, **self.kwds)
+            self._make_ts_plot(data)
         else:
+            from pandas.core.frame import DataFrame
             lines = []
             labels = []
             x = self._get_xticks(convert_period=True)
@@ -1391,6 +1476,16 @@ def _make_plot(self):
                 kwds = self.kwds.copy()
                 self._maybe_add_color(colors, kwds, style, i)
 
+                for err_kw in ['xerr', 'yerr']:
+                    # user provided label-matched dataframe of errors
+                    if err_kw in kwds:
+                        if isinstance(kwds[err_kw], (DataFrame, dict)):
+                            if label in kwds[err_kw].keys():
+                              kwds[err_kw] = kwds[err_kw][label]
+                            else: del kwds[err_kw]
+                        elif kwds[err_kw] is not None:
+                            kwds[err_kw] = kwds[err_kw][i]
+
                 label = com.pprint_thing(label)  # .encode('utf-8')
 
                 mask = com.isnull(y)
@@ -1399,10 +1494,11 @@ def _make_plot(self):
                     y = np.ma.masked_where(mask, y)
 
                 kwds['label'] = label
-                if style is None:
-                    args = (ax, x, y)
-                else:
+                # prevent style kwarg from going to errorbar, where it is unsupported
+                if style is not None and plotf.__name__=='plot':
                     args = (ax, x, y, style)
+                else:
+                    args = (ax, x, y)
 
                 newline = plotf(*args, **kwds)[0]
                 lines.append(newline)
@@ -1422,6 +1518,8 @@ def _make_plot(self):
 
     def _make_ts_plot(self, data, **kwargs):
         from pandas.tseries.plotting import tsplot
+        from pandas.core.frame import DataFrame
+
         kwargs = kwargs.copy()
         colors = self._get_colors()
 
@@ -1430,8 +1528,15 @@ def _make_ts_plot(self, data, **kwargs):
         labels = []
 
         def _plot(data, col_num, ax, label, style, **kwds):
-            newlines = tsplot(data, plotf, ax=ax, label=label,
-                                style=style, **kwds)
+
+            if plotf.__name__=='plot':
+                newlines = tsplot(data, plotf, ax=ax, label=label,
+                                    style=style, **kwds)
+            # errorbar function does not support style argument
+            elif plotf.__name__=='errorbar':
+                newlines = tsplot(data, plotf, ax=ax, label=label,
+                                    **kwds)
+
             ax.grid(self.grid)
             lines.append(newlines[0])
 
@@ -1444,19 +1549,33 @@ def _plot(data, col_num, ax, label, style, **kwds):
             ax = self._get_ax(0)  # self.axes[0]
             style = self.style or ''
             label = com.pprint_thing(self.label)
-            kwds = kwargs.copy()
+            kwds = self.kwds.copy()
             self._maybe_add_color(colors, kwds, style, 0)
 
+            if 'yerr' in kwds:
+                kwds['yerr'] = kwds['yerr'][0]
+
             _plot(data, 0, ax, label, self.style, **kwds)
+
         else:
             for i, col in enumerate(data.columns):
                 label = com.pprint_thing(col)
                 ax = self._get_ax(i)
                 style = self._get_style(i, col)
-                kwds = kwargs.copy()
+                kwds = self.kwds.copy()
 
                 self._maybe_add_color(colors, kwds, style, i)
 
+                # key-matched DataFrame of errors
+                if 'yerr' in kwds:
+                    yerr = kwds['yerr']
+                    if isinstance(yerr, (DataFrame, dict)):
+                        if col in yerr.keys():
+                            kwds['yerr'] = yerr[col]
+                        else: del kwds['yerr']
+                    else:
+                        kwds['yerr'] = yerr[i]
+
                 _plot(data[col], i, ax, label, style, **kwds)
 
         self._make_legend(lines, labels)
@@ -1581,6 +1700,7 @@ def f(ax, x, y, w, start=None, log=self.log, **kwds):
 
     def _make_plot(self):
         import matplotlib as mpl
+        from pandas import DataFrame, Series
 
         # mpl decided to make their version string unicode across all Python
         # versions for mpl >= 1.3 so we have to call str here for python 2
@@ -1599,10 +1719,25 @@ def _make_plot(self):
 
         for i, (label, y) in enumerate(self._iter_data()):
             ax = self._get_ax(i)
-            label = com.pprint_thing(label)
             kwds = self.kwds.copy()
             kwds['color'] = colors[i % ncolors]
 
+            for err_kw in ['xerr', 'yerr']:
+                if err_kw in kwds:
+                    # user provided label-matched dataframe of errors
+                    if isinstance(kwds[err_kw], (DataFrame, dict)):
+                        if label in kwds[err_kw].keys():
+                          kwds[err_kw] = kwds[err_kw][label]
+                        else: del kwds[err_kw]
+                    elif kwds[err_kw] is not None:
+                        kwds[err_kw] = kwds[err_kw][i]
+
+            label = com.pprint_thing(label)
+
+            if (('yerr' in kwds) or ('xerr' in kwds)) \
+                and (kwds.get('ecolor') is None):
+                kwds['ecolor'] = mpl.rcParams['xtick.color']
+
             start = 0
             if self.log:
                 start = 1
@@ -1694,6 +1829,9 @@ def plot_frame(frame=None, x=None, y=None, subplots=False, sharex=True,
     x : label or position, default None
     y : label or position, default None
         Allows plotting of one column versus another
+    yerr : DataFrame (with matching labels), Series, list-type (tuple, list,
+        ndarray), or str of column name containing y error values
+    xerr : similar functionality as yerr, but for x error values
     subplots : boolean, default False
         Make separate subplots for each time series
     sharex : boolean, default True
@@ -1807,6 +1945,15 @@ def plot_frame(frame=None, x=None, y=None, subplots=False, sharex=True,
             label = kwds.pop('label', label)
             ser = frame[y]
             ser.index.name = label
+
+            for kw in ['xerr', 'yerr']:
+                if (kw in kwds) and \
+                (isinstance(kwds[kw], str) or com.is_integer(kwds[kw])):
+                    try:
+                        kwds[kw] = frame[kwds[kw]]
+                    except (IndexError, KeyError, TypeError):
+                        pass
+
             return plot_series(ser, label=label, kind=kind,
                                use_index=use_index,
                                rot=rot, xticks=xticks, yticks=yticks,
@@ -1876,6 +2023,8 @@ def plot_series(series, label=None, kind='line', use_index=True, rot=None,
     -----
     See matplotlib documentation online for more on this subject
     """
+
+    from pandas import DataFrame
     kind = _get_standard_kind(kind.lower().strip())
     if kind == 'line':
         klass = LinePlot
@@ -2611,6 +2760,7 @@ def _maybe_convert_date(x):
         x = conv_func(x)
     return x
 
+
 if __name__ == '__main__':
     # import pandas.rpy.common as com
     # sales = com.load_data('sanfrancisco.home.sales', package='nutshell')


Test patch:
diff --git a/pandas/tests/test_graphics.py b/pandas/tests/test_graphics.py
--- a/pandas/tests/test_graphics.py
+++ b/pandas/tests/test_graphics.py
@@ -360,6 +360,35 @@ def test_dup_datetime_index_plot(self):
         s = Series(values, index=index)
         _check_plot_works(s.plot)
 
+    @slow
+    def test_errorbar_plot(self):
+
+        s = Series(np.arange(10))
+        s_err = np.random.randn(10)
+
+        # test line and bar plots
+        kinds = ['line', 'bar']
+        for kind in kinds:
+            _check_plot_works(s.plot, yerr=Series(s_err), kind=kind)
+            _check_plot_works(s.plot, yerr=s_err, kind=kind)
+            _check_plot_works(s.plot, yerr=s_err.tolist(), kind=kind)
+
+        _check_plot_works(s.plot, xerr=s_err)
+
+        # test time series plotting
+        ix = date_range('1/1/2000', '1/1/2001', freq='M')
+        ts = Series(np.arange(12), index=ix)
+        ts_err = Series(np.random.randn(12), index=ix)
+
+        _check_plot_works(ts.plot, yerr=ts_err)
+
+        # check incorrect lengths and types
+        with tm.assertRaises(ValueError):
+            s.plot(yerr=np.arange(11))
+
+        s_err = ['zzz']*10
+        with tm.assertRaises(TypeError):
+            s.plot(yerr=s_err)
 
 @tm.mplskip
 class TestDataFramePlots(tm.TestCase):
@@ -1015,6 +1044,104 @@ def test_allow_cmap(self):
             df.plot(kind='hexbin', x='A', y='B', cmap='YlGn',
                          colormap='BuGn')
 
+    def test_errorbar_plot(self):
+
+        d = {'x': np.arange(12), 'y': np.arange(12, 0, -1)}
+        df = DataFrame(d)
+        d_err = {'x': np.ones(12)*0.2, 'y': np.ones(12)*0.4}
+        df_err = DataFrame(d_err)
+
+        # check line plots
+        _check_plot_works(df.plot, yerr=df_err, logy=True)
+        _check_plot_works(df.plot, yerr=df_err, logx=True, logy=True)
+
+        kinds = ['line', 'bar', 'barh']
+        for kind in kinds:
+            _check_plot_works(df.plot, yerr=df_err['x'], kind=kind)
+            _check_plot_works(df.plot, yerr=d_err, kind=kind)
+            _check_plot_works(df.plot, yerr=df_err, xerr=df_err, kind=kind)
+            _check_plot_works(df.plot, yerr=df_err['x'], xerr=df_err['x'], kind=kind)
+            _check_plot_works(df.plot, yerr=df_err, xerr=df_err, subplots=True, kind=kind)
+
+        _check_plot_works((df+1).plot, yerr=df_err, xerr=df_err, kind='bar', log=True)
+
+        # yerr is raw error values
+        _check_plot_works(df['y'].plot, yerr=np.ones(12)*0.4)
+        _check_plot_works(df.plot, yerr=np.ones((2, 12))*0.4)
+
+        # yerr is column name
+        df['yerr'] = np.ones(12)*0.2
+        _check_plot_works(df.plot, y='y', x='x', yerr='yerr')
+
+        with tm.assertRaises(ValueError):
+            df.plot(yerr=np.random.randn(11))
+
+        df_err = DataFrame({'x': ['zzz']*12, 'y': ['zzz']*12})
+        with tm.assertRaises(TypeError):
+            df.plot(yerr=df_err)
+
+    @slow
+    def test_errorbar_with_integer_column_names(self):
+        # test with integer column names
+        df = DataFrame(np.random.randn(10, 2))
+        df_err = DataFrame(np.random.randn(10, 2))
+        _check_plot_works(df.plot, yerr=df_err)
+        _check_plot_works(df.plot, y=0, yerr=1)
+
+    @slow
+    def test_errorbar_with_partial_columns(self):
+        df = DataFrame(np.random.randn(10, 3))
+        df_err = DataFrame(np.random.randn(10, 2), columns=[0, 2])
+        kinds = ['line', 'bar']
+        for kind in kinds:
+            _check_plot_works(df.plot, yerr=df_err, kind=kind)
+
+        ix = date_range('1/1/2000', periods=10, freq='M')
+        df.set_index(ix, inplace=True)
+        df_err.set_index(ix, inplace=True)
+        _check_plot_works(df.plot, yerr=df_err, kind='line')
+
+    @slow
+    def test_errorbar_timeseries(self):
+
+        d = {'x': np.arange(12), 'y': np.arange(12, 0, -1)}
+        d_err = {'x': np.ones(12)*0.2, 'y': np.ones(12)*0.4}
+
+        # check time-series plots
+        ix = date_range('1/1/2000', '1/1/2001', freq='M')
+        tdf = DataFrame(d, index=ix)
+        tdf_err = DataFrame(d_err, index=ix)
+
+        kinds = ['line', 'bar', 'barh']
+        for kind in kinds:
+            _check_plot_works(tdf.plot, yerr=tdf_err, kind=kind)
+            _check_plot_works(tdf.plot, yerr=d_err, kind=kind)
+            _check_plot_works(tdf.plot, y='y', kind=kind)
+            _check_plot_works(tdf.plot, y='y', yerr='x', kind=kind)
+            _check_plot_works(tdf.plot, yerr=tdf_err, kind=kind)
+            _check_plot_works(tdf.plot, kind=kind, subplots=True)
+
+
+    def test_errorbar_asymmetrical(self):
+
+        np.random.seed(0)
+        err = np.random.rand(3, 2, 5)
+
+        data = np.random.randn(5, 3)
+        df = DataFrame(data)
+
+        ax = df.plot(yerr=err, xerr=err/2)
+
+        self.assertEqual(ax.lines[7].get_ydata()[0], data[0,1]-err[1,0,0])
+        self.assertEqual(ax.lines[8].get_ydata()[0], data[0,1]+err[1,1,0])
+
+        self.assertEqual(ax.lines[5].get_xdata()[0], -err[1,0,0]/2)
+        self.assertEqual(ax.lines[6].get_xdata()[0], err[1,1,0]/2)
+
+        with tm.assertRaises(ValueError):
+            df.plot(yerr=err.T)
+
+        tm.close()
 
 @tm.mplskip
 class TestDataFrameGroupByPlots(tm.TestCase):