Repository: pandas-dev/pandas
Pull request: BUG: repair 'style' kwd handling in DataFrame.plot (#21003)
- [x] closes #21003
- [x] tests added / passed
- [x] passes `black pandas`
    => `command not found`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
    => `command not found`
- [x] whatsnew entry


Problem statement:
pandas plotting raises ValueError on style strings that should be valid according to spec.
#### Code Sample

```python
import pandas as pd
from numpy.random import random
import matplotlib.pyplot as plt

data = random((7,4))
fmt = 'd'

plt.plot(data, fmt, color='green') # works as expected (no error)

df = pd.DataFrame(data)
df.plot(color='green', style=fmt)

# The previous line raises ValueError:
# Cannot pass 'style' string with a color symbol and 'color' keyword argument.
# Please use one or the other or pass 'style' without a color symbol
```

#### Full Stack Trace
```python
Traceback (most recent call last):

  File "<ipython-input-178-b2322b55aff5>", line 1, in <module>
    runfile('G:/irreplacable stuff/Scripts/playground/playground.py', wdir='G:/irreplacable stuff/Scripts/playground')

  File "C:\ProgramData\Anaconda2\lib\site-packages\spyder\utils\site\sitecustomize.py", line 705, in runfile
    execfile(filename, namespace)

  File "C:\ProgramData\Anaconda2\lib\site-packages\spyder\utils\site\sitecustomize.py", line 87, in execfile
    exec(compile(scripttext, filename, 'exec'), glob, loc)

  File "G:/irreplacable stuff/Scripts/playground/playground.py", line 30, in <module>
    df.plot(color='green', style='d')

  File "C:\ProgramData\Anaconda2\lib\site-packages\pandas\plotting\_core.py", line 2677, in __call__
    sort_columns=sort_columns, **kwds)

  File "C:\ProgramData\Anaconda2\lib\site-packages\pandas\plotting\_core.py", line 1902, in plot_frame
    **kwds)

  File "C:\ProgramData\Anaconda2\lib\site-packages\pandas\plotting\_core.py", line 1727, in _plot
    plot_obj = klass(data, subplots=subplots, ax=ax, kind=kind, **kwds)

  File "C:\ProgramData\Anaconda2\lib\site-packages\pandas\plotting\_core.py", line 931, in __init__
    MPLPlot.__init__(self, data, **kwargs)

  File "C:\ProgramData\Anaconda2\lib\site-packages\pandas\plotting\_core.py", line 182, in __init__
    self._validate_color_args()

  File "C:\ProgramData\Anaconda2\lib\site-packages\pandas\plotting\_core.py", line 215, in _validate_color_args
    "Cannot pass 'style' string with a color "

ValueError: Cannot pass 'style' string with a color symbol and 'color' keyword argument. Please use one or the other or pass 'style' without a color symbol
```
#### Problem description

`df.plot` should just pass the `style` kwarg as `fmt` arg to `matplotlib.axes.plot` but it does does some extra (sloppy) validation where it thinks that valid marker style symbols are color symbols and raises an error if the color is already defined elsewhere. The problem is clearly in `_core.py`, line 213.

This problem affects the following standard marker styles (key corresponds to `m` in the example code):
```python
{u'd': u'thin_diamond',
 u'h': u'hexagon1',
 u'o': u'circle',
 u'p': u'pentagon',
 u's': u'square',
 u'v': u'triangle_down',
 u'x': u'x'}
```

#### Expected Output
`df.plot(color='green', style=fmt)` should simply plot the plot the same way as `plt.plot(data, fmt, color='green')` without raising errors. All legal values for `fmt` arg in `pyplot.plot` should be legal for the `style` kwarg in `df.plot`.

#### Output of ``pd.show_versions()``

<details>
INSTALLED VERSIONS
------------------
commit: None
python: 2.7.14.final.0
python-bits: 64
OS: Windows
OS-release: 10
machine: AMD64
processor: Intel64 Family 6 Model 45 Stepping 7, GenuineIntel
byteorder: little
LC_ALL: None
LANG: en
LOCALE: None.None

pandas: 0.22.0
pytest: 3.5.0
pip: 9.0.3
setuptools: 39.0.1
Cython: 0.28.2
numpy: 1.14.2
scipy: 1.0.1
pyarrow: None
xarray: None
IPython: 5.6.0
sphinx: 1.7.2
patsy: 0.5.0
dateutil: 2.7.2
pytz: 2018.4
blosc: None
bottleneck: 1.2.1
tables: 3.4.2
numexpr: 2.6.4
feather: None
matplotlib: 2.2.2
openpyxl: 2.5.2
xlrd: 1.1.0
xlwt: 1.3.0
xlsxwriter: 1.0.4
lxml: 4.2.1
bs4: 4.6.0
html5lib: 1.0.1
sqlalchemy: 1.2.6
pymysql: None
psycopg2: None
jinja2: 2.10
s3fs: None
fastparquet: None
pandas_gbq: None
pandas_datareader: None
</details>

Hints from issue discussion:
I think this can be solved by changing the regex at `_core.py` line 213 and line 670 to `'^[bgrcmykw]'` or perhaps pulling the valid color letters directly from `matplotlib.colors.BASE_COLORS`.
By Pandas 1.0.2 the issue has been partly fixed and the code sample runs fine now. However the same problem still persists when passing a list of colors as illustrated in the code sample below.
```
import pandas as pd
from numpy.random import random
import matplotlib.pyplot as plt

data = random((7,4))
fmt = 'd'

plt.plot(data, fmt, color='green') # works as expected (no error)

df = pd.DataFrame(data)
color = ['yellow', 'red', 'green', 'blue']
df.plot(color=color, style=fmt)
```

This can be fixed by changing pandas/plotting/matplotlib/core.py in the `_apply_style_colors` method line 727 from

`nocolor_style = style is None or re.match("[a-z]+", style) is None`

to

`nocolor_style = style is None or re.match("[bgrcmykw]+", style) is None`.

However, a real solution should pull the color letters from `matplotlib.colors.BASE_COLORS`.

By the way, I'm the same guy as Austrianguy. E-mail inboxes sometimes disappear. And passwords are forgotten.
I'm working on the bug fix. However I ran into the following issue when writing the test cases. The code above only fails during rendering. I included my code from 2018-05-10 and 2020-04-23 as tests in `pandas/tests/plotting/test_frame.py`. As expected the 2018 test passes. But unexpectedly the 2020 test passes too. I'm quite sure that is because when running the test suite the plot never renders so there's no opportunity for the error to raise. In fact the test suite doesn't even import matplotlib. Two options for the test suite: A) import matplotlib in the test. B) check that the output line colors are correct. I think I'll go for B.

Here's the traceback from running my 2020 code (copied from above) directly in Spyder:
```
Traceback (most recent call last):
  File "C:\ProgramData\Anaconda3\lib\site-packages\matplotlib\backends\backend_qt5.py", line 508, in _draw_idle
    self.draw()
  File "C:\ProgramData\Anaconda3\lib\site-packages\matplotlib\backends\backend_agg.py", line 388, in draw
    self.figure.draw(self.renderer)
  File "C:\ProgramData\Anaconda3\lib\site-packages\matplotlib\artist.py", line 38, in draw_wrapper
    return draw(artist, renderer, *args, **kwargs)
  File "C:\ProgramData\Anaconda3\lib\site-packages\matplotlib\figure.py", line 1709, in draw
    renderer, self, artists, self.suppressComposite)
  File "C:\ProgramData\Anaconda3\lib\site-packages\matplotlib\image.py", line 135, in _draw_list_compositing_images
    a.draw(renderer)
  File "C:\ProgramData\Anaconda3\lib\site-packages\matplotlib\artist.py", line 38, in draw_wrapper
    return draw(artist, renderer, *args, **kwargs)
  File "C:\ProgramData\Anaconda3\lib\site-packages\matplotlib\axes\_base.py", line 2647, in draw
    mimage._draw_list_compositing_images(renderer, self, artists)
  File "C:\ProgramData\Anaconda3\lib\site-packages\matplotlib\image.py", line 135, in _draw_list_compositing_images
    a.draw(renderer)
  File "C:\ProgramData\Anaconda3\lib\site-packages\matplotlib\artist.py", line 38, in draw_wrapper
    return draw(artist, renderer, *args, **kwargs)
  File "C:\ProgramData\Anaconda3\lib\site-packages\matplotlib\lines.py", line 812, in draw
    self.get_markeredgecolor(), self._alpha)
  File "C:\ProgramData\Anaconda3\lib\site-packages\matplotlib\colors.py", line 177, in to_rgba
    rgba = _to_rgba_no_colorcycle(c, alpha)
  File "C:\ProgramData\Anaconda3\lib\site-packages\matplotlib\colors.py", line 240, in _to_rgba_no_colorcycle
    raise ValueError("Invalid RGBA argument: {!r}".format(orig_c))
ValueError: Invalid RGBA argument: ['yellow', 'red', 'green', 'blue']
Traceback (most recent call last):
  File "C:\ProgramData\Anaconda3\lib\site-packages\matplotlib\backends\backend_qt5.py", line 508, in _draw_idle
    self.draw()
  File "C:\ProgramData\Anaconda3\lib\site-packages\matplotlib\backends\backend_agg.py", line 388, in draw
    self.figure.draw(self.renderer)
  File "C:\ProgramData\Anaconda3\lib\site-packages\matplotlib\artist.py", line 38, in draw_wrapper
    return draw(artist, renderer, *args, **kwargs)
  File "C:\ProgramData\Anaconda3\lib\site-packages\matplotlib\figure.py", line 1709, in draw
    renderer, self, artists, self.suppressComposite)
  File "C:\ProgramData\Anaconda3\lib\site-packages\matplotlib\image.py", line 135, in _draw_list_compositing_images
    a.draw(renderer)
  File "C:\ProgramData\Anaconda3\lib\site-packages\matplotlib\artist.py", line 38, in draw_wrapper
    return draw(artist, renderer, *args, **kwargs)
  File "C:\ProgramData\Anaconda3\lib\site-packages\matplotlib\axes\_base.py", line 2647, in draw
    mimage._draw_list_compositing_images(renderer, self, artists)
  File "C:\ProgramData\Anaconda3\lib\site-packages\matplotlib\image.py", line 135, in _draw_list_compositing_images
    a.draw(renderer)
  File "C:\ProgramData\Anaconda3\lib\site-packages\matplotlib\artist.py", line 38, in draw_wrapper
    return draw(artist, renderer, *args, **kwargs)
  File "C:\ProgramData\Anaconda3\lib\site-packages\matplotlib\lines.py", line 812, in draw
    self.get_markeredgecolor(), self._alpha)
  File "C:\ProgramData\Anaconda3\lib\site-packages\matplotlib\colors.py", line 177, in to_rgba
    rgba = _to_rgba_no_colorcycle(c, alpha)
  File "C:\ProgramData\Anaconda3\lib\site-packages\matplotlib\colors.py", line 240, in _to_rgba_no_colorcycle
    raise ValueError("Invalid RGBA argument: {!r}".format(orig_c))
ValueError: Invalid RGBA argument: ['yellow', 'red', 'green', 'blue']
```
related PR that was supposed to fix these issues: #29122

Commit message: black

manually made edits suggested by `black`.

Patch to review:
diff --git a/doc/source/whatsnew/v1.1.0.rst b/doc/source/whatsnew/v1.1.0.rst
index 07849702c646d..2ca9fc42edde0 100644
--- a/doc/source/whatsnew/v1.1.0.rst
+++ b/doc/source/whatsnew/v1.1.0.rst
@@ -593,6 +593,7 @@ Plotting
 - Bug in :meth:`DataFrame.plot.hist` where weights are not working for multiple columns (:issue:`33173`)
 - Bug in :meth:`DataFrame.boxplot` and :meth:`DataFrame.plot.boxplot` lost color attributes of ``medianprops``, ``whiskerprops``, ``capprops`` and ``medianprops`` (:issue:`30346`)
 - Bug in :meth:`DataFrame.plot.scatter` that when adding multiple plots with different ``cmap``, colorbars alway use the first ``cmap`` (:issue:`33389`)
+- Bug in :meth:`DataFrame.plot` where valid style kwds were rejected (:issue:`21003`)
 
 
 Groupby/resample/rolling
diff --git a/pandas/plotting/_matplotlib/core.py b/pandas/plotting/_matplotlib/core.py
index 46941e437a4ce..7d4b5c214dc8c 100644
--- a/pandas/plotting/_matplotlib/core.py
+++ b/pandas/plotting/_matplotlib/core.py
@@ -1,4 +1,3 @@
-import re
 from typing import Optional
 import warnings
 
@@ -193,7 +192,6 @@ def __init__(
         self._validate_color_args()
 
     def _validate_color_args(self):
-        import matplotlib.colors
 
         if (
             "color" in self.kwds
@@ -226,13 +224,25 @@ def _validate_color_args(self):
                 styles = [self.style]
             # need only a single match
             for s in styles:
-                for char in s:
-                    if char in matplotlib.colors.BASE_COLORS:
-                        raise ValueError(
-                            "Cannot pass 'style' string with a color symbol and "
-                            "'color' keyword argument. Please use one or the other or "
-                            "pass 'style' without a color symbol"
-                        )
+                if self._color_in_style(s):
+                    raise ValueError(
+                        "Cannot pass 'style' string with a color symbol and "
+                        "'color' keyword argument. Please use one or the "
+                        "other or pass 'style' without a color symbol"
+                    )
+
+    @staticmethod
+    def _color_in_style(style):
+        """
+        Is there a color letter in the style string?
+
+        Returns
+        -------
+        bool
+        """
+        from matplotlib.colors import BASE_COLORS
+
+        return not set(BASE_COLORS).isdisjoint(style)
 
     def _iter_data(self, data=None, keep_index=False, fillna=None):
         if data is None:
@@ -723,7 +733,7 @@ def _apply_style_colors(self, colors, kwds, col_num, label):
                 style = self.style
 
         has_color = "color" in kwds or self.colormap is not None
-        nocolor_style = style is None or re.match("[a-z]+", style) is None
+        nocolor_style = style is None or not self._color_in_style(style)
         if (has_color or self.subplots) and nocolor_style:
             if isinstance(colors, dict):
                 kwds["color"] = colors[label]
diff --git a/pandas/tests/plotting/test_frame.py b/pandas/tests/plotting/test_frame.py
index c84a09f21f46b..e55357be24389 100644
--- a/pandas/tests/plotting/test_frame.py
+++ b/pandas/tests/plotting/test_frame.py
@@ -204,6 +204,23 @@ def test_color_and_style_arguments(self):
         with pytest.raises(ValueError):
             df.plot(color=["red", "black"], style=["k-", "r--"])
 
+    def test_color_and_marker(self):
+        # GH21003 2018-05-10
+        df = DataFrame(np.random.random((7, 4)))
+        # combining a color string and a marker letter should be allowed
+        df.plot(color="green", style="d")  # d for diamond
+
+    def test_color_list_and_marker(self):
+        # GH21003 2018-04-23
+        df = DataFrame(np.random.random((7, 4)))
+        # combining a color list and a marker letter should be allowed
+        color = ["yellow", "red", "green", "blue"]
+        ax = df.plot(color=color, style="d")
+        output_colors = [line.get_color() for line in ax.lines]
+        assert output_colors == color  # each "line" is one of the four colors
+        # In the Github bug the color list was passed to MPL like this:
+        # ax.lines[i].get_color() == ['yellow', 'red', 'green', 'blue']
+
     def test_nonnumeric_exclude(self):
         df = DataFrame({"A": ["x", "y", "z"], "B": [1, 2, 3]})
         ax = df.plot()
diff --git a/pandas/tests/plotting/test_series.py b/pandas/tests/plotting/test_series.py
index 5341878d4986e..fab9a9c331185 100644
--- a/pandas/tests/plotting/test_series.py
+++ b/pandas/tests/plotting/test_series.py
@@ -933,4 +933,4 @@ def test_plot_no_numeric_data(self):
     def test_style_single_ok(self):
         s = pd.Series([1, 2])
         ax = s.plot(style="s", color="C3")
-        assert ax.lines[0].get_color() == ["C3"]
+        assert ax.lines[0].get_color() == "C3"


Code review comments:
diff_hunk: @@ -593,6 +593,7 @@ Plotting
 - Bug in :meth:`DataFrame.plot.hist` where weights are not working for multiple columns (:issue:`33173`)
 - Bug in :meth:`DataFrame.boxplot` and :meth:`DataFrame.plot.boxplot` lost color attributes of ``medianprops``, ``whiskerprops``, ``capprops`` and ``medianprops`` (:issue:`30346`)
 - Bug in :meth:`DataFrame.plot.scatter` that when adding multiple plots with different ``cmap``, colorbars alway use the first ``cmap`` (:issue:`33389`)
+- Bug in :meth:`DataFrame.plot` where valid style kwds were rejected (:issue:`21003`)
path: doc/source/whatsnew/v1.1.0.rst
text: @user1:
Can you be more specific? This is a user-facing note

diff_hunk: @@ -204,6 +204,23 @@ def test_color_and_style_arguments(self):
         with pytest.raises(ValueError):
             df.plot(color=["red", "black"], style=["k-", "r--"])
 
+    def test_color_and_marker(self):
+        # GH21003 2018-05-10
path: pandas/tests/plotting/test_frame.py
text: @user1:
What's `2018-05-10`? I've not seen dates elsewhere in the codebase, I think it should be removed

@author:
I clarified these two lines.

diff_hunk: @@ -226,13 +224,25 @@ def _validate_color_args(self):
                 styles = [self.style]
             # need only a single match
             for s in styles:
-                for char in s:
-                    if char in matplotlib.colors.BASE_COLORS:
-                        raise ValueError(
-                            "Cannot pass 'style' string with a color symbol and "
-                            "'color' keyword argument. Please use one or the other or "
-                            "pass 'style' without a color symbol"
-                        )
+                if self._color_in_style(s):
+                    raise ValueError(
+                        "Cannot pass 'style' string with a color symbol and "
+                        "'color' keyword argument. Please use one or the "
+                        "other or pass 'style' without a color symbol"
+                    )
+
+    @staticmethod
+    def _color_in_style(style):
+        """
+        Is there a color letter in the style string?
+
+        Returns
+        -------
+        bool
+        """
+        from matplotlib.colors import BASE_COLORS
+
+        return not set(BASE_COLORS).isdisjoint(style)
path: pandas/plotting/_matplotlib/core.py
text: @user1:
Is it necessary to add a new stastic method (and 10+ lines of code) for a one-liner? IMO just replacing the guard with `set(BASE_COLORS).isdisjoint(style)` should be enough, with the new import replacing the old one

EDIT
----
@author on a second thought, this is probably alright here, as it means you you don't have to duplicate code later in `_apply_style_colors`.

Can you add type annotations? I think `style` is a string here, so
```python
_color_in_style(style: str) -> bool:
```

Also, the short summary in the docstring should start with an infinitive verb, see https://pandas.pydata.org/docs/development/contributing_docstring.html

@author:
Working on it! With type annotations it looks way cleaner than with the lengthy docstring which is a little silly in a private method. This would be the first type annotation in `core.py`, yey! Not sure if the triple double quotes should go on separate lines or in the singe docstring line. In `core.py` you see it done both ways. I'll put them on separate lines, taking the docstring guide literally.

@user1:
:laughing: yeah, they're encouraged for new development, though old code doesn't necessarily have them