Repository: pandas-dev/pandas
Pull request: BUG: Enable plotting with PeriodIndex with arbitrary frequencies, resolves #14763
- [x] closes #14763 
- [x] tests added / passed
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`

Enables plotting of data with PeriodIndex with frequencies which are multiples of the `data.index.freq.rule_code`.
Hopefully I'll find some time to include tests within the next 3 weeks. Since this is my first pull requests, I may need some additional time to write tests.

Problem statement:
Period Index plotting method, non standard intervals
#### Code Sample, a copy-pastable example if possible
```python
import numpy as np, pandas as pd
idx = pd.period_range('2000-01-01', '2000-01-05', freq='6H')
s = pd.Series(np.random.randn(len(idx)), index=idx)
s.plot()
```
#### Problem description
Plotting a period index with 'D' or 'H' works fine, but changing the interval to 6-hr throws an error: `IncompatibleFrequency: Input has different freq=6H from PeriodIndex(freq=H)` 

#### Expected Output
Should make a nice plot. Using the same example above with alternative frequencies of 'D' or 'H' plots as expected, but 3H, 6H, and even 24H throw an error.

#### Output

<details>
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
C:\Users\jaq\AppData\Local\Continuum\Anaconda\envs\py35\lib\site-packages\pandas\tseries\period.py in _from_arraylike(cls, data, freq, tz)
    263                         try:
--> 264                             data = com._ensure_int64(data)
    265                         except (TypeError, ValueError):

pandas\src\generated.pyx in pandas.algos.ensure_int64 (pandas\algos.c:64659)()

pandas\src\generated.pyx in pandas.algos.ensure_int64 (pandas\algos.c:64550)()

TypeError: int() argument must be a string, a bytes-like object or a number, not 'pandas._period.Period'

During handling of the above exception, another exception occurred:

IncompatibleFrequency                     Traceback (most recent call last)
<ipython-input-103-18275fd04456> in <module>()
      2 idx = ps.period_range('2000-01-01', '2000-01-05', freq='6H')
      3 s = ps.Series(np.random.randn(len(idx)), index=idx)
----> 4 s.plot()

C:\Users\jaq\AppData\Local\Continuum\Anaconda\envs\py35\lib\site-packages\pandas\tools\plotting.py in __call__(self, kind, ax, figsize, use_index, title, grid, legend, style, logx, logy, loglog, xticks, yticks, xlim, ylim, rot, fontsize, colormap, table, yerr, xerr, label, secondary_y, **kwds)
   3564                            colormap=colormap, table=table, yerr=yerr,
   3565                            xerr=xerr, label=label, secondary_y=secondary_y,
-> 3566                            **kwds)
   3567     __call__.__doc__ = plot_series.__doc__
   3568 

C:\Users\jaq\AppData\Local\Continuum\Anaconda\envs\py35\lib\site-packages\pandas\tools\plotting.py in plot_series(data, kind, ax, figsize, use_index, title, grid, legend, style, logx, logy, loglog, xticks, yticks, xlim, ylim, rot, fontsize, colormap, table, yerr, xerr, label, secondary_y, **kwds)
   2643                  yerr=yerr, xerr=xerr,
   2644                  label=label, secondary_y=secondary_y,
-> 2645                  **kwds)
   2646 
   2647 

C:\Users\jaq\AppData\Local\Continuum\Anaconda\envs\py35\lib\site-packages\pandas\tools\plotting.py in _plot(data, x, y, subplots, ax, kind, **kwds)
   2439         plot_obj = klass(data, subplots=subplots, ax=ax, kind=kind, **kwds)
   2440 
-> 2441     plot_obj.generate()
   2442     plot_obj.draw()
   2443     return plot_obj.result

C:\Users\jaq\AppData\Local\Continuum\Anaconda\envs\py35\lib\site-packages\pandas\tools\plotting.py in generate(self)
   1026         self._compute_plot_data()
   1027         self._setup_subplots()
-> 1028         self._make_plot()
   1029         self._add_table()
   1030         self._make_legend()

C:\Users\jaq\AppData\Local\Continuum\Anaconda\envs\py35\lib\site-packages\pandas\tools\plotting.py in _make_plot(self)
   1705                              stacking_id=stacking_id,
   1706                              is_errorbar=is_errorbar,
-> 1707                              **kwds)
   1708             self._add_legend_handle(newlines[0], label, index=i)
   1709 

C:\Users\jaq\AppData\Local\Continuum\Anaconda\envs\py35\lib\site-packages\pandas\tools\plotting.py in _ts_plot(cls, ax, x, data, style, **kwds)
   1743         ax._plot_data.append((data, cls._kind, kwds))
   1744 
-> 1745         lines = cls._plot(ax, data.index, data.values, style=style, **kwds)
   1746         # set date formatter, locators and rescale limits
   1747         format_dateaxis(ax, ax.freq)

C:\Users\jaq\AppData\Local\Continuum\Anaconda\envs\py35\lib\site-packages\pandas\tools\plotting.py in _plot(cls, ax, x, y, style, column_num, stacking_id, **kwds)
   1720             cls._initialize_stacker(ax, stacking_id, len(y))
   1721         y_values = cls._get_stacked_values(ax, stacking_id, y, kwds['label'])
-> 1722         lines = MPLPlot._plot(ax, x, y_values, style=style, **kwds)
   1723         cls._update_stacker(ax, stacking_id, y)
   1724         return lines

C:\Users\jaq\AppData\Local\Continuum\Anaconda\envs\py35\lib\site-packages\pandas\tools\plotting.py in _plot(cls, ax, x, y, style, is_errorbar, **kwds)
   1340             else:
   1341                 args = (x, y)
-> 1342             return ax.plot(*args, **kwds)
   1343 
   1344     def _get_index_name(self):

C:\Users\jaq\AppData\Local\Continuum\Anaconda\envs\py35\lib\site-packages\matplotlib\__init__.py in inner(ax, *args, **kwargs)
   1816                     warnings.warn(msg % (label_namer, func.__name__),
   1817                                   RuntimeWarning, stacklevel=2)
-> 1818             return func(ax, *args, **kwargs)
   1819         pre_doc = inner.__doc__
   1820         if pre_doc is None:

C:\Users\jaq\AppData\Local\Continuum\Anaconda\envs\py35\lib\site-packages\matplotlib\axes\_axes.py in plot(self, *args, **kwargs)
   1381 
   1382         for line in self._get_lines(*args, **kwargs):
-> 1383             self.add_line(line)
   1384             lines.append(line)
   1385 

C:\Users\jaq\AppData\Local\Continuum\Anaconda\envs\py35\lib\site-packages\matplotlib\axes\_base.py in add_line(self, line)
   1701             line.set_clip_path(self.patch)
   1702 
-> 1703         self._update_line_limits(line)
   1704         if not line.get_label():
   1705             line.set_label('_line%d' % len(self.lines))

C:\Users\jaq\AppData\Local\Continuum\Anaconda\envs\py35\lib\site-packages\matplotlib\axes\_base.py in _update_line_limits(self, line)
   1723         Figures out the data limit of the given line, updating self.dataLim.
   1724         """
-> 1725         path = line.get_path()
   1726         if path.vertices.size == 0:
   1727             return

C:\Users\jaq\AppData\Local\Continuum\Anaconda\envs\py35\lib\site-packages\matplotlib\lines.py in get_path(self)
    936         """
    937         if self._invalidy or self._invalidx:
--> 938             self.recache()
    939         return self._path
    940 

C:\Users\jaq\AppData\Local\Continuum\Anaconda\envs\py35\lib\site-packages\matplotlib\lines.py in recache(self, always)
    619     def recache(self, always=False):
    620         if always or self._invalidx:
--> 621             xconv = self.convert_xunits(self._xorig)
    622             if ma.isMaskedArray(self._xorig):
    623                 x = ma.asarray(xconv, np.float_).filled(np.nan)

C:\Users\jaq\AppData\Local\Continuum\Anaconda\envs\py35\lib\site-packages\matplotlib\artist.py in convert_xunits(self, x)
    188         if ax is None or ax.xaxis is None:
    189             return x
--> 190         return ax.xaxis.convert_units(x)
    191 
    192     def convert_yunits(self, y):

C:\Users\jaq\AppData\Local\Continuum\Anaconda\envs\py35\lib\site-packages\matplotlib\axis.py in convert_units(self, x)
   1442             return x
   1443 
-> 1444         ret = self.converter.convert(x, self.units, self)
   1445         return ret
   1446 

C:\Users\jaq\AppData\Local\Continuum\Anaconda\envs\py35\lib\site-packages\pandas\tseries\converter.py in convert(values, units, axis)
    120             return values.map(lambda x: get_datevalue(x, axis.freq))
    121         if com.is_period_arraylike(values):
--> 122             return PeriodIndex(values, freq=axis.freq).values
    123         if isinstance(values, (list, tuple, np.ndarray, Index)):
    124             return [get_datevalue(x, axis.freq) for x in values]

C:\Users\jaq\AppData\Local\Continuum\Anaconda\envs\py35\lib\site-packages\pandas\tseries\period.py in __new__(cls, data, ordinal, freq, start, end, periods, copy, name, tz, **kwargs)
    188                                                  freq, kwargs)
    189         else:
--> 190             ordinal, freq = cls._from_arraylike(data, freq, tz)
    191             data = np.array(ordinal, dtype=np.int64, copy=False)
    192 

C:\Users\jaq\AppData\Local\Continuum\Anaconda\envs\py35\lib\site-packages\pandas\tseries\period.py in _from_arraylike(cls, data, freq, tz)
    265                         except (TypeError, ValueError):
    266                             data = com._ensure_object(data)
--> 267                             data = _get_ordinals(data, freq)
    268 
    269         return data, freq

C:\Users\jaq\AppData\Local\Continuum\Anaconda\envs\py35\lib\site-packages\pandas\tseries\period.py in _get_ordinals(data, freq)
     42     f = lambda x: Period(x, freq=freq).ordinal
     43     if isinstance(data[0], Period):
---> 44         return period.extract_ordinals(data, freq)
     45     else:
     46         return lib.map_infer(data, f)

pandas\src\period.pyx in pandas._period.extract_ordinals (pandas\src\period.c:7030)()

IncompatibleFrequency: Input has different freq=6H from PeriodIndex(freq=H)
</details>

Hints from issue discussion:
Would it be possible to include this in the 0.20.4 release? It is making it difficult to [plot timeseries data](https://pandas.pydata.org/pandas-docs/stable/visualization.html#basic-plotting-plot):

https://stackoverflow.com/questions/43206554/typeerror-float-argument-must-be-a-string-or-a-number-not-period

I am getting Type error even when the time series period is set to 'D':
![screenshot from 2017-10-11 14-26-35](https://user-images.githubusercontent.com/17307/31438067-3f116c20-ae90-11e7-96ba-e33879af00b2.png)

this would require a community pull request to fix
OK, I am just not sure of the underlying cause. Would you help point me in the right direction?
Any news on this?
@scootty1 pandas is completely volunteer

you are welcome to make a PE for any issue
@jreback I know. I tried to find where the bug is coming from and since it is my first "guess" on a matter like this, it took me quite some time...
One thing I found confusing was somewhere between `pandas/plotting/_timeseries._maybe_resample` and/or `pandas/plotting/_core._ts_plot`. The frequency of the x-axis `ax.xaxis.freq` seems to be set to the rule code, for example `H` for a frequency of the index of `6H`. But this also seems to be the case for plotting `DateTimeIndex` with a freq of `6H`, so I guess this is ok.

Imho the bug is located in `pandas/plotting/_timeseries._maybe_convert_index`. Line 266 only checks for DatetimeIndex. It is assumed that the PeriodIndex already has the correct ax freq set. I changed the code in `_maybe_convert_index` to:

```
def _maybe_convert_index(ax, data):
    # tsplot converts automatically, but don't want to convert index
    # over and over for DataFrames
    if isinstance(data.index, (ABCDatetimeIndex, ABCPeriodIndex)):
        freq = getattr(data.index, 'freq', None)

        if freq is None:
            freq = getattr(data.index, 'inferred_freq', None)
        if isinstance(freq, DateOffset):
            freq = freq.rule_code

        if freq is None:
            freq = _get_ax_freq(ax)

        if freq is None:
            raise ValueError('Could not get frequency alias for plotting')

        freq = get_base_alias(freq)
        freq = frequencies.get_period_alias(freq)
        if isinstance(data.index, ABCDatetimeIndex):
            data = data.to_period(freq=freq)
        elif isinstance(data.index, ABCPeriodIndex):
            data.index = data.index.asfreq(freq)
    return data
```
Plotting some tests seems to be working...

I'll try to make a pull request. But since this is my first pull request, it may take me some time...

Commit message: reverted existing tests, created new

Patch to review:
diff --git a/pandas/plotting/_timeseries.py b/pandas/plotting/_timeseries.py
index f836bd0cd52d7..4460537f79b80 100644
--- a/pandas/plotting/_timeseries.py
+++ b/pandas/plotting/_timeseries.py
@@ -262,7 +262,7 @@ def _get_index_freq(data):
 def _maybe_convert_index(ax, data):
     # tsplot converts automatically, but don't want to convert index
     # over and over for DataFrames
-    if isinstance(data.index, ABCDatetimeIndex):
+    if isinstance(data.index, (ABCDatetimeIndex, ABCPeriodIndex)):
         freq = getattr(data.index, 'freq', None)
 
         if freq is None:
@@ -279,7 +279,10 @@ def _maybe_convert_index(ax, data):
         freq = get_base_alias(freq)
         freq = frequencies.get_period_alias(freq)
 
-        data = data.to_period(freq=freq)
+        if isinstance(data.index, ABCDatetimeIndex):
+            data = data.to_period(freq=freq)
+        elif isinstance(data.index, ABCPeriodIndex):
+            data.index = data.index.asfreq(freq=freq)
     return data
 
 
diff --git a/pandas/tests/plotting/test_datetimelike.py b/pandas/tests/plotting/test_datetimelike.py
index c451228b5b319..204e20f4d4aaa 100644
--- a/pandas/tests/plotting/test_datetimelike.py
+++ b/pandas/tests/plotting/test_datetimelike.py
@@ -1,5 +1,6 @@
 """ Test cases for time series specific (freq conversion, etc) """
 from datetime import date, datetime, time, timedelta
+import itertools
 import pickle
 import sys
 
@@ -33,6 +34,13 @@ def setup_method(self, method):
         self.period_df = [DataFrame(np.random.randn(len(x), 3), index=x,
                                     columns=['A', 'B', 'C'])
                           for x in idx]
+        mults = [1, 23]  # setup periods with multiples of freq.rule_code
+        freq = [str(mlt) + frq for frq, mlt in itertools.product(freq, mults)]
+        idx = [period_range('12/31/1999', freq=x, periods=100) for x in freq]
+        self.period_mlt_ser = [Series(np.random.randn(len(x)), x) for x in idx]
+        self.period_mlt_df = [DataFrame(np.random.randn(len(x), 3), index=x,
+                                        columns=['A', 'B', 'C'])
+                              for x in idx]
 
         freq = ['S', 'T', 'H', 'D', 'W', 'M', 'Q-DEC', 'A', '1B30Min']
         idx = [date_range('12/31/1999', freq=x, periods=100) for x in freq]
@@ -210,6 +218,11 @@ def test_line_plot_period_series(self):
         for s in self.period_ser:
             _check_plot_works(s.plot, s.index.freq)
 
+    @pytest.mark.slow
+    def test_line_plot_period_mlt_series(self):
+        for s in self.period_mlt_ser:
+            _check_plot_works(s.plot, s.index.freq.rule_code)
+
     @pytest.mark.slow
     def test_line_plot_datetime_series(self):
         for s in self.datetime_ser:
@@ -220,6 +233,12 @@ def test_line_plot_period_frame(self):
         for df in self.period_df:
             _check_plot_works(df.plot, df.index.freq)
 
+    @pytest.mark.slow
+    def test_line_plot_period_mlt_frame(self):
+        for df in self.period_mlt_df:
+            freq = df.index.asfreq(df.index.freq.rule_code).freq
+            _check_plot_works(df.plot, freq)
+
     @pytest.mark.slow
     def test_line_plot_datetime_frame(self):
         for df in self.datetime_df:


Code review comments:
diff_hunk: @@ -33,6 +34,13 @@ def setup_method(self, method):
         self.period_df = [DataFrame(np.random.randn(len(x), 3), index=x,
                                     columns=['A', 'B', 'C'])
                           for x in idx]
+        mults = [1, 23]  # setup periods with multiples of freq.rule_code
path: pandas/tests/plotting/test_datetimelike.py
text: @user1:
Instead of adding this to the setup can you keep it self-contained to the new test you are adding and simply parametrize for DataFrame and Series?

I realize you are just copying this style from the existing tests but that makes it much harder to reason about the test than it needs to be (this module is due for a cleanup)

@author:
Ok, could you please give me a short example? As I stated in my first comment, this is my first pull request. Some more advice may save us both time. :)
So you mean I should remove lines 37 to 45 from test_datetimelike.py and instead add the same to `test_line_plot_period_mlt_series`, as in:
```
@user2.mark.slow
def test_line_plot_period_mlt_series(self):
    freq = ['S', 'T', 'H', 'D', 'W', 'M', 'Q', 'A']
    mults = [1, 23]
    freqs = [str(mlt) + frq for frq, mlt in itertools.product(freq, mults)]
    idx = [period_range('12/31/1999', freq=x, periods=100) for x in freqs]
    self.period_mlt_ser = [Series(np.random.randn(len(x)), x) for x in idx]
    for s in self.period_mlt_ser:
        _check_plot_works(s.plot, s.index.freq.rule_code)
```
And the same for `test_line_plot_period_mlt_frame`.

@user3:
Agreed this is due for a cleanup, but that can wait till later. For now, a compromise: These objects are only used in one test each, so I think you can do

```
def test_line_plot_period_mlt_series(self):


```
for mult in [1, 23]:
    for base in self.freq:
        idx = period_range(..., freq=freq)
        s = Series(1, idx)
        _check_plot_works(s.plot, s.index.freq.rule_code)
```

That'll require a few other changes like making `freq` a class attribute, but should ease the later refactor.

Also, can you clarify the variable names (`mlt -> multiple`, etc.)