Repository: pandas-dev/pandas

Issue:
ENH/API: Decide what to return Period - Period subtraction
#### Code Sample, a copy-pastable example if possible

Split from #13071. Currently it returns int and loses freq info.

```
pd.Period('2011-03', freq='M') - pd.Period('2011-01', freq='M') 
#2
```
#### Expected Output

Current ideas are:
1. Return `Timedelta` adding support of `Monthly` freq.
2. Return `Perioddelta` newly created.
3. Return `DateOffset`.
#### output of `pd.show_versions()`

on current master.

CC: @jreback @MaximilianR 



Discussion:
I don't think we want to add other units like months to Timedelta without a full overhauling of time units.

I also don't think we want to add a new PeriodDelta type.

So I guess that leaves returns DateOffsets. Are there any differences of Periods that could not be represented with DateOffsets? The other reasonable choice is to simply raise a TypeError.

> I don't think we want to add other units like months to Timedelta without a full overhauling of time units.

I am not sure this will work, but what is the issue here

> >  I don't think we want to add other units like months to Timedelta without a full overhauling of time units.
> 
> I am not sure this will work, but what is the issue here

A month has not a well defined length, so does not fit in the current concept of Timedelta's.

@sinhrks Do you want to do this for 0.19? (changing to DateOffset, if possible) Or push to next release?

I'm not sure I understand what the problem is here - I *like* the fact that it returns an int. You know what the implied freq is anyway so I don't see much of an issue with losing the freq info.
```python
In [8]: p0 = pd.Period('2011-01', freq='M')
   ...: p1 = pd.Period('2011-03', freq='M')

In [9]: dt = p1 - p0

In [10]: assert p1 == p0 + dt
```

I suppose there's a type-safety argument to be made in that you could prevent a `PeriodDelta('2M')` from being added to a PeriodIndex of a different frequency, but I just don't see that ever really happening in practice and whatever benefit that brings has to be weighted against the increased api size and maintenance burden.

FWIW the old `scikits.timeseries` simply returned ints for the difference and that worked well in practice... for me at least.
It would also be nice if you could subtract PeriodIndexes and get an array of ints:
```python
In [14]: periods = pd.period_range('01-Jan-2017', '01-Jan-2020', freq='Q')

In [15]: periods - periods
Traceback (most recent call last):

  File "<ipython-input-15-5a6917b72e26>", line 1, in <module>
    periods - periods

  File "C:\Python\lib\site-packages\pandas\tseries\base.py", line 658, in __sub__
    typ2=type(other).__name__))

TypeError: cannot subtract PeriodIndex and PeriodIndex
```
@dhirschfeld as indicated above, returning an array (or scalar) of ``DateOffsets`` is the only solution here. Integers are the result of the implementation leaking thru here.

Resolving patch:
diff --git a/pandas/core/indexes/datetimelike.py b/pandas/core/indexes/datetimelike.py
--- a/pandas/core/indexes/datetimelike.py
+++ b/pandas/core/indexes/datetimelike.py
@@ -13,7 +13,8 @@
 import numpy as np
 
 from pandas._libs import lib, iNaT, NaT, Timedelta
-from pandas._libs.tslibs.period import Period
+from pandas._libs.tslibs.period import (Period, IncompatibleFrequency,
+                                        _DIFFERENT_FREQ_INDEX)
 from pandas._libs.tslibs.timedeltas import delta_to_nanoseconds
 from pandas._libs.tslibs.timestamps import round_ns
 
@@ -784,6 +785,41 @@ def _sub_nat(self):
     def _sub_period(self, other):
         return NotImplemented
 
+    def _sub_period_array(self, other):
+        """
+        Subtract one PeriodIndex from another.  This is only valid if they
+        have the same frequency.
+
+        Parameters
+        ----------
+        other : PeriodIndex
+
+        Returns
+        -------
+        result : np.ndarray[object]
+            Array of DateOffset objects; nulls represented by NaT
+        """
+        if not is_period_dtype(self):
+            raise TypeError("cannot subtract {dtype}-dtype to {cls}"
+                            .format(dtype=other.dtype,
+                                    cls=type(self).__name__))
+
+        if not len(self) == len(other):
+            raise ValueError("cannot subtract indices of unequal length")
+        if self.freq != other.freq:
+            msg = _DIFFERENT_FREQ_INDEX.format(self.freqstr, other.freqstr)
+            raise IncompatibleFrequency(msg)
+
+        new_values = checked_add_with_arr(self.asi8, -other.asi8,
+                                          arr_mask=self._isnan,
+                                          b_mask=other._isnan)
+
+        new_values = np.array([self.freq * x for x in new_values])
+        if self.hasnans or other.hasnans:
+            mask = (self._isnan) | (other._isnan)
+            new_values[mask] = NaT
+        return new_values
+
     def _add_offset(self, offset):
         raise com.AbstractMethodError(self)
 
@@ -894,7 +930,7 @@ def __add__(self, other):
                 return self._add_datelike(other)
             elif is_integer_dtype(other):
                 result = self._addsub_int_array(other, operator.add)
-            elif is_float_dtype(other):
+            elif is_float_dtype(other) or is_period_dtype(other):
                 # Explicitly catch invalid dtypes
                 raise TypeError("cannot add {dtype}-dtype to {cls}"
                                 .format(dtype=other.dtype,
@@ -953,6 +989,9 @@ def __sub__(self, other):
             elif is_datetime64_dtype(other) or is_datetime64tz_dtype(other):
                 # DatetimeIndex, ndarray[datetime64]
                 result = self._sub_datelike(other)
+            elif is_period_dtype(other):
+                # PeriodIndex
+                result = self._sub_period_array(other)
             elif is_integer_dtype(other):
                 result = self._addsub_int_array(other, operator.sub)
             elif isinstance(other, Index):


Test patch:
diff --git a/pandas/tests/indexes/datetimes/test_arithmetic.py b/pandas/tests/indexes/datetimes/test_arithmetic.py
--- a/pandas/tests/indexes/datetimes/test_arithmetic.py
+++ b/pandas/tests/indexes/datetimes/test_arithmetic.py
@@ -762,6 +762,17 @@ def test_sub_period(self, freq):
         with pytest.raises(TypeError):
             p - idx
 
+    @pytest.mark.parametrize('op', [operator.add, ops.radd,
+                                    operator.sub, ops.rsub])
+    @pytest.mark.parametrize('pi_freq', ['D', 'W', 'Q', 'H'])
+    @pytest.mark.parametrize('dti_freq', [None, 'D'])
+    def test_dti_sub_pi(self, dti_freq, pi_freq, op):
+        # GH#20049 subtracting PeriodIndex should raise TypeError
+        dti = pd.DatetimeIndex(['2011-01-01', '2011-01-02'], freq=dti_freq)
+        pi = dti.to_period(pi_freq)
+        with pytest.raises(TypeError):
+            op(dti, pi)
+
     def test_ufunc_coercions(self):
         idx = date_range('2011-01-01', periods=3, freq='2D', name='x')
 
diff --git a/pandas/tests/indexes/period/test_arithmetic.py b/pandas/tests/indexes/period/test_arithmetic.py
--- a/pandas/tests/indexes/period/test_arithmetic.py
+++ b/pandas/tests/indexes/period/test_arithmetic.py
@@ -258,6 +258,57 @@ def test_comp_nat(self, dtype):
 
 
 class TestPeriodIndexArithmetic(object):
+    # ---------------------------------------------------------------
+    # __add__/__sub__ with PeriodIndex
+    # PeriodIndex + other is defined for integers and timedelta-like others
+    # PeriodIndex - other is defined for integers, timedelta-like others,
+    #   and PeriodIndex (with matching freq)
+
+    def test_pi_add_iadd_pi_raises(self):
+        rng = pd.period_range('1/1/2000', freq='D', periods=5)
+        other = pd.period_range('1/6/2000', freq='D', periods=5)
+
+        # An earlier implementation of PeriodIndex addition performed
+        # a set operation (union).  This has since been changed to
+        # raise a TypeError. See GH#14164 and GH#13077 for historical
+        # reference.
+        with pytest.raises(TypeError):
+            rng + other
+
+        with pytest.raises(TypeError):
+            rng += other
+
+    def test_pi_sub_isub_pi(self):
+        # GH#20049
+        # For historical reference see GH#14164, GH#13077.
+        # PeriodIndex subtraction originally performed set difference,
+        # then changed to raise TypeError before being implemented in GH#20049
+        rng = pd.period_range('1/1/2000', freq='D', periods=5)
+        other = pd.period_range('1/6/2000', freq='D', periods=5)
+
+        off = rng.freq
+        expected = pd.Index([-5 * off] * 5)
+        result = rng - other
+        tm.assert_index_equal(result, expected)
+
+        rng -= other
+        tm.assert_index_equal(rng, expected)
+
+    def test_pi_sub_pi_with_nat(self):
+        rng = pd.period_range('1/1/2000', freq='D', periods=5)
+        other = rng[1:].insert(0, pd.NaT)
+        assert other[1:].equals(rng[1:])
+
+        result = rng - other
+        off = rng.freq
+        expected = pd.Index([pd.NaT, 0 * off, 0 * off, 0 * off, 0 * off])
+        tm.assert_index_equal(result, expected)
+
+    def test_pi_sub_pi_mismatched_freq(self):
+        rng = pd.period_range('1/1/2000', freq='D', periods=5)
+        other = pd.period_range('1/6/2000', freq='H', periods=5)
+        with pytest.raises(period.IncompatibleFrequency):
+            rng - other
 
     # -------------------------------------------------------------
     # Invalid Operations
@@ -379,17 +430,6 @@ def test_pi_sub_offset_array(self, box):
             with tm.assert_produces_warning(PerformanceWarning):
                 anchored - pi
 
-    def test_pi_add_iadd_pi_raises(self):
-        rng = pd.period_range('1/1/2000', freq='D', periods=5)
-        other = pd.period_range('1/6/2000', freq='D', periods=5)
-
-        # previously performed setop union, now raises TypeError (GH14164)
-        with pytest.raises(TypeError):
-            rng + other
-
-        with pytest.raises(TypeError):
-            rng += other
-
     def test_pi_add_iadd_int(self, one):
         # Variants of `one` for #19012
         rng = pd.period_range('2000-01-01 09:00', freq='H', periods=10)
@@ -419,18 +459,6 @@ def test_pi_sub_intlike(self, five):
         exp = rng + (-five)
         tm.assert_index_equal(result, exp)
 
-    def test_pi_sub_isub_pi_raises(self):
-        # previously performed setop, now raises TypeError (GH14164)
-        # TODO needs to wait on #13077 for decision on result type
-        rng = pd.period_range('1/1/2000', freq='D', periods=5)
-        other = pd.period_range('1/6/2000', freq='D', periods=5)
-
-        with pytest.raises(TypeError):
-            rng - other
-
-        with pytest.raises(TypeError):
-            rng -= other
-
     def test_pi_sub_isub_offset(self):
         # offset
         # DateOffset
diff --git a/pandas/tests/indexes/timedeltas/test_arithmetic.py b/pandas/tests/indexes/timedeltas/test_arithmetic.py
--- a/pandas/tests/indexes/timedeltas/test_arithmetic.py
+++ b/pandas/tests/indexes/timedeltas/test_arithmetic.py
@@ -305,6 +305,18 @@ def test_tdi_sub_period(self, freq):
         with pytest.raises(TypeError):
             p - idx
 
+    @pytest.mark.parametrize('op', [operator.add, ops.radd,
+                                    operator.sub, ops.rsub])
+    @pytest.mark.parametrize('pi_freq', ['D', 'W', 'Q', 'H'])
+    @pytest.mark.parametrize('tdi_freq', [None, 'H'])
+    def test_dti_sub_pi(self, tdi_freq, pi_freq, op):
+        # GH#20049 subtracting PeriodIndex should raise TypeError
+        tdi = pd.TimedeltaIndex(['1 hours', '2 hours'], freq=tdi_freq)
+        dti = pd.Timestamp('2018-03-07 17:16:40') + tdi
+        pi = dti.to_period(pi_freq)
+        with pytest.raises(TypeError):
+            op(dti, pi)
+
     # -------------------------------------------------------------
     # TimedeltaIndex.shift is used by __add__/__sub__