Repository: numpy/numpy
Pull request: BUG: fix ma.diff not preserving mask when using append/prepend
<!--         ----------------------------------------------------------------
                MAKE SURE YOUR PR GETS THE ATTENTION IT DESERVES!
                ----------------------------------------------------------------

*  FORMAT IT RIGHT:
      https://www.numpy.org/devdocs/dev/development_workflow.html#writing-the-commit-message

*  IF IT'S A NEW FEATURE OR API CHANGE, TEST THE WATERS:
      https://www.numpy.org/devdocs/dev/development_workflow.html#get-the-mailing-list-s-opinion

*  HIT ALL THE GUIDELINES:
      https://numpy.org/devdocs/dev/index.html#guidelines

*  WHAT TO DO IF WE HAVEN'T GOTTEN BACK TO YOU:
      https://www.numpy.org/devdocs/dev/development_workflow.html#getting-your-pr-reviewed
-->

Closes https://github.com/numpy/numpy/issues/22465.

Ported CORE diff to MA
- Kept only relevant append/prepend code
- Internally call CORE diff
- docstring mentions mask preservation + change return type to MaskedArray
- docstring examples adapted to MA module

Added 2 unit tests



Problem statement:
BUG: Numpy ma diff skipping the mask when using argument 'prepend'
### Describe the issue:

Hi ,

I have realized that when I pass a masked array in numpy ma diff ```np.ma.diff``` and prepend a masked intial value, the mask is deleted and replaced (in my case for 0. values). Despite being the ```fill_value``` different from 0.

I would expect that ```np.ma.diff``` keeps the mask after calculating the difference, specifically if you are prepending a masked array that keeps the same structure than the input array.
See below:

### Reproduce the code example:

```python
import numpy as np
# a is the dummy array
a = np.random.randn(5,10,10)
a = np.ma.masked_where(a<.1,a)
print(a.mask.any()) # returns True

b = np.ma.diff(a, prepend=a[:1,...],axis=0)
print(b.mask.any()) # returns False (unexpected)

# What I would expect / my workaround
c = np.ma.diff(np.ma.append(a[:1], a, axis=0),axis=0)
print(c.mask.any()) # returns True (as expected)
```

### NumPy/Python version information:

My numpy version is: '1.23.4' 
Python version is: 3.10.6

### Context for the issue:

Shouldn't np.ma module's objects respect always the input mask (if it is properly referenced in its arguments)?


Thanks a lot and keep on with the amazing numpy!

Hints from issue discussion:

Commit message: DEV: add pyi

Patch to review:
diff --git a/doc/release/upcoming_changes/22776.improvement.rst b/doc/release/upcoming_changes/22776.improvement.rst
new file mode 100644
index 000000000000..669fcff161a9
--- /dev/null
+++ b/doc/release/upcoming_changes/22776.improvement.rst
@@ -0,0 +1,6 @@
+Fix ``np.ma.diff`` not preserving the mask when called with arguments prepend/append.
+-------------------------------------------------------------------------------------
+Calling ``np.ma.diff`` with arguments prepend and/or append now returns a 
+``MaskedArray`` with the input mask preserved.
+
+Previously, a ``MaskedArray`` without the mask was returned.
\ No newline at end of file
diff --git a/numpy/ma/core.py b/numpy/ma/core.py
index 59ba3a593931..02799baed6ff 100644
--- a/numpy/ma/core.py
+++ b/numpy/ma/core.py
@@ -7341,6 +7341,126 @@ def size(obj, axis=None):
 size.__doc__ = np.size.__doc__
 
 
+def diff(a, /, n=1, axis=-1, prepend=np._NoValue, append=np._NoValue):
+    """
+    Calculate the n-th discrete difference along the given axis.
+    The first difference is given by ``out[i] = a[i+1] - a[i]`` along
+    the given axis, higher differences are calculated by using `diff`
+    recursively.
+    Preserves the input mask.
+
+    .. versionadded:: 1.25.0
+
+    Parameters
+    ----------
+    a : array_like
+        Input array
+    n : int, optional
+        The number of times values are differenced. If zero, the input
+        is returned as-is.
+    axis : int, optional
+        The axis along which the difference is taken, default is the
+        last axis.
+    prepend, append : array_like, optional
+        Values to prepend or append to `a` along axis prior to
+        performing the difference.  Scalar values are expanded to
+        arrays with length 1 in the direction of axis and the shape
+        of the input array in along all other axes.  Otherwise the
+        dimension and shape must match `a` except along axis.
+
+    Returns
+    -------
+    diff : MaskedArray
+        The n-th differences. The shape of the output is the same as `a`
+        except along `axis` where the dimension is smaller by `n`. The
+        type of the output is the same as the type of the difference
+        between any two elements of `a`. This is the same as the type of
+        `a` in most cases. A notable exception is `datetime64`, which
+        results in a `timedelta64` output array.
+
+    See Also
+    --------
+    numpy.diff : Equivalent function in the top-level NumPy module.
+
+    Notes
+    -----
+    Type is preserved for boolean arrays, so the result will contain
+    `False` when consecutive elements are the same and `True` when they
+    differ.
+
+    For unsigned integer arrays, the results will also be unsigned. This
+    should not be surprising, as the result is consistent with
+    calculating the difference directly:
+
+    >>> u8_arr = np.array([1, 0], dtype=np.uint8)
+    >>> np.ma.diff(u8_arr)
+    array([255], dtype=uint8)
+    >>> u8_arr[1,...] - u8_arr[0,...]
+    255
+
+    If this is not desirable, then the array should be cast to a larger
+    integer type first:
+
+    >>> i16_arr = u8_arr.astype(np.int16)
+    >>> np.ma.diff(i16_arr)
+    array([-1], dtype=int16)
+
+    Examples
+    --------
+    >>> a = np.array([1, 2, 3, 4, 7, 0, 2, 3])
+    >>> x = np.ma.masked_where(a < 2, a)
+    >>> np.ma.diff(x)
+    masked_array(data=[--, 1, 1, 3, --, --, 1],
+            mask=[ True, False, False, False,  True,  True, False],
+        fill_value=999999)
+
+    >>> np.ma.diff(x, n=2)
+    masked_array(data=[--, 0, 2, --, --, --],
+                mask=[ True, False, False,  True,  True,  True],
+        fill_value=999999)
+
+    >>> a = np.array([[1, 3, 1, 5, 10], [0, 1, 5, 6, 8]])
+    >>> x = np.ma.masked_equal(a, value=1)
+    >>> np.ma.diff(x)
+    masked_array(
+        data=[[--, --, --, 5],
+                [--, --, 1, 2]],
+        mask=[[ True,  True,  True, False],
+                [ True,  True, False, False]],
+        fill_value=1)
+
+    >>> np.ma.diff(x, axis=0)
+    masked_array(data=[[--, --, --, 1, -2]],
+            mask=[[ True,  True,  True, False, False]],
+        fill_value=1)
+
+    """
+    combined = []
+    if prepend is not np._NoValue:
+        prepend = np.ma.asanyarray(prepend)
+        if prepend.ndim == 0:
+            shape = list(a.shape)
+            shape[axis] = 1
+            prepend = np.broadcast_to(prepend, tuple(shape))
+        combined.append(prepend)
+
+    combined.append(a)
+
+    if append is not np._NoValue:
+        append = np.ma.asanyarray(append)
+        if append.ndim == 0:
+            shape = list(a.shape)
+            shape[axis] = 1
+            append = np.broadcast_to(append, tuple(shape))
+        combined.append(append)
+
+    if len(combined) > 1:
+        a = np.ma.concatenate(combined, axis)
+
+    # GH 22465 np.diff without prepend/append preserves the mask 
+    return np.diff(a, n, axis)
+
+
 ##############################################################################
 #                            Extra functions                                 #
 ##############################################################################
@@ -8281,12 +8401,6 @@ def __call__(self, *args, **params):
     np_ret='clipped_array : ndarray',
     np_ma_ret='clipped_array : MaskedArray',
 )
-diff = _convert2ma(
-    'diff',
-    params=dict(fill_value=None, hardmask=False),
-    np_ret='diff : ndarray',
-    np_ma_ret='diff : MaskedArray',
-)
 empty = _convert2ma(
     'empty',
     params=dict(fill_value=None, hardmask=False),
diff --git a/numpy/ma/core.pyi b/numpy/ma/core.pyi
index 94a91da85a46..58d73a231a1f 100644
--- a/numpy/ma/core.pyi
+++ b/numpy/ma/core.pyi
@@ -7,7 +7,6 @@ from numpy import (
     amin as amin,
     bool_ as bool_,
     expand_dims as expand_dims,
-    diff as diff,
     clip as clip,
     indices as indices,
     ones_like as ones_like,
@@ -429,6 +428,7 @@ def resize(x, new_shape): ...
 def ndim(obj): ...
 def shape(obj): ...
 def size(obj, axis=...): ...
+def diff(a, /, n=..., axis=..., prepend=..., append=...): ...
 def where(condition, x=..., y=...): ...
 def choose(indices, choices, out=..., mode=...): ...
 def round_(a, decimals=..., out=...): ...
diff --git a/numpy/ma/tests/test_core.py b/numpy/ma/tests/test_core.py
index bc897d7314a5..135c3e5310e8 100644
--- a/numpy/ma/tests/test_core.py
+++ b/numpy/ma/tests/test_core.py
@@ -4091,6 +4091,33 @@ def test_mean_overflow(self):
                          mask=np.zeros((10000, 10000)))
         assert_equal(a.mean(), 65535.0)
 
+    def test_diff_with_prepend(self):
+        # GH 22465
+        x = np.array([1, 2, 2, 3, 4, 2, 1, 1])
+
+        a = np.ma.masked_equal(x[3:], value=2)
+        a_prep = np.ma.masked_equal(x[:3], value=2)
+        diff1 = np.ma.diff(a, prepend=a_prep, axis=0)
+
+        b = np.ma.masked_equal(x, value=2)
+        diff2 = np.ma.diff(b, axis=0)
+
+        assert_(np.ma.allequal(diff1, diff2))
+
+    def test_diff_with_append(self):
+        # GH 22465
+        x = np.array([1, 2, 2, 3, 4, 2, 1, 1])
+
+        a = np.ma.masked_equal(x[:3], value=2)
+        a_app = np.ma.masked_equal(x[3:], value=2)
+        diff1 = np.ma.diff(a, append=a_app, axis=0)
+
+        b = np.ma.masked_equal(x, value=2)
+        diff2 = np.ma.diff(b, axis=0)
+
+        assert_(np.ma.allequal(diff1, diff2))
+
+
 class TestMaskedArrayMathMethodsComplex:
     # Test class for miscellaneous MaskedArrays methods.
     def setup_method(self):


Code review comments:
diff_hunk: @@ -7341,6 +7341,126 @@ def size(obj, axis=None):
 size.__doc__ = np.size.__doc__
 
 
+def diff(a, /, n=1, axis=-1, prepend=np._NoValue, append=np._NoValue):
+    """
+    Calculate the n-th discrete difference along the given axis.
+    The first difference is given by ``out[i] = a[i+1] - a[i]`` along
+    the given axis, higher differences are calculated by using `diff`
+    recursively.
+    Preserves the input mask.
+
+    .. versionadded:: 1.25.0
+
path: numpy/ma/core.py
text: @user1:
```suggestion
```
Sorry my bad, since it was auto-wrapping before, this is basically a bug fix...