Repository: dask/dask

Issue:
un-numpy-like behavior of `da.pad` with stats 
`da.pad` with `mode` set to a statistic (e.g., `minimum`) a) doesn't match numpy and b) gives a confusing error message when it fails. The source of this problem is how `da.pad` tries to calculate the `stat_length` parameter when that kwarg is not specified.  

If I have the time, I can put together a PR to fix this. It should not be too hard. The problem is this line: https://github.com/dask/dask/blob/master/dask/array/creation.py#L1007, which blindly uses `args[0]` as `stat_length` without an earlier check for whether `stat_length` has been supplied as a `kwarg`. 

MRE: 
```
import dask.array as da
# this works:
arr_2d = da.zeros((1,1))
da.pad(arr_2d, pad_width=1, mode='minimum')

# error: `TypeError: `pad_value` must be composed of integral typed values.`
arr_3d  = da.zeros((1,1,1))
da.pad(arr_3d, pad_width=1, mode='minimum')

# In numpy, pad with mode=$statistic uses an extra kwarg called `stat_length, which defaults to None. 
# In `da.pad`, passing stat_length=None doesn't work. This produces the same confusing error about the non-integral type of the pad_value.
da.pad(arr_3d, pad_width=1, mode='minimum', stat_length=None)

# this gives equivalent behavior to np.pad with stat_length=None:
da.pad(arr_3d, pad_width=1, mode='minimum', stat_length=arr_3d.shape)
```



Discussion:
@d-v-b thanks for raising this issue and investigating the `da.pad` internals! I'm able to reproduce the same `TypeError`. 

> If I have the time, I can put together a PR to fix this

That would be much appreciated, thanks! 

cc @jakirkham 
I started working on a PR to resolve this issue. I found many instances where the internals of `da.pad` diverges from `np.pad`. An attractive strategy for the PR would be to bring the internals of `da.pad` into alignment with `np.pad`, but I wonder: was there a conscious decision to diverge from numpy in this function? if so, then I would need to take my PR in another direction.

Additionally, I notice a lot of useful private helper functions in `np.lib.arraypad`, e.g. https://github.com/numpy/numpy/blob/v1.17.0/numpy/lib/arraypad.py#L504. Is there a reason not to use these helper functions in `da.pad`? 

@jakirkham 

Let me just start by saying that I think Dask is a really cool project :-)

@d-v-b Cool you are working on this. I'm currently working on implementing a pad method for [xarray](https://github.com/pydata/xarray/issues/2605), I saw some of the unittest fail and wrote a little script to compare numpy as dask's implementation. I saw a couple of ways that numpy and xarray seem to differ:

1. The issue you mention above where the default stat_length kwarg fails.
2. When using `pad_widths= ((2, 3),)` all modes fail with the same error message as above. 
3. When using `mode="mean"`, Dask converts the array to floats, while numpy keeps the array as integers.
4. Modes `symmetric`, `reflect` and `wrap`  can result in output arrays that have a different shape than the numpy equivalent. 

Do you think I should create several seperate issues in the Dask issue tracker?

This is the script I used to test this:
```
import dask.array as da
import numpy as np

np_arr = np.arange(2*3*4).reshape(2,3,4)
da_arr = da.from_array(np_arr, chunks="auto")

for pad_widths in [2, (2,), ((2, 3),), ((3, 1), (0,0), (2, 0))]:
    for mode in ["constant", "edge", "linear_ramp", "maximum", "mean", "minimum", "reflect", "symmetric", "wrap"]: #[median, empty]

        np_result = np.pad(np_arr, pad_widths, mode=mode)
        try:
            da_result = da.pad(da_arr, pad_widths, mode=mode).compute()
        except Exception as e:
            print("ERROR", mode, str(pad_widths))
            print(e)
            continue

        try:
            assert np_result.dtype == da_result.dtype, "dtype does not match"
            assert np_result.shape == da_result.shape, "shapes do not match"
            np.testing.assert_equal(np_result, da_result), "values do not match"
            print("successfull:", mode, str(pad_widths))
        except AssertionError as e:
            print("mismatch:", mode, str(pad_widths))
            print(e)
```
@mark-boer I haven't touched my code for fixing this in a while -- my progress is stalled while waiting for answers to the questions I asked in my last comment on the issue. 

That being said, I think there should be tests that catch these problems, so I assume additions to the test suite would be appreciated (and maybe we can catch @jakirkham's attention :) 
In general, I think we should assume that undocumented differences between the two are unintentional bugs. PRs fixing these (small, independent ones each focused on a specific issue, if possible) would be welcome.
When I wrote this, I think we matched NumPy pretty closely. Since then NumPy revamped how they pad arrays to be more performant ( https://github.com/numpy/numpy/pull/11358 ) with some follow-up PRs. It's possible something changed through that churn. I don't have any objection to either raising an issue with NumPy if they changed behavior or aligning to it on our end.
Ah, thanks for the historical context @jakirkham, that explains a lot. If I have time soon I'll look into bringing dask into alignment with numpy in this area.
Thanks @d-v-b! 😄

Resolving patch:
diff --git a/dask/array/creation.py b/dask/array/creation.py
--- a/dask/array/creation.py
+++ b/dask/array/creation.py
@@ -835,7 +835,7 @@ def expand_pad_value(array, pad_value):
         and len(pad_value) == 2
         and all(isinstance(pw, Number) for pw in pad_value)
     ):
-        pad_value = tuple((pad_value[0], pad_value[1]) for _ in range(array.ndim))
+        pad_value = array.ndim * (tuple(pad_value),)
     elif (
         isinstance(pad_value, Sequence)
         and len(pad_value) == array.ndim
@@ -843,7 +843,15 @@ def expand_pad_value(array, pad_value):
         and all((len(pw) == 2) for pw in pad_value)
         and all(all(isinstance(w, Number) for w in pw) for pw in pad_value)
     ):
-        pad_value = tuple((pw[0], pw[1]) for pw in pad_value)
+        pad_value = tuple(tuple(pw) for pw in pad_value)
+    elif (
+        isinstance(pad_value, Sequence)
+        and len(pad_value) == 1
+        and isinstance(pad_value[0], Sequence)
+        and len(pad_value[0]) == 2
+        and all(isinstance(pw, Number) for pw in pad_value[0])
+    ):
+        pad_value = array.ndim * (tuple(pad_value[0]),)
     else:
         raise TypeError("`pad_value` must be composed of integral typed values.")
 
@@ -1117,7 +1125,7 @@ def pad(array, pad_width, mode, **kwargs):
     pad_width = expand_pad_value(array, pad_width)
 
     if mode in ["maximum", "mean", "median", "minimum"]:
-        kwargs.setdefault("stat_length", array.shape)
+        kwargs.setdefault("stat_length", tuple((n, n) for n in array.shape))
     elif mode == "constant":
         kwargs.setdefault("constant_values", 0)
     elif mode == "linear_ramp":


Test patch:
diff --git a/dask/array/tests/test_creation.py b/dask/array/tests/test_creation.py
--- a/dask/array/tests/test_creation.py
+++ b/dask/array/tests/test_creation.py
@@ -10,7 +10,7 @@
 import dask.array as da
 from dask.array.core import normalize_chunks
 from dask.array.utils import assert_eq, same_keys, AxisError
-from dask.array.numpy_compat import _numpy_117
+from dask.array.numpy_compat import _numpy_117, _numpy_118
 
 
 @pytest.mark.parametrize(
@@ -719,6 +719,66 @@ def test_pad(shape, chunks, pad_width, mode, kwargs):
         assert_eq(np_r, da_r)
 
 
+@pytest.mark.parametrize("dtype", [np.uint8, np.int16, np.float32, bool])
+@pytest.mark.parametrize(
+    "pad_widths", [2, (2,), (2, 3), ((2, 3),), ((3, 1), (0, 0), (2, 0))]
+)
+@pytest.mark.parametrize(
+    "mode",
+    [
+        "constant",
+        "edge",
+        pytest.param(
+            "linear_ramp",
+            marks=pytest.mark.skipif(
+                not _numpy_118, reason="numpy changed pad behaviour"
+            ),
+        ),
+        "maximum",
+        pytest.param(
+            "mean",
+            marks=pytest.mark.skip(
+                reason="Bug dask changes the dtype to float: https://github.com/dask/dask/issues/5303"
+            ),
+        ),
+        "minimum",
+        pytest.param(
+            "reflect",
+            marks=pytest.mark.skip(
+                reason="Bug when pad_width is larger than dimension: https://github.com/dask/dask/issues/5303"
+            ),
+        ),
+        pytest.param(
+            "symmetric",
+            marks=pytest.mark.skip(
+                reason="Bug when pad_width is larger than dimension: https://github.com/dask/dask/issues/5303"
+            ),
+        ),
+        pytest.param(
+            "wrap",
+            marks=pytest.mark.skip(
+                reason="Bug when pad_width is larger than dimension: https://github.com/dask/dask/issues/5303"
+            ),
+        ),
+        pytest.param("median", marks=pytest.mark.skip(reason="Not implemented"),),
+        pytest.param(
+            "empty",
+            marks=pytest.mark.skip(
+                reason="Empty leads to undefined values, which may be different"
+            ),
+        ),
+    ],
+)
+def test_pad_3d_data(dtype, pad_widths, mode):
+    np_a = np.arange(2 * 3 * 4).reshape(2, 3, 4).astype(dtype)
+    da_a = da.from_array(np_a, chunks="auto")
+
+    np_r = np.pad(np_a, pad_widths, mode=mode)
+    da_r = da.pad(da_a, pad_widths, mode=mode)
+
+    assert_eq(np_r, da_r)
+
+
 @pytest.mark.parametrize("kwargs", [{}, {"scaler": 2}])
 def test_pad_udf(kwargs):
     def udf_pad(vector, pad_width, iaxis, kwargs):