Repository: dask/dask

Issue:
Can only specify boundary conditions per-axis with `ghost.ghost`
As asked on [SO](http://stackoverflow.com/questions/31616830/individual-dask-array-boundaries-with-ghosting/31617671)

The current `ghost.ghost` function allows specifying a few different types of B.C. but they're always the same for both y-boundaries and both x-boundaries.  This makes perfect sense for periodic boundaries, but less so for Dirichlet boundaries.  

As @mrocklin commented, `ghost.ghost_internal` can accomplish most of this but it would be good if it were exposed in `ghost.ghost`, e.g.

`g = da.ghost.ghost(x, depth={0: 1, 1: 1}, boundary={0: 100, 1: 100, 2: 0, 3: 0})`



Discussion:
Pinging @stefanv

In the scikit-image use-case, a feature that would be useful is to only ghost "to the inside" of the array, i.e. to ghost, but to crop the ghosted array to the original image extents.

The reason for this is that scikit-image has specialized handling of boundaries, which sometimes causes artefacts.  If ghosting pads out those edges, these (desired) artefacts will disappear.

@stefanv that is currently possible with [`ghost_internal`](https://github.com/ContinuumIO/dask/blob/master/dask/array/ghost.py#L99). 

Perhaps this would be best exposed by adding a `internal_depth` keyword argument to `da.ghost.ghost`. Which takes dict keyed with axes and valued with depths. If an axis in this dictionary is not specified then it will be taken from the `depth` kwarg. And if just the `internal_depth` is specified for an axis, the external boundary will be zero.

Note I mixed up `boundary` and `depth` above, but have since edited the comment to fix this.^

There should be an easy way to say: "do what you normally do, just don't go outside the array boundaries"

Ideally this option ends up within the `da.Array.map_overlap` function which, I think, has the most user friendly API so far.  I think that this function came out of @cowlicks early work with skimage.  

At the moment the `boundary=None` option defaults to `reflect`.  Perhaps `None` should actually mean "no boundary."

@mrocklin there are currently two arguments for specifying the ghosting. `depth` and `boundary`. Which, more verbosely, mean "depth of ghosting" and "external ghosting type". If someone specifies a ghosting depth, but no boundary type, I think they would still expect some ghosting to happen.

One reason `reflect` might be the default is because this is the behavior for some skimage filters.

@stefanv we could have an `internal_ghosting_only=True/False` kwarg in ghost. However I think an `internal_depth` kwarg would be valuable since it would allow the user to have internal ghosting which differs from external ghosting.

>  If someone specifies a ghosting depth, but no boundary type, I think they would still expect some ghosting to happen

I agree.  But would the expect ghosting on the boundary?

> One reason reflect might be the default is because this is the behavior for some skimage filters.

This is a good reason to keep the default as `reflect`.  I wonder if we should set the default to `boundary='reflect'` in the function signature and then use `boundary=None` to mean internal ghosting only.

@gforsyth you _can_ specify the _external_ boundary conditions per axis.

``` python
In [1]: import numpy as np; import dask.array as da

In [2]: a = da.from_array(np.arange(16).reshape(4, 4), chunks=(2,2))

In [3]: g = da.ghost.ghost(a, depth={0: 1, 1:2}, boundary={0:42, 1:'reflect'})

In [4]: g.compute()
Out[4]: 
array([[42, 42, 42, 42, 42, 42, 42, 42, 42, 42, 42, 42],
       [ 1,  0,  0,  1,  2,  3,  0,  1,  2,  3,  3,  2],
       [ 5,  4,  4,  5,  6,  7,  4,  5,  6,  7,  7,  6],
       [ 9,  8,  8,  9, 10, 11,  8,  9, 10, 11, 11, 10],
       [ 5,  4,  4,  5,  6,  7,  4,  5,  6,  7,  7,  6],
       [ 9,  8,  8,  9, 10, 11,  8,  9, 10, 11, 11, 10],
       [13, 12, 12, 13, 14, 15, 12, 13, 14, 15, 15, 14],
       [42, 42, 42, 42, 42, 42, 42, 42, 42, 42, 42, 42]])
```

But you _cannot_ specify the _internal_ boundary conditions, the internal boundary conditions just copy the elements of the chunks next to them. We could possibly change this, but I'm not sure this is what you want.

And currently the `depth` argument is the depth of the boundary for both internal and external boundaries. We could easily change this so this by adding a `internal_depth` argument which would default to be the same as `depth` if it is not provided.

I suspect that @gforsyth wanted to have different boundary conditions on the left vs right or top vs bottom, which is not currently supported.

That's what I was thinking of.  I was just playing around with solving PDEs in Dask and if you're doing something that's elliptic and you want to set one side of your domain to 0 and the other 3 to 100 that doesn't work the way ghost is currently set up.

@gforsyth Perhaps a good api here would be to allow tuples as values in the `boundary` argument like:

```
In [1]: import numpy as np; import dask.array as da

In [2]: a = da.from_array(np.arange(16).reshape(4, 4), chunks=(2,2))

In [3]: g = da.ghost.ghost(a, depth={0: 1, 1:2}, boundary={0: (33, 42), 1:('reflect', 66)}

In [4]: g.compute()
Out[4]: 
array([[33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 66],
       [ 1,  0,  0,  1,  2,  3,  0,  1,  2,  3,  3, 66],
       [ 5,  4,  4,  5,  6,  7,  4,  5,  6,  7,  7, 66],
       [ 9,  8,  8,  9, 10, 11,  8,  9, 10, 11, 11, 66],
       [ 5,  4,  4,  5,  6,  7,  4,  5,  6,  7,  7, 66],
       [ 9,  8,  8,  9, 10, 11,  8,  9, 10, 11, 11, 66],
       [13, 12, 12, 13, 14, 15, 12, 13, 14, 15, 15, 66],
       [42, 42, 42, 42, 42, 42, 42, 42, 42, 42, 42, 66]])
```

Note that doing something like this with the `depth` argument would make internal depth ambiguous.

If I understand correctly this is distinct from what @stefanv wants.

I think that would work well for specifying the external boundaries.  Does it make any sense to also have a `depth` tuple possible?  You might need to have a reflection that's 2-deep for a Neumann boundary but only need a 1-deep for Dirichlet on the opposite boundary.

Hrrr.  Just realized that that would necessitate a separate depth kwarg for internal vs. external boundaries otherwise it's going to be _very_ ambiguous.

Noting your example array

```
In [4]: g.compute()
Out[4]: 
array([[33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 33, 66],
       [ 1,  0,  0,  1,  2,  3,  0,  1,  2,  3,  3, 66],
       [ 5,  4,  4,  5,  6,  7,  4,  5,  6,  7,  7, 66],
       [ 9,  8,  8,  9, 10, 11,  8,  9, 10, 11, 11, 66],
       [ 5,  4,  4,  5,  6,  7,  4,  5,  6,  7,  7, 66],
       [ 9,  8,  8,  9, 10, 11,  8,  9, 10, 11, 11, 66],
       [13, 12, 12, 13, 14, 15, 12, 13, 14, 15, 15, 66],
       [42, 42, 42, 42, 42, 42, 42, 42, 42, 42, 42, 66]])
```

the external boundary depth doesn't particularly matter -- if it were the difference between a Neumann and Dirichlet boundary the extra bit of memory required to pad out a Dirichlet boundary one column further than necessary is negligible.

That said, `g[:-2]` seems to be a reflect while `g[:-1]` is Dirichlet.  I imagine it's just an editing error from mocking up the boundary tuple example but so long as both `g[:-2]` and `g[:-1]` were both 66 in the above example that would cover the use cases I have.

One thing to keep in mind when evaluating all these new options, is that
preferably all keywords should be usable in combination with each other.
The API becomes terribly convoluted when you have "use this keyword unless
than one is specified, in which case we change the meaning of the first or
ignore it".

Agreed.  My suggestion for a depth tuple is unnecessary (and also ignored @cowlicks pointing out the inherent ambiguity in such a setup).  I think allowing tuples in the `boundary` argument is sufficient.

@gforsyth I'm giving a talk this weekend ([about dask](https://pygotham.org/2015/talks/169/going-parallel-and-larger/)), so I'll probably get to this Monday. Unless someone beats me to it.

@stefanv how is this API for your usecase: `boundary=None` will mean no external boundaries, `boundary={0:(1, None)}` would pad the left boundary with ones, and not add a boundary on the right, and it will default to `boundary='reflect'.

Perfect.


Resolving patch:
diff --git a/dask/array/core.py b/dask/array/core.py
--- a/dask/array/core.py
+++ b/dask/array/core.py
@@ -1060,9 +1060,10 @@ def map_overlap(self, func, depth, boundary=None, trim=True, **kwargs):
         depth: int, tuple, or dict
             The number of cells that each block should share with its neighbors
             If a tuple or dict this can be different per axis
-        boundary: str
-            how to handle the boundaries.  Values include 'reflect', 'periodic'
-            or any constant value like 0 or np.nan
+        boundary: str, tuple, dict
+            how to handle the boundaries.  Values include 'reflect',
+            'periodic', 'nearest', 'none', or any constant value like 0 or
+            np.nan
         trim: bool
             Whether or not to trim the excess after the map function.  Set this
             to false if your mapping function does this for you.
@@ -1080,6 +1081,24 @@ def map_overlap(self, func, depth, boundary=None, trim=True, **kwargs):
         >>> y = x.map_overlap(derivative, depth=1, boundary=0)
         >>> y.compute()
         array([ 1,  0,  1,  1,  0,  0, -1, -1,  0])
+
+        >>> import dask.array as da
+        >>> x = np.arange(16).reshape((4, 4))
+        >>> d = da.from_array(x, chunks=(2, 2))
+        >>> d.map_overlap(lambda x: x + x.size, depth=1).compute()
+        array([[16, 17, 18, 19],
+               [20, 21, 22, 23],
+               [24, 25, 26, 27],
+               [28, 29, 30, 31]])
+
+        >>> func = lambda x: x + x.size
+        >>> depth = {0: 1, 1: 1}
+        >>> boundary = {0: 'reflect', 1: 'none'}
+        >>> d.map_overlap(func, depth, boundary).compute()  # doctest: +NORMALIZE_WHITESPACE
+        array([[ 12.,  13.,  14.,  15.],
+               [ 16.,  17.,  18.,  19.],
+               [ 20.,  21.,  22.,  23.],
+               [ 24.,  25.,  26.,  27.]])
         """
         from .ghost import map_overlap
         return map_overlap(self, func, depth, boundary, trim, **kwargs)
diff --git a/dask/array/ghost.py b/dask/array/ghost.py
--- a/dask/array/ghost.py
+++ b/dask/array/ghost.py
@@ -268,11 +268,14 @@ def boundaries(x, depth=None, kind=None):
         if d == 0:
             continue
 
-        if kind.get(i) == 'periodic':
+        this_kind = kind.get(i, 'none')
+        if this_kind == 'none':
+            continue
+        elif this_kind == 'periodic':
             x = periodic(x, i, d)
-        elif kind.get(i) == 'reflect':
+        elif this_kind == 'reflect':
             x = reflect(x, i, d)
-        elif kind.get(i) == 'nearest':
+        elif this_kind == 'nearest':
             x = nearest(x, i, d)
         elif i in kind:
             x = constant(x, i, d, kind[i])
@@ -291,7 +294,8 @@ def ghost(x, depth, boundary):
     depth: dict
         The size of the shared boundary per axis
     boundary: dict
-        The boundary condition on each axis
+        The boundary condition on each axis. Options are 'reflect', 'periodic',
+        'nearest', 'none', an integer will fill the boundary with that integer.
 
     The axes dict informs how many cells to overlap between neighboring blocks
     {0: 2, 2: 5} means share two cells in 0 axis, 5 cells in 2 axis
@@ -330,20 +334,11 @@ def ghost(x, depth, boundary):
            [100, 100, 100, 100, 100, 100, 100, 100, 100, 100, 100, 100],
            [100, 100, 100, 100, 100, 100, 100, 100, 100, 100, 100, 100]])
     """
-    if isinstance(depth, int):
-        depth = (depth,) * x.ndim
-    if isinstance(depth, tuple):
-        depth = dict(zip(range(x.ndim), depth))
-
-    if boundary is None:
-        boundary = 'reflect'
-    if not isinstance(boundary, (tuple, dict)):
-        boundary = (boundary,) * x.ndim
-    if isinstance(boundary, tuple):
-        boundary = dict(zip(range(x.ndim), boundary))
+    depth2 = coerce_depth(x.ndim, depth)
+    boundary2 = coerce_boundary(x.ndim, boundary)
 
     # is depth larger than chunk size?
-    depth_values = [depth.get(i, 0) for i in range(x.ndim)]
+    depth_values = [depth2.get(i, 0) for i in range(x.ndim)]
     for d, c in zip(depth_values, x.chunks):
         if d > min(c):
             raise ValueError("The overlapping depth %d is larger than your\n"
@@ -351,30 +346,73 @@ def ghost(x, depth, boundary):
                              "with a larger chunk size or a chunk size that\n"
                              "more evenly divides the shape of your array." %
                              (d, min(c)))
-    x2 = boundaries(x, depth, boundary)
-    x3 = ghost_internal(x2, depth)
-    trim = dict((k, v*2 if boundary.get(k, None) is not None else 0)
-                for k, v in depth.items())
+    x2 = boundaries(x, depth2, boundary2)
+    x3 = ghost_internal(x2, depth2)
+    trim = dict((k, v*2 if boundary2.get(k, 'none') != 'none' else 0)
+                for k, v in depth2.items())
     x4 = chunk.trim(x3, trim)
     return x4
 
 
+def add_dummy_padding(x, depth, boundary):
+    """
+    Pads an array which has 'none' as the boundary type.
+    Used to simplify trimming arrays which use 'none'.
+
+    >>> import dask.array as da
+    >>> x = da.arange(6, chunks=3)
+    >>> add_dummy_padding(x, {0: 1}, {0: 'none'}).compute()  # doctest: +NORMALIZE_WHITESPACE
+    array([..., 0., 1., 2., 3., 4., 5., ...])
+    """
+    for k, v in boundary.items():
+        if v == 'none':
+            d = depth[k]
+
+            empty_shape = list(x.shape)
+            empty_shape[k] = d
+
+            empty_chunks = list(x.chunks)
+            empty_chunks[k] = (d,)
+
+            empty = wrap.empty(empty_shape, chunks=empty_chunks)
+
+            out_chunks = list(x.chunks)
+            ax_chunks = list(out_chunks[k])
+            ax_chunks[0] += d
+            ax_chunks[-1] += d
+            out_chunks[k] = ax_chunks
+
+            x = concatenate([empty, x, empty], axis=k)
+            x = x.rechunk(out_chunks)
+    return x
+
+
 def map_overlap(x, func, depth, boundary=None, trim=True, **kwargs):
+    depth2 = coerce_depth(x.ndim, depth)
+    boundary2 = coerce_boundary(x.ndim, boundary)
+
+    g = ghost(x, depth=depth2, boundary=boundary2)
+    g2 = g.map_blocks(func, **kwargs)
+    if trim:
+        g3 = add_dummy_padding(g2, depth2, boundary2)
+        return trim_internal(g3, depth2)
+    else:
+        return g2
+
+
+def coerce_depth(ndim, depth):
     if isinstance(depth, int):
-        depth = (depth,) * x.ndim
+        depth = (depth,) * ndim
     if isinstance(depth, tuple):
-        depth = dict(zip(range(x.ndim), depth))
+        depth = dict(zip(range(ndim), depth))
+    return depth
 
+
+def coerce_boundary(ndim, boundary):
     if boundary is None:
         boundary = 'reflect'
     if not isinstance(boundary, (tuple, dict)):
-        boundary = (boundary,) * x.ndim
+        boundary = (boundary,) * ndim
     if isinstance(boundary, tuple):
-        boundary = dict(zip(range(x.ndim), boundary))
-
-    g = ghost(x, depth=depth, boundary=boundary)
-    g2 = g.map_blocks(func, **kwargs)
-    if trim:
-        return trim_internal(g2, depth)
-    else:
-        return g2
+        boundary = dict(zip(range(ndim), boundary))
+    return boundary


Test patch:
diff --git a/dask/array/tests/test_ghost.py b/dask/array/tests/test_ghost.py
--- a/dask/array/tests/test_ghost.py
+++ b/dask/array/tests/test_ghost.py
@@ -175,7 +175,25 @@ def test_ghost():
     assert same_keys(g, ghost(d, depth={0: 2, 1: 1},
                              boundary={0: 100, 1: 'reflect'}))
 
-    g = ghost(d, depth={0: 2, 1: 1}, boundary={0: 100})
+    g = ghost(d, depth={0: 2, 1: 1}, boundary={0: 100, 1: 'none'})
+    expected = np.array(
+      [[100, 100, 100, 100, 100, 100, 100, 100, 100, 100],
+       [100, 100, 100, 100, 100, 100, 100, 100, 100, 100],
+       [  0,   1,   2,   3,   4,   3,   4,   5,   6,   7],
+       [  8,   9,  10,  11,  12,  11,  12,  13,  14,  15],
+       [ 16,  17,  18,  19,  20,  19,  20,  21,  22,  23],
+       [ 24,  25,  26,  27,  28,  27,  28,  29,  30,  31],
+       [ 32,  33,  34,  35,  36,  35,  36,  37,  38,  39],
+       [ 40,  41,  42,  43,  44,  43,  44,  45,  46,  47],
+       [ 16,  17,  18,  19,  20,  19,  20,  21,  22,  23],
+       [ 24,  25,  26,  27,  28,  27,  28,  29,  30,  31],
+       [ 32,  33,  34,  35,  36,  35,  36,  37,  38,  39],
+       [ 40,  41,  42,  43,  44,  43,  44,  45,  46,  47],
+       [ 48,  49,  50,  51,  52,  51,  52,  53,  54,  55],
+       [ 56,  57,  58,  59,  60,  59,  60,  61,  62,  63],
+       [100, 100, 100, 100, 100, 100, 100, 100, 100, 100],
+       [100, 100, 100, 100, 100, 100, 100, 100, 100, 100]])
+    assert eq(g, expected)
     assert g.chunks == ((8, 8), (5, 5))
 
 
@@ -185,6 +203,14 @@ def test_map_overlap():
     y = x.map_overlap(lambda x: x + len(x), depth=2)
     assert eq(y, np.arange(10) + 5 + 2 + 2)
 
+    x = np.arange(16).reshape((4, 4))
+    d = da.from_array(x, chunks=(2, 2))
+    exp1 = d.map_overlap(lambda x: x + x.size, depth=1).compute()
+    exp2 = d.map_overlap(lambda x: x + x.size, depth={0: 1, 1: 1},\
+                    boundary={0: 'reflect', 1: 'none'}).compute()
+    assert eq(exp1, x + 16)
+    assert eq(exp2, x + 12)
+
 
 def test_nearest_ghost():
     a = np.arange(144).reshape(12, 12).astype(float)
@@ -313,3 +339,14 @@ def test_bad_depth_raises():
     depth = {0: 4, 1: 2}
 
     pytest.raises(ValueError, ghost, darr, depth=depth, boundary=1)
+
+
+def test_none_boundaries():
+    x = da.from_array(np.arange(16).reshape(4, 4), chunks=(2, 2))
+    exp = boundaries(x, 2, {0: 'none', 1: 33})
+    res = np.array(
+          [[33, 33,  0,  1,  2,  3, 33, 33],
+           [33, 33,  4,  5,  6,  7, 33, 33],
+           [33, 33,  8,  9, 10, 11, 33, 33],
+           [33, 33, 12, 13, 14, 15, 33, 33]])
+    assert eq(exp, res)