Repository: dask/dask

Issue:
explicitly pass in names for delayed functions
tl;dr I want to pass the uniquely identifying name of delayed function instead of having it automatically generated via a hash of the function and arguments.

I'm using delayed to build workflows.  The result of a function that I'm delaying depends on some but not all of its arguments.  For example, a function which performs a parallel reduction might have a `num_workers` argument that doesn't affect the output.  Right now, delayed hashes it anyways and forces re-computation.  I want to avoid that.

The real use-case is a bit more complicated: I'm making some system calls from within the delayed functions that run simulations on supercomputers.  This calculations are very expensive, so I want a bit more control over which tasks are considered to have the same output.

The behavior I'm looking for [is here](https://github.com/maxhutch/nekpy/blob/names/nekpy/dask/tasks.py#L60), but it doesn't work: the names in the dask are still of the form `<function name>-<hash of something>`.  Tracing the code I get lost in DelayedLeaf, which seems to ignore its name argument.



Discussion:
You can provide the name yourself explicitly using the `dask_key_name` kwarg:

``` python
In [104]: @delayed(pure=True)
   .....: def foo(a, b, c=0):
   .....:     return a + b + c
   .....:

In [105]: foo
Out[105]: Delayed('fooeda225def65bca4b5be25f9f12f1bbaa')

In [106]: foo(1, 2, 3)
Out[106]: Delayed('foo-2b595b88120141ef301c7ccab85a386b')

In [107]: foo(1, 2, 3, dask_key_name='foo')
Out[107]: Delayed('foo')
```

Is that an acceptable solution here?

When does that kwarg need to be passed?  How does that differ from the intended behavior of the `name` argument to `delayed`?

The `name` kwarg sets the name in the underlying graph for the object being wrapped. The `dask_key_name` kwarg sets the key name for the result of a `__call__` on a delayed object. The following might clarify things:

``` python
In [112]: def foo(a, b, c=0):
   .....:     return a + b + c
   .....:

In [113]: dfoo = delayed(foo, name='foo', pure=True)

In [114]: dfoo
Out[114]: Delayed('foo')

In [115]: dfoo(1, 2, c=3)
Out[115]: Delayed('foo-7012413eafb786ebcdd430865623469c')

In [116]: dfoo(1, 2, c=3, dask_key_name='foo_called')
Out[116]: Delayed('foo_called')
```

Yeah, that works and makes sense.  I think its the currying that threw me off.  
Could `dask_key_name` get promoted to the docs?  This is really helpful functionality.

Good point. Adding.


Resolving patch:
diff --git a/dask/delayed.py b/dask/delayed.py
--- a/dask/delayed.py
+++ b/dask/delayed.py
@@ -123,7 +123,8 @@ def delayed(obj, name=None, pure=False):
     obj : object
         The function or object to wrap
     name : string or hashable, optional
-        The key to use in the underlying graph. Defaults to hashing content.
+        The key to use in the underlying graph for the wrapped object. Defaults
+        to hashing content.
     pure : bool, optional
         Indicates whether calling the resulting ``Delayed`` object is a pure
         operation. If True, arguments to the call are hashed to produce
@@ -176,14 +177,38 @@ def delayed(obj, name=None, pure=False):
     >>> out1.key == out2.key
     True
 
+    The key name of the result of calling a delayed object is determined by
+    hashing the arguments by default. To explicitly set the name, you can use
+    the ``dask_key_name`` keyword when calling the function:
+
+    >>> add(1, 2)    # doctest: +SKIP
+    Delayed('add-3dce7c56edd1ac2614add714086e950f')
+    >>> add(1, 2, dask_key_name='three')
+    Delayed('three')
+
+    Note that objects with the same key name are assumed to have the same
+    result. If you set the names explicitly you should make sure your key names
+    are different for different results.
+
+    >>> add(1, 2, dask_key_name='three')
+    >>> add(2, 1, dask_key_name='three')
+    >>> add(2, 2, dask_key_name='four')
+
     ``delayed`` can also be applied to objects to make operations on them lazy:
 
     >>> a = delayed([1, 2, 3])
-    >>> type(a) == Delayed
+    >>> isinstance(a, Delayed)
     True
     >>> a.compute()
     [1, 2, 3]
 
+    The key name of a delayed object is hashed by default. To explicitly set
+    the name, you can use the ``name`` keyword:
+
+    >>> a = delayed([1, 2, 3], name='mylist')
+    >>> a
+    Delayed('mylist')
+
     Delayed results act as a proxy to the underlying object. Many operators
     are supported:
 
@@ -209,6 +234,13 @@ def delayed(obj, name=None, pure=False):
 
     >>> a.count(2, pure=True).key == a.count(2, pure=True).key
     True
+
+    As with function calls, method calls also support the ``dask_key_name``
+    keyword:
+
+    >>> a.count(2, dask_key_name="count_2")
+    Delayed("count_2")
+
     """
     if isinstance(obj, Delayed):
         return obj
@@ -315,7 +347,11 @@ def __iter__(self):
 
     def __call__(self, *args, **kwargs):
         pure = kwargs.pop('pure', False)
-        return delayed(apply, pure=pure)(self, args, kwargs)
+        name = kwargs.pop('dask_key_name', None)
+        func = delayed(apply, pure=pure)
+        if name is not None:
+            return func(self, args, kwargs, dask_key_name=name)
+        return func(self, args, kwargs)
 
     def __bool__(self):
         raise TypeError("Truth of Delayed objects is not supported")


Test patch:
diff --git a/dask/tests/test_delayed.py b/dask/tests/test_delayed.py
--- a/dask/tests/test_delayed.py
+++ b/dask/tests/test_delayed.py
@@ -58,6 +58,8 @@ def test_methods():
     assert a.split(' ').compute() == ['a', 'b', 'c', 'd', 'e']
     assert a.upper().replace('B', 'A').split().count('A').compute() == 2
     assert a.split(' ', pure=True).key == a.split(' ', pure=True).key
+    o = a.split(' ', dask_key_name='test')
+    assert o.key == 'test'
 
 
 def test_attributes():