Repository: dask/dask

Issue:
Inheritance of Purity
(As discussed here: http://stackoverflow.com/questions/42773134/inheritance-of-purity/42773643)

I have a question about inheritance of function purity. For example, consider this case:

```python
In [1]: from dask import delayed

In [2]: myArr = delayed(np.ones, pure=True)((10,10))

In [3]: myArr
Out[3]: Delayed('ones-850ff41a84b309775d01d5f3e5c4d1c4')

In [4]: myArr = delayed(np.ones, pure=True)((10,10))

In [5]: myArr
Out[5]: Delayed('ones-850ff41a84b309775d01d5f3e5c4d1c4')

In [6]: myArr.shape
Out[6]: Delayed('getattr-de14f312-2605-4fc4-a419-c376520f73b4')

In [7]: myArr.shape
Out[7]: Delayed('getattr-9e547a62-0bc6-43e5-8d3c-cac532b73511')

In [8]: delayed(getattr, pure=True)(myArr, 'shape')
Out[8]: Delayed('getattr-5224be928bad33c9778d4f96d610cc37')

In [9]: delayed(getattr, pure=True)(myArr, 'shape')
Out[9]: Delayed('getattr-5224be928bad33c9778d4f96d610cc37')

```

I would expect line [6] and [7] to yield the same key, since they are accessing an attribute from a delayed instance declared pure. However, it is not the case, and I must explicitly declare the `getattr` method to be pure for this to work, as seen in lines [8] and [9].

As discussed in the stackoverflow link, this seems to be purely an issue of `dask` semantics. I would be inclined to voting for purity to be inherited. But I am a new user, so I would appreciate feedback/comments on this. Thanks!

PS: The eventual use case is for dask distributed. It seems that the way purity is treated in this case may possibly be different than the case listed here.

Discussion:
mrocklin: +1 from me

Patch:
diff --git a/dask/delayed.py b/dask/delayed.py
index d54d9b941..a9cbf6360 100644
--- a/dask/delayed.py
+++ b/dask/delayed.py
@@ -240,9 +240,29 @@ def delayed(obj, name=None, pure=False, nout=None, traverse=True):
     >>> res.compute()  # doctest: +SKIP
     AttributeError("'list' object has no attribute 'not_a_real_method'")
 
-    Methods are assumed to be impure by default, meaning that subsequent calls
-    may return different results. To assume purity, set `pure=True`. This
-    allows sharing of any intermediate values.
+    "Magic" methods (e.g. operators and attribute access) are assumed to be
+    pure, meaning that subsequent calls must return the same results.  This is
+    not overrideable. To invoke an impure attribute or operator, you'd need to
+    use it in a delayed function with ``pure=False``.
+
+    >>> class Incrementer(object):
+    ...     def __init__(self):
+    ...         self._n = 0
+    ...     @property
+    ...     def n(self):
+    ...         self._n += 1
+    ...         return self._n
+    ...
+    >>> x = delayed(Incrementer())
+    >>> x.n.key == x.n.key
+    True
+    >>> get_n = delayed(lambda x: x.n, pure=False)
+    >>> get_n(x).key == get_n(x).key
+    False
+
+    In contrast, methods are assumed to be impure by default, meaning that
+    subsequent calls may return different results. To assume purity, set
+    `pure=True`. This allows sharing of any intermediate values.
 
     >>> a.count(2, pure=True).key == a.count(2, pure=True).key
     True
@@ -252,7 +272,6 @@ def delayed(obj, name=None, pure=False, nout=None, traverse=True):
 
     >>> a.count(2, dask_key_name="count_2")
     Delayed("count_2")
-
     """
     if isinstance(obj, Delayed):
         return obj
@@ -346,7 +365,7 @@ class Delayed(base.Base):
     def __getattr__(self, attr):
         if attr.startswith('_'):
             raise AttributeError("Attribute {0} not found".format(attr))
-        return DelayedAttr(self, attr, 'getattr-%s' % tokenize(self, attr))
+        return DelayedAttr(self, attr)
 
     def __setattr__(self, attr, val):
         if attr in self.__slots__:
@@ -436,10 +455,10 @@ class DelayedLeaf(Delayed):
 class DelayedAttr(Delayed):
     __slots__ = ('_obj', '_attr', '_key')
 
-    def __init__(self, obj, attr, key):
+    def __init__(self, obj, attr):
         self._obj = obj
         self._attr = attr
-        self._key = key
+        self._key = 'getattr-%s' % tokenize(obj, attr, pure=True)
 
     @property
     def dask(self):


Test patch:
diff --git a/dask/tests/test_delayed.py b/dask/tests/test_delayed.py
index 11f00a428..bfea68244 100644
--- a/dask/tests/test_delayed.py
+++ b/dask/tests/test_delayed.py
@@ -69,6 +69,7 @@ def test_methods():
 
 def test_attributes():
     a = delayed(2 + 1j)
+    assert a.real._key == a.real._key
     assert a.real.compute() == 2
     assert a.imag.compute() == 1
     assert (a.real + a.imag).compute() == 3