Repository: pantsbuild/pants

Issue:
Audit/defend-against use of @memoized_property with large instances
`@memoized_property` uses the instance of the class that it is being used with as a key into a dict on the class. This means that when `@memoized_property` is used with "large" instances (ie, those with expensive `__hash__`/`__eq__`), things can become accidentally quadratic.

We should audit for cases like this, and ideally find a way to guard against them.


Discussion:
Also, when used in a daemon context you have a memory leak since instances that have cached will have a ref cycle with the class. Pretty sure the right answer is to fix classmethods to cache on the class, instance methods on the instance, etc.
> Also, when used in a daemon context you have a memory leak since instances that have cached will have a ref cycle with the class. Pretty sure the right answer is to fix classmethods to cache on the class, instance methods on the instance, etc.

This is a great problem/solution combo -- are there any perf or correctness differences with using class/instance attrs to cache vs a dict? Or is that the point of python? Dicts and objects are the "same" in js obv but I'm not sure about how true that is for python.

> This means that when `@memoized_property` is used with "large" instances (ie, those with expensive `__hash__`/`__eq__`), things can become accidentally quadratic.

Could we cache the `__hash__`? It seems like that would address the specific issue fixed in #6551, since `datatype`s are immutable. I also don't understand what "accidentally quadratic" is specifically referring to here unless you're referring to the general phenomenon, which would make perfect sense.

I found [this still-open python bug from 2006](https://bugs.python.org/issue1462796) (linked from [this stackoverflow question which says caching the `__hash__` is cool and fun and everyone does it](https://stackoverflow.com/questions/3787405/python-is-there-any-reason-not-to-cache-an-objects-hash)) to discuss caching the `__hash__` for tuples, which it looks like could have made #6551 unnecessary, for example (although I think the separation of types introduced in that PR is useful regardless, and I also don't know if caching the `__hash__` of arbitrary tuples is obviously the right thing to do).
> Pretty sure the right answer is to fix classmethods to cache on the class, instance methods on the instance, etc.

Not sure if this is fud, but: part of the reason to extend `namedtuple` as we do was to get a proper/tested implementation of `__slots__` to reduce memory usage, etc. I think that the reason that `@memoized_property` is currently working on datatype is that it doesn't attempt to actually attach any additional data to the instance.
Gotcha. Maybe there is a more clever way to get around the rolling object id problem.
> I think that the reason that @memoized_property is currently working on datatype is that it doesn't attempt to actually attach any additional data to the instance

*whispers* What if we added a slot for the memoization??? Either in `datatype`, or `@memoized_property` (or if necessary a new method) could do some class magic (editing the class to have a slot for the property's field, using the property's slot to store the dict and/or using another slot for the memo dict)?

> the rolling object id problem

Is this just referring to the ref cycle issue you mentioned above, or a more general issue?
> Is this just referring to the ref cycle issue you mentioned above, or a more general issue?

https://github.com/pantsbuild/pants/blob/f6cf76d5687d2fcdc3c6cbaf60d99a2db63231a8/src/python/pants/util/memo.py#L34-L39

I've got a straightforward fix for this that cuts out the instances `__hash__` and `__eq__` completely but will not address the long standing strong retention problem.

Resolving patch:
diff --git a/src/python/pants/util/memo.py b/src/python/pants/util/memo.py
--- a/src/python/pants/util/memo.py
+++ b/src/python/pants/util/memo.py
@@ -24,6 +24,29 @@ def equal_args(*args, **kwargs):
   return key
 
 
+class InstanceKey(object):
+  """An equality wrapper for an arbitrary object instance.
+
+  This wrapper leverages `id` and `is` for fast `__hash__` and `__eq__` but both of these rely on
+  the object in question not being gc'd since both `id` and `is` rely on the instance address which
+  can be recycled; so we retain a strong reference to the instance to ensure no recycling can occur.
+  """
+
+  def __init__(self, instance):
+    self._instance = instance
+    self._hash = id(instance)
+
+  def __hash__(self):
+    return self._hash
+
+  def __eq__(self, other):
+    if self._instance is other:
+      return True
+    if isinstance(other, InstanceKey):
+      return self._instance is other._instance
+    return False
+
+
 def per_instance(*args, **kwargs):
   """A memoized key factory that works like `equal_args` except that the first parameter's identity
   is used when forming the key.
@@ -31,14 +54,7 @@ def per_instance(*args, **kwargs):
   This is a useful key factory when you want to enforce memoization happens per-instance for an
   instance method in a class hierarchy that defines a custom `__hash__`/`__eq__`.
   """
-  # For methods, the cache should be per-instance, so we take the id of the self/cls argument
-  # instead of relying on `==` since different instances may evaluate as `==`.  Additionally, we
-  # pair the id with the instance to ensure the instance is not GC'd since `id` allows for re-use
-  # of ids under GC.
-  instance = args[0]
-  unique_retained_instance = (id(instance), instance)
-
-  instance_and_rest = (unique_retained_instance,) + args[1:]
+  instance_and_rest = (InstanceKey(args[0]),) + args[1:]
   return equal_args(*instance_and_rest, **kwargs)
 
 


Test patch: