Retain/release logic has moved to be invalidation-based, rather than queue-based
Fix a race condition when cancelling lots of operations