reactiveui/ReactiveUI — pull request #394 by npnelson (2013-11-04 14:08:46+00:00)
Title: Release function could block in ObservableAsyncMRUCache 

Fix to ObservableAsyncMRUCache where a call to a release function could block if the key in the cache was evicted while still in flight.

This change fixes the blocking issue, but takes us down what could be a deep rabbit hole that I'm not sure how to fix.

Imagine this (contrived) scenario:

ObservableAsyncMRUCache with max size of 1 and a release function defined.

1) Call AsyncGet key=1 and imagine this function will take 1 second to return.
2) Call AsyncGet key=2 - it evicts the inflight request for key = 1. This is where the previous code would block, because it called the release function for key=1 before key = 1 was back.
3) Call AsyncGet key = 1 again (after key=2 comes back but before your second is up from step 1) and this time have it come back quickly. The cache is not aware of the previous inflight request for key =1 because it got evicted at step 2.
4) Now key=1 is sitting in the cache with a result, but once your second is up from step 1, the release function will be called, even though key=1 is in the cache.

I couldn't come up with a clean test to demonstrate this (plus I didn't want to check in a failing test without a fix). I also don't grok the inner workings of the cache enough to prevent it from evicting an inflight request (even if that means letting the cache grow beyond maxsize). If you could prevent the cache from evicting an inflight request, that might do it, but it seems that you would have to add a timeout parameter somewhere and if the function doesn't come back before the timeout expires, evict the item from the cache and throw a timeout exception to subscribers.