More double cache fixes

This PR fixes 2 remaining issues in our double cache implementation:
1) Average calculation with heavy seeking/cache swapping doesn't work properly when the "other" cache is already (almost) full;
2) We would wait for data although the position was already available in our other cache.