Memoize association load state