Make sure async relationships which are already loaded get linked correctly