Build relationships based on remote ids