Retrieve the neighbor list even if we can't retrieve the full object view