Factor out some commonalities between ToOne and ToMany deref fields