Don't create grouping element for to-one relations.

Fixes 79953026, @afroisalreadyinu can you take a look?