Don't create grouping element for to-one relations.

Fixes 79953026, @afroisalreadyinu can you take a look?
If we are writing a JSON blob column, this might not have an `id` attribute, so this will KeyError.