Gather terms for each column in Sets to avoid adding them twice