To avoid the "Duplicate id in list" error and fix the tests.
Moving the removal of duplicates before the grouping, otherwise we go back to having the chunking error (because a Set doesn't respect the order).