Only reorder collection if it is grouped

Only need to use work around of GROUP BY + COUNT queries if we are actually using grouping.

This is a squash-merge of #2864