Use 'by' instead of array_agg(distinct) for factors

It turns out that the latter uses lots of memory and is slow.