Speed up regrouping (column swap)

Signed-off-by: Jeroen Meijer <jjgmeijer@gmail.com>