Ensure columns keep their position in sortInfo

This problem only affect external sorting via setting the sortInfo
ojbect.