Do an actual swap in the collection on card move