Remove duplicate properties during store migration from 2.1 to 2.2

We should also add a dedicated test for this method.