Remove duplicate properties during store migration from 2.1 to 2.2

This is addressed by #3502