Fix old schema migration so that it is idempotent

Change-Id: I28fbed7108d19594084409a02ec9b8a4f8f3d404