upgrade to new schema, and some necessary modifications