harden the migrate scripts - character encoding, html stripping and near-complete coverage of data (excepting fields deliberately excluded)