Add overlay files to remove on upgrade, partially fix #228