Remove the build_root when the prefix changes.