Avoid sys.path pollution when switching interpreters

Fixes https://github.com/pypa/virtualenv/issues/671