Only delete cached Python outside the virtualenv