Don't lazy load the virtualenvwrapper

Fix robbyrussell#2355.