Remove vagrant user outside Vagrant

Fixes #359