Cleanup build dir after successful switch

Fixes #1006