Fix git branch not going away when leaving a git dir