Only restore working directory is existing PWD isn't HOME.