Fix work dir removal.

Removing work dir needs getting out of it first.