compose tree: Cleanup workdir always

Particularly if we hit the cached case, but we might as well always do
it.