Only remove the temporary directory if it has a non-empty name.