Cleanup redundant directory after creating tarball