[stable6] Cleanup handling of temporary files

Backport for #11709