delete CA file

In both cases, let's only `unlink()` if we created the file previously.