Deal with possible dual-submission of CSR

Probably only a problem on CA nodes, but this commit makes it seem to
work in all the scenarios we're using it for.