Make sure temp FPR file gets deleted and avoid leaking file descriptors