Clean up after package_signer monitor.