PRIVKEY and PUBKEY are not server-specific data structures so they should not be free'd from here.