Remove gpg-agent plugin and use my own handling for it