Remove the ssh-agent plugin from the default plugin list since the gpg-agent will also work as such (in addition to GPG)