Only the GPG submodule needs p11-kit access for accessing keyservers