Repository: OfflineIMAP/offlineimap

Issue #573: CERTIFICATE_VERIFY_FAILED but openssl s_client works with same ca certs file
Created: 2018-08-29T14:55:50Z
General informations

system/distribution (with version): Linux user 4.17.0-1-amd64 #1 SMP Debian 4.17.6-2 (2018-07-15) x86_64 GNU/Linux
offlineimap version (offlineimap -V): offlineimap v7.2.1, imaplib2 v2.57 (system), Python v2.7.15, OpenSSL 1.1.1-pre9 (beta) 21 Aug 2018
Python version: v2.7.15
server name or domain: imap.gmail.com
CLI options: -dALL

Configuration file offlineimaprc
[general]
accounts = [username]
fsync = false
sockettimeout = 30

[Account [username]]
localrepository = [username]local
remoterepository = [username]remote
synclabels = yes
labelsheader = X-Keywords
presynchook = ~/.offlineimap/presync.sh
maxconnections=1

[Repository [username]local]
type = GmailMaildir
localfolders = ~/.mail
folderfilter = lambda folder: folder in ['INBOX','sent','archive','bin']
nametrans = lambda folder: {'archive': '[Gmail]/All Mail', 'sent': '[Gmail]/Sent Mail', 'bin': '[Gmail]/Bin',}.get(folder, folder)
maxage = 60

[Repository [username]remote]
type = Gmail
maxconnections=2
remoteuser = [username]@gmail.com
realdelete=no
folderfilter = lambda foldername: foldername in ['INBOX','[Gmail]/All Mail', '[Gmail]/Sent Mail',]
nametrans = lambda folder: { '[Gmail]/All Mail':'archive', '[Gmail]/Bin':'bin','[Gmail]/Sent Mail':'sent',}.get(folder, folder)
ssl = true
sslcacertfile = /etc/ssl/certs/ca-certificates.crt
remotepass = [pass]
maxage = 60

pythonfile (if any)
None

Logs, error
offlineimap -dALL

OfflineIMAP 7.2.1
  Licensed under the GNU GPL v2 or any later version (with an OpenSSL exception)
imaplib2 v2.57 (system), Python v2.7.15, OpenSSL 1.1.1-pre9 (beta) 21 Aug 2018
Now debugging for imap: IMAP protocol debugging
Now debugging for maildir: Maildir repository debugging
Now debugging for thread: Threading debugging
Now debugging for : Other offlineimap related sync messages
Account sync [username]:
 [thread]: Register new thread 'Account sync [username]' (account '[username]')
 [imap]: Using authentication mechanisms ['GSSAPI', 'XOAUTH2', 'CRAM-MD5', 'PLAIN', 'LOGIN']
 [maildir]: MaildirRepository initialized, sep is '.'
 *** Processing account [username]
 Calling hook: ~/.offlineimap/presync.sh
 Hook stdout: 
Cleaning up emacs temp and autosave files in drafts...

Hook stderr:

 Hook return code: 0
 Establishing connection to imap.gmail.com:993 ([username]remote)
 [imap]: [username]remote: level 'tls_compat', version 'None'
 ERROR: Unknown SSL protocol connecting to host 'imap.gmail.com' for repository '[username]remote'. OpenSSL responded:
[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:726)
 ['  File "/usr/share/offlineimap/offlineimap/accounts.py", line 283, in syncrunner\n    self.__sync()\n', '  File "/usr/share/offlineimap/offlineimap/accounts.py", line 359, in __sync\n    remoterepos.getfolders()\n', '  File "/usr/share/offlineimap/offlineimap/repository/IMAP.py", line 452, in getfolders\n    imapobj = self.imapserver.acquireconnection()\n', '  File "/usr/share/offlineimap/offlineimap/imapserver.py", line 633, in acquireconnection\n    exc_info()[2])\n', '  File "/usr/share/offlineimap/offlineimap/imapserver.py", line 547, in acquireconnection\n    af=self.af,\n', '  File "/usr/share/offlineimap/offlineimap/imaplibutil.py", line 194, in __init__\n    super(WrappedIMAP4_SSL, self).__init__(*args, **kwargs)\n', '  File "/usr/lib/python2.7/dist-packages/imaplib2.py", line 2182, in __init__\n    IMAP4.__init__(self, host, port, debug, debug_file, identifier, timeout, debug_buf_lvl)\n', '  File "/usr/lib/python2.7/dist-packages/imaplib2.py", line 360, in __init__\n    self.open(host, port)\n', '  File "/usr/share/offlineimap/offlineimap/imaplibutil.py", line 202, in open\n    super(WrappedIMAP4_SSL, self).open(host, port)\n', '  File "/usr/lib/python2.7/dist-packages/imaplib2.py", line 2195, in open\n    self.ssl_wrap_socket()\n', '  File "/usr/lib/python2.7/dist-packages/imaplib2.py", line 547, in ssl_wrap_socket\n    self.sock = ssl.wrap_socket(self.sock, self.keyfile, self.certfile, ca_certs=self.ca_certs, cert_reqs=cert_reqs, ssl_version=ssl_version)\n', '  File "/usr/lib/python2.7/ssl.py", line 949, in wrap_socket\n    ciphers=ciphers)\n', '  File "/usr/lib/python2.7/ssl.py", line 617, in __init__\n    self.do_handshake()\n', '  File "/usr/lib/python2.7/ssl.py", line 846, in do_handshake\n    self._sslobj.do_handshake()\n']
 *** Finished account '[username]' in 0:00
[thread]: Unregister thread 'Account sync [username]'
ERROR: Exceptions occurred during the run!
ERROR: Unknown SSL protocol connecting to host 'imap.gmail.com' for repository '[username]remote'. OpenSSL responded:
[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:726)

Traceback:
  File "/usr/share/offlineimap/offlineimap/accounts.py", line 283, in syncrunner
    self.__sync()
  File "/usr/share/offlineimap/offlineimap/accounts.py", line 359, in __sync
    remoterepos.getfolders()
  File "/usr/share/offlineimap/offlineimap/repository/IMAP.py", line 452, in getfolders
    imapobj = self.imapserver.acquireconnection()
  File "/usr/share/offlineimap/offlineimap/imapserver.py", line 633, in acquireconnection
    exc_info()[2])
  File "/usr/share/offlineimap/offlineimap/imapserver.py", line 547, in acquireconnection
    af=self.af,
  File "/usr/share/offlineimap/offlineimap/imaplibutil.py", line 194, in __init__
    super(WrappedIMAP4_SSL, self).__init__(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/imaplib2.py", line 2182, in __init__
    IMAP4.__init__(self, host, port, debug, debug_file, identifier, timeout, debug_buf_lvl)
  File "/usr/lib/python2.7/dist-packages/imaplib2.py", line 360, in __init__
    self.open(host, port)
  File "/usr/share/offlineimap/offlineimap/imaplibutil.py", line 202, in open
    super(WrappedIMAP4_SSL, self).open(host, port)
  File "/usr/lib/python2.7/dist-packages/imaplib2.py", line 2195, in open
    self.ssl_wrap_socket()
  File "/usr/lib/python2.7/dist-packages/imaplib2.py", line 547, in ssl_wrap_socket
    self.sock = ssl.wrap_socket(self.sock, self.keyfile, self.certfile, ca_certs=self.ca_certs, cert_reqs=cert_reqs, ssl_version=ssl_version)
  File "/usr/lib/python2.7/ssl.py", line 949, in wrap_socket
    ciphers=ciphers)
  File "/usr/lib/python2.7/ssl.py", line 617, in __init__
    self.do_handshake()
  File "/usr/lib/python2.7/ssl.py", line 846, in do_handshake
    self._sslobj.do_handshake()

OpennSSL connection:
SSL_CERT_DIR="" openssl s_client -connect imap.gmail.com:993 -showcerts -CAfile /etc/ssl/certs/ca-certificates.crt

CONNECTED(00000005)
depth=2 OU = GlobalSign Root CA - R2, O = GlobalSign, CN = GlobalSign
verify return:1
depth=1 C = US, O = Google Trust Services, CN = Google Internet Authority G3
verify return:1
depth=0 C = US, ST = California, L = Mountain View, O = Google LLC, CN = imap.gmail.com
verify return:1
---
Certificate chain
 0 s:C = US, ST = California, L = Mountain View, O = Google LLC, CN = imap.gmail.com
   i:C = US, O = Google Trust Services, CN = Google Internet Authority G3
-----BEGIN CERTIFICATE-----
MIIEgjCCA2qgAwIBAgIILq49eT5KrPEwDQYJKoZIhvcNAQELBQAwVDELMAkGA1UE
BhMCVVMxHjAcBgNVBAoTFUdvb2dsZSBUcnVzdCBTZXJ2aWNlczElMCMGA1UEAxMc
R29vZ2xlIEludGVybmV0IEF1dGhvcml0eSBHMzAeFw0xODA4MTQwNzQ4MTdaFw0x
ODEwMjMwNzM4MDBaMGgxCzAJBgNVBAYTAlVTMRMwEQYDVQQIDApDYWxpZm9ybmlh
MRYwFAYDVQQHDA1Nb3VudGFpbiBWaWV3MRMwEQYDVQQKDApHb29nbGUgTExDMRcw
FQYDVQQDDA5pbWFwLmdtYWlsLmNvbTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCC
AQoCggEBAMWF7EuRPI8Zi3emP7fYCRQ/9k5nzNbJ8G62vub5EAjJi0gF9qX5Oe9O
rjBJmgrWo7iXD2Y/iVBqqw057B2C+oEURkCY200WTOwLZZUlxTXB2BFXnbFMwdof
56C1g7dwZswYzowBygitDmUIOR0LdY+3P3b36fZVFI0npHccCjwzKgGK8905Q2il
Ht+6aFZea7wSXshEE5aPe+nZYbsKYwW3kna/Q2cZ+PWJAjgovsQp6VlseIz7Huay
VEztuggKQEM+jffTEr3hpt32p+2awyHvw8p9rVXXEwB6nWvB3VEa8IV4WHZ0JPKj
Gia/fbIHMGUs8BV2dUUh0aEHHAhEVLECAwEAAaOCAUIwggE+MBMGA1UdJQQMMAoG
CCsGAQUFBwMBMBkGA1UdEQQSMBCCDmltYXAuZ21haWwuY29tMGgGCCsGAQUFBwEB
BFwwWjAtBggrBgEFBQcwAoYhaHR0cDovL3BraS5nb29nL2dzcjIvR1RTR0lBRzMu
Y3J0MCkGCCsGAQUFBzABhh1odHRwOi8vb2NzcC5wa2kuZ29vZy9HVFNHSUFHMzAd
BgNVHQ4EFgQU+LBcUvLD/WgEVXkoDFbW+dp4VYQwDAYDVR0TAQH/BAIwADAfBgNV
HSMEGDAWgBR3wrhQmmd2drEtwobQg6B+pn66SzAhBgNVHSAEGjAYMAwGCisGAQQB
1nkCBQMwCAYGZ4EMAQICMDEGA1UdHwQqMCgwJqAkoCKGIGh0dHA6Ly9jcmwucGtp
Lmdvb2cvR1RTR0lBRzMuY3JsMA0GCSqGSIb3DQEBCwUAA4IBAQCUgE+G0HYFNtFF
HA+7lJYgI775FSCusfP9LTABTLaNr+761vWTW/rW8KgviDvkpW4095SKRhCwj21T
xt67/bdcrP0Y8q7E3Tjlg5leB0gZbQzOlucBdlVxrACvdj8XSvvqoKN6f6DHXYG3
suoMh9CISRB0fhJ4R1u3wRRI1qTt7guuAtngTenPM2b1xPXOhDA2P++4ZAoAZJEK
KLe9EwG+lAniut1qrNc0sH8N5HRAoP3uvEKqovTMXBOQgZ1/n8UkPKvGswCnk6v0
7SuTO6VwyQezjubg053iJSevLr7MMVNcYf1pbwLLegbhHYQieI81VrO+0bf5kPIJ
9ojXL67W
-----END CERTIFICATE-----
 1 s:C = US, O = Google Trust Services, CN = Google Internet Authority G3
   i:OU = GlobalSign Root CA - R2, O = GlobalSign, CN = GlobalSign
-----BEGIN CERTIFICATE-----
MIIEXDCCA0SgAwIBAgINAeOpMBz8cgY4P5pTHTANBgkqhkiG9w0BAQsFADBMMSAw
HgYDVQQLExdHbG9iYWxTaWduIFJvb3QgQ0EgLSBSMjETMBEGA1UEChMKR2xvYmFs
U2lnbjETMBEGA1UEAxMKR2xvYmFsU2lnbjAeFw0xNzA2MTUwMDAwNDJaFw0yMTEy
MTUwMDAwNDJaMFQxCzAJBgNVBAYTAlVTMR4wHAYDVQQKExVHb29nbGUgVHJ1c3Qg
U2VydmljZXMxJTAjBgNVBAMTHEdvb2dsZSBJbnRlcm5ldCBBdXRob3JpdHkgRzMw
ggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDKUkvqHv/OJGuo2nIYaNVW
XQ5IWi01CXZaz6TIHLGp/lOJ+600/4hbn7vn6AAB3DVzdQOts7G5pH0rJnnOFUAK
71G4nzKMfHCGUksW/mona+Y2emJQ2N+aicwJKetPKRSIgAuPOB6Aahh8Hb2XO3h9
RUk2T0HNouB2VzxoMXlkyW7XUR5mw6JkLHnA52XDVoRTWkNty5oCINLvGmnRsJ1z
ouAqYGVQMc/7sy+/EYhALrVJEA8KbtyX+r8snwU5C1hUrwaW6MWOARa8qBpNQcWT
kaIeoYvy/sGIJEmjR0vFEwHdp1cSaWIr6/4g72n7OqXwfinu7ZYW97EfoOSQJeAz
AgMBAAGjggEzMIIBLzAOBgNVHQ8BAf8EBAMCAYYwHQYDVR0lBBYwFAYIKwYBBQUH
AwEGCCsGAQUFBwMCMBIGA1UdEwEB/wQIMAYBAf8CAQAwHQYDVR0OBBYEFHfCuFCa
Z3Z2sS3ChtCDoH6mfrpLMB8GA1UdIwQYMBaAFJviB1dnHB7AagbeWbSaLd/cGYYu
MDUGCCsGAQUFBwEBBCkwJzAlBggrBgEFBQcwAYYZaHR0cDovL29jc3AucGtpLmdv
b2cvZ3NyMjAyBgNVHR8EKzApMCegJaAjhiFodHRwOi8vY3JsLnBraS5nb29nL2dz
cjIvZ3NyMi5jcmwwPwYDVR0gBDgwNjA0BgZngQwBAgIwKjAoBggrBgEFBQcCARYc
aHR0cHM6Ly9wa2kuZ29vZy9yZXBvc2l0b3J5LzANBgkqhkiG9w0BAQsFAAOCAQEA
HLeJluRT7bvs26gyAZ8so81trUISd7O45skDUmAge1cnxhG1P2cNmSxbWsoiCt2e
ux9LSD+PAj2LIYRFHW31/6xoic1k4tbWXkDCjir37xTTNqRAMPUyFRWSdvt+nlPq
wnb8Oa2I/maSJukcxDjNSfpDh/Bd1lZNgdd/8cLdsE3+wypufJ9uXO1iQpnh9zbu
FIwsIONGl1p3A8CgxkqI/UAih3JaGOqcpcdaCIzkBaR9uYQ1X4k2Vg5APRLouzVy
7a8IVk6wuy6pm+T7HT4LY8ibS5FEZlfAFLSW8NwsVz9SBK2Vqn1N0PIMn5xA6NZV
c7o835DLAFshEWfC7TIe3g==
-----END CERTIFICATE-----
---
Server certificate
subject=C = US, ST = California, L = Mountain View, O = Google LLC, CN = imap.gmail.com

issuer=C = US, O = Google Trust Services, CN = Google Internet Authority G3

---
No client certificate CA names sent
Peer signing digest: SHA256
Peer signature type: RSA-PSS
Server Temp Key: X25519, 253 bits
---
SSL handshake has read 2954 bytes and written 391 bytes
Verification: OK
---
New, TLSv1.2, Cipher is ECDHE-RSA-CHACHA20-POLY1305
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : ECDHE-RSA-CHACHA20-POLY1305
    Session-ID: FA4360FD71496FB6C5CA2BFABB56046EE7CE96EB974ABD39864CCC7BD0CEEE30
    Session-ID-ctx: 
    Master-Key: D9F328F69BA96E84135CFFC209BC2991544D5C21B5732E851E8798A52850DCB7464839945E30E896F2B3CFCDB66128EC
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    TLS session ticket lifetime hint: 100800 (seconds)
    TLS session ticket:
    0000 - 00 70 2f 91 25 bb 23 fa-bd 77 cd f9 13 d5 63 b4   .p/.%.#..w....c.
    0010 - 53 cb 0b d4 a2 2b a8 cf-9a b6 59 17 21 64 18 da   S....+....Y.!d..
    0020 - 0f 5b 53 59 80 05 e9 b5-7e 1b da df 94 bf 0c 29   .[SY....~......)
    0030 - 07 29 b4 86 07 4e 08 07-5f d3 8e a4 3a a4 79 ee   .)...N.._...:.y.
    0040 - 83 cf 06 d7 c9 ad 02 b5-b8 a7 fc 32 f1 cf e1 92   ...........2....
    0050 - e4 d1 fa f5 47 a6 65 ae-f1 82 3b 0f ef 32 4a 58   ....G.e...;..2JX
    0060 - 65 11 fa 5e 84 98 7e a0-62 40 20 c4 7f de bf 85   e..^..~.b@ .....
    0070 - 7f 38 48 54 7b 64 b5 b9-d1 b4 f2 b2 c6 f8 52 90   .8HT{d........R.
    0080 - f1 60 20 22 61 39 fd ab-54 ec 6b 15 eb 47 70 09   .` "a9..T.k..Gp.
    0090 - 8b 58 6e 68 0c 99 27 c6-36 ad 0e 79 2c 45 92 80   .Xnh..'.6..y,E..
    00a0 - 17 b9 32 a6 2d 27 0d df-a2 a2 7b 8d 80 4d c1 5e   ..2.-'....{..M.^
    00b0 - 53 90 25 cb f2 65 d3 d7-56 43 c8 de fa 5b a4 80   S.%..e..VC...[..
    00c0 - 28 da 9c 66 71 f0 dd 53-72 30 c2 22 17 03 a3 24   (..fq..Sr0."...$
    00d0 - ef 65 10 db bc 20 c2 ed-d5 4b                     .e... ...K

    Start Time: 1535552960
    Timeout   : 7200 (sec)
    Verify return code: 0 (ok)
    Extended master secret: yes
---
* OK Gimap ready for requests from 158.223.28.49 q23-v6mb381565940wmc

Pre-sync hook (presync.sh; not relevant)
#!/bin/bash
## Things to do before a sync by offline imap

shopt -s extglob

# Delete emacs temp and autosaves from the Drafts folder:
echo
echo "Cleaning up emacs temp and autosave files in drafts..."
for file in /Users/isaac/.mail/drafts/cur/*~;  do rm -vf "$file" ;done
for file in /Users/isaac/.mail/drafts/cur/\#*; do rm -vf "$file" ;done

Steps to reproduce the error
With this config, run offlineimap -dALL
Note that SSL_CERT_DIR="" openssl s_client -connect imap.gmail.com:993 -showcerts -CAfile /etc/ssl/certs/ca-certificates.crt successfully verifies the certificate with the same ca-certificates.crt as defined in .offlineimaprc.

Duplicate issue #591: IMAP4 protocol error: socket error: <type 'exceptions.IOError'>
Created: 2019-01-06T14:06:33Z
General Informations

system/distribution (with version):
$ uname -a

Linux localhost 4.19.13-1-lts #1 SMP Sun Dec 30 07:38:47 CET 2018 x86_64 GNU/Linux

$ lsb_release -a

LSB Version:    1.4
Distributor ID: ArchLabs
Description:    ArchLabs Linux
Release:        2018.12.17
Codename:       ArchLabs


offlineimap version (offlineimap -V):
$ offlineimap -V
offlineimap v7.2.2, imaplib2 v2.57 (bundled), Python v2.7.15, OpenSSL 1.1.1a  20 Nov 2018


Python version:
$ python --version
Python 3.7.2
┌ ~
└> $ python2 --version
Python 2.7.15


server name or domain:
CLI options:

Configuration file offlineimaprc
[general]
accounts = accountname
pythonfile = ~/.config/mutt/credentials/imappwd.py

[Account accountname]
localrepository = accountname-local
remoterepository = accountname-remote
#autorefresh = 10
maxage = 720
ignorelabels = \Inbox, \Starred, \Sent, \Draft, \Spam, \Trash, \Important

[Repository accountname-remote]
auth_mechanisms = LOGIN
type = Gmail
remoteuser = admin
remotepasseval = mailpasswd("accountname")
remoteport = 993
starttls = no
ssl = yes
folderfilter = lambda foldername: foldername not in ['[Gmail]/All Mail']
sslcacertfile = /etc/ssl/certs/ca-certificates.crt

oauth2_client_id = ID
oauth2_client_secret = SECRET
oauth2_access_token = TOKEN
oauth2_request_url = https://accounts.google.com/o/oauth2/token
oauth2_refresh_token = SECRET

remotepassfile = ~/accountname

transporttunnel = openssl s_client -host myimap -port 993 -quiet

cert_fingerprint=f3043dd689a2e7dddfbef82703a6c65ea9b634c1

[Repository accountname-local]
type = GmailMaildir
localfolders = ~/.mail/accountname

pythonfile (if any)
Logs, error
Establishing connection to tunnel:openssl s_client -host myimap -port 993
140692074840576:error:2008F002:BIO routines:BIO_lookup_ex:system lib:crypto
connect:errno=22
ERROR: While attempting to sync account 'accountname'
IMAP4 protocol error: socket error: <type 'exceptions.IOError'> - Hang up

Steps to reproduce the error
$ offlineimap -a accountname

OfflineIMAP 7.2.2
Licensed under the GNU GPL v2 or any later version (with an OpenSSL excep
imaplib2 v2.57 (bundled), Python v2.7.15, OpenSSL 1.1.1a  20 Nov 2018
Account sync accountname:
*** Processing account accountname
Establishing connection to tunnel:openssl s_client -host myimap -port 993
140692074840576:error:2008F002:BIO routines:BIO_lookup_ex:system lib:crypto
connect:errno=22
ERROR: While attempting to sync account 'accountname'
IMAP4 protocol error: socket error: <type 'exceptions.IOError'> - Hang up
*** Finished account 'accountname' in 0:10
ERROR: Exceptions occurred during the run!
ERROR: While attempting to sync account 'accountname'
IMAP4 protocol error: socket error: <type 'exceptions.IOError'> - Hang up

Traceback:
File "/usr/lib/python2.7/site-packages/offlineimap/accounts.py", line 283
self.__sync()
File "/usr/lib/python2.7/site-packages/offlineimap/accounts.py", line 359
remoterepos.getfolders()
File "/usr/lib/python2.7/site-packages/offlineimap/repository/IMAP.py", l
imapobj = self.imapserver.acquireconnection()
File "/usr/lib/python2.7/site-packages/offlineimap/imapserver.py", line 5
use_socket=self.proxied_socket,
File "/usr/lib/python2.7/site-packages/offlineimap/imaplibutil.py", line
IMAP4.__init__(self, tunnelcmd, **kwargs)
File "/usr/lib/python2.7/site-packages/offlineimap/bundled_imaplib2.py",
self.welcome = self._request_push(name='welcome', tag='continuation').g
File "/usr/lib/python2.7/site-packages/offlineimap/bundled_imaplib2.py",
raise typ(exc_fmt % str(val))

Closing comment (2019-01-17T11:12:32Z): Seems related to #573