Repository: ansible/ansible

Issue:
StartTLS is not offered mail

##### SUMMARY
Notification module Mail not sending mails.
with 2.5.5 was ok.

##### ISSUE TYPE
 - Bug Report

##### COMPONENT NAME
!component =lib/ansible/modules/notification/mail.py

##### ANSIBLE VERSION
```
ansible 2.6.0
  config file = /etc/ansible/ansible.cfg
  configured module search path = [u'/root/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python2.7/site-packages/ansible
  executable location = ./ansible
  python version = 2.7.5 (default, Apr 11 2018, 07:36:10) [GCC 4.8.5 20150623 (Red Hat 4.8.5-28)]
```

##### CONFIGURATION

##### OS / ENVIRONMENT
centos-release-7-5.1804.el7.centos.2.x86_64


##### STEPS TO REPRODUCE
<!--- For bugs, show exactly how to reproduce the problem, using a minimal test-case.
For new features, show how the feature would be used. -->

<!--- Paste example playbooks or commands between quotes below -->
```yaml
- hosts: localhost
  tasks:
    - name: Sending an e-mail using StartTLS to the remote machine
      mail:
        host: mail.example.lv
        port: 587
        from: user@example.lv
        to: admin <admin@example.lv>
        subject: Ansible-report
        body: System {{ ansible_hostname }} has been successfully provisioned.
        secure: starttls
        username: user@example.lv
        password: example
```

<!--- You can also paste gist.github.com links for larger files -->

##### EXPECTED RESULTS
Mail sent successfully

##### ACTUAL RESULTS
with option starttls result: StartTLS is not offered on server.
`````
The full traceback is:
  File "/tmp/ansible_2ndNol/ansible_module_mail.py", line 252, in main
    code, smtpmessage = smtp.connect(host, port=port)
  File "/usr/lib64/python2.7/smtplib.py", line 315, in connect
    self.sock = self._get_socket(host, port, self.timeout)
  File "/usr/lib64/python2.7/smtplib.py", line 795, in _get_socket
    new_socket = ssl.wrap_socket(new_socket, self.keyfile, self.certfile)
  File "/usr/lib64/python2.7/ssl.py", line 936, in wrap_socket
    ciphers=ciphers)
  File "/usr/lib64/python2.7/ssl.py", line 611, in __init__
    self.do_handshake()
  File "/usr/lib64/python2.7/ssl.py", line 833, in do_handshake
    self._sslobj.do_handshake()
`````
with option try: No Suitable authentication method was found on server.
````` 
The full traceback is:
  File "/tmp/ansible_C0vbqy/ansible_module_mail.py", line 286, in main
    smtp.login(username, password)
  File "/usr/lib64/python2.7/smtplib.py", line 617, in login
    raise SMTPException("No suitable authentication method found.")
`````



Discussion:
Files identified in the description:
None

If these files are inaccurate, please update the `component name` section of the description or use the `!component` bot command.

[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: components_banner --->
Files identified in the description:
* [lib/ansible/modules/notification/mail.py](https://github.com/ansible/ansible/blob/devel/lib/ansible/modules/notification/mail.py)

If these files are inaccurate, please update the `component name` section of the description or use the `!component` bot command.

[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: components_banner --->
cc @dagwieers
[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: notify --->
The change that is likely to have caused this regression is this: #37098
As you can see one of our concerns was that we have no integration tests.
Feel free to review that PR and propose a fix.

cc @bcoca 
@agrisvv In your first traceback I don't see an exception being raised (the code was hoping for `ssl.SSLError`). The fix could be as simple as catching that specific exception and continuing to the starttls attempt.

needs_info
As @dagwieers mentions, we would need the actual exception. I've tested this with my own TLS enabled mail server w/o issues and I cannot reproduce, so we need more information.

needs_info
cometed out try and except in py module and error at last line:

`````
The full traceback is:
Traceback (most recent call last):
  File "/tmp/ansible_X_P_wl/ansible_module_mail.py", line 367, in <module>
    main()
  File "/tmp/ansible_X_P_wl/ansible_module_mail.py", line 252, in main
    code, smtpmessage = smtp.connect(host, port=port)
  File "/usr/lib64/python2.7/smtplib.py", line 315, in connect
    self.sock = self._get_socket(host, port, self.timeout)
  File "/usr/lib64/python2.7/smtplib.py", line 795, in _get_socket
    new_socket = ssl.wrap_socket(new_socket, self.keyfile, self.certfile)
  File "/usr/lib64/python2.7/ssl.py", line 936, in wrap_socket
    ciphers=ciphers)
  File "/usr/lib64/python2.7/ssl.py", line 611, in __init__
    self.do_handshake()
  File "/usr/lib64/python2.7/ssl.py", line 833, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLError: [SSL: UNKNOWN_PROTOCOL] unknown protocol (_ssl.c:579)
`````

same with option try or starttls. server are MS exchange, also  gmail -- same result.

are you getting a 'failed' message or a direct traceback from module? 
modified /usr/lib/python2.7/site-packages/ansible/modules/notification/mail.py
and runed playboook with -vvvv get failed=1 
with this script i receive email from same server where ansible.
``````py
import smtplib

server = smtplib.SMTP('mail.example.lv', 587)
server.starttls()
server.login("smtp@example.lv", "example")

msg = "YOUR MESSAGE!"
server.sendmail("smtp@example.lv", "example@example.lv", msg)
server.quit()
``````
@agrisvv You're not doing a separate `smtp.connect()` and that may be an improvement over the existing code. Just for testing, can you modify your simple script to perform it similarly to our code using `smtp.connect()` ? Let's see if we can make a simple reproducer first.
so here are modified scripts what worked with my server.

what i think:
1. part one end  with ssl.SSLError.
2. On port 587 work only TLS. Port 465 are for SSL
3. this function has_extn('STARTTLS') to me always return false on gmail also.. maybe could get by return code.
`````py

import os
import smtplib
import ssl
import traceback
from email import encoders
from email.utils import parseaddr, formataddr
from email.mime.base import MIMEBase
from email.mime.multipart import MIMEMultipart
from email.mime.text import MIMEText
from email.header import Header


def main():

    username = "smtp@domain.com"
    password = "xxxxxxxxxxx"
    host = "mail.domain.com"
    port = "587"
    sender = "smtp@domain.com"
    recipients = gggg@domain.com"
    subject = "xxxxx"
    body = "sssss"
    charset = "UTF8"
    secure = "try"
    timeout = 10
    code = 0
    secure_state = False
    sender_phrase, sender_addr = parseaddr(sender)

    if not body:
        body = subject
   
    if secure != 'never':
          smtp = smtplib.SMTP_SSL(timeout=timeout)
     #   code, smtpmessage = smtp.connect(host, port)
      #  secure_state = True
    #ssl.SSLError: [SSL: UNKNOWN_PROTOCOL] unknown protocol (_ssl.c:579)
       
                    

    if not secure_state:
      smtp = smtplib.SMTP(timeout=timeout)
      code, smtpmessage = smtp.connect(host, port=port)


    #this was always false....
    print smtp.has_extn('STARTTLS')
    print code

    if int(code) > 0:
        if not secure_state and secure in ('starttls', 'try'):
            #if smtp.has_extn('STARTTLS'):
               
                    smtp.starttls()
                    secure_state = True
               
            #else:
             #   if secure == 'starttls':
               #     print 'StartTLS is not offered on server'
       
        smtp.ehlo()
      
    if username and password:
        if smtp.has_extn('AUTH'):
            #try:
                smtp.login(username, password)
           # except smtplib.SMTPAuthenticationError:
            #    module.fail_json(rc=1, msg='Authentication to %s:%s failed, please check your username and/or password' % (host, port))
            #except smtplib.SMTPException:
            #    module.fail_json(rc=1, msg='No Suitable authentication method was found on %s:%s' % (host, port))
        else:
            print "No Authentication on the server at"

    if not secure_state and (username and password):
        print 'Username and Password was sent without encryption'

    msg = MIMEMultipart(_charset=charset)
    msg['From'] = formataddr((sender_phrase, sender_addr))
    msg['Subject'] = Header(subject, charset)
    msg.preamble = "Multipart message"
 
    part = MIMEText(body + "\n\n")
    msg.attach(part)
    composed = msg.as_string()
    result = smtp.sendmail(sender_addr, "vvvvv@domain.com", composed)
    smtp.quit()


if __name__ == '__main__':
    main()

``````
and only after
```` 
smtp.ehlo()
``````

you can get smtp.has_extn('STARTTLS') as true
This cold resolve problem and dose not impact other cases.
Because smtp.has_extn function not work without smtp.ehlo.
Existing duplicate block  smtp.ehlo  after smtp.starttls() also need to leave - don't remove.

before
`````py
if int(code) > 0:

```````
add line 
``````py
   try:
     smtp.ehlo()
   except smtplib.SMTPException as e:
            module.fail_json(rc=1, msg='Helo failed for host %s:%s: %s' % (host, port, to_native(e)), exception=traceback.format_exc())

Resolving patch:
diff --git a/lib/ansible/modules/notification/mail.py b/lib/ansible/modules/notification/mail.py
--- a/lib/ansible/modules/notification/mail.py
+++ b/lib/ansible/modules/notification/mail.py
@@ -263,6 +263,11 @@ def main():
     except smtplib.SMTPException as e:
         module.fail_json(rc=1, msg='Unable to Connect %s:%s: %s' % (host, port, to_native(e)), exception=traceback.format_exc())
 
+    try:
+        smtp.ehlo()
+    except smtplib.SMTPException as e:
+            module.fail_json(rc=1, msg='Helo failed for host %s:%s: %s' % (host, port, to_native(e)), exception=traceback.format_exc())
+
     if int(code) > 0:
         if not secure_state and secure in ('starttls', 'try'):
             if smtp.has_extn('STARTTLS'):
@@ -272,13 +277,13 @@ def main():
                 except smtplib.SMTPException as e:
                     module.fail_json(rc=1, msg='Unable to start an encrypted session to %s:%s: %s' %
                                      (host, port, to_native(e)), exception=traceback.format_exc())
+                try:
+                    smtp.ehlo()
+                except smtplib.SMTPException as e:
+                    module.fail_json(rc=1, msg='Helo failed for host %s:%s: %s' % (host, port, to_native(e)), exception=traceback.format_exc())
             else:
                 if secure == 'starttls':
                     module.fail_json(rc=1, msg='StartTLS is not offered on server %s:%s' % (host, port))
-        try:
-            smtp.ehlo()
-        except smtplib.SMTPException as e:
-            module.fail_json(rc=1, msg='Helo failed for host %s:%s: %s' % (host, port, to_native(e)), exception=traceback.format_exc())
 
     if username and password:
         if smtp.has_extn('AUTH'):


Test patch: