Repository: privacyidea/privacyidea

Issue:
Feature request: Email OTP template
At the moment the email body send to the user is very limited, no multiline and no templates, just like for a text message where it is ok to have single line text. In email OTP messages I could imagine having multiline text as emailtext or even templates.

I have tried to break lines using \n and json list as emailtext: ["Test line 1", "Text line 2"]


Discussion:
You are right. This could be done similar to the ``notification handler``, where we can define the body and also the if it is *plain* or *html*.

# Problem A: Policy

The email text is part of a policy. The "action" in the policy do not allow such complicated configuration.
We *could* add email templates (in a database table) and then change the policy to select one of these templates.

The templates could contain:

* template subject
* template body
* (mailserver to use)
* plain/html

# Problem B: Security

I do not really encourage to send emails, as this is everything else than secure. 
So as I am not convinced by this concept, my personal prio is veeery low for this. ;-)
Nevertheless: I think *A* is a sensible approach.
But not till something increases my personal priority.
There any update about that feature?
No. Personal priority has not been increased, yet.
Maybe you can tell a bit more about your use case?
And why and what you want to send? (As you are not the original poster, you might shed some new light on this)

We need to format an email with the user's otp, today we only change
emailtext and subjecttext in the policy, but we cannot format the text, i
tried \n\r too, so, we can't format message body of email, only a single
line text.

A email template will be very important feature in my opinion of course :).

Att.
Kleber

2017-12-15 19:01 GMT-02:00 Cornelius Kölbel <notifications@github.com>:

> No. Personal priority has not been increased, yet.
> Maybe you can tell a bit more about your use case?
> And why and what you want to send? (As you are not the original poster,
> you might shed some new light on this)
>
> —
> You are receiving this because you commented.
> Reply to this email directly, view it on GitHub
> <https://github.com/privacyidea/privacyidea/issues/684#issuecomment-352110783>,
> or mute the thread
> <https://github.com/notifications/unsubscribe-auth/AA5b-5MqNrEYLYwKY7ufjJewu8Xsvy9pks5tAt4TgaJpZM4NF23j>
> .
>

Why do you need to? How should your email look like?

You can replace ``<otp>`` and ``<serial>``.

https://github.com/privacyidea/privacyidea/blob/master/privacyidea/lib/tokens/emailtoken.py#L363
We need format email that is send to user, like that: 

*Hi, 
Here is your otp code: 564458 
Thank you* 

That is impossible to break lines today, I know I can replace <otp>, but we cannot break lines to format an email more legible!. 

Today the email is sended in only one line, like that: *Hi Here is your otp code: 564458. Thank you* 

Att. Kleber
A simple template processor like Jinja2 would do the job. Users could
implement whatever text or html with as much information they want.
Personally, I would require to add a footer with legal information and
contact support phone and email address. Not sure if it could be possible
to greet like:

Dear <firstName> <lastName>,
...

Regards,
Raoul

On Wed, Dec 20, 2017 at 2:11 AM, Kleber Rocha <notifications@github.com>
wrote:

> We need format email that is send to user, like that:
>
>
>
> *Hi,*
>
>
> *Here is your otp code: 564458*
> *Thank you*
>
> That is impossible to break lines today, I know I can replace <otp>, but We
> cannot break lines to format an email more legible!.
>
> T
>
> Att.
> Kleber
>
>
> 2017-12-19 19:37 GMT-02:00 Cornelius Kölbel <notifications@github.com>:
>
> > Why do you need to? How should your email look like?
> >
> > You can replace <otp> and <serial>.
> >
> > https://github.com/privacyidea/privacyidea/blob/
> > master/privacyidea/lib/tokens/emailtoken.py#L363
> >
> > —
> > You are receiving this because you commented.
> > Reply to this email directly, view it on GitHub
> > <https://github.com/privacyidea/privacyidea/issues/684#issuecomment-
> 352893779>,
> > or mute the thread
> > <https://github.com/notifications/unsubscribe-
> auth/AA5b-7DzlVCHjMQ_lNx0jCcf-7QmCu4aks5tCCyOgaJpZM4NF23j>
> > .
>
> >
>
> —
> You are receiving this because you authored the thread.
> Reply to this email directly, view it on GitHub
> <https://github.com/privacyidea/privacyidea/issues/684#issuecomment-352935107>,
> or mute the thread
> <https://github.com/notifications/unsubscribe-auth/AAD5iwXduqAOzOsKiG5a6FWqfTTyrS9Jks5tCF6pgaJpZM4NF23j>
> .
>

I think we are getting closer here.

@rthill @klinux you told me that you need a template and that you need a template, because you need line breaks. But I still do not know, why you need line breaks. Why should your email look like:

<em>
Hi Raoul Hill,

your totally secure one time password, which was sent to you over an unencryted media, is:
123456

Disclaimer: Please do not print this email. Save the environment.
If this email was not ment for you - since you are the hacker -  please inform the sender and destroy this email!
</em>

I would like to understand your "why", so that I get an idea of the bigger picture. This might finally increase the priority of this. Well, there are also other ways to increase the priority.

>  Why do you need line breaks, html and a footer?

In regards to jinja: Yes there are a lot of possibilities to do templates. But as I pointed out initially the problem is that this is a simple policy, that does not allow such long texts. I.e. we would need to add extra data schema for templates and so on and so on. And currently the priority is to low, to justify the enhancement in the database schema.
As mentioned, priorities are volatile and can change one day.

Thanks a lot for any additional input.
I do not agree that it is sent over unencrypted media when using a known
infrastructure with TLS enabled.

BTW, I would need this as users are not as techie as we are. Having the
possibility to add additional text will help to guide people, especially
when switching to privacyIdea from an old legacy system or when
implementing new stuff. It will save us lots of training hours.

Raoul

On Wed, Dec 20, 2017 at 8:32 AM, Cornelius Kölbel <notifications@github.com>
wrote:

> I think we are getting closer here.
>
> @rthill <https://github.com/rthill> @klinux <https://github.com/klinux>
> you told me that you need a template and that you need a template, because
> you need line breaks. But I still do not know, why you need line breaks.
> Why should your email look like:
>
>
> * Hi Raoul Hill, your totally secure one time password, which was sent to
> you over an unencryted media, is: 123456 *
>
>
>
> *Disclaimer: Please do not print this email. Save the environment. If this
> email was not ment for you - since you are the hacker - please inform the
> sender and destroy this email! *
>
> I would like to understand your "why", so that I get an idea of the bigger
> picture. This might finally increase the priority of this. Well, there are
> also other ways to increase the priority.
>
> Why do you need line breaks, html and a footer?
>
> In regards to jinja: Yes there are a lot of possibilities to do templates.
> But as I pointed out initially the problem is that this is a simple policy,
> that does not allow such long texts. I.e. we would need to add extra data
> schema for templates and so on and so on. And currently the priority is to
> low, to justify the enhancement in the database schema.
> As mentioned, priorities are volatile and can change one day.
>
> Thanks a lot for any additional input.
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> <https://github.com/privacyidea/privacyidea/issues/684#issuecomment-352987213>,
> or mute the thread
> <https://github.com/notifications/unsubscribe-auth/AAD5i7xHrLAAewRtX-vG7rCE9A5wArU_ks5tCLgPgaJpZM4NF23j>
> .
>

Hi Roul,
thanks a lot. Makes perfectly sense.
Kind regards
Cornelius
https://www.youtube.com/watch?v=g8IVI0sZ6F8

After some internal discussion we might do it this way. 
In the action "emailtext" we could provide the URI to a local template file.
This way we keep this encapsulated in the email token code, to not mess with the policy structure and do not need to enhance the DB schema.


Resolving patch:
diff --git a/privacyidea/lib/tokens/emailtoken.py b/privacyidea/lib/tokens/emailtoken.py
--- a/privacyidea/lib/tokens/emailtoken.py
+++ b/privacyidea/lib/tokens/emailtoken.py
@@ -1,5 +1,6 @@
 # -*- coding: utf-8 -*-
-#
+#  2018-01-21 Cornelius Kölbel <cornelius.koelbel@netknights.it>
+#             Add email templates
 #  2015-12-29 Cornelius Kölbel <cornelius.koelbel@netknights.it>
 #             Use privacyidea.lib.smtpserver instead of smtplib
 #  2015-10-12 Cornelius Kölbel <cornelius.koelbel@netknights.it>
@@ -141,9 +142,10 @@ def get_class_info(key=None, ret='all'):
                'policy': {SCOPE.AUTH: {
                    EMAILACTION.EMAILTEXT: {
                        'type': 'str',
-                       'desc': _('The text that will be send via EMail for'
+                       'desc': _('The text that will be sent via EMail for'
                                  ' an EMail token. Use <otp> and <serial> '
-                                 'as parameters.')},
+                                 'as parameters. You may also specify a filename '
+                                 'as email template starting with "file:".')},
                    EMAILACTION.EMAILSUBJECT: {
                        'type': 'str',
                        'desc': _('The subject of the EMail for'
@@ -255,7 +257,7 @@ def create_challenge(self, transactionid=None, options=None):
                 info = ("The PIN was correct, but the "
                         "EMail could not be sent: %r" % e)
                 log.warning(info)
-                log.debug("{0!s}".format(traceback.format_exc(e)))
+                log.debug(u"{0!s}".format(traceback.format_exc(e)))
                 return_message = info
 
         return success, return_message, transactionid, attributes
@@ -326,6 +328,16 @@ def _get_email_text_or_subject(options,
                 message = messages[0]
 
         message = message.format(challenge=options.get("challenge"))
+        if message.startswith("file:"):
+            # We read the template from the file.
+            try:
+                with open(message[5:], "r") as f:
+                    message = f.read()
+            except Exception as e:  # pragma: no cover
+                message = default
+                log.warning(u"Failed to read email template: {0!r}".format(e))
+                log.debug(u"{0!s}".format(traceback.format_exc()))
+
         return message
 
     @staticmethod
@@ -380,15 +392,20 @@ def _compose_email(self, message="<otp>", subject="Your OTP"):
         message = message.replace("<otp>", otp)
         message = message.replace("<serial>", serial)
 
+        message = message.format(otp=otp, serial=serial)
+
         subject = subject.replace("<otp>", otp)
         subject = subject.replace("<serial>", serial)
 
+        subject = subject.format(otp=otp, serial=serial)
+
         log.debug("sending Email to {0!r} ".format(recipient))
 
         identifier = get_from_config("email.identifier")
         if identifier:
             # New way to send email
-            ret = send_email_identifier(identifier, recipient, subject, message)
+            ret = send_email_identifier(identifier, recipient, subject, message,
+                                        mimetype="text/html")
         else:
             # old way to send email / DEPRECATED
             mailserver = get_from_config("email.mailserver", "localhost")


Test patch:
diff --git a/tests/test_lib_tokens_email.py b/tests/test_lib_tokens_email.py
--- a/tests/test_lib_tokens_email.py
+++ b/tests/test_lib_tokens_email.py
@@ -2,6 +2,7 @@
 This test file tests the lib.tokens.smstoken
 """
 PWFILE = "tests/testdata/passwords"
+TEMPLATE_FILE = "tests/testdata/emailtemplate.html"
 
 from .base import MyTestCase, FakeFlaskG
 from privacyidea.lib.resolver import (save_resolver)
@@ -10,8 +11,10 @@
 from privacyidea.lib.tokenclass import DATE_FORMAT
 from privacyidea.lib.tokens.emailtoken import EmailTokenClass, EMAILACTION
 from privacyidea.models import (Token, Config, Challenge)
-from privacyidea.lib.config import (set_privacyidea_config, set_prepend_pin)
+from privacyidea.lib.config import (set_privacyidea_config, set_prepend_pin,
+                                    delete_privacyidea_config)
 from privacyidea.lib.policy import set_policy, SCOPE, PolicyClass
+from privacyidea.lib.smtpserver import add_smtpserver, delete_smtpserver
 import datetime
 from dateutil.tz import tzlocal
 import smtpmock
@@ -316,6 +319,7 @@ def test_17_challenge_token(self):
     def test_18_challenge_request(self):
         smtpmock.setdata(response={"pi_tester@privacyidea.org": (200, 'OK')})
         transactionid = "123456098712"
+        # send the email with the old configuration
         set_privacyidea_config("email.mailserver", "localhost")
         set_privacyidea_config("email.username", "user")
         set_privacyidea_config("email.username", "password")
@@ -368,6 +372,28 @@ def test_19_emailtext(self):
         r = token.check_otp("287922", options=options)
         self.assertTrue(r > 0, r)
 
+        # create a EMAILTEXT policy with template
+        p = set_policy(name="emailtext",
+                       action="{0!s}=file:{1!s}".format(EMAILACTION.EMAILTEXT, TEMPLATE_FILE),
+                       scope=SCOPE.AUTH)
+        self.assertTrue(p > 0)
+
+        g = FakeFlaskG()
+        P = PolicyClass()
+        g.policy_object = P
+        options = {"g": g}
+        smtpmock.setdata(response={"pi_tester@privacyidea.org": (200, "OK")})
+        transactionid = "123456098714"
+        db_token = Token.query.filter_by(serial=self.serial1).first()
+        token = EmailTokenClass(db_token)
+        email_text = token._get_email_text_or_subject(options, EMAILACTION.EMAILTEXT)
+        self.assertTrue("<p>Hello,</p>" in email_text)
+        c = token.create_challenge(transactionid, options=options)
+        self.assertTrue(c[0], c)
+        display_message = c[1]
+        self.assertTrue(c[3].get("state"), transactionid)
+        self.assertEqual(display_message, "Enter the OTP from the Email:")
+
     @smtpmock.activate
     def test_20_sending_email_exception(self):
         smtpmock.setdata(exception=True)
@@ -388,3 +414,24 @@ def test_21_test_email_config(self):
                                          "email.recipient": "user@example.com"})
         self.assertEqual(r[0], True)
         self.assertEqual(r[1], TEST_SUCCESSFUL)
+
+    @smtpmock.activate
+    def test_22_new_email_config(self):
+        smtpmock.setdata(response={"pi_tester@privacyidea.org": (200, 'OK')})
+        transactionid = "123456098717"
+        # send the email with the new configuration
+        r = add_smtpserver(identifier="myServer", server="1.2.3.4")
+        set_privacyidea_config("email.identifier", "myServer")
+        db_token = Token.query.filter_by(serial=self.serial1).first()
+        token = EmailTokenClass(db_token)
+        self.assertTrue(token.check_otp("123456", 1, 10) == -1)
+        c = token.create_challenge(transactionid)
+        self.assertTrue(c[0], c)
+        otp = c[1]
+        self.assertTrue(c[3].get("state"), transactionid)
+
+        # check for the challenges response
+        r = token.check_challenge_response(passw=otp)
+        self.assertTrue(r, r)
+        delete_smtpserver("myServer")
+        delete_privacyidea_config("email.identifier")
diff --git a/tests/testdata/emailtemplate.html b/tests/testdata/emailtemplate.html
new file mode 100644
--- /dev/null
+++ b/tests/testdata/emailtemplate.html
@@ -0,0 +1,10 @@
+<p>Hello,</p>
+
+<p>
+You may use the OTP value {otp} with your token.
+{serial}.
+</p>
+
+<p>
+Your admins
+</p>