Repository: matrix-org/synapse

Issue:
the email pusher is broken in 0.33.7
If you leave `email_templates` unset, or set to `res/templates`, you get the following error when the pusher tries to send an email:

```
Traceback (most recent call last):
  File synapse/push/emailpusher.py, line 125, in _process
  File /home/matrix/.synapse/local/lib/python2.7/site-packages/twisted/internet/defer.py, line 1416, in _inlineCallbacks
    result = result.throwExceptionIntoGenerator(g)
  File /home/matrix/.synapse/local/lib/python2.7/site-packages/twisted/python/failure.py, line 491, in throwExceptionIntoGenerator
    return g.throw(self.type, self.value, self.tb)
  File synapse/push/emailpusher.py, line 177, in _unsafe_process
  File /home/matrix/.synapse/local/lib/python2.7/site-packages/twisted/internet/defer.py, line 1416, in _inlineCallbacks
    result = result.throwExceptionIntoGenerator(g)
  File /home/matrix/.synapse/local/lib/python2.7/site-packages/twisted/python/failure.py, line 491, in throwExceptionIntoGenerator
    return g.throw(self.type, self.value, self.tb)
  File synapse/push/emailpusher.py, line 287, in send_notification
  File /home/matrix/.synapse/local/lib/python2.7/site-packages/twisted/internet/defer.py, line 1418, in _inlineCallbacks
    result = g.send(result)
  File synapse/push/mailer.py, line 178, in send_notification_mail
  File /home/matrix/.synapse/local/lib/python2.7/site-packages/jinja2/environment.py, line 1008, in render
    return self.environment.handle_exception(exc_info, True)
  File /home/matrix/.synapse/local/lib/python2.7/site-packages/jinja2/environment.py, line 780, in handle_exception
    reraise(exc_type, exc_value, tb)
  File synapse/res/templates/notif_mail.html, line 5, in top-level template code
  File /home/matrix/.synapse/local/lib/python2.7/site-packages/jinja2/loaders.py, line 235, in get_source
    raise TemplateNotFound(template)
TemplateNotFound: mail.css
```



Discussion:
A workaround for now is to explicitly set `email_templates` to the full path of wherever `synapse/res/templates` is on your system.
This was introduced by https://github.com/matrix-org/synapse/pull/4052.

So, some backstory is necessary here:

The default config (before #4052) gave an example setting of `res/templates` for `email_templates`. However, *this never worked*, because synapse changes directory to `/` once it starts (at least if it is run with the `--daemonize` arg, as synctl does). In practice therefore, anyone who uses email notifications will have set `email_templates` to an absolute path.

In #4052, I had the great idea of solving #4014 and https://github.com/matrix-org/package-synapse-debian/issues/37 by including the example templates as package data, and using them from there if `email_templates` was unset (or, in an attempt to preserve backwards-compatibility, if it was set to `res/templates` - not that that will have helped anyone in practice, as per the above).

However, it turns out that attempting to load the email templates from the package data has exactly the same problem as loading them from a relative path: the supporting templates like `mail.css` can't be found once synapse changes directory.


Resolving patch:
diff --git a/synapse/config/emailconfig.py b/synapse/config/emailconfig.py
--- a/synapse/config/emailconfig.py
+++ b/synapse/config/emailconfig.py
@@ -19,18 +19,12 @@
 import email.utils
 import logging
 import os
-import sys
-import textwrap
 
-from ._base import Config
+import pkg_resources
 
-logger = logging.getLogger(__name__)
+from ._base import Config, ConfigError
 
-TEMPLATE_DIR_WARNING = """\
-WARNING: The email notifier is configured to look for templates in '%(template_dir)s',
-but no templates could be found there. We will fall back to using the example templates;
-to get rid of this warning, leave 'email.template_dir' unset.
-"""
+logger = logging.getLogger(__name__)
 
 
 class EmailConfig(Config):
@@ -78,20 +72,22 @@ def read_config(self, config):
             self.email_notif_template_html = email_config["notif_template_html"]
             self.email_notif_template_text = email_config["notif_template_text"]
 
-            self.email_template_dir = email_config.get("template_dir")
-
-            # backwards-compatibility hack
-            if (
-                self.email_template_dir == "res/templates"
-                and not os.path.isfile(
-                    os.path.join(self.email_template_dir, self.email_notif_template_text)
+            template_dir = email_config.get("template_dir")
+            # we need an absolute path, because we change directory after starting (and
+            # we don't yet know what auxilliary templates like mail.css we will need).
+            # (Note that loading as package_resources with jinja.PackageLoader doesn't
+            # work for the same reason.)
+            if not template_dir:
+                template_dir = pkg_resources.resource_filename(
+                    'synapse', 'res/templates'
                 )
-            ):
-                t = TEMPLATE_DIR_WARNING % {
-                    "template_dir": self.email_template_dir,
-                }
-                print(textwrap.fill(t, width=80) + "\n", file=sys.stderr)
-                self.email_template_dir = None
+            template_dir = os.path.abspath(template_dir)
+
+            for f in self.email_notif_template_text, self.email_notif_template_html:
+                p = os.path.join(template_dir, f)
+                if not os.path.isfile(p):
+                    raise ConfigError("Unable to find email template file %s" % (p, ))
+            self.email_template_dir = template_dir
 
             self.email_notif_for_new_users = email_config.get(
                 "notif_for_new_users", True
diff --git a/synapse/push/mailer.py b/synapse/push/mailer.py
--- a/synapse/push/mailer.py
+++ b/synapse/push/mailer.py
@@ -526,12 +526,8 @@ def load_jinja2_templates(config):
     Returns:
         (notif_template_html, notif_template_text)
     """
-    logger.info("loading jinja2")
-
-    if config.email_template_dir:
-        loader = jinja2.FileSystemLoader(config.email_template_dir)
-    else:
-        loader = jinja2.PackageLoader('synapse', 'res/templates')
+    logger.info("loading email templates from '%s'", config.email_template_dir)
+    loader = jinja2.FileSystemLoader(config.email_template_dir)
     env = jinja2.Environment(loader=loader)
     env.filters["format_ts"] = format_ts_filter
     env.filters["mxc_to_http"] = _create_mxc_to_http_filter(config)


Test patch: