Repository: getnikola/nikola

Issue:
Unclear error when file referenced in template doesn't exist
### Environment (if reporting a bug)

**Python Version:** 3.5

**Nikola Version:** current git

**Operating System:** Debian

### Description:

I created `templates/page.tmpl` with `nikola theme -c page.tmpl` and added the line
```
<%namespace name="ui" file="crumbs.tmpl" import="bar"/>
```

now it seems that `crumbs.tmpl` is not available for bootstrap4, ok.  But the error message is really bad:

```
Scanning posts........done!
[2018-04-19T17:25:04Z] ERROR: Nikola: Error loading tasks. An unhandled exception occurred.
[2018-04-19T17:25:04Z] ERROR: Nikola: TypeError: invalid file: None
```

With  `NIKOLA_DEBUG=1`:

```
[2018-04-19T17:25:28Z] ERROR: Nikola: Error loading tasks. An unhandled exception occurred.
Traceback (most recent call last):
  File "/home/tbm/.local/lib/python3.5/site-packages/doit/doit_cmd.py", line 177, in run
    return command.parse_execute(args)
  File "/home/tbm/.local/lib/python3.5/site-packages/doit/cmd_base.py", line 127, in parse_execute
    return self.execute(params, args)
  File "/home/tbm/.local/lib/python3.5/site-packages/doit/cmd_base.py", line 389, in execute
    self, params, args)
  File "/home/tbm/.local/lib/python3.5/site-packages/nikola/__main__.py", line 277, in load_tasks
    self.nikola.gen_tasks('render_site', "Task", 'Group of tasks to render the site.'))
  File "/home/tbm/.local/lib/python3.5/site-packages/doit/loader.py", line 313, in generate_tasks
    for task_dict, x_doc in flat_generator(gen_result, gen_doc):
  File "/home/tbm/.local/lib/python3.5/site-packages/doit/loader.py", line 26, in flat_generator
    for item in gen:
  File "/home/tbm/.local/lib/python3.5/site-packages/nikola/nikola.py", line 1942, in gen_tasks
    for task in flatten(pluginInfo.plugin_object.gen_tasks()):
  File "/home/tbm/.local/lib/python3.5/site-packages/nikola/nikola.py", line 1936, in flatten
    for t in task:
  File "/home/tbm/.local/lib/python3.5/site-packages/nikola/plugins/task/pages.py", line 66, in gen_tasks
    for task in self.site.generic_page_renderer(lang, post, kw["filters"], context):
  File "/home/tbm/.local/lib/python3.5/site-packages/nikola/nikola.py", line 2256, in generic_page_renderer
    url_type=post.url_type)
  File "/home/tbm/.local/lib/python3.5/site-packages/nikola/nikola.py", line 2161, in generic_renderer
    file_deps += self.template_system.template_deps(template_name)
  File "/home/tbm/.local/lib/python3.5/site-packages/nikola/plugins/template/mako.py", line 129, in template_deps
    deps += [fname] + self.get_deps(fname)
  File "/home/tbm/.local/lib/python3.5/site-packages/nikola/plugins/template/mako.py", line 73, in get_deps
    text = util.read_file(filename)
  File "/home/tbm/.local/lib/python3.5/site-packages/mako/util.py", line 365, in read_file
    fp = open(path, mode)
TypeError: invalid file: None
```


Discussion:
Maybe I should file a bug in mako to produce a better error message (which file failed to load) and nikola could improve the message by saying "the following error occurred while loading template X".
What is filename when it's used in text = util.read_file(filename) ?

I am guessing crumb.tmpl?
> What is filename when it's used in text = util.read_file(filename) ?

> I am guessing crumb.tmpl?

Actually, no.  For a file that doesn't exist (like crumbs.tmpl), get_deps() is called with None for the filename.

In template_deps(), you have:

```
            dep_filenames = self.get_deps(template.filename)
```

and that includes filenames plus None for the file that doesn't exist.

```
dep_filenames:  ['/home/tbm/.local/lib/python3.5/site-packages/nikola/data/themes/base/templates/post_helper.tmpl', '/home/tbm/.local/lib/python3.5/site-packages/nikola/data/themes/base/templates/post_header.tmpl', '/home/tbm/.local/lib/python3.5/site-packages/nikola/data/themes/base/templates/comments_helper.tmpl', '/home/tbm/.local/lib/python3.5/site-packages/nikola/data/themes/base/templates/math_helper.tmpl', None, '/home/tbm/.local/lib/python3.5/site-packages/nikola/data/themes/bootstrap4/templates/post.tmpl']
```

This is because get_template_path() returns None in that case.

So someone in that path should produce a proper error message.  Maybe get_string_deps()?

This change results in an error (probably because `crumbs.tmpl` is not converted to a filepath, i.e. None in our case):
```
[2018-04-21T09:08:06Z] ERROR: Nikola: FileNotFoundError: [Errno 2] No such file or directory: 'crumbs.tmpl'
```

```
--- a/nikola/plugins/template/mako.py
+++ b/nikola/plugins/template/mako.py
@@ -65,7 +65,9 @@ class MakoTemplates(TemplateSystem):
         # Some templates will include "foo.tmpl" and we need paths, so normalize them
         # using the template lookup
         for i, d in enumerate(deps):
-            deps[i] = self.get_template_path(d)
+            dep = self.get_template_path(d)
+            if dep:
+                deps[i] = dep
         return deps
 
     def get_deps(self, filename):
```

but maybe this is better:

```
[2018-04-21T09:10:38Z] ERROR: mako: Template crumbs.tmpl referenced in template templates/page.tmpl doesn't exist
[2018-04-21T09:10:39Z] ERROR: Nikola: FileNotFoundError: [Errno 2] No such file or directory: 'crumbs.tmpl'
```

with:

```
--- a/nikola/plugins/template/mako.py
+++ b/nikola/plugins/template/mako.py
@@ -65,7 +65,12 @@ class MakoTemplates(TemplateSystem):
         # Some templates will include "foo.tmpl" and we need paths, so normalize them
         # using the template lookup
         for i, d in enumerate(deps):
-            deps[i] = self.get_template_path(d)
+            dep = self.get_template_path(d)
+            if dep:
+                deps[i] = dep
+            else:
+                LOGGER.error("Template {0} referenced in template {1} doesn't exist",
+                    d, filename)
         return deps
 
     def get_deps(self, filename):
```

Does that look sane?  If so, I can open a PR.
(The original patch was wrong. I made some mistakes when I manually applied the changes to my nikola git checkout after editing the copy in `~/.local`. I've updated the patch)
I remember this or something similar which had me bitten as well (might have been with jinja2 templates, though). I'd really like the second patch (the one with the more precise error message) to be in Nikola. One suggestion maybe: how about saying `Cannot find template [...] referenced in [...]` instead of `Template [...] referenced in [...] doesn't exist`? After all, it could also be that a user put it into a wrong place :-)
@felixfontein I can change the message and I'll check if the other templates have the same issue.

One question though: when I use LOGGER.error() my error message is shown but the later error about the file not existing is shown as well. IT seems LOGGER.error() doesn't exit immediately.  Is there anything I should do differently or is this behaviour ok?
I just tested jinja templates, and there the error message is more helpful:
```
[2018-04-21T12:08:28Z] ERROR: Nikola: Error loading tasks. An unhandled exception occurred.
[2018-04-21T12:08:28Z] ERROR: Nikola: jinja2.exceptions.TemplateNotFound: asdf.tmpl
```

And yes, `LOGGER.error()` does not exist (it definitely should not, after all it is a logging call). You can either quit at that point, or just keep the other error message. I personally don't mind having both errors. @Kwpolska, @ralsina, any opinions?
Having both errors is ok for me.

Resolving patch:
diff --git a/nikola/plugins/template/mako.py b/nikola/plugins/template/mako.py
--- a/nikola/plugins/template/mako.py
+++ b/nikola/plugins/template/mako.py
@@ -65,7 +65,12 @@ def get_string_deps(self, text, filename=None):
         # Some templates will include "foo.tmpl" and we need paths, so normalize them
         # using the template lookup
         for i, d in enumerate(deps):
-            deps[i] = self.get_template_path(d)
+            dep = self.get_template_path(d)
+            if dep:
+                deps[i] = dep
+            else:
+                LOGGER.error("Cannot find template {0} referenced in {1}",
+                             d, filename)
         return deps
 
     def get_deps(self, filename):


Test patch: