Repository: getpelican/pelican

Issue:
RFE: Make it possible to include content files from other content files
Hi guys,
I have several pages which have the same boiler plate text. It's a warning/disclaimer notice which gets included where necessary. As far as I can see it is not possible to do this inside my content files. 
For example instead of linking to another file with `[disclaimer]({filename}disclaimer.md]` I want to be able to write `{include}disclaimer.md` and the text will be automatically included in the current page being processed. 

Please let me know if this exists (or partially exists). I intend to work on this feature b/c I need it.



Discussion:
hello? Is this syntax can only be used in markdown files? or any kind of textfiles, like .rst or plain .txt, in the content folder?
I have noticed that this is specially useful for organizing large posts.

I've used a .html file to test it. As far as I can tell the curly braces syntax (also {filename}, {category}) can be used in any kind of text file. I still need to hear code review from the project maintainers though.

@mitchtbaum - I've received a notification about you commenting on this issue but the comment got lost somehow. 

@atodorov: Sorry about that.  I had started to comment, and somehow this form submitted before finishing it, so I wrote another one saying that happened and deleted both.  #tangled-web-problems

I looked further into this, and aside from Pelican's current link syntax and potential additions and improvements which we can address in another issue, I believe I have already produced a good solution for your use case.  I need to fix upstream dependencies so I can make stable links to it (here and in Pelican's plugin meta-repo) and finish documentation, so I am working on that now.  // Temporarily, you can find usable, pre-released code in https://github.com/mitchtbaum?tab=repositories  //

In a nutshell, you would create a 'notice' (article with that category) for each one you want to use, and you would **reference** your desired notice from any article with a `notice: warning` or `notice: whatever` metadata entry, and then in your article template or specific category template, you would include something like this chunk: 

```
{% if article.notice %}
  {% for reference in article.notice %}
    <div id="notice">
      <b>Notice:</b> {{ reference.body }}
    </div>
  {% endfor %}
{% endif %}
```

How would that work for you?

---

Aside:  You can see a small writeup before I had worked out this `reference` plugin's full design in #1169.  I will mark that as complete when I get this code into a repo where we can continue to work on it, #1488.

Note: edited Jinja example to add div wrapper and changed styling to wrap only leading text saying what it is.

> How would that work for you?

It won't as far as I can understand. Correct me if I'm wrong but your solution requires me to have a template in which to reference the text I need to include, right ?

What I want to do is include text inside my content files (e.g. html pages or markdown posts). I don't want to mess about with templates.

References take place only in content.  They are simple key-value pairs going from one piece of content to another using metadata.

So for instance, `/content/notice/bring-water.md` could have body text "Bring a water bottle!", and `/content/travel/kalahari.md` could have `title: Kalahari Desert`, `notice: bring-water`, and `body` text like "It gets dry there, like any desert."

When your content (markdown, etc) gets rendered into a desired output (html, etc), it will go wherever you put it and have whatever style you give it, as per your theme.

imho, data feels most comfortable (so to speak) when it is kept separate from any logic about where to put it and how it could look.

@atodorov 

> I have several pages which have the same boiler plate text. It's a warning/disclaimer notice which gets included where necessary

I wrote the [jinja2content](https://github.com/leotrs/pelican-jinja2content) plugin for this purpose. You can see it in action [here](https://github.com/leotrs/erdos/blob/master/content/challenges/graphs/nodes_edges.md). The file includes several others by using a jinja2 `{% include %}` statement.

The plugin also provides full jinja2 functionality (before the theme's templates are rendered, so the content that is passed onto the plugin does not have the `include`s any more).

cc. @mitchtbaum 

PR #1909 was submitted to address this and is currently awaiting review from @getpelican/reviewers.


Resolving patch:
diff --git a/pelican/generators.py b/pelican/generators.py
--- a/pelican/generators.py
+++ b/pelican/generators.py
@@ -155,16 +155,15 @@ def get_files(self, paths, exclude=[], extensions=None):
 
             if os.path.isdir(root):
                 for dirpath, dirs, temp_files in os.walk(
-                        root, followlinks=True):
-                    drop = []
+                        root, topdown=True, followlinks=True):
                     excl = exclusions_by_dirpath.get(dirpath, ())
-                    for d in dirs:
+                    # We copy the `dirs` list as we will modify it in the loop:
+                    for d in list(dirs):
                         if (d in excl or
                             any(fnmatch.fnmatch(d, ignore)
                                 for ignore in ignores)):
-                            drop.append(d)
-                    for d in drop:
-                        dirs.remove(d)
+                            if d in dirs:
+                                dirs.remove(d)
 
                     reldir = os.path.relpath(dirpath, self.path)
                     for f in temp_files:


Test patch:
diff --git a/pelican/tests/support.py b/pelican/tests/support.py
--- a/pelican/tests/support.py
+++ b/pelican/tests/support.py
@@ -188,7 +188,7 @@ class LogCountHandler(BufferingHandler):
     """Capturing and counting logged messages."""
 
     def __init__(self, capacity=1000):
-        logging.handlers.BufferingHandler.__init__(self, capacity)
+        super(LogCountHandler, self).__init__(capacity)
 
     def count_logs(self, msg=None, level=None):
         return len([