internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: getnikola/nikola

Issue:
Mako template dependencies don't support nesting
A page that uses `story.tmpl` gets the following 21 dependencies:

```python
['story.tmpl', 'post_helper.tmpl', 'math_helper.tmpl', 'post_header.tmpl', 'post_helper.tmpl', 'comments_helper.tmpl', 'comments_helper.tmpl', 'comments_helper_disqus.tmpl', 'comments_helper_intensedebate.tmpl', 'comments_helper_muut.tmpl', 'comments_helper_facebook.tmpl', 'comments_helper_isso.tmpl', 'comments_helper_commento.tmpl', 'math_helper.tmpl', 'post.tmpl', 'post_helper.tmpl', 'post_header.tmpl', 'comments_helper.tmpl', 'math_helper.tmpl', 'ui_helper.tmpl', 'base.tmpl']
```

But if you use `page.tmpl`, which just inherits from `story.tmpl`, the list is much shorter (7 files):

```python
['page.tmpl', 'story.tmpl', 'post_helper.tmpl', 'post_header.tmpl', 'comments_helper.tmpl', 'math_helper.tmpl', 'post.tmpl']
```
Mako template dependencies don't support nesting
A page that uses `story.tmpl` gets the following 21 dependencies:

```python
['story.tmpl', 'post_helper.tmpl', 'math_helper.tmpl', 'post_header.tmpl', 'post_helper.tmpl', 'comments_helper.tmpl', 'comments_helper.tmpl', 'comments_helper_disqus.tmpl', 'comments_helper_intensedebate.tmpl', 'comments_helper_muut.tmpl', 'comments_helper_facebook.tmpl', 'comments_helper_isso.tmpl', 'comments_helper_commento.tmpl', 'math_helper.tmpl', 'post.tmpl', 'post_helper.tmpl', 'post_header.tmpl', 'comments_helper.tmpl', 'math_helper.tmpl', 'ui_helper.tmpl', 'base.tmpl']
```

But if you use `page.tmpl`, which just inherits from `story.tmpl`, the list is much shorter (7 files):

```python
['page.tmpl', 'story.tmpl', 'post_helper.tmpl', 'post_header.tmpl', 'comments_helper.tmpl', 'math_helper.tmpl', 'post.tmpl']
```


Discussion:

Resolving patch:
diff --git a/nikola/plugins/template/mako.py b/nikola/plugins/template/mako.py
--- a/nikola/plugins/template/mako.py
+++ b/nikola/plugins/template/mako.py
@@ -131,9 +131,10 @@ def template_deps(self, template_name):
             dep_filenames = self.get_deps(template.filename)
             deps = [template.filename]
             for fname in dep_filenames:
-                deps += [fname] + self.get_deps(fname)
-            self.cache[template_name] = deps
-        return list(self.cache[template_name])
+                # yes, it uses forward slashes on Windows
+                deps += self.template_deps(fname.split('/')[-1])
+            self.cache[template_name] = list(set(deps))
+        return self.cache[template_name]
 
     def get_template_path(self, template_name):
         """Get the path to a template or return None."""


Test patch:
Repository: getnikola/nikola

Issue:
Changes to EXIF_WHITELIST does not trigger gallery rebuild
<!--
Before creating an issue:
* make sure you are using an up-to-date version of Nikola
* search for existing issues that might be related

Make sure to:
* provide information about your environment (below)
* include all the output you get, and any other information related to your problem

Nikola v7.6.4, as provided by Ubuntu, is NOT SUPPORTED.
If you are using this version, you should upgrade: https://getnikola.com/getting-started.html
-->

### Environment

**Python Version:**
3.8.1

**Nikola Version:**
8.0.4

**Operating System:**
Windows 10

### Description:
Changes to the `conf.py` variable `EXIF_WHITELIST` trigger a rebuild of all images in the `images` folder.  It does not, however, cause any rebuild of images in the `gallery` folder.  In order to change the EXIF data for galleries you must perform a full site rebuild.


Discussion:
It seems to trigger a rebuild of the HTML files and RSS feeds of galleries but not of the images themselves.

![image](https://user-images.githubusercontent.com/1579/78379748-3a85ee00-75a9-11ea-8aae-aa76c6790933.png)


Resolving patch:
diff --git a/nikola/plugins/task/galleries.py b/nikola/plugins/task/galleries.py
--- a/nikola/plugins/task/galleries.py
+++ b/nikola/plugins/task/galleries.py
@@ -571,7 +571,10 @@ def create_target_images(self, img, input_path):
             ],
             'clean': True,
             'uptodate': [utils.config_changed({
-                1: self.kw['thumbnail_size']
+                1: self.kw['thumbnail_size'],
+                2: self.kw['preserve_exif_data'],
+                3: self.kw['exif_whitelist'],
+                4: self.kw['preserve_icc_profiles'],
             }, 'nikola.plugins.task.galleries:resize_thumb')],
         }, self.kw['filters'])
 
@@ -587,7 +590,10 @@ def create_target_images(self, img, input_path):
             ],
             'clean': True,
             'uptodate': [utils.config_changed({
-                1: self.kw['max_image_size']
+                1: self.kw['max_image_size'],
+                2: self.kw['preserve_exif_data'],
+                3: self.kw['exif_whitelist'],
+                4: self.kw['preserve_icc_profiles'],
             }, 'nikola.plugins.task.galleries:resize_max')],
         }, self.kw['filters'])
 


Test patch:
Repository: getnikola/nikola

Issue:
Soundcloud plugin lacks license information
### Environment

**Nikola Version:** git master

### Description:

The module `nikola.plugins.compile.rest.soundcloud` does not contain any license information.
Is this intended or simply has been forgotten?


Discussion:

Resolving patch:
diff --git a/nikola/plugins/compile/rest/soundcloud.py b/nikola/plugins/compile/rest/soundcloud.py
--- a/nikola/plugins/compile/rest/soundcloud.py
+++ b/nikola/plugins/compile/rest/soundcloud.py
@@ -1,5 +1,29 @@
 # -*- coding: utf-8 -*-
 
+# Copyright © 2012-2020 Roberto Alsina and others.
+
+# Permission is hereby granted, free of charge, to any
+# person obtaining a copy of this software and associated
+# documentation files (the "Software"), to deal in the
+# Software without restriction, including without limitation
+# the rights to use, copy, modify, merge, publish,
+# distribute, sublicense, and/or sell copies of the
+# Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice
+# shall be included in all copies or substantial portions of
+# the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY
+# KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE
+# WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR
+# PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS
+# OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR
+# OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR
+# OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
+# SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+
 """SoundCloud directive for reStructuredText."""
 
 from docutils import nodes


Test patch:
Repository: getnikola/nikola

Issue:
Hyphenation adds (sometimes) soft hyphens at the beginning of words.
Example:

![image](https://user-images.githubusercontent.com/1579/78097710-558f0d00-73b3-11ea-841e-8c30ac5b251a.png)



Discussion:

Resolving patch:
diff --git a/nikola/post.py b/nikola/post.py
--- a/nikola/post.py
+++ b/nikola/post.py
@@ -1115,8 +1115,14 @@ def insert_hyphens(node, hyphenator):
         text = getattr(node, attr)
         if not text:
             continue
-        new_data = ' '.join([hyphenator.inserted(w, hyphen='\u00AD')
-                             for w in text.split(' ')])
+
+        lines = text.splitlines()
+        new_data = "\n".join(
+            [
+                " ".join([hyphenator.inserted(w, hyphen="\u00AD") for w in line.split(" ")])
+                for line in lines
+            ]
+        )
         # Spaces are trimmed, we have to add them manually back
         if text[0].isspace():
             new_data = ' ' + new_data


Test patch:
Repository: getnikola/nikola

Issue:
Dependency issue in gallery building with -n4
Building my site, if I use multiprocessing, I get errors about specific images "not existing".

It seems to me like the page tries to get the image size but the image is not processed yet. Probably a missing dependency.


Discussion:

Resolving patch:
diff --git a/nikola/plugins/task/galleries.py b/nikola/plugins/task/galleries.py
--- a/nikola/plugins/task/galleries.py
+++ b/nikola/plugins/task/galleries.py
@@ -308,7 +308,7 @@ def gen_tasks(self):
                 yield utils.apply_filters({
                     'basename': self.name,
                     'name': dst,
-                    'file_dep': file_dep,
+                    'file_dep': file_dep + dest_img_list,
                     'targets': [dst],
                     'actions': [
                         (self.render_gallery_index, (


Test patch:
Repository: getnikola/nikola

Issue:
Galleries with index.txt and without title get name "index"
And it's probably better if they have the name of the folder instead.


Discussion:

Resolving patch:
diff --git a/nikola/plugins/task/galleries.py b/nikola/plugins/task/galleries.py
--- a/nikola/plugins/task/galleries.py
+++ b/nikola/plugins/task/galleries.py
@@ -503,8 +503,9 @@ def parse_index(self, gallery, input_folder, output_folder):
             # index.txt file would be errorneously named `index`
             # (warning: galleries titled index and filenamed differently
             #  may break)
-            if post.title == 'index':
-                post.title = os.path.split(gallery)[1]
+            if post.title() == 'index':
+                for lang in post.meta.keys():
+                    post.meta[lang]['title'] = os.path.split(gallery)[1]
             # Register the post (via #2417)
             self.site.post_per_input_file[index_path] = post
         else:


Test patch:
Repository: getnikola/nikola

Issue:
Add support for default previewimage
The previewimage meta tag is used for Open Graph metadata, among other things. However, not all pages and blog posts will have their own image. It would be cool to be able to provide a generic **default image via the config**, without having to add it manually to all pages that don't feature one already.

(New pages and posts can be generated with the matching previewimage line, of course, but retrofitting all old files is a pain.)


Discussion:
I concur with this need and I add that it would be desirable to have a default previewimage _per section_, if the site has sections. For instance a section can be a periodic column by some author and a photo of the author could be the default preview image.
PRs for either or both the features are welcome. This could be generalized to per-category/site-wide default post metadata.

Resolving patch:
diff --git a/nikola/nikola.py b/nikola/nikola.py
--- a/nikola/nikola.py
+++ b/nikola/nikola.py
@@ -475,6 +475,7 @@ def __init__(self, **config):
             'LINK_CHECK_WHITELIST': [],
             'LISTINGS_FOLDERS': {'listings': 'listings'},
             'LOGO_URL': '',
+            'DEFAULT_PREVIEW_IMAGE': None,
             'NAVIGATION_LINKS': {},
             'NAVIGATION_ALT_LINKS': {},
             'MARKDOWN_EXTENSIONS': ['fenced_code', 'codehilite', 'extra'],
diff --git a/nikola/post.py b/nikola/post.py
--- a/nikola/post.py
+++ b/nikola/post.py
@@ -138,6 +138,7 @@ def __init__(
         self._dependency_uptodate_fragment = defaultdict(list)
         self._dependency_uptodate_page = defaultdict(list)
         self._depfile = defaultdict(list)
+        self._default_preview_image = self.config['DEFAULT_PREVIEW_IMAGE']
         if metadata_extractors_by is None:
             self.metadata_extractors_by = {'priority': {}, 'source': {}}
         else:
@@ -931,9 +932,8 @@ def previewimage(self, lang=None):
             lang = nikola.utils.LocaleBorg().current_lang
 
         image_path = self.meta[lang]['previewimage']
-
         if not image_path:
-            return None
+            return self._default_preview_image
 
         # This is further parsed by the template, because we don’t have access
         # to the URL replacer here.  (Issue #1473)


Test patch:
Repository: getnikola/nikola

Issue:
Hyphenation sometimes uses the wrong language.
If you have a blog supporting en / es (en default) and you have a page that is only in "es" and hyphenation enabled, then that "es" content will be visible as "en" content (ok) BUT it will be hyphenated as english, which is totally wrong.

A fix for this may be quite involved.


Discussion:

Resolving patch:
diff --git a/nikola/post.py b/nikola/post.py
--- a/nikola/post.py
+++ b/nikola/post.py
@@ -703,16 +703,21 @@ def translated_base_path(self, lang):
         return get_translation_candidate(self.config, self.base_path, lang)
 
     def _translated_file_path(self, lang):
-        """Return path to the translation's file, or to the original."""
+        """Get path to a post's translation.
+
+        Returns path to the translation's file, or to as good a file as it can
+        plus "real" language of the text.
+        """
         if lang in self.translated_to:
             if lang == self.default_lang:
-                return self.base_path
+                return self.base_path, lang
             else:
-                return get_translation_candidate(self.config, self.base_path, lang)
+                return get_translation_candidate(self.config, self.base_path, lang), lang
         elif lang != self.default_lang:
-            return self.base_path
+            return self.base_path, self.default_lang
         else:
-            return get_translation_candidate(self.config, self.base_path, sorted(self.translated_to)[0])
+            real_lang = sorted(self.translated_to)[0]
+            return get_translation_candidate(self.config, self.base_path, real_lang), real_lang
 
     def text(self, lang=None, teaser_only=False, strip_html=False, show_read_more_link=True,
              feed_read_more_link=False, feed_links_append_query=None):
@@ -729,7 +734,7 @@ def text(self, lang=None, teaser_only=False, strip_html=False, show_read_more_li
         """
         if lang is None:
             lang = nikola.utils.LocaleBorg().current_lang
-        file_name = self._translated_file_path(lang)
+        file_name, real_lang = self._translated_file_path(lang)
 
         # Yes, we compile it and screw it.
         # This may be controversial, but the user (or someone) is asking for the post text
@@ -754,7 +759,7 @@ def text(self, lang=None, teaser_only=False, strip_html=False, show_read_more_li
         document.make_links_absolute(base_url)
 
         if self.hyphenate:
-            hyphenate(document, lang)
+            hyphenate(document, real_lang)
 
         try:
             data = lxml.html.tostring(document.body, encoding='unicode')
@@ -837,7 +842,7 @@ def paragraph_count(self):
         if self._paragraph_count is None:
             # duplicated with Post.text()
             lang = nikola.utils.LocaleBorg().current_lang
-            file_name = self._translated_file_path(lang)
+            file_name, _ = self._translated_file_path(lang)
             with io.open(file_name, "r", encoding="utf8") as post_file:
                 data = post_file.read().strip()
             try:


Test patch:
Repository: getnikola/nikola

Issue:
Add "get post contents" method to Post
To support "posts without sources" (#2659) we need to add a method to get the post's raw content, similar to `text()` (`source(lang)` maybe?)

It also simplifies compiler code, while we're at it.


Discussion:

Resolving patch:
diff --git a/nikola/plugin_categories.py b/nikola/plugin_categories.py
--- a/nikola/plugin_categories.py
+++ b/nikola/plugin_categories.py
@@ -349,8 +349,8 @@ def split_metadata(self, data: str, post=None, lang=None) -> (str, str):
 
         if isinstance(extractor, MetadataExtractor):
             return extractor.split_metadata_from_text(data)
-        else:
-            return data, data
+        # Ouch!
+        return data, data
 
     def get_compiler_extensions(self) -> list:
         """Activate all the compiler extension plugins for a given compiler and return them."""
diff --git a/nikola/post.py b/nikola/post.py
--- a/nikola/post.py
+++ b/nikola/post.py
@@ -748,9 +748,23 @@ def _translated_file_path(self, lang):
             real_lang = sorted(self.translated_to)[0]
             return get_translation_candidate(self.config, self.base_path, real_lang), real_lang
 
+    def source(self, lang=None):
+        """Read the post and return its source."""
+        if lang is None:
+            lang = nikola.utils.LocaleBorg().current_lang
+
+        source = self.translated_source_path(lang)
+        with open(source, encoding='utf-8') as inf:
+            data = inf.read()
+        if self.is_two_file:  # Metadata is not here
+            source_data = data
+        else:
+            source_data = self.compiler.split_metadata(data, self, lang)[1]
+        return source_data
+
     def text(self, lang=None, teaser_only=False, strip_html=False, show_read_more_link=True,
              feed_read_more_link=False, feed_links_append_query=None):
-        """Read the post file for that language and return its contents.
+        """Read the post file for that language and return its compiled contents.
 
         teaser_only=True breaks at the teaser marker and returns only the teaser.
         strip_html=True removes HTML tags


Test patch:
Repository: getnikola/nikola

Issue:
line numbers in code listings can't be disabled in 8.1.0
### Environment

**Python Version:**
3.7.7

**Nikola Version:**
8.1.0

**Operating System:**
Fedora 32

### Description:
This started over at Tweitter (https://twitter.com/zhenech/status/1278613632725782528) and I was asked to report a bug here.

With the upgrade to 8.1, the code listings in my site (https://github.com/evgeni/die-welt.net) started to get line numbers and I can't figure out a way to revert back to the 8.0 behaviour. I wouldn't mind the numbers that much, but the way the tables get transferred to RSS clients make it really hard to read the code there.

To summarize the thread:
* using Markdown as input for my posts, usually with braced code blocks (but also tried normal indented ones as shown in the CodeHilite docs for "not numbered")
* setting `MARKDOWN_EXTENSION_CONFIGS = { 'codehilite': {'linenos': False, 'linenums': False} }` in `conf.py` didn't have an effect either
* 8.0.4 used to generate `<pre class="code literal-block"><span></span><code>…` whereas 8.1.0 started to create a `<table>…`

Thanks a ton for your support and Nikola!


Discussion:

Resolving patch:
diff --git a/nikola/plugins/task/listings.py b/nikola/plugins/task/listings.py
--- a/nikola/plugins/task/listings.py
+++ b/nikola/plugins/task/listings.py
@@ -134,7 +134,9 @@ def render_listing(in_name, out_name, input_folder, output_folder, folders=[], f
                         except Exception:
                             lexer = TextLexer()
                         fd.seek(0)
-                    code = highlight(fd.read(), lexer, utils.NikolaPygmentsHTML(in_name))
+                    code = highlight(
+                        fd.read(), lexer,
+                        utils.NikolaPygmentsHTML(in_name, linenos='table'))
                 title = os.path.basename(in_name)
             else:
                 code = ''
diff --git a/nikola/utils.py b/nikola/utils.py
--- a/nikola/utils.py
+++ b/nikola/utils.py
@@ -1629,7 +1629,11 @@ def __init__(self, anchor_ref=None, classes=None, **kwargs):
                 anchor_ref, lang=LocaleBorg().current_lang, force=True)
         self.nclasses = classes
         kwargs['cssclass'] = 'code'
+        if not kwargs.get('linenos'):
+            # Default to no line numbers (Issue #3426)
+            kwargs['linenos'] = False
         if kwargs.get('linenos') not in {'table', 'inline', 'ol', False}:
+            # Map invalid values to table
             kwargs['linenos'] = 'table'
         kwargs['anchorlinenos'] = kwargs['linenos'] == 'table'
         kwargs['nowrap'] = False


Test patch:
Repository: getnikola/nikola

Issue:
The post_list plugin prevents 'else' functionality in templates
<!--
Before creating an issue:
* make sure you are using an up-to-date version of Nikola
* search for existing issues that might be related

Make sure to:
* provide information about your environment (below)
* include all the output you get, and any other information related to your problem

Nikola v7.6.4, as provided by Ubuntu, is NOT SUPPORTED.
If you are using this version, you should upgrade: https://getnikola.com/getting-started.html
-->

### Environment

**Python Version:**

3.7.8

**Nikola Version:**

8.1.1

**Operating System:**

Mac OS Catalina (10.15.5) / Ubuntu 19.10 

### Description:

In the default template for the `post-list` plugin, namely `post_list_directive.tmpl`

```python
{% if posts %}
    <ul class="post-list">
    ...
```
Which suggests that there is some possibility that the template will be called with no posts.

While in `list_post.tmpl`, which you can also use with `post-list`, we have this:

```python
    {% if posts %}
    <ul class="postlist">
    {% for post in posts %}
        <li><time class="listdate" datetime="{{ post.formatted_date('webiso') }}" title="{{ post.formatted_date(date_format)|e }}">{{ post.formatted_date(date_format)|e }}</time> <a href="{{ post.permalink() }}" class="listtitle">{{ post.title()|e }}</a></li>
    {% endfor %}
    </ul>
    {% else %}
    <p>{{ messages("No posts found.") }}</p>
    {% endif %}
```

Which is obviously expected to be able to handle the situation when there are no posts.

However, when the plugin returns no posts, the `else` block is not executed. In fact, it appears that the template is not called at all when no posts are returned. 

This is because of these lines in `post_list.py`, at around lines 221-222:

```python
        if not posts:
            return '', []
```

It seems that because the empty values are returned, processing is not passed to the template. Removing those lines fixes the problem and allows the template's `else` clause to work.

I can't see that this change breaks anything else, so I'll submit a pull request for it, unless someone has an objection.






Discussion:

Resolving patch:
diff --git a/nikola/plugins/shortcode/post_list.py b/nikola/plugins/shortcode/post_list.py
--- a/nikola/plugins/shortcode/post_list.py
+++ b/nikola/plugins/shortcode/post_list.py
@@ -218,9 +218,6 @@ def handler(self, start=None, stop=None, reverse=False, tags=None, require_all_t
 
             posts += [post]
 
-        if not posts:
-            return '', []
-
         template_deps = site.template_system.template_deps(template)
         if state:
             # Register template as a dependency (Issue #2391)


Test patch: