internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: getnikola/nikola

Issue:
post-list vs post_list
Is it `post-list` or `post_list`?

With 8.0.0ebta1, this:
```
{{% post-list categories=phd %}}{{% /post-list %}}
```

 gives:

```
[2018-05-09T17:21:03Z] ERROR: Nikola: Unknown shortcode post-list (started at line 1, column 1)
```

This works:

```
{{% post_list categories=phd %}}{{% /post_list %}}
```

I don't know if this is a RST vs Markdown difference but the manual shows `post-list` even for the non-RST example.



Discussion:
Good question. I suppose it's better to use post_list since post-list could be problematic in some contexts.

In any case we can configure the shortcode and the directive to accept both names.
There might be a reason for using dashes in the reST thing, or there might not. But yes, supporting both ways would be nice if possible.

Resolving patch:
diff --git a/nikola/plugins/compile/rest/post_list.py b/nikola/plugins/compile/rest/post_list.py
--- a/nikola/plugins/compile/rest/post_list.py
+++ b/nikola/plugins/compile/rest/post_list.py
@@ -46,6 +46,7 @@ def set_site(self, site):
         """Set Nikola site."""
         self.site = site
         directives.register_directive('post-list', PostListDirective)
+        directives.register_directive('post_list', PostListDirective)
         PostListDirective.site = site
         return super(Plugin, self).set_site(site)
 
diff --git a/nikola/plugins/shortcode/post_list.py b/nikola/plugins/shortcode/post_list.py
--- a/nikola/plugins/shortcode/post_list.py
+++ b/nikola/plugins/shortcode/post_list.py
@@ -120,6 +120,11 @@ class PostListShortcode(ShortcodePlugin):
 
     name = "post_list"
 
+    def set_site(self, site):
+        """Set the site."""
+        super(PostListShortcode, self).set_site(site)
+        site.register_shortcode('post-list', self.handler)
+
     def handler(self, start=None, stop=None, reverse=False, tags=None, require_all_tags=False, categories=None,
                 sections=None, slugs=None, post_type='post', type=False,
                 lang=None, template='post_list_directive.tmpl', sort=None,


Test patch:
Repository: getnikola/nikola

Issue:
should keywords be treated as string (for smartjoin)
**Nikola Version:** 8.0.0beta1

I get the following:
```
    callable_(context, *args, **kwargs)
  File "/home/tbm/homepage/cache/.mako.tmp/base.tmpl.py", line 73, in render_body
    context['self'].extra_head(**pageargs)
  File "/home/tbm/homepage/cache/.mako.tmp/post.tmpl.py", line 164, in render_extra_head
    __M_writer(filters.html_escape(str(smartjoin(', ', post.meta('keywords')))))
  File "/home/tbm/.local/lib/python3.5/site-packages/nikola/utils.py", line 1884, in smartjoin
    return join_char.join(string_or_iterable)
TypeError: sequence item 4: expected str instance, int found
```

caused by:
```
keywords: [Debian, QNAP, TS-409, Orion, 5281]
```

5281 is interpreted as an integer instead of a string.  I wonder if all keywords should be converted to strings.



Discussion:
Yeah, all strings makes sense.
How about changing `smartjoin` as follows:
- if the input is an array or iterable, convert all arguments to strings with `str()` and join them;
- if the input is anything else, apply `str()` and return it.

This is a bit more general, and won't fail if the input happens to be an integer instead of a string (which can happen with other metadata formats, and keywords/tags such as `404`).
@felixfontein :+1:

Anyone wants to do this?
I'll do it.
Thank you @Kwpolska and @felixfontein!

Resolving patch:
diff --git a/nikola/utils.py b/nikola/utils.py
--- a/nikola/utils.py
+++ b/nikola/utils.py
@@ -69,7 +69,7 @@
     husl = None
 
 from blinker import signal
-from collections import defaultdict, Callable, OrderedDict
+from collections import defaultdict, Callable, OrderedDict, Iterable
 from importlib import reload as _reload
 from logbook.compat import redirect_logging
 from logbook.more import ExceptionHandler, ColorizedStderrHandler
@@ -1870,17 +1870,21 @@ def sort_posts(posts, *keys):
 
 
 def smartjoin(join_char: str, string_or_iterable) -> str:
-    """Join string_or_iterable with join_char if it is not a string already.
+    """Join string_or_iterable with join_char if it is iterable; otherwise converts it to string.
 
     >>> smartjoin('; ', 'foo, bar')
     'foo, bar'
     >>> smartjoin('; ', ['foo', 'bar'])
     'foo; bar'
+    >>> smartjoin(' to ', ['count', 42])
+    'count to 42'
     """
     if isinstance(string_or_iterable, (unicode_str, bytes_str)):
         return string_or_iterable
+    elif isinstance(string_or_iterable, Iterable):
+        return join_char.join([str(e) for e in string_or_iterable])
     else:
-        return join_char.join(string_or_iterable)
+        return str(string_or_iterable)
 
 
 def _smartjoin_filter(string_or_iterable, join_char: str) -> str:


Test patch:
Repository: getnikola/nikola

Issue:
Ignore .DS_Store files
Reported by email:

```
This is my first attempt at using Nikola. I just installed 

Nikola v7.8.15 

on a Mac running OSX 10.12.6. 

I used pip install inside an Anaconda environment. Then I ran: 

$ nikola init --demo ~/nik4


and accepted all of the defaults. From inside ~/nik4:

$ nikola build

There were a series of copy_assets:... steps that seemed to be fine, but then I got an error on the render_listings step: 

.  copy_assets:output/assets/css/code.css

.  render_listings:output/listings/index.html

.  render_listings:output/listings/.DS_Store.html

TaskError - taskid:render_listings:output/listings/.DS_Store.html

PythonAction Error

Traceback (most recent call last):

  File "/anaconda3/lib/python3.6/site-packages/doit/action.py", line 424, in execute

    returned_value = self.py_callable(*self.args, **kwargs)

  File "/anaconda3/lib/python3.6/site-packages/nikola/plugins/task/listings.py", line 138, in render_listing

    code = highlight(fd.read(), lexer, utils.NikolaPygmentsHTML(in_name))

  File "/anaconda3/lib/python3.6/codecs.py", line 321, in decode

    (result, consumed) = self._buffer_decode(data, self.errors, final)

UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb7 in position 309: invalid start byte



########################################

render_listings:output/listings/.DS_Store.html <stdout>:




Any suggestions about what I can try next? I made sure not to use Finder, but OSX creates the .DS_Store files anyway. 
```


Discussion:

Resolving patch:
diff --git a/nikola/plugins/task/listings.py b/nikola/plugins/task/listings.py
--- a/nikola/plugins/task/listings.py
+++ b/nikola/plugins/task/listings.py
@@ -222,6 +222,8 @@ def render_listing(in_name, out_name, input_folder, output_folder, folders=[], f
                     'clean': True,
                 }, self.kw["filters"])
                 for f in files:
+                    if f == '.DS_Store':
+                        continue
                     ext = os.path.splitext(f)[-1]
                     if ext in ignored_extensions:
                         continue


Test patch:
Repository: getnikola/nikola

Issue:
remove monkey-patched DoitCommand.help
Nikola has a monkey-patched version of DoitCommand.help method [1].
This should not be necessary anymore since doit 0.31 after see https://github.com/pydoit/doit/issues/184

I am doing some changes (WIP) on the help mechanism https://github.com/pydoit/doit/issues/261. 
With this monkey-patch you guys will not benefit from the new code...
 and I want to remove `opt.help_doc()` (that would break your code).

[1] https://github.com/getnikola/nikola/blame/8a71bee3e215ae201eff3830476e382731b8f95f/nikola/plugin_categories.py#L158




Discussion:

Resolving patch:
diff --git a/nikola/plugin_categories.py b/nikola/plugin_categories.py
--- a/nikola/plugin_categories.py
+++ b/nikola/plugin_categories.py
@@ -30,6 +30,7 @@
 import os
 import sys
 
+import doit
 import logbook
 from doit.cmd_base import Command as DoitCommand
 from yapsy.IPlugin import IPlugin
@@ -173,7 +174,9 @@ def help(self):
     return "\n".join(text)
 
 
-DoitCommand.help = help
+# we need to patch DoitCommand.help with doit <0.31.0
+if doit.__version__ < (0, 31, 0):
+    DoitCommand.help = help
 
 
 class BaseTask(BasePlugin):


Test patch:
Repository: getnikola/nikola

Issue:
gallery: don't generate index page on empty output folder
Empty output folder allows galleries to be placed at the site's top directory,
instead of inside e.g. a "galleries" subdirectory.  But without this change, the
plugin would want to generate an index page that collides with the site's main
index page.

### Pull Request Checklist

- [X] I’ve read the [guidelines for contributing](https://github.com/getnikola/nikola/blob/master/CONTRIBUTING.rst).
- [X] I updated AUTHORS.txt and CHANGES.txt (if the change is non-trivial) and documentation (if applicable).
- [X] I tested my changes.

### Description

I don't think this requires a documentation change?  Attempting to use an empty output folder for galleries I think just follows naturally without needing it explicitly mentioned?


Discussion:

Resolving patch:
diff --git a/nikola/plugins/task/galleries.py b/nikola/plugins/task/galleries.py
--- a/nikola/plugins/task/galleries.py
+++ b/nikola/plugins/task/galleries.py
@@ -366,7 +366,12 @@ def find_galleries(self):
         self.gallery_list = []
         for input_folder, output_folder in self.kw['gallery_folders'].items():
             for root, dirs, files in os.walk(input_folder, followlinks=True):
-                self.gallery_list.append((root, input_folder, output_folder))
+                # If output folder is empty, the top-level gallery
+                # index will collide with the main page for the site.
+                # Don't generate the top-level gallery index in that
+                # case.
+                if output_folder or root != input_folder:
+                    self.gallery_list.append((root, input_folder, output_folder))
 
     def create_galleries_paths(self):
         """Given a list of galleries, put their paths into self.gallery_links."""


Test patch:
Repository: getnikola/nikola

Issue:
Responsive youtube embed
I'm trying to set an embedded youtube link to 100% width in a .rst file. Is this possible?

I've tried:

```
.. youtube:: 3XsQCkF1SrE
   :align: center
   :width: 100%
```



Discussion:
We just set width="whatever you type". Check if the `width` attribute of iframes can take a `100%` value, and perhaps check how other people achieved that 100% width?
@Kwpolska Thanks for your reply!

I get this error:
```
System Message: ERROR/3 (<string>, line 5)

Error in "youtube" directive: invalid option value: (option: "width"; value: '"100%"') invalid literal for int() with base 10: '100%'.

.. youtube:: 3XsQCkF1SrE
   :align: center
   :width: "100%"
```
(I also tried "100%")

http://www.tagindex.net/html/frame/i_width.html tells me iframes can take %s as width. 
We declare the type of `width` and `height` to be integers, perhaps we should make them strings instead. Can you make a PR for this?
Yes I can :) thanks for your help!

Resolving patch:
diff --git a/nikola/plugins/compile/rest/youtube.py b/nikola/plugins/compile/rest/youtube.py
--- a/nikola/plugins/compile/rest/youtube.py
+++ b/nikola/plugins/compile/rest/youtube.py
@@ -67,8 +67,8 @@ class Youtube(Directive):
     has_content = True
     required_arguments = 1
     option_spec = {
-        "width": directives.positive_int,
-        "height": directives.positive_int,
+        "width": directives.unchanged,
+        "height": directives.unchanged,
         "align": _align_choice
     }
 
@@ -80,7 +80,7 @@ def run(self):
             'width': 560,
             'height': 315,
         }
-        options.update(self.options)
+        options.update({k: v for k, v in self.options.items() if v})
         if self.options.get('align') in _align_options_base:
             options['align'] = ' align-' + self.options['align']
         else:


Test patch:
Repository: getnikola/nikola

Issue:
Add option to use --cloak-email-addresses feature in rst2html
### Description:

Per recommendation from ralsina in the IRC chatroom, I am requesting an additional option to use the `cloak-email-addresses` feature provided by rst2html. 


Discussion:
I think it would make more sense to support the option syntax from docutils' config file in nikola's conf.py instead (see http://docutils.sourceforge.net/docs/user/config.html#example)

In fact ... if you put that in a docutils.conf right next to conf.py ... it should work, I think?
Confirmed. I honestly didn't think to check whether docutils was configurable this way. Nice!
Actually, my quick test (just running `rst2html` directly) doesn't bear out when actually running `nikola build`. For whatever reason, the `docutils.conf` isn't be honored.
Ok then, I will make it happen.
Great!

Out of curiosity, you mentioned a couple of options initially. It seems like the best approach uses docutils.conf rather than adding anything to conf.py. Is that your plan?
Not sure yet :-)

On Tue, Dec 11, 2018 at 11:27 AM Daniel Moch <notifications@github.com>
wrote:

> Great!
>
> Out of curiosity, you mentioned a couple of options initially. It seems
> like the best approach uses docutils.conf rather than adding anything to
> conf.py. Is that your plan?
>
> —
> You are receiving this because you were assigned.
> Reply to this email directly, view it on GitHub
> <https://github.com/getnikola/nikola/issues/3188#issuecomment-446220816>,
> or mute the thread
> <https://github.com/notifications/unsubscribe-auth/AAAGK-Or0EnsVY8zYFCm1QzbBOTj28ieks5u38DRgaJpZM4ZBwZZ>
> .
>


Resolving patch:
diff --git a/nikola/plugins/compile/rest/__init__.py b/nikola/plugins/compile/rest/__init__.py
--- a/nikola/plugins/compile/rest/__init__.py
+++ b/nikola/plugins/compile/rest/__init__.py
@@ -230,6 +230,8 @@ def observer(msg):
 class NikolaReader(docutils.readers.standalone.Reader):
     """Nikola-specific docutils reader."""
 
+    config_section = 'nikola'
+
     def __init__(self, *args, **kwargs):
         """Initialize the reader."""
         self.transforms = kwargs.pop('transforms', [])
@@ -303,7 +305,7 @@ def rst2html(source, source_path=None, source_class=docutils.io.StringInput,
              destination_path=None, reader=None,
              parser=None, parser_name='restructuredtext', writer=None,
              writer_name='html', settings=None, settings_spec=None,
-             settings_overrides=None, config_section=None,
+             settings_overrides=None, config_section='nikola',
              enable_exit_status=None, logger=None, l_add_ln=0, transforms=None):
     """Set up & run a ``Publisher``, and return a dictionary of document parts.
 


Test patch:
Repository: getnikola/nikola

Issue:
Strange rendering of markdown pages
As discussed on IRC: I'm running last nikola (7d142c907b761db03eea160956635b94503b1a8b), with python 3.6.7 (Ubuntu 18.04).

When generating my site https://github.com/hardening/hardening-consulting.com (mdx_video module required), I noticed a strange rendering bug, if I change the beginning of the post (https://github.com/hardening/hardening-consulting.com/blob/master/posts/2018-twisted-tricks.wp) to

```
<img class="alignright" src="/images/twisted.png" width="100px">

En travaillant sur [topka][topka], j'ai eu l'occasion d'utiliser [twisted][twisted]. Les tutoriaux de twisted
```
(`&nbsp;` removed) then the `img` tag is not rendered in the output page. 

Even more strange, if I move lower the `img` in the post, then `img` is not rendered and the text around is removed. Most of my other posts have the same structure and work fine.


Discussion:
just curious, is it reproducible elsewhere ?
Reproduced. It looks like the first paragraph gets removed. This should not happen with 2-file posts.
This was happening only in non-translated posts on your blog. Since Nikola couldn’t find a `.en.meta` file, it set the status for the post to one-file. In one-file mode, we delete everything before the first empty line in output to hide the metadata from view.

Fixed by making the one-file decision only basing on the default language, unless we manage to find a non-default two-file post. PR #3195.

Resolving patch:
diff --git a/nikola/post.py b/nikola/post.py
--- a/nikola/post.py
+++ b/nikola/post.py
@@ -1029,8 +1029,7 @@ def get_meta(post, lang):
     # If meta file exists, use it
     metafile_meta, used_extractor = get_metadata_from_meta_file(post.metadata_path, post, config, lang, metadata_extractors_by)
 
-    if not metafile_meta:
-        post.is_two_file = False
+    is_two_file = bool(metafile_meta)
 
     # Fetch compiler metadata.
     compiler_meta = {}
@@ -1042,7 +1041,7 @@ def get_meta(post, lang):
         meta.update(compiler_meta)
 
     # Meta files and inter-file metadata override compiler metadata
-    if not post.is_two_file:
+    if not metafile_meta:
         new_meta, used_extractor = get_metadata_from_file(post.source_path, post, config, lang, metadata_extractors_by)
         meta.update(new_meta)
     else:
@@ -1068,6 +1067,10 @@ def get_meta(post, lang):
             meta['title'] = os.path.splitext(
                 os.path.basename(post.source_path))[0]
 
+    # Set one-file status basing on default language only (Issue #3191)
+    if is_two_file or lang is None:
+        post.is_two_file = is_two_file
+
     return meta, used_extractor
 
 


Test patch:
Repository: getnikola/nikola

Issue:
new_post cannot use a directory name as a path
<!--
Before creating an issue:
* make sure you are using an up-to-date version of Nikola
* search for existing issues that might be related

Make sure to:
* provide information about your environment (below)
* include all the output you get, and any other information related to your problem

Nikola v7.6.4, as provided by Ubuntu, is NOT SUPPORTED.
If you are using this version, you should upgrade: https://getnikola.com/getting-started.html
-->

### Environment

**Python Version:** Python 3.7.1

**Nikola Version:** Nikola v8.0.1

**Operating System:** Windows 10

### Description:

The command "nikola new_post directory" is supposed to create a new post in the specified directory and append the filename for you. However using it causes the error:

    ERROR: new_post: Unknown post extension , maybe you need to install a plugin or enable an existing one?

There is no error when the file format is specified with -f.

### Solution:

The possibility to create new posts with only a directory as a path was added with commit https://github.com/getnikola/nikola/commit/ca1ab1bc9a058dcfeaec2efdc1e7790ef7e56b4d (see Issue https://github.com/getnikola/nikola/issues/2651). It seems that the later commit https://github.com/getnikola/nikola/pull/2799/commits/274cf88dde014bbca506fa42a0eab816689a0404 (see Issue https://github.com/getnikola/nikola/issues/2798) is causing the bug: it checks if the "path" variable is empty and it assumes a non-empty variable always contains a file extension. Adding a clause for this specific case would solve the issue.


Discussion:

Resolving patch:
diff --git a/nikola/plugins/command/new_post.py b/nikola/plugins/command/new_post.py
--- a/nikola/plugins/command/new_post.py
+++ b/nikola/plugins/command/new_post.py
@@ -264,7 +264,7 @@ def _execute(self, options, args):
         if "@" in content_format:
             content_format, content_subformat = content_format.split("@")
 
-        if not content_format and path:
+        if not content_format and path and not os.path.isdir(path):
             # content_format not specified. If path was given, use
             # it to guess (Issue #2798)
             extension = os.path.splitext(path)[-1]


Test patch:
Repository: getnikola/nikola

Issue:
Cannot use "ignored_assets" properly on Windows with custom themes
<!--
Before creating an issue:
* make sure you are using an up-to-date version of Nikola
* search for existing issues that might be related

Make sure to:
* provide information about your environment (below)
* include all the output you get, and any other information related to your problem

Nikola v7.6.4, as provided by Ubuntu, is NOT SUPPORTED.
If you are using this version, you should upgrade: https://getnikola.com/getting-started.html
-->

### Environment

**Python Version:** Python 3.7.1

**Nikola Version:** Nikola v8.0.1

**Operating System:** Windows 10

### Description:

The option "ignored_assets" used in theme meta files does not work properly on Windows: the relative path is only recognized when using a backslash, i.e. css\theme.css instead of css/theme.css ([see doc](https://getnikola.com/theming.html#theme-meta-files)).

### Solution:

Normalize the path to the convention of the operating system when parsing the theme meta file.


Discussion:

Resolving patch:
diff --git a/nikola/plugins/task/copy_assets.py b/nikola/plugins/task/copy_assets.py
--- a/nikola/plugins/task/copy_assets.py
+++ b/nikola/plugins/task/copy_assets.py
@@ -67,7 +67,7 @@ def gen_tasks(self):
         theme_ini = utils.parse_theme_meta(main_theme)
         if theme_ini:
             ignored_assets = theme_ini.get("Nikola", "ignored_assets", fallback='').split(',')
-            ignored_assets = [asset_name.strip() for asset_name in ignored_assets]
+            ignored_assets = [os.path.normpath(asset_name.strip()) for asset_name in ignored_assets]
         else:
             ignored_assets = []
 


Test patch:
Repository: getnikola/nikola

Issue:
Make ARCHIVE_ settings translatable
Currently it seems that the settings ARCHIVE_PATH and ARCHIVE_FILENAME can not be translated. In line with settings like INDEX_PATH it would be nice to be able to translate this. Currently I set ARCHIVE_FILENAME to "index.html" so that it is not visible, but the path will still be the same in all languages.

Thank you for your consideration!


Discussion:

Resolving patch:
diff --git a/nikola/nikola.py b/nikola/nikola.py
--- a/nikola/nikola.py
+++ b/nikola/nikola.py
@@ -616,6 +616,8 @@ def __init__(self, **config):
                                       'INDEXES_PRETTY_PAGE_URL',
                                       'THEME_CONFIG',
                                       # PATH options (Issue #1914)
+                                      'ARCHIVE_PATH',
+                                      'ARCHIVE_FILENAME',
                                       'TAG_PATH',
                                       'TAGS_INDEX_PATH',
                                       'CATEGORY_PATH',
diff --git a/nikola/plugins/task/archive.py b/nikola/plugins/task/archive.py
--- a/nikola/plugins/task/archive.py
+++ b/nikola/plugins/task/archive.py
@@ -126,12 +126,12 @@ def get_classification_friendly_name(self, classification, lang, only_last_compo
 
     def get_path(self, classification, lang, dest_type='page'):
         """Return a path for the given classification."""
-        components = [self.site.config['ARCHIVE_PATH']]
+        components = [self.site.config['ARCHIVE_PATH'](lang)]
         if classification:
             components.extend(classification)
             add_index = 'always'
         else:
-            components.append(self.site.config['ARCHIVE_FILENAME'])
+            components.append(self.site.config['ARCHIVE_FILENAME'](lang))
             add_index = 'never'
         return [_f for _f in components if _f], add_index
 


Test patch: