internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: getnikola/nikola

Issue:
reST listings should also have source links
This will make it easy to download listings displayed in posts.



Discussion:
what does this mean ?

#2452 #1995 c7106a3e3c7ad7730416ea3666a03b8f6516c93c but in reST listings (inside posts).


Resolving patch:
diff --git a/nikola/plugins/compile/rest/listing.py b/nikola/plugins/compile/rest/listing.py
--- a/nikola/plugins/compile/rest/listing.py
+++ b/nikola/plugins/compile/rest/listing.py
@@ -136,6 +136,7 @@ def set_site(self, site):
         # leaving these to make the code directive work with
         # docutils < 0.9
         CodeBlock.site = site
+        Listing.site = site
         directives.register_directive('code', CodeBlock)
         directives.register_directive('code-block', CodeBlock)
         directives.register_directive('sourcecode', CodeBlock)
@@ -189,8 +190,11 @@ def run(self):
             self.content = fileobject.read().splitlines()
         self.state.document.settings.record_dependencies.add(fpath)
         target = urlunsplit(("link", 'listing', fpath.replace('\\', '/'), '', ''))
+        src_target = urlunsplit(("link", 'listing_source', fpath.replace('\\', '/'), '', ''))
+        src_label = self.site.MESSAGES('Source')
         generated_nodes = (
-            [core.publish_doctree('`{0} <{1}>`_'.format(_fname, target))[0]])
+            [core.publish_doctree('`{0} <{1}>`_  `({2}) <{3}>`_' .format(
+                _fname, target, src_label, src_target))[0]])
         generated_nodes += self.get_code_from_file(fileobject)
         return generated_nodes
 
diff --git a/nikola/plugins/task/listings.py b/nikola/plugins/task/listings.py
--- a/nikola/plugins/task/listings.py
+++ b/nikola/plugins/task/listings.py
@@ -54,6 +54,7 @@ def register_output_name(self, input_folder, rel_name, rel_output_name):
     def set_site(self, site):
         """Set Nikola site."""
         site.register_path_handler('listing', self.listing_path)
+        site.register_path_handler('listing_source', self.listing_source_path)
 
         # We need to prepare some things for the listings path handler to work.
 
@@ -255,6 +256,22 @@ def render_listing(in_name, out_name, input_folder, output_folder, folders=[], f
                             'clean': True,
                         }, self.kw["filters"])
 
+    def listing_source_path(self, name, lang):
+        """A link to the source code for a listing.
+
+        It will try to use the file name if it's not ambiguous, or the file path.
+
+        Example:
+
+        link://listing_source/hello.py => /listings/tutorial/hello.py
+
+        link://listing_source/tutorial/hello.py => /listings/tutorial/hello.py
+        """
+        result = self.listing_path(name, lang)
+        if result[-1].endswith('.html'):
+            result[-1] = result[-1][:-5]
+        return result
+
     def listing_path(self, namep, lang):
         """A link to a listing.
 


Test patch:
Repository: getnikola/nikola

Issue:
Nikola generates invalid html5 when markdown footnote extension is used
The [default output type](http://pythonhosted.org/Markdown/reference.html#markdown) of the python markdown library is xhtml1. The 4 templates that ship with Nikola have <!DOCTYPE html> which defines them as html5, so I'm assuming that we're intending to generate html5.

When the footnote markdown extension is used, it generates invalid html5 according to the w3c validator.

`<a class="footnote-ref" href="..." rev="footnote">...</a>`

(rev="footnote" is valid html4, but not html5)

The markdown library indicates that this is invalid html5 (https://github.com/waylan/Python-Markdown/blob/master/markdown/extensions/footnotes.py#L149) so we can trigger the correct behaviour by setting the output_format.

Given the markdown library does not make much use of the output_format variable, I don't think this is likely to materially change the output for many people at all - https://github.com/waylan/Python-Markdown/search?utf8=%E2%9C%93&q=output_format)



Discussion:

Resolving patch:
diff --git a/nikola/plugins/compile/markdown/__init__.py b/nikola/plugins/compile/markdown/__init__.py
--- a/nikola/plugins/compile/markdown/__init__.py
+++ b/nikola/plugins/compile/markdown/__init__.py
@@ -78,7 +78,7 @@ def compile_html(self, source, dest, is_two_file=True):
                 data = in_file.read()
             if not is_two_file:
                 _, data = self.split_metadata(data)
-            output = markdown(data, self.extensions)
+            output = markdown(data, self.extensions, output_format="html5")
             output, shortcode_deps = self.site.apply_shortcodes(output, filename=source, with_dependencies=True, extra_context=dict(post=post))
             out_file.write(output)
         if post is None:


Test patch:
Repository: getnikola/nikola

Issue:
Plugin dependencies
Plugins (from the plugins repo) should be able to depend on each other.

Plugin dependencies
Plugins (from the plugins repo) should be able to depend on each other.



Discussion:
About implementing this... just an extra plugin metadata field? That should do the trick, right? Then the implementation in the plugin installer just calls itself again for all dependencies.

@Kwpolska if that sounds good, I can do it quick

Or a `requirements-plugins.txt` file. Go ahead.

About implementing this... just an extra plugin metadata field? That should do the trick, right? Then the implementation in the plugin installer just calls itself again for all dependencies.

@Kwpolska if that sounds good, I can do it quick

Or a `requirements-plugins.txt` file. Go ahead.


Resolving patch:
diff --git a/nikola/plugins/command/plugin.py b/nikola/plugins/command/plugin.py
--- a/nikola/plugins/command/plugin.py
+++ b/nikola/plugins/command/plugin.py
@@ -265,6 +265,7 @@ def do_install(self, url, name, show_install_notes=True):
                       'package manager.')
             else:
                 LOGGER.info('Dependency installation succeeded.')
+
         reqnpypath = os.path.join(dest_path, 'requirements-nonpy.txt')
         if os.path.exists(reqnpypath):
             LOGGER.notice('This plugin has third-party '
@@ -280,6 +281,25 @@ def do_install(self, url, name, show_install_notes=True):
 
             print('You have to install those yourself or through a package '
                   'manager.')
+
+        req_plug_path = os.path.join(dest_path, 'requirements-plugins.txt')
+        if os.path.exists(req_plug_path):
+            LOGGER.notice('This plugin requires other Nikola plugins.')
+            LOGGER.info('Installing plugins using Nikola...')
+            print('Contents of the requirements-nonpy.txt file:\n')
+            try:
+                with io.open(req_plug_path, 'r', encoding='utf-8') as inf:
+                    for plugname in inf.readlines():
+                        self.do_install(url, plugname, show_install_notes)
+            except subprocess.CalledProcessError:
+                LOGGER.error('Could not install a plugin.')
+                print('Contents of the requirements-plugins.txt file:\n')
+                with io.open(req_plug_path, 'r', encoding='utf-8') as fh:
+                    print(utils.indent(fh.read(), 4 * ' '))
+                print('You have to install those yourself manually.')
+            else:
+                LOGGER.info('Dependency installation succeeded.')
+
         confpypath = os.path.join(dest_path, 'conf.py.sample')
         if os.path.exists(confpypath) and show_install_notes:
             LOGGER.notice('This plugin has a sample config file.  Integrate it with yours in order to make this plugin work!')


Test patch:
Repository: getnikola/nikola

Issue:
url_replacer() mangles bare hash links
`<a href="#">something</a>` is mangled into `<a href="index.html">something</a>`, as is `<a href="#a">something</a>` unless there exists an element with `id="a"`. This causes a lot of problems for JavaScript-powered links. I have a simple example patch that fixes this.


Discussion:

Resolving patch:
diff --git a/nikola/nikola.py b/nikola/nikola.py
--- a/nikola/nikola.py
+++ b/nikola/nikola.py
@@ -1381,6 +1381,10 @@ def url_replacer(self, src, dst, lang=None, url_type=None):
         lang is used for language-sensitive URLs in link://
         url_type is used to determine final link appearance, defaulting to URL_TYPE from config
         """
+        # Avoid mangling links within the page
+        if dst.startswith('#'):
+            return dst
+
         parsed_src = urlsplit(src)
         src_elems = parsed_src.path.split('/')[1:]
         dst_url = urlparse(dst)


Test patch:
Repository: getnikola/nikola

Issue:
Archives (CREATE_MONTHLY_ARCHIVE = True) think all months are "None"
https://irclogs.getnikola.com/2015/

To reproduce locally: get https://github.com/getnikola/irclogs-site create any random files in `posts/*.rst`.


Discussion:

Resolving patch:
diff --git a/nikola/plugins/task/archive.py b/nikola/plugins/task/archive.py
--- a/nikola/plugins/task/archive.py
+++ b/nikola/plugins/task/archive.py
@@ -97,7 +97,7 @@ def get_classification_friendly_name(self, classification, lang, only_last_compo
         elif len(classification) == 1:
             return classification[0]
         elif len(classification) == 2:
-            nikola.utils.LocaleBorg().get_month_name(int(classification[1]), lang)
+            return nikola.utils.LocaleBorg().get_month_name(int(classification[1]), lang)
         else:
             # Fallback
             return '/'.join(classification)


Test patch:
Repository: getnikola/nikola

Issue:
Post.text with strip_html=True fails for empty posts
```pytb
Scanning posts......done!
.  local_search:output/assets/js/tipuesearch_content.json
########################################
TaskError - taskid:local_search:output/assets/js/tipuesearch_content.json
PythonAction Error
Traceback (most recent call last):
  File "/home/kwpolska/virtualenvs/nikola-py3/lib/python3.5/site-packages/doit/action.py", line 383, in execute
    returned_value = self.py_callable(*self.args, **kwargs)
  File "/home/kwpolska/nikola-logs/plugins/localsearch/localsearch/__init__.py", line 77, in save_data
    text = post.text(lang, strip_html=True)
  File "/home/kwpolska/git/nikola/nikola/post.py", line 677, in text
    data = content.text_content().strip()  # No whitespace wanted.
  File "/home/kwpolska/virtualenvs/nikola-py3/lib/python3.5/site-packages/lxml/html/__init__.py", line 411, in text_content
    return _collect_string_content(self)
  File "src/lxml/xpath.pxi", line 433, in lxml.etree.XPath.__call__ (src/lxml/lxml.etree.c:168402)
  File "src/lxml/apihelpers.pxi", line 67, in lxml.etree._rootNodeOrRaise (src/lxml/lxml.etree.c:15759)
ValueError: Input object is not an XML element: HtmlComment
```

Caused by this file:

```html
<!--
.. title: Search
.. slug: search
.. date: 2015-07-20 14:18:28 UTC
.. tags:
.. category:
.. link:
.. description:
.. type: text
-->
```

To fix this, all I had to do was to add a newline (which was probably triggering the metadata comment removal thing), but we should protect against those crashes.


Discussion:

Resolving patch:
diff --git a/nikola/post.py b/nikola/post.py
--- a/nikola/post.py
+++ b/nikola/post.py
@@ -687,7 +687,7 @@ def text(self, lang=None, teaser_only=False, strip_html=False, show_read_more_li
                 # Not all posts have a body. For example, you may have a page statically defined in the template that does not take content as input.
                 content = lxml.html.fromstring(data)
                 data = content.text_content().strip()  # No whitespace wanted.
-            except lxml.etree.ParserError:
+            except (lxml.etree.ParserError, ValueError):
                 data = ""
         elif data:
             if self.demote_headers:


Test patch:
Repository: getnikola/nikola

Issue:
Per page pretty url config is not working
I have disabled pretty urls in `conf.py`

```py
PRETTY_URLS = False
```

For some posts, I don't want this, so I have set 

```
.. pretty_url: True
```

But Nikola is ignoring this.


Discussion:
I think it only works the other way around: if you set `PRETTY_URLS` to `True`, you can disable it for a specific post by specifying `.. pretty_url: False`. At least that's what the code does. From what I understand, that's also what the manual is saying.
This seems to be intentional, and our docs state that you can *disable* PRETTY_URLS on a per-page basic. However, adding support should not be hard — but is that necessary? If you need URLs like those only for a few pages, you can create those pretty easily with `posts/foo/index.rst` with the slug `index`.

Resolving patch:
diff --git a/nikola/post.py b/nikola/post.py
--- a/nikola/post.py
+++ b/nikola/post.py
@@ -277,12 +277,13 @@ def __repr__(self):
         return '<Post: {0!r} {1}>'.format(self.source_path, m.hexdigest())
 
     def _has_pretty_url(self, lang):
-        if self.pretty_urls and \
-                self.meta[lang].get('pretty_url', '') != 'False' and \
-                self.meta[lang]['slug'] != 'index':
-            return True
+        m = self.meta[lang].get('pretty_url', '')
+        if m:
+            # match is a non-empty string, overides anything
+            return m == 'True'
         else:
-            return False
+            # use PRETTY_URLS, unless the slug is 'index'
+            return self.pretty_urls and self.meta[lang]['slug'] != 'index'
 
     @property
     def is_mathjax(self):


Test patch:
Repository: getnikola/nikola

Issue:
Suggestion for adding a generator meta tag
I have been using [wappalyzer](https://www.wappalyzer.com/) for a number of years now and what I personally find impressive about it is that it gives me the opportunity to discover new programs or projects every time I browse a website; thus making me a new user to that particular set of tools, like many other users of it.

I have noticed that projects like Hugo, Blogger, Jekyll, and so forth are detected by wappalyzer, whereas Nikola websites are not.

After a couple of seconds of investigation I have noticed that Nikola does not use a `<meta name="generator" content="Nikola 7.8.1">` meta tag.

In my humble, yet honest opinion, if we added this specific meta tag in Nikola, it would broaden its user base even further.

What do you guys think?


Discussion:
I’m :+1:, but without the version number (requires rebuilding every single page).

Resolving patch:
diff --git a/nikola/nikola.py b/nikola/nikola.py
--- a/nikola/nikola.py
+++ b/nikola/nikola.py
@@ -1642,7 +1642,7 @@ def generic_rss_renderer(self, lang, title, link, description, timeline, output_
             link=utils.encodelink(link),
             description=description,
             lastBuildDate=datetime.datetime.utcnow(),
-            generator='https://getnikola.com/',
+            generator='Nikola (getnikola.com)',
             language=lang
         )
 


Test patch:
Repository: getnikola/nikola

Issue:
new install - new_page fails
steps to reproduce:

1. install nikola.
2. init a new project and cd to it's dir
3. run "nikola new_page" throws an exception.
4. however, run "nikola new_post -p" and it works just fine!??

i am puzzled, why does "new_post -p" work and "new_page" fail???

the exception trace i get is:
Traceback (most recent call last):
  File "/home/smg/.virtualenvs_py3/nikola/lib/python3.5/site-packages/doit/doit_cmd.py", line 167, in run
    return command.parse_execute(args)
  File "/home/smg/.virtualenvs_py3/nikola/lib/python3.5/site-packages/doit/cmd_base.py", line 120, in parse_execute
    return self.execute(params, args)
  File "/home/smg/.virtualenvs_py3/nikola/lib/python3.5/site-packages/nikola/plugin_categories.py", line 130, in execute
    return self._execute(options, args)
  File "/home/smg/.virtualenvs_py3/nikola/lib/python3.5/site-packages/nikola/plugins/command/new_page.py", line 113, in _execute
    return p.execute(options, args)
  File "/home/smg/.virtualenvs_py3/nikola/lib/python3.5/site-packages/nikola/plugin_categories.py", line 130, in execute
    return self._execute(options, args)
  File "/home/smg/.virtualenvs_py3/nikola/lib/python3.5/site-packages/nikola/plugins/command/new_post.py", line 244, in _execute
    date_path_opt = options['date-path']
KeyError: 'date-path'




Discussion:
oh and i'm installing latest nikola via pip...  v7.8.2,   with [extras].

Resolving patch:
diff --git a/nikola/plugins/command/new_page.py b/nikola/plugins/command/new_page.py
--- a/nikola/plugins/command/new_page.py
+++ b/nikola/plugins/command/new_page.py
@@ -107,6 +107,7 @@ def _execute(self, options, args):
         options['tags'] = ''
         options['schedule'] = False
         options['is_page'] = True
+        options['date-path'] = False
         # Even though stuff was split into `new_page`, it’s easier to do it
         # there not to duplicate the code.
         p = self.site.plugin_manager.getPluginByName('new_post', 'Command').plugin_object


Test patch:
Repository: getnikola/nikola

Issue:
Add option to disable meta generator tag
See #2620.


Discussion:
@felixfontein :+1: 

Resolving patch:
diff --git a/nikola/nikola.py b/nikola/nikola.py
--- a/nikola/nikola.py
+++ b/nikola/nikola.py
@@ -612,6 +612,7 @@ def __init__(self, **config):
             'GITHUB_DEPLOY_BRANCH': 'gh-pages',
             'GITHUB_REMOTE_NAME': 'origin',
             'GITHUB_COMMIT_SOURCE': False,  # WARNING: conf.py.in overrides this with True for backwards compatibility
+            'META_GENERATOR_TAG': True,
         }
 
         # set global_context for template rendering
@@ -1204,6 +1205,7 @@ def _set_global_context_from_config(self):
         self._GLOBAL_CONTEXT['posts_section_name'] = self.config.get('POSTS_SECTION_NAME')
         self._GLOBAL_CONTEXT['posts_section_title'] = self.config.get('POSTS_SECTION_TITLE')
         self._GLOBAL_CONTEXT['sort_posts'] = utils.sort_posts
+        self._GLOBAL_CONTEXT['meta_generator_tag'] = self.config.get('META_GENERATOR_TAG')
 
         # IPython theme configuration.  If a website has ipynb enabled in post_pages
         # we should enable the IPython CSS (leaving that up to the theme itself).


Test patch:
Repository: getnikola/nikola

Issue:
allow specifying of section only via path, then let nikola append filename
There is currently no easy way to specify a section for a new post.
`nikola new_post posts/sectionname/here-i-have-to-think-of-the-slug-of-the-title-already.md` and then specifying the Title in nikola seems error prone. It would be much nicer if one could specify the section-path only and the filename would then be appended: `nikola new_post posts/sectionname`.


Discussion:
But Nikola needs to create the file to let you edit it, there is no way around that.

There is no reason why the filename you edit and the slug have to match, tho.
What @t-animal wants, and what is feasible IMO, is for `nikola new_post posts/directory` to create `posts/directory/slug.rst` automatically.
Yes @Kwpolska is right, sorry if I did not state that clearly
I still don't get it. what would "slug" be in that case? index?
No, it would be whatever the user-provided title slugifies to. So basically, what we do when no path is specified, but with the user choosing the exact location for their new post.
maybe i used the wrong words here. when I create a new post using `nikola new_post` i am asked for a title of that post. the resulting file will be named similarly to the title, let's call that filename `f`. when i create a post in a specific section by specifying `nikola new_post posts/sectionname/<f>` I have to think of how nikola creates `f`. i think it would be easier to do `nikola new_post posts/sectionname/`, be asked for a title and let nikola decide how the new file should be called.

Resolving patch:
diff --git a/nikola/plugins/command/new_post.py b/nikola/plugins/command/new_post.py
--- a/nikola/plugins/command/new_post.py
+++ b/nikola/plugins/command/new_post.py
@@ -311,7 +311,14 @@ def _execute(self, options, args):
                     path = path.decode(sys.stdin.encoding)
                 except (AttributeError, TypeError):  # for tests
                     path = path.decode('utf-8')
-            slug = utils.slugify(os.path.splitext(os.path.basename(path))[0], lang=self.site.default_lang)
+            if os.path.isdir(path):
+                # If the user provides a directory, add the file name generated from title (Issue #2651)
+                slug = utils.slugify(title, lang=self.site.default_lang)
+                pattern = os.path.basename(entry[0])
+                suffix = pattern[1:]
+                path = os.path.join(path, slug + suffix)
+            else:
+                slug = utils.slugify(os.path.splitext(os.path.basename(path))[0], lang=self.site.default_lang)
 
         if isinstance(author, utils.bytes_str):
                 try:


Test patch:
Repository: getnikola/nikola

Issue:
Posts with paths not relative to cwd put their cache in unusual places
If posts use absolute paths, or paths that refer to parent directories (eg. when provided by custom scanners), the cache ends up appearing in weird places, eg. alongside the source files.


Discussion:

Resolving patch:
diff --git a/nikola/post.py b/nikola/post.py
--- a/nikola/post.py
+++ b/nikola/post.py
@@ -118,6 +118,9 @@ def __init__(
         self.pretty_urls = self.config['PRETTY_URLS']
         self.source_path = source_path  # posts/blah.txt
         self.post_name = os.path.splitext(source_path)[0]  # posts/blah
+        _relpath = os.path.relpath(self.post_name)
+        if _relpath != self.post_name:
+            self.post_name = _relpath.replace('..' + os.sep, '_..' + os.sep)
         # cache[\/]posts[\/]blah.html
         self.base_path = os.path.join(self.config['CACHE_FOLDER'], self.post_name + ".html")
         # cache/posts/blah.html


Test patch:
Repository: getnikola/nikola

Issue:
`nikola plugin --list-installed` is hard to read
The readability of the output of `nikola plugin --list-installed` can be improved by adding colors, e.g., plugin names have different color than plugin paths.


Discussion:

Resolving patch:
diff --git a/nikola/plugins/command/plugin.py b/nikola/plugins/command/plugin.py
--- a/nikola/plugins/command/plugin.py
+++ b/nikola/plugins/command/plugin.py
@@ -179,9 +179,18 @@ def list_installed(self):
         plugins.sort()
         print('Installed Plugins:')
         print('------------------')
+        maxlength = max(len(i[0]) for i in plugins)
+        if self.site.colorful:
+            formatstring = '\x1b[1m{0:<{2}}\x1b[0m  at {1}'
+        else:
+            formatstring = '{0:<{2}}  at {1}'
         for name, path in plugins:
-            print('{0} at {1}'.format(name, path))
-        print('\n\nAlso, you have disabled these plugins: {}'.format(self.site.config['DISABLED_PLUGINS']))
+            print(formatstring.format(name, path, maxlength))
+        dp = self.site.config['DISABLED_PLUGINS']
+        if dp:
+            print('\n\nAlso, you have disabled these plugins: {}'.format(', '.join(dp)))
+        else:
+            print('\n\nNo plugins are disabled.')
         return 0
 
     def do_upgrade(self, url):


Test patch: