internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: getnikola/nikola

Issue:
Post type metadata cannot be overridden by plugins
I'm writing a plugin that needs to set the `type` metadata (specifically to "micro", for microblog posts). But as it stands, The value that the plugin sets always gets replaced by the default value, "text". So I'm making a core change under this number to let my plugin work.


Discussion:

Resolving patch:
diff --git a/nikola/plugins/command/new_post.py b/nikola/plugins/command/new_post.py
--- a/nikola/plugins/command/new_post.py
+++ b/nikola/plugins/command/new_post.py
@@ -243,6 +243,7 @@ def _execute(self, options, args):
         date_path_opt = options['date-path']
         date_path_auto = self.site.config['NEW_POST_DATE_PATH'] and content_type == 'post'
         date_path_format = self.site.config['NEW_POST_DATE_PATH_FORMAT'].strip('/')
+        post_type = options.get('type', 'text')
 
         if wants_available:
             self.print_compilers()
@@ -363,7 +364,7 @@ def _execute(self, options, args):
             'tags': tags,
             'link': '',
             'description': '',
-            'type': 'text',
+            'type': post_type,
         }
 
         if not path:
@@ -429,7 +430,7 @@ def _execute(self, options, args):
         else:
             compiler_plugin.create_post(
                 txt_path, content=content, onefile=onefile, title=title,
-                slug=slug, date=date, tags=tags, is_page=is_page, **metadata)
+                slug=slug, date=date, tags=tags, is_page=is_page, type=post_type, **metadata)
 
         event = dict(path=txt_path)
 


Test patch:
Repository: getnikola/nikola

Issue:
Nikola strips away query strings with URL_TYPE='full_path'
### Environment

**Python Version:3.8.2**

**Nikola Version:8.1.1**

**Operating System:Ubuntu 20.04.1 LTS**

### Description:

As title says, with `URL_TYPE='full_path'`, Nikola strips away query strings from links. Is that intentional?

#### Example:
The relevant line from the template:
`<script src="/assets/js/test.js?version={{ assets_version }}"></script>`

- Result with `URL_TYPE='rel_path'`:  
`<script src="assets/js/all.js?version=1597143702"></script>`
- Result with `URL_TYPE='full_path'`:  
`<script src=/assets/js/all.js></script>`
- Result with `URL_TYPE='absolute'`:  
`<script src="https://example.org/assets/js/all.js?version=1597143702"></script>`


Discussion:

Resolving patch:
diff --git a/nikola/nikola.py b/nikola/nikola.py
--- a/nikola/nikola.py
+++ b/nikola/nikola.py
@@ -1589,7 +1589,7 @@ def url_replacer(self, src, dst, lang=None, url_type=None):
                 return dst
             elif url_type == 'full_path':
                 dst = urljoin(self.config['BASE_URL'], dst.lstrip('/'))
-                return urlparse(dst).path
+                return utils.full_path_from_urlparse(urlparse(dst))
             else:
                 return "#"
 
@@ -1604,10 +1604,7 @@ def url_replacer(self, src, dst, lang=None, url_type=None):
             dst = urljoin(self.config['BASE_URL'], dst.lstrip('/'))
             if url_type == 'full_path':
                 parsed = urlparse(urljoin(self.config['BASE_URL'], dst.lstrip('/')))
-                if parsed.fragment:
-                    dst = '{0}#{1}'.format(parsed.path, parsed.fragment)
-                else:
-                    dst = parsed.path
+                dst = utils.full_path_from_urlparse(parsed)
             return dst
 
         # Now both paths are on the same site and absolute
diff --git a/nikola/utils.py b/nikola/utils.py
--- a/nikola/utils.py
+++ b/nikola/utils.py
@@ -861,6 +861,16 @@ def encodelink(iri):
     encoded_link = urlunparse(link.values())
     return encoded_link
 
+
+def full_path_from_urlparse(parsed) -> str:
+    """Given urlparse output, return the full path (with query and fragment)."""
+    dst = parsed.path
+    if parsed.query:
+        dst = "{0}?{1}".format(dst, parsed.query)
+    if parsed.fragment:
+        dst = "{0}#{1}".format(dst, parsed.fragment)
+    return dst
+
 # A very slightly safer version of zip.extractall that works on
 # python < 2.6
 


Test patch:
Repository: getnikola/nikola

Issue:
reST doc directive: slugs with slashes in no longer work
At some point in the previous few versions, slugs with slashes in no longer work - links pointing to them have their slashes removed before the link is resolved, leading to an error like `"swowndata" slug doesn't exist` for a slug called `sw/owndata`. I can't find this being mentioned in the release notes or the other issues here, so I assume this was done by accident, is a bug, and should be fixed.


Discussion:
Please provide more details. Is this a slug in post meta, or is it a reference with doc or magic links?
It's an issue with links - if you `nikola build` on a checkout of https://github.com/infinity0/droid-hacks you can hopefully reproduce it.
Reproduced, bug was introduced in commit 22b29404241992816399c9ff4116ccb1572650e1.

Resolving patch:
diff --git a/nikola/plugins/compile/rest/doc.py b/nikola/plugins/compile/rest/doc.py
--- a/nikola/plugins/compile/rest/doc.py
+++ b/nikola/plugins/compile/rest/doc.py
@@ -47,16 +47,8 @@ def set_site(self, site):
         return super().set_site(site)
 
 
-def _doc_link(rawtext, text, options={}, content=[]):
-    """Handle the doc role."""
-    # split link's text and post's slug in role content
-    has_explicit_title, title, slug = split_explicit_title(text)
-    if '#' in slug:
-        slug, fragment = slug.split('#', 1)
-    else:
-        fragment = None
-    slug = slugify(slug)
-    # check if the slug given is part of our blog posts/pages
+def _find_post(slug):
+    """Find a post with the given slug in posts or pages."""
     twin_slugs = False
     post = None
     for p in doc_role.site.timeline:
@@ -66,6 +58,23 @@ def _doc_link(rawtext, text, options={}, content=[]):
             else:
                 twin_slugs = True
                 break
+    return post, twin_slugs
+
+
+def _doc_link(rawtext, text, options={}, content=[]):
+    """Handle the doc role."""
+    # split link's text and post's slug in role content
+    has_explicit_title, title, slug = split_explicit_title(text)
+    if '#' in slug:
+        slug, fragment = slug.split('#', 1)
+    else:
+        fragment = None
+
+    # Look for the unslugified input first, then try to slugify (Issue #3450)
+    post, twin_slugs = _find_post(slug)
+    if post is None:
+        slug = slugify(slug)
+        post, twin_slugs = _find_post(slug)
 
     try:
         if post is None:


Test patch:
Repository: getnikola/nikola

Issue:
nbconvert 6 and jinja2.exceptions.TemplateNotFound: static/style.css
### Environment

**Python Version:** 3.8.5, 3.7.9

**Nikola Version:** 8.1.1 and 8.0.4

**Operating System:** Windows 10, WSL2 (running through miniconda on ubuntu)

### Description:

Hello - I'd like to raise this issue unless it's been resolved since 8.1.1 (current version on PyPI).

When running `nikola build` or `nikola build -a` with all cache deleted, the following error comes up when starting to process `.ipynb` posts:

```jinja2.exceptions.TemplateNotFound: static/style.css```

This worked for me prior to a system factory reset and reinstallation of software, so I started downgrading to versions I knew worked, e.g. python 3.8.5 to 3.7.9 and Nikola 8.1.1 to 8.0.4 in hopes of a quick fix. I couldn't find much through a web search, most of it referred to Flask.

I was checking the versions of Jinja2 and nbconvert in hopes of finding something there, and I noticed that `nbconvert` was at `6.0.2`. I downgraded it to a version I knew worked, `5.6.1`, and everything is running smoothly again!


Discussion:

Resolving patch:
diff --git a/nikola/plugins/compile/ipynb.py b/nikola/plugins/compile/ipynb.py
--- a/nikola/plugins/compile/ipynb.py
+++ b/nikola/plugins/compile/ipynb.py
@@ -31,11 +31,13 @@
 import os
 
 try:
+    import nbconvert
     from nbconvert.exporters import HTMLExporter
     import nbformat
     current_nbformat = nbformat.current_nbformat
     from jupyter_client import kernelspec
     from traitlets.config import Config
+    NBCONVERT_VERSION_MAJOR = int(nbconvert.__version__.partition(".")[0])
     flag = True
 except ImportError:
     flag = None
@@ -60,7 +62,10 @@ def _compile_string(self, nb_json):
         c = Config(get_default_jupyter_config())
         c.merge(Config(self.site.config['IPYNB_CONFIG']))
         if 'template_file' not in self.site.config['IPYNB_CONFIG'].get('Exporter', {}):
-            c['Exporter']['template_file'] = 'basic.tpl'  # not a typo
+            if NBCONVERT_VERSION_MAJOR >= 6:
+                c['Exporter']['template_file'] = 'classic/base.html.j2'
+            else:
+                c['Exporter']['template_file'] = 'basic.tpl'  # not a typo
         exportHtml = HTMLExporter(config=c)
         body, _ = exportHtml.from_notebook_node(nb_json)
         return body


Test patch:
Repository: getnikola/nikola

Issue:
Native support for Utterances comment system
See https://utteranc.es/

Utterances gives you a script tag that you can add to your site, but it's not clear where that should go in the conf.py. 


Discussion:
Adding a new comment system requires changes to templates, namely `comments_helper.tmpl` and `comments_helper_NAMEOFSYSTEM.tmpl`. Looking at the page, it shouldn’t be hard to integrate this with Nikola. Would you be willing to take this?
What would be the best way to do it? Utterances has different configuration options than the other systems. 
You can use a GLOBAL_CONTEXT variable like isso does, or perhaps try to fit it in COMMENT_SYSTEM_ID.
`GLOBAL_CONTEXT` seems like it should work. However, there is one configuration option that is required (either `issue-term` or `issue-number`). If you don't specify it, it will show an error in the console log. Can we make it show an error on the Nikola build? 

Resolving patch:
diff --git a/nikola/nikola.py b/nikola/nikola.py
--- a/nikola/nikola.py
+++ b/nikola/nikola.py
@@ -100,6 +100,7 @@
         'isso',
         'muut',
         'commento',
+        'utterances',
     ],
     'TRANSLATIONS': {
         'af': 'Afrikaans',
@@ -836,6 +837,12 @@ def __init__(self, **config):
             utils.LOGGER.error('CATEGORY_PAGES_FOLLOW_DESTPATH requires CATEGORY_ALLOW_HIERARCHIES = True, CATEGORY_OUTPUT_FLAT_HIERARCHY = False.')
             sys.exit(1)
 
+        # The Utterances comment system has a required configuration value
+        if self.config.get('COMMENT_SYSTEM') == 'utterances':
+            utterances_config = self.config.get('GLOBAL_CONTEXT', {}).get('utterances_config', {})
+            if not ('issue-term' in utterances_config or 'issue-number' in utterances_config):
+                utils.LOGGER.error("COMMENT_SYSTEM = 'utterances' must have either GLOBAL_CONTEXT['utterances_config']['issue-term'] or GLOBAL_CONTEXT['utterances_config']['issue-term'] defined.")
+
         # Handle CONTENT_FOOTER and RSS_COPYRIGHT* properly.
         # We provide the arguments to format in CONTENT_FOOTER_FORMATS and RSS_COPYRIGHT_FORMATS.
         self.config['CONTENT_FOOTER'].langformat(self.config['CONTENT_FOOTER_FORMATS'])


Test patch:
Repository: getnikola/nikola

Issue:
Error when using LINK_CHECK_WHITELIST
<!--
Before creating an issue:
* make sure you are using an up-to-date version of Nikola
* search for existing issues that might be related

Make sure to:
* provide information about your environment (below)
* include all the output you get, and any other information related to your problem

Nikola v7.6.4, as provided by Ubuntu, is NOT SUPPORTED.
If you are using this version, you should upgrade: https://getnikola.com/getting-started.html
-->

### Environment

**Python Version:** 3.8

**Nikola Version:** 8.1

**Operating System:** Linux

### Description:


When using

```python
LINK_CHECK_WHITELIST = [
    r'http://127\.0\.0\.1',
]
```

And running `nikola check -l`, you get

```
[2020-10-10 10:31:23] ERROR: check: Error with: .../index.html cannot use a string pattern on a by
tes-like object

```

Looks like the bug is around here

https://github.com/getnikola/nikola/blob/38bd1c08e50b36da3c7bd9e7c3fbeccdddb0cd61/nikola/plugins/command/check.py#L276-L278

But I'm not familiar enough with the codebase to know why target is a byte string or where that conversion is happening.


Discussion:

Resolving patch:
diff --git a/nikola/plugins/command/check.py b/nikola/plugins/command/check.py
--- a/nikola/plugins/command/check.py
+++ b/nikola/plugins/command/check.py
@@ -274,7 +274,7 @@ def analyze(self, fname, find_sources=False, check_remote=False):
                         continue
 
                     # Skip whitelisted targets
-                    if any(re.search(_, target) for _ in self.whitelist):
+                    if any(pattern.search(target) for pattern in self.whitelist):
                         continue
 
                     # Check the remote link works
@@ -320,7 +320,7 @@ def analyze(self, fname, find_sources=False, check_remote=False):
                         target_filename = os.path.abspath(
                             os.path.join(os.path.dirname(filename).encode('utf-8'), unquoted_target))
 
-                elif url_type in ('full_path', 'absolute'):
+                else:
                     relative = False
                     if url_type == 'absolute':
                         # convert to 'full_path' case, ie url relative to root
@@ -348,7 +348,12 @@ def analyze(self, fname, find_sources=False, check_remote=False):
                         fs_rel_path = fs_relpath_from_url_path(url_rel_path)
                         target_filename = os.path.join(self.site.config['OUTPUT_FOLDER'], fs_rel_path)
 
-                if any(re.search(x, target_filename) for x in self.whitelist):
+                if isinstance(target_filename, str):
+                    target_filename_str = target_filename
+                else:
+                    target_filename_str = target_filename.decode("utf-8", errors="surrogateescape")
+
+                if any(pattern.search(target_filename_str) for pattern in self.whitelist):
                     continue
 
                 elif target_filename not in self.existing_targets:


Test patch:
Repository: getnikola/nikola

Issue:
Incorrect src attribute for images in featured posts in bootblog4 theme
<!--
Before creating an issue:
* make sure you are using an up-to-date version of Nikola
* search for existing issues that might be related

Make sure to:
* provide information about your environment (below)
* include all the output you get, and any other information related to your problem

Nikola v7.6.4, as provided by Ubuntu, is NOT SUPPORTED.
If you are using this version, you should upgrade: https://getnikola.com/getting-started.html
-->

### Environment

**Python Version:** 3.8.5

**Nikola Version:** 8.1.1

**Operating System:** Ubuntu 20.10

### Description:

It appears that Nikola 8.1.1 has introduced either a bug or a backwards-incompatible change that makes images in featured posts using the bootblog4 template render incorrecty.

My featured posts include these headings: 
```
.. status: featured
.. previewimage: /galleries/face-a-day-2020/20200817-me.png
```

My nikola site is rendered under a path: 
```python
# conf.py
SITE_URL = "https://carles.barrobes.com/drawing/"
# BASE_URL not set
```

- With nikola 8.0.4, this would generate a correct `src` attribute `galleries/face-a-day-2020/20200817-me.png`
- In nikola 8.1.1, this generates a relative path that doesn't work `drawing/galleries/face-a-day-2020/20200817-me.png` as it generates an invalid image URL of `https://carles.barrobes.com/drawing/drawing/galleries/face-a-day-2020/20200817-me.png` (duplicated `drawing` in the path).

The source of the issue seems to have been introduced here: https://github.com/getnikola/nikola/commit/e559cfda090edacbca0769395703fdbde50691cb




Discussion:
The diff link doesn’t work, please link to the specific commit.
Hmmm. Possibly this commit then https://github.com/getnikola/nikola/commit/e559cfda090edacbca0769395703fdbde50691cb

Resolving patch:
diff --git a/nikola/post.py b/nikola/post.py
--- a/nikola/post.py
+++ b/nikola/post.py
@@ -1085,11 +1085,13 @@ def previewimage(self, lang=None):
 
         image_path = self.meta[lang]['previewimage']
         if not image_path:
-            return self._default_preview_image
+            image_path = self._default_preview_image
 
-        # This is further parsed by the template, because we don’t have access
-        # to the URL replacer here.  (Issue #1473)
-        return image_path
+        if not image_path or image_path.startswith("/"):
+            # Paths starting with slashes are expected to be root-relative, pass them directly.
+            return image_path
+        # Other paths are relative to the permalink. The path will be made prettier by the URL replacer later.
+        return urljoin(self.permalink(lang), image_path)
 
     def source_ext(self, prefix=False):
         """Return the source file extension.


Test patch:
Repository: getnikola/nikola

Issue:
UnboundLocalError exception in `nikola deploy`
<!--
Before creating an issue:
* make sure you are using an up-to-date version of Nikola
* search for existing issues that might be related

Make sure to:
* provide information about your environment (below)
* include all the output you get, and any other information related to your problem

Nikola v7.6.4, as provided by Ubuntu, is NOT SUPPORTED.
If you are using this version, you should upgrade: https://getnikola.com/getting-started.html
-->

### Environment

**Python Version:**

Python 3.7.9

**Nikola Version:**

Nikola v8.1.2 (installed from git master)

**Operating System:** 

FreeBSD 12.2-RELEASE amd64

### Description:

On running `nikola deploy` on a new site, after rsync completes, this exception is thrown:

```python
  File "[...]/nikola/nikola/plugins/command/deploy.py", line 99, in _execute
    self._emit_deploy_event(last_deploy, new_deploy, clean, undeployed_posts)
UnboundLocalError: local variable 'clean' referenced before assignment
```

I looked at the source and noticed that `clean` is only set inside a condition:

https://github.com/getnikola/nikola/blob/d0ef5b68d0667f635ecb2bc0d5b92e480b5e4b25/nikola/plugins/command/deploy.py#L50-L56

To try and fix it, I changed the source to:

```python
        if last_deploy is not None:
            last_deploy = dateutil.parser.parse(last_deploy)
            clean = False
        else:
            clean = True
```

That leads to another exception elsewhere:

```python
  File "[...]/nikola/nikola/plugins/command/deploy.py", line 131, in _emit_deploy_event
    if last_deploy.tzinfo is None:
AttributeError: 'NoneType' object has no attribute 'tzinfo'
```

Which was fixed by changing the source code at:

https://github.com/getnikola/nikola/blob/d0ef5b68d0667f635ecb2bc0d5b92e480b5e4b25/nikola/plugins/command/deploy.py#L98-L99

to:

```python
        new_deploy = datetime.utcnow()
        if last_deploy is None:
            last_deploy = new_deploy
        self._emit_deploy_event(last_deploy, new_deploy, clean, undeployed_posts)
```

After this the deploy command has worked as I expected.


Discussion:

Resolving patch:
diff --git a/nikola/plugins/command/deploy.py b/nikola/plugins/command/deploy.py
--- a/nikola/plugins/command/deploy.py
+++ b/nikola/plugins/command/deploy.py
@@ -54,6 +54,8 @@ def _execute(self, command, args):
         if last_deploy is not None:
             last_deploy = dateutil.parser.parse(last_deploy)
             clean = False
+        else:
+            clean = True
 
         if self.site.config['COMMENT_SYSTEM'] and self.site.config['COMMENT_SYSTEM_ID'] == 'nikolademo':
             self.logger.warning("\nWARNING WARNING WARNING WARNING\n"
@@ -96,6 +98,8 @@ def _execute(self, command, args):
         self.logger.info("Successful deployment")
 
         new_deploy = datetime.utcnow()
+        if last_deploy is None:
+            last_deploy = new_deploy
         self._emit_deploy_event(last_deploy, new_deploy, clean, undeployed_posts)
 
         # Store timestamp of successful deployment


Test patch:
Repository: getnikola/nikola

Issue:
Add full GLOBAL_CONTEXT support to the post_list plugin
<!--
Before creating an issue:
* make sure you are using an up-to-date version of Nikola
* search for existing issues that might be related

Describe your requested features as precisely as possible. -->

I've got some data and functions in `GLOBAL_CONTEXT` that I'd like to use in a custom post list template.  Right now, it appears that only the locale's date format is passed along to the template context.

Would you accept a PR to make all of the `GLOBAL_CONTEXT` available to the plugin?


Discussion:
We’re rebuilding a ton of stuff on GLOBAL_CONTEXT change anyway, so I guess we could add post-lists to that.

Resolving patch:
diff --git a/nikola/plugins/shortcode/post_list.py b/nikola/plugins/shortcode/post_list.py
--- a/nikola/plugins/shortcode/post_list.py
+++ b/nikola/plugins/shortcode/post_list.py
@@ -145,6 +145,7 @@ def handler(self, start=None, stop=None, reverse=False, tags=None, require_all_t
 
         if self_post:
             self_post.register_depfile("####MAGIC####TIMELINE", lang=lang)
+            self_post.register_depfile("####MAGIC####CONFIG:GLOBAL_CONTEXT", lang=lang)
 
         # If we get strings for start/stop, make them integers
         if start is not None:
@@ -227,7 +228,8 @@ def handler(self, start=None, stop=None, reverse=False, tags=None, require_all_t
             for d in template_deps:
                 self_post.register_depfile(d, lang=lang)
 
-        template_data = {
+        template_data = site.GLOBAL_CONTEXT.copy()
+        template_data.update({
             'lang': lang,
             'posts': posts,
             # Need to provide str, not TranslatableSetting (Issue #2104)
@@ -235,7 +237,7 @@ def handler(self, start=None, stop=None, reverse=False, tags=None, require_all_t
             'post_list_id': post_list_id,
             'messages': site.MESSAGES,
             '_link': site.link,
-        }
+        })
         output = site.template_system.render_template(
             template, None, template_data)
         return output, template_deps


Test patch: