internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: getnikola/nikola

Issue:
nikola tabcompletion should be able to be ran outside a Nikola site
Running `nikola tabcompletion --hardcode-tasks -s zsh` or `nikola tabcompletion` give me: `ERROR: Nikola: This command needs to run inside an existing Nikola site.`
I would like it to be run outside a Nikola site, however I'm not sure if it's possible. In my case would like to put the tab-completion files for bash and zsh in a package to avoid every user to do this by hand. Is it possible, or at least only when used with the `--hardcode-tasks` option?



Discussion:
In order to support this, we would need to add an exception for this command. The command is provided by doit.

@schettino72, is `tabcompletion` functional without any tasks defined (only commands)?

We can do it by inheriting that command class, probably. I'll take a shot at it now.


Resolving patch:
diff --git a/nikola/__main__.py b/nikola/__main__.py
--- a/nikola/__main__.py
+++ b/nikola/__main__.py
@@ -44,6 +44,7 @@
 from doit.cmd_help import Help as DoitHelp
 from doit.cmd_run import Run as DoitRun
 from doit.cmd_clean import Clean as DoitClean
+from doit.cmd_completion import TabCompletion
 from doit.cmd_auto import Auto as DoitAuto
 from logbook import NullHandler
 from blinker import signal
@@ -359,7 +360,7 @@ def run(self, cmd_args):
                     LOGGER.info('Did you mean "{}" or "{}"?'.format('", "'.join(best_sugg[:-1]), best_sugg[-1]))
             return 3
 
-        if sub_cmds[args[0]] is not Help and not isinstance(sub_cmds[args[0]], Command):  # Is a doit command
+        if not sub_cmds[args[0]] in (Help, TabCompletion) and not isinstance(sub_cmds[args[0]], Command):
             if not self.nikola.configured:
                 LOGGER.error("This command needs to run inside an "
                              "existing Nikola site.")


Test patch:
Repository: getnikola/nikola

Issue:
Add support for cssminifier.com and javascript-minifier.com
They offer a nice service to minify CSS/JS and it's easy to use.



Discussion:

Resolving patch:
diff --git a/nikola/filters.py b/nikola/filters.py
--- a/nikola/filters.py
+++ b/nikola/filters.py
@@ -26,7 +26,6 @@
 
 """Utility functions to help run filters on files."""
 
-from .utils import req_missing
 from functools import wraps
 import os
 import io
@@ -39,6 +38,9 @@
     import typogrify.filters as typo
 except ImportError:
     typo = None  # NOQA
+import requests
+
+from .utils import req_missing, LOGGER
 
 
 def apply_to_binary_file(f):
@@ -274,3 +276,29 @@ def php_template_injection(data):
         return phpdata
     else:
         return data
+
+
+@apply_to_text_file
+def cssminify(data):
+    """Minify CSS using http://cssminifier.com/."""
+    try:
+        url = 'http://cssminifier.com/raw'
+        _data = {'input': data}
+        response = requests.post(url, data=_data)
+        return response.text
+    except Exception as exc:
+        LOGGER.error("can't use cssminifier.com: {}", exc)
+        return data
+
+
+@apply_to_text_file
+def jsminify(data):
+    """Minify JS using http://javascript-minifier.com/."""
+    try:
+        url = 'http://javascript-minifier.com/raw'
+        _data = {'input': data}
+        response = requests.post(url, data=_data)
+        return response.text
+    except Exception as exc:
+        LOGGER.error("can't use javascript-minifier.com: {}", exc)
+        return data


Test patch:
Repository: getnikola/nikola

Issue:
Author pages show stories
example: https://getnikola.com/authors/roberto-alsina.html → anything between 7.5.0 and 7.6.1 (most of which were written by me, but that’s not important)

cc @jjconti



Discussion:
The wrong author is because I am the default :-) Changing it to "The Nikola Team".

Now, go to https://getnikola.com/authors/the-nikola-team.html to see the bug

I’d rather keep the default author as “Roberto Alsina” and <ins>override it in</ins> the pages.  Most of the posts are actually written by you, and you should get credit.

Hey, I am part of the team. If I want to keep credit I will add me ;-)

You really **should** get credit. I’ll change the default back and fix the pages.

COMMIT WAR! SHINY AND CHROME! My name is already everywhere in the sources even for files I never touched :-D

I really believe in credit where it’s due. This project would not exist without you.

(And there are files without your name AFAIK)


Resolving patch:
diff --git a/nikola/plugins/task/authors.py b/nikola/plugins/task/authors.py
--- a/nikola/plugins/task/authors.py
+++ b/nikola/plugins/task/authors.py
@@ -292,5 +292,6 @@ def _posts_per_author(self):
         if self.posts_per_author is None:
             self.posts_per_author = defaultdict(list)
             for post in self.site.timeline:
-                self.posts_per_author[post.author()].append(post)
+                if post.is_post:
+                    self.posts_per_author[post.author()].append(post)
         return self.posts_per_author


Test patch:
Repository: getnikola/nikola

Issue:
Improve archive
When listing years or months, show the count of posts.

Example:
- 2004 (25)
- 2005 (23)

Or
- January (4)
- February (8)



Discussion:

Resolving patch:
diff --git a/nikola/plugins/task/archive.py b/nikola/plugins/task/archive.py
--- a/nikola/plugins/task/archive.py
+++ b/nikola/plugins/task/archive.py
@@ -175,10 +175,10 @@ def gen_tasks(self):
                 if not kw["create_monthly_archive"] or kw["create_full_archives"]:
                     yield self._generate_posts_task(kw, year, lang, posts, title, deps_translatable)
                 else:
-                    months = set([(m.split('/')[1], self.site.link("archive", m, lang)) for m in self.site.posts_per_month.keys() if m.startswith(str(year))])
+                    months = set([(m.split('/')[1], self.site.link("archive", m, lang), len(self.site.posts_per_month[m])) for m in self.site.posts_per_month.keys() if m.startswith(str(year))])
                     months = sorted(list(months))
                     months.reverse()
-                    items = [[nikola.utils.LocaleBorg().get_month_name(int(month), lang), link] for month, link in months]
+                    items = [[nikola.utils.LocaleBorg().get_month_name(int(month), lang), link, count] for month, link, count in months]
                     yield self._prepare_task(kw, year, lang, None, items, "list.tmpl", title, deps_translatable)
 
             if not kw["create_monthly_archive"] and not kw["create_full_archives"] and not kw["create_daily_archive"]:
@@ -219,7 +219,7 @@ def gen_tasks(self):
             years.sort(reverse=True)
             kw['years'] = years
             for lang in kw["translations"]:
-                items = [(y, self.site.link("archive", y, lang)) for y in years]
+                items = [(y, self.site.link("archive", y, lang), len(self.site.posts_per_year[y])) for y in years]
                 yield self._prepare_task(kw, None, lang, None, items, "list.tmpl", kw["messages"][lang]["Archive"])
 
     def archive_path(self, name, lang, is_feed=False):
diff --git a/nikola/plugins/task/indexes.py b/nikola/plugins/task/indexes.py
--- a/nikola/plugins/task/indexes.py
+++ b/nikola/plugins/task/indexes.py
@@ -204,7 +204,8 @@ def cat_path(i, displayed_i, num_pages, force_addition, extension=None):
                             should_render = False
                         else:
                             context["items"].append((post.title(lang),
-                                                     post.permalink(lang)))
+                                                     post.permalink(lang),
+                                                     None))
 
                     if should_render:
                         task = self.site.generic_post_list_renderer(lang, post_list,


Test patch:
Repository: getnikola/nikola

Issue:
DATE_FORMAT should be translatable.
`DATE_FORMAT` should be translatable. (See #2029)



Discussion:

Resolving patch:
diff --git a/nikola/nikola.py b/nikola/nikola.py
--- a/nikola/nikola.py
+++ b/nikola/nikola.py
@@ -553,6 +553,8 @@ def __init__(self, **config):
                                       # PATH options (Issue #1914)
                                       'TAG_PATH',
                                       'CATEGORY_PATH',
+                                      'DATE_FORMAT',
+                                      'JS_DATE_FORMAT',
                                       )
 
         self._GLOBAL_CONTEXT_TRANSLATABLE = ('blog_author',
@@ -564,10 +566,20 @@ def __init__(self, **config):
                                              'social_buttons_code',
                                              'search_form',
                                              'body_end',
-                                             'extra_head_data',)
+                                             'extra_head_data',
+                                             'date_format',
+                                             'js_date_format',)
         # WARNING: navigation_links SHOULD NOT be added to the list above.
         #          Themes ask for [lang] there and we should provide it.
 
+        # We first have to massage JS_DATE_FORMAT, otherwise we run into trouble
+        if 'JS_DATE_FORMAT' in self.config:
+            if isinstance(self.config['JS_DATE_FORMAT'], dict):
+                for k in self.config['JS_DATE_FORMAT']:
+                    self.config['JS_DATE_FORMAT'][k] = json.dumps(self.config['JS_DATE_FORMAT'][k])
+            else:
+                self.config['JS_DATE_FORMAT'] = json.dumps(self.config['JS_DATE_FORMAT'])
+
         for i in self.TRANSLATABLE_SETTINGS:
             try:
                 self.config[i] = utils.TranslatableSetting(i, self.config[i], self.config['TRANSLATIONS'])
@@ -924,7 +936,7 @@ def _set_global_context(self):
             'SHOW_SOURCELINK')
         self._GLOBAL_CONTEXT['extra_head_data'] = self.config.get('EXTRA_HEAD_DATA')
         self._GLOBAL_CONTEXT['date_fanciness'] = self.config.get('DATE_FANCINESS')
-        self._GLOBAL_CONTEXT['js_date_format'] = json.dumps(self.config.get('JS_DATE_FORMAT'))
+        self._GLOBAL_CONTEXT['js_date_format'] = self.config.get('JS_DATE_FORMAT')
         self._GLOBAL_CONTEXT['colorbox_locales'] = LEGAL_VALUES['COLORBOX_LOCALES']
         self._GLOBAL_CONTEXT['momentjs_locales'] = LEGAL_VALUES['MOMENTJS_LOCALES']
         self._GLOBAL_CONTEXT['hidden_tags'] = self.config.get('HIDDEN_TAGS')


Test patch:
Repository: getnikola/nikola

Issue:
SVG images in galleries don't work
Galleries plugin (and scale_images plugin) explicitly claims to support SVG:

```
image_ext_list_builtin = ['.jpg', '.png', '.jpeg', '.gif', '.svg', '.bmp', '.tiff']
```

But it fails:

```
.  render_galleries:output/galleries/test/normal_vectors.thumbnail.svg
########################################
TaskError - taskid:render_galleries:output/galleries/test/normal_vectors.thumbnail.svg
PythonAction Error
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/doit/action.py", line 360, in execute
    returned_value = self.py_callable(*self.args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/Nikola-7.3.0-py2.7.egg/nikola/image_processing.py", line 55, in resize_image
    im = Image.open(src)
  File "/usr/lib/python2.7/dist-packages/PIL/Image.py", line 2288, in open
    % (filename if filename else fp))
IOError: cannot identify image file u'galleries/test/normal_vectors.svg'
```

Should probably be handled by simply copying the SVG and setting desired size in HTML code instead.



Discussion:
We would still need to figure out the SVG size to get the proper aspect ratio.

PS. 

>   File "/usr/local/lib/python2.7/dist-packages/Nikola-7.3.0-py2.7.egg/

Don’t use `easy_install`.


Resolving patch:
diff --git a/nikola/image_processing.py b/nikola/image_processing.py
--- a/nikola/image_processing.py
+++ b/nikola/image_processing.py
@@ -52,7 +52,7 @@ class ImageProcessor(object):
 
     def resize_image(self, src, dst, max_size, bigger_panoramas=True):
         """Make a copy of the image in the requested size."""
-        if not Image:
+        if not Image or os.path.splitext(src)[1] in ['.svg', '.svgz']:
             utils.copy_file(src, dst)
             return
         im = Image.open(src)
diff --git a/nikola/plugins/task/galleries.py b/nikola/plugins/task/galleries.py
--- a/nikola/plugins/task/galleries.py
+++ b/nikola/plugins/task/galleries.py
@@ -563,9 +563,12 @@ def url_from_path(p):
         for img, thumb, title in zip(img_list, thumbs, img_titles):
             w, h = _image_size_cache.get(thumb, (None, None))
             if w is None:
-                im = Image.open(thumb)
-                w, h = im.size
-                _image_size_cache[thumb] = w, h
+                if os.path.splitext(thumb)[1] in ['.svg', '.svgz']:
+                    w, h = 200, 200
+                else:
+                    im = Image.open(thumb)
+                    w, h = im.size
+                    _image_size_cache[thumb] = w, h
             # Thumbs are files in output, we need URLs
             photo_array.append({
                 'url': url_from_path(img),


Test patch:
Repository: getnikola/nikola

Issue:
post.section_link ignores languages
In [11]: site.timeline[30].section_link('pl')
Out[11]: '/nieskategoryzowane/'

The correct output should be '/pl/nieskategoryzowane/'. I’m using sections from metadata, but I think this is a problem with the path-based implementation, too.  I’ll attempt to fix it by myself.



Discussion:

Resolving patch:
diff --git a/nikola/post.py b/nikola/post.py
--- a/nikola/post.py
+++ b/nikola/post.py
@@ -745,11 +745,16 @@ def section_color(self, lang=None):
 
     def section_link(self, lang=None):
         """Return the link to the post's section."""
+        if lang is None:
+            lang = nikola.utils.LocaleBorg().current_lang
+
         slug = self.section_slug(lang)
+        t = os.path.normpath(self.translations[lang])
+        if t == '.':
+            t = ''
+        link = '/' + '/'.join(i for i in (t, slug) if i) + '/'
         if not self.pretty_urls:
-            link = urljoin('/' + slug + '/', self.index_file)
-        else:
-            link = '/' + slug + '/'
+            link = urljoin(link, self.index_file)
         link = utils.encodelink(link)
         return link
 


Test patch:
Repository: getnikola/nikola

Issue:
The "filters" metadata typogrify doesn't work?
I tried to apply typogrify that way and could not. Either it doesn't work or it needs better docs.

So, according to the code, this should use the typogrify filter:

```
.. filters: filters.typogrify
```

And ... it does! Sort of.

1) In cache/posts/foo.html -> the post has smart quotes
2) In output/posts/foo/index.html? Nope.
3) site.timeline[0].text()? No quotes

~~Found the guilty party: LXML~~ NOPE

Now for real. This is what happens:

`smartypants('hello "world"')` works.

But!

`smartypants('hello &quot;world&quot;')` DOES NOT WORK.



Discussion:
What is the problem?  `&quot;` == `"`; does typogrify not understand that, or?

I thought typogrify added those `&quot;` in the 1st place... didn't it? Hmmmm

Typogrify would have added “” or their entity equivalents: `&ldquo;` `&#8220;` / `&rdquo;` `&#8221;`

Ok, then I spent a while chasing the wrong bunny. I'll try again tonight :-)

Browsers are not intelligent enough to guess what quote to apply where; and even all the algorithms (be it Microsoft Word, WordPress or Typogrify) can fuck up too.


Resolving patch:
diff --git a/nikola/filters.py b/nikola/filters.py
--- a/nikola/filters.py
+++ b/nikola/filters.py
@@ -34,6 +34,7 @@
 import tempfile
 import shlex
 
+import lxml
 try:
     import typogrify.filters as typo
 except ImportError:
@@ -236,6 +237,7 @@ def typogrify(data):
     if typo is None:
         req_missing(['typogrify'], 'use the typogrify filter')
 
+    data = _normalize_html(data)
     data = typo.amp(data)
     data = typo.widont(data)
     data = typo.smartypants(data)
@@ -253,6 +255,7 @@ def typogrify_sans_widont(data):
     if typo is None:
         req_missing(['typogrify'], 'use the typogrify_sans_widont filter')
 
+    data = _normalize_html(data)
     data = typo.amp(data)
     data = typo.smartypants(data)
     # Disabled because of typogrify bug where it breaks <title>
@@ -302,3 +305,15 @@ def jsminify(data):
     except Exception as exc:
         LOGGER.error("can't use javascript-minifier.com: {}", exc)
         return data
+
+
+def _normalize_html(data):
+    """Pass HTML through LXML to clean it up, if possible."""
+    try:
+        data = lxml.html.tostring(lxml.html.fragment_fromstring(data), encoding='unicode')
+    except:
+        pass
+    return data
+
+
+normalize_html = apply_to_text_file(_normalize_html)
diff --git a/nikola/plugins/task/posts.py b/nikola/plugins/task/posts.py
--- a/nikola/plugins/task/posts.py
+++ b/nikola/plugins/task/posts.py
@@ -116,7 +116,7 @@ def tl_ch():
                             pass
                     else:
                         flist.append(f)
-                yield utils.apply_filters(task, {os.path.splitext(dest): flist})
+                yield utils.apply_filters(task, {os.path.splitext(dest)[-1]: flist})
 
     def dependence_on_timeline(self, post, lang):
         """Check if a post depends on the timeline."""


Test patch:
Repository: getnikola/nikola

Issue:
Changing section colors does not rebuild things
I want to use section colors in all post headers, and I have to run `nikola build -a` to get the changes to appear everywhere.



Discussion:
We should probably just add POSTS_SECTION_COLORS to the global context, right?


Resolving patch:
diff --git a/nikola/nikola.py b/nikola/nikola.py
--- a/nikola/nikola.py
+++ b/nikola/nikola.py
@@ -945,6 +945,7 @@ def _set_global_context(self):
         self._GLOBAL_CONTEXT['hidden_categories'] = self.config.get('HIDDEN_CATEGORIES')
         self._GLOBAL_CONTEXT['hidden_authors'] = self.config.get('HIDDEN_AUTHORS')
         self._GLOBAL_CONTEXT['url_replacer'] = self.url_replacer
+        self._GLOBAL_CONTEXT['post_section_colors'] = self.config.get('POST_SECTION_COLORS')
 
         # IPython theme configuration.  If a website has ipynb enabled in post_pages
         # we should enable the IPython CSS (leaving that up to the theme itself).


Test patch:
Repository: getnikola/nikola

Issue:
Changing section colors does not rebuild things
I want to use section colors in all post headers, and I have to run `nikola build -a` to get the changes to appear everywhere.



Discussion:
We should probably just add POSTS_SECTION_COLORS to the global context, right?

We actually need all the `POSTS_SECTION_*` things there, for other things to get rebuilt.  I’ll add them in.


Resolving patch:
diff --git a/nikola/nikola.py b/nikola/nikola.py
--- a/nikola/nikola.py
+++ b/nikola/nikola.py
@@ -441,6 +441,7 @@ def __init__(self, **config):
             'POSTS': (("posts/*.txt", "posts", "post.tmpl"),),
             'POSTS_SECTIONS': True,
             'POSTS_SECTION_ARE_INDEXES': True,
+            'POSTS_SECTION_COLORS': {},
             'POSTS_SECTION_DESCRIPTIONS': "",
             'POSTS_SECTION_FROM_META': False,
             'POSTS_SECTION_NAME': "",
@@ -945,7 +946,13 @@ def _set_global_context(self):
         self._GLOBAL_CONTEXT['hidden_categories'] = self.config.get('HIDDEN_CATEGORIES')
         self._GLOBAL_CONTEXT['hidden_authors'] = self.config.get('HIDDEN_AUTHORS')
         self._GLOBAL_CONTEXT['url_replacer'] = self.url_replacer
-        self._GLOBAL_CONTEXT['post_section_colors'] = self.config.get('POST_SECTION_COLORS')
+        self._GLOBAL_CONTEXT['posts_sections'] = self.config.get('POSTS_SECTIONS')
+        self._GLOBAL_CONTEXT['posts_section_are_indexes'] = self.config.get('POSTS_SECTION_ARE_INDEXES')
+        self._GLOBAL_CONTEXT['posts_section_colors'] = self.config.get('POSTS_SECTION_COLORS')
+        self._GLOBAL_CONTEXT['posts_section_descriptions'] = self.config.get('POSTS_SECTION_DESCRIPTIONS')
+        self._GLOBAL_CONTEXT['posts_section_from_meta'] = self.config.get('POSTS_SECTION_FROM_META')
+        self._GLOBAL_CONTEXT['posts_section_name'] = self.config.get('POSTS_SECTION_NAME')
+        self._GLOBAL_CONTEXT['posts_section_title'] = self.config.get('POSTS_SECTION_TITLE')
 
         # IPython theme configuration.  If a website has ipynb enabled in post_pages
         # we should enable the IPython CSS (leaving that up to the theme itself).


Test patch:
Repository: getnikola/nikola

Issue:
Deprecate `Post.section_link()`
`site.link()` aka `link://` should be used instead.

(via #2069)



Discussion:

Resolving patch:
diff --git a/nikola/post.py b/nikola/post.py
--- a/nikola/post.py
+++ b/nikola/post.py
@@ -744,7 +744,9 @@ def section_color(self, lang=None):
         return utils.colorize_str_from_base_color(slug, base)
 
     def section_link(self, lang=None):
-        """Return the link to the post's section."""
+        """Return the link to the post's section (deprecated)."""
+        utils.LOGGER.warning("Post.section_link is deprecated. Please use " +
+                             "site.link('index_section_path', post.section_slug()) instead.")
         if lang is None:
             lang = nikola.utils.LocaleBorg().current_lang
 


Test patch:
Repository: getnikola/nikola

Issue:
.. template: directive is not used in translated pages (.en.rst)
I have a multilanguage site and I have some pages (under `stories/`) which are translated. For example:
- home.rst
- home.en.rst

(the repo is here: https://github.com/humitos/argentinaenpython.com.ar)

If I use `.. template: englishpage.tmpl` in the `home.en.rst` doesn't work. Nikola just use the same template to render both files.



Discussion:
The fix here is to make Post.template_name respect the locale. Should be a 2-liner, give or take a line.


Resolving patch:
diff --git a/nikola/nikola.py b/nikola/nikola.py
--- a/nikola/nikola.py
+++ b/nikola/nikola.py
@@ -1734,6 +1734,7 @@ def scan_posts(self, really=False, ignore_quit=False, quiet=False):
 
     def generic_page_renderer(self, lang, post, filters, context=None):
         """Render post fragments to final HTML pages."""
+        utils.LocaleBorg().set_locale(lang)
         context = context.copy() if context else {}
         deps = post.deps(lang) + \
             self.template_system.template_deps(post.template_name)
diff --git a/nikola/post.py b/nikola/post.py
--- a/nikola/post.py
+++ b/nikola/post.py
@@ -323,7 +323,8 @@ def next_post(self, v):
     @property
     def template_name(self):
         """Return template name for this post."""
-        return self.meta('template') or self._template_name
+        lang = nikola.utils.LocaleBorg().current_lang
+        return self.meta[lang]['template'] or self._template_name
 
     def formatted_date(self, date_format, date=None):
         """Return the formatted date as unicode."""


Test patch:
Repository: getnikola/nikola

Issue:
${messages("xxx")} not working in post-list template
When I use ${messages("xxx")} tag inside post-list template. I get an error during site compilation.

Error message is the following : File "/opt/nikola/projects/konilabs-web/cache/.mako.tmp/post_list_thumb.tmpl.py", line 72, in render_content __M_writer(unicode(messages("Read More"))) TypeError: 'Undefined' object is not callable 



Discussion:

Resolving patch:
diff --git a/nikola/plugins/compile/rest/post_list.py b/nikola/plugins/compile/rest/post_list.py
--- a/nikola/plugins/compile/rest/post_list.py
+++ b/nikola/plugins/compile/rest/post_list.py
@@ -205,6 +205,7 @@ def run(self):
             # Need to provide str, not TranslatableSetting (Issue #2104)
             'date_format': self.site.GLOBAL_CONTEXT.get('date_format')[lang],
             'post_list_id': post_list_id,
+            'messages': self.site.MESSAGES,
         }
         output = self.site.template_system.render_template(
             template, None, template_data)


Test patch: