internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: getnikola/nikola

Issue:
rst.css overrides pygments .code by specifying pre.code
At the moment any attempt to change `CODE_COLOR_SCHEME` does not alter the colours because rst.css has more specific CSS rules, `pre.code`, than the ones supplied by pygments, `.code`.

Can be reproduced by using a basic theme and trying to change `CODE_COLOR_SCHEME`. Notice that the background colors don't really change and the foreground ones change only slightly.



Discussion:
we should nuke [rst.css@247](https://github.com/getnikola/nikola/blob/master/nikola/data/themes/base/assets/css/rst.css#L247) to fix this.


Resolving patch:
diff --git a/nikola/plugins/task/copy_assets.py b/nikola/plugins/task/copy_assets.py
--- a/nikola/plugins/task/copy_assets.py
+++ b/nikola/plugins/task/copy_assets.py
@@ -75,7 +75,7 @@ def create_code_css():
                 formatter = get_formatter_by_name('html', style=kw["code_color_scheme"])
                 utils.makedirs(os.path.dirname(code_css_path))
                 with codecs.open(code_css_path, 'wb+', 'utf8') as outf:
-                    outf.write(formatter.get_style_defs('.code'))
+                    outf.write(formatter.get_style_defs('pre.code'))
                     outf.write("table.codetable { width: 100%;} td.linenos {text-align: right; width: 4em;}")
 
             task = {


Test patch:
Repository: getnikola/nikola

Issue:
reST listings can’t number lines, while code-blocks do it incompatibly
1. One can’t do a `.. listing::` in Nikola with line numbers.
2. In `.. code-block::`s, one must use the reST `:number-lines:` option syntax.  However, Sphinx wants `:linenos:` instead. (#770)



Discussion:
Adding both options to listing is easy-ish, I'll do it.

It should also be possible to add :linenos: to CodeBlock I guess.

It should.  If you re-implement `CodeBlock.run()`.  It’s ~~nicely~~ hard-coded.

listing supports :number-lines: I just tried it.

It’s probably because I used `:linenos:`, but the second issue still stands.


Resolving patch:
diff --git a/nikola/plugins/compile/rest/listing.py b/nikola/plugins/compile/rest/listing.py
--- a/nikola/plugins/compile/rest/listing.py
+++ b/nikola/plugins/compile/rest/listing.py
@@ -56,6 +56,18 @@ def run(self):
 
 from nikola.plugin_categories import RestExtension
 
+# Add sphinx compatibility option
+CodeBlock.option_spec['linenos'] = directives.unchanged
+
+
+class FlexibleCodeBlock(CodeBlock):
+
+    def run(self):
+        if 'linenos' in self.options:
+            self.options['number-lines'] = self.options['linenos']
+        return super(FlexibleCodeBlock, self).run()
+CodeBlock = FlexibleCodeBlock
+
 
 class Plugin(RestExtension):
 
@@ -71,6 +83,11 @@ def set_site(self, site):
         directives.register_directive('listing', Listing)
         return super(Plugin, self).set_site(site)
 
+# Add sphinx compatibility option
+listing_spec = Include.option_spec
+listing_spec['linenos'] = directives.unchanged
+print(listing_spec)
+
 
 class Listing(Include):
     """ listing directive: create a highlighted block of code from a file in listings/
@@ -84,6 +101,7 @@ class Listing(Include):
     has_content = False
     required_arguments = 1
     optional_arguments = 1
+    option_spec = listing_spec
 
     def run(self):
         fname = self.arguments.pop(0)
@@ -91,6 +109,8 @@ def run(self):
         fpath = os.path.join('listings', fname)
         self.arguments.insert(0, fpath)
         self.options['code'] = lang
+        if 'linenos' in self.options:
+            self.options['number-lines'] = self.options['linenos']
         with codecs_open(fpath, 'rb+', 'utf8') as fileobject:
             self.content = fileobject.read().splitlines()
         self.state.document.settings.record_dependencies.add(fpath)


Test patch:
Repository: getnikola/nikola

Issue:
[Question] Nikola is supporting another tag locations?
I using Nikola v5.5.1.

I wish instead:

```
output / TRANSLATION[lang] / TAG_PATH / tag_name.html
```

Using this:

```
output / TRANSLATION[lang] / TAG_PATH / tag_name / index.html
```

Nikola this is supporting?
## 

Sorry for my bad english.



Discussion:
Does not look like it; it ignores the `PRETTY_URLS` setting that would normally do exactly that.

Yes, PRETTY_URLS shouls do that and it doesn't. Scheduling for 6.1


Resolving patch:
diff --git a/nikola/plugins/task/tags.py b/nikola/plugins/task/tags.py
--- a/nikola/plugins/task/tags.py
+++ b/nikola/plugins/task/tags.py
@@ -311,9 +311,17 @@ def tag_index_path(self, name, lang):
                               self.site.config['INDEX_FILE']] if _f]
 
     def tag_path(self, name, lang):
-        return [_f for _f in [self.site.config['TRANSLATIONS'][lang],
-                              self.site.config['TAG_PATH'], self.slugify_name(name) + ".html"] if
-                _f]
+        if self.site.config['PRETTY_URLS']:
+            return [_f for _f in [
+                self.site.config['TRANSLATIONS'][lang],
+                self.site.config['TAG_PATH'],
+                self.slugify_name(name),
+                self.site.config['INDEX_FILE']] if _f]
+        else:
+            return [_f for _f in [
+                self.site.config['TRANSLATIONS'][lang],
+                self.site.config['TAG_PATH'],
+                self.slugify_name(name) + ".html"] if _f]
 
     def tag_rss_path(self, name, lang):
         return [_f for _f in [self.site.config['TRANSLATIONS'][lang],


Test patch:
Repository: getnikola/nikola

Issue:
Act sanely with tags that have different “real” names but the same slugs
via nikola-discuss: https://groups.google.com/forum/#!topic/nikola-discuss/7KZv9EsNmKs



Discussion:
I think we should collapse those tags into one and give a warning.


Resolving patch:
diff --git a/nikola/nikola.py b/nikola/nikola.py
--- a/nikola/nikola.py
+++ b/nikola/nikola.py
@@ -928,7 +928,8 @@ def scan_posts(self):
             return
         seen = set([])
         print("Scanning posts", end='', file=sys.stderr)
-        lower_case_tags = set([])
+        slugged_tags = set([])
+        quit = False
         for wildcard, destination, template_name, use_in_feeds in \
                 self.config['post_pages']:
             print(".", end='', file=sys.stderr)
@@ -977,16 +978,16 @@ def scan_posts(self):
                         self.posts_per_month[
                             '{0}/{1:02d}'.format(post.date.year, post.date.month)].append(post.source_path)
                         for tag in post.alltags:
-                            if tag.lower() in lower_case_tags:
+                            if utils.slugify(tag) in slugged_tags:
                                 if tag not in self.posts_per_tag:
                                     # Tags that differ only in case
                                     other_tag = [k for k in self.posts_per_tag.keys() if k.lower() == tag.lower()][0]
-                                    utils.LOGGER.error('You have cases that differ only in upper/lower case: {0} and {1}'.format(tag, other_tag))
+                                    utils.LOGGER.error('You have tags that are too similar: {0} and {1}'.format(tag, other_tag))
                                     utils.LOGGER.error('Tag {0} is used in: {1}'.format(tag, post.source_path))
                                     utils.LOGGER.error('Tag {0} is used in: {1}'.format(other_tag, ', '.join(self.posts_per_tag[other_tag])))
-                                    sys.exit(1)
+                                    quit = True
                             else:
-                                lower_case_tags.add(tag.lower())
+                                slugged_tags.add(utils.slugify(tag))
                             self.posts_per_tag[tag].append(post.source_path)
                         self.posts_per_category[post.meta('category')].append(post.source_path)
                     else:
@@ -1005,6 +1006,8 @@ def scan_posts(self):
             p.prev_post = post_timeline[i + 1]
         self._scanned = True
         print("done!", file=sys.stderr)
+        if quit:
+            sys.exit(1)
 
     def generic_page_renderer(self, lang, post, filters):
         """Render post fragments to final HTML pages."""


Test patch:
Repository: getnikola/nikola

Issue:
Invalid permalinks for system pages (index, categories etc) in secondary language
Nikola 6.3.0.
I am trying to mark active link in menu, doing it like this:

```
    %for url, text in navigation_links[lang]:
        %if rel_link(permalink, url) == "#":
            <a class="active">...</a>
        %else:
            <a>...</a>
        %endif
    %endfor
```

But this does not work, because in secondary language permalinks for system pages are like this:

```
/./en/index.html
/./en/categories/index.html
/./en/galleries/galleries/index.html
```

While in config this urls are usually like this:

```
/en/index.html
/en/categories/index.html
/en/galleries/galleries/index.html
```

This URLs are not equal, so I can't properly check for active page.
Permalinks of my pages (in pages/posts directory) are correct, this issue is only for system pages and only for non-default languages.

As a workaround i added "/." to urls of system pages, but definetly it is a bug.
Thanks.



Discussion:
That's probably a missing normpath call somewhere, I'll take a look :-)

And indeed...

```
In [3]: SITE.link('index', '', 'es')
Out[3]: u'/./es/index-.html'
```


Resolving patch:
diff --git a/nikola/nikola.py b/nikola/nikola.py
--- a/nikola/nikola.py
+++ b/nikola/nikola.py
@@ -814,6 +814,7 @@ def path(self, kind, name, lang=None, is_link=False):
             lang = utils.LocaleBorg().current_lang
 
         path = self.path_handlers[kind](name, lang)
+        path = [os.path.normpath(p) for p in path if p != '.']  # Fix Issue #1028
 
         if is_link:
             link = '/' + ('/'.join(path))


Test patch:
Repository: getnikola/nikola

Issue:
Magic link for linking posts using filename
Right now, if you want to link to another post, you must know the slug. If you don't know/remember, you must find the file -> open it -> check the slug.

I suggest adding something like `link://filename` or `link://file`.



Discussion:
I’m not sure whether this would be 100% useful (and I personally suggest having filenames and slugs identical).  Also, the syntax might collide with another one: we have direct links in pages with this syntax.  For example, if you say `link:///favicon.ico`, the HTML will contain `/favicon.ico` regardless of your settings for relative links etc.

But I’m not closing this just yet.  @ralsina, your opinion?

The proposal is to have `link://filename/foo/bar.rst` -> the file generated from bar.rst and it's similar to `link://slug/bar`

The main benefit would be to make imports from pelican or other static systems easier.


Resolving patch:
diff --git a/nikola/nikola.py b/nikola/nikola.py
--- a/nikola/nikola.py
+++ b/nikola/nikola.py
@@ -96,6 +96,7 @@ def __init__(self, **config):
         self.path_handlers = {
             'slug': self.slug_path,
             'post_path': self.post_path,
+            'filename': self.filename_path,
         }
 
         self.strict = False
@@ -806,6 +807,7 @@ def path(self, kind, name, lang=None, is_link=False):
         * listing (name is the source code file name)
         * post_path (name is 1st element in a POSTS/PAGES tuple)
         * slug (name is the slug of a post or story)
+        * filename (name is the source filename of a post/story, in DEFAULT_LANG, relative to conf.py)
 
         The returned value is always a path relative to output, like
         "categories/whatever.html"
@@ -850,6 +852,16 @@ def slug_path(self, name, lang):
                 utils.LOGGER.warning('Ambiguous path request for slug: {0}'.format(name))
             return [_f for _f in results[0].permalink(lang).split('/') if _f]
 
+    def filename_path(self, name, lang):
+        """filename path handler"""
+        results = [p for p in self.timeline if p.source_path == name]
+        if not results:
+            utils.LOGGER.warning("Can't resolve path request for filename: {0}".format(name))
+        else:
+            if len(results) > 1:
+                utils.LOGGER.error("Ambiguous path request for filename: {0}".format(name))
+            return [_f for _f in results[0].permalink(lang).split('/') if _f]
+
     def register_path_handler(self, kind, f):
         if kind in self.path_handlers:
             utils.LOGGER.warning('Conflicting path handlers for kind: {0}'.format(kind))


Test patch:
Repository: getnikola/nikola

Issue:
Allow passing options to sass/less compilers
via @inoatech on Twitter: https://twitter.com/inoatech/status/429541811632107520



Discussion:

Resolving patch:
diff --git a/nikola/nikola.py b/nikola/nikola.py
--- a/nikola/nikola.py
+++ b/nikola/nikola.py
@@ -185,6 +185,7 @@ def __init__(self, **config):
             'INDEX_PATH': '',
             'IPYNB_CONFIG': {},
             'LESS_COMPILER': 'lessc',
+            'LESS_OPTIONS': [],
             'LICENSE': '',
             'LINK_CHECK_WHITELIST': [],
             'LISTINGS_FOLDER': 'listings',
@@ -204,6 +205,7 @@ def __init__(self, **config):
             'RSS_PATH': '',
             'RSS_TEASERS': True,
             'SASS_COMPILER': 'sass',
+            'SASS_OPTIONS': [],
             'SEARCH_FORM': '',
             'SLUG_TAG_PATH': True,
             'SOCIAL_BUTTONS_CODE': SOCIAL_BUTTONS_CODE,
diff --git a/nikola/plugins/task/build_less.py b/nikola/plugins/task/build_less.py
--- a/nikola/plugins/task/build_less.py
+++ b/nikola/plugins/task/build_less.py
@@ -46,6 +46,7 @@ class BuildLess(Task):
     def gen_tasks(self):
         """Generate CSS out of LESS sources."""
         self.compiler_name = self.site.config['LESS_COMPILER']
+        self.compiler_options = self.site.config['LESS_OPTIONS']
 
         kw = {
             'cache_folder': self.site.config['CACHE_FOLDER'],
@@ -93,7 +94,7 @@ def compile_target(target, dst):
             src = os.path.join(kw['cache_folder'], self.sources_folder, target)
             run_in_shell = sys.platform == 'win32'
             try:
-                compiled = subprocess.check_output([self.compiler_name, src], shell=run_in_shell)
+                compiled = subprocess.check_output([self.compiler_name] + self.compiler_options + [src], shell=run_in_shell)
             except OSError:
                 utils.req_missing([self.compiler_name],
                                   'build LESS files (and use this theme)',
diff --git a/nikola/plugins/task/build_sass.py b/nikola/plugins/task/build_sass.py
--- a/nikola/plugins/task/build_sass.py
+++ b/nikola/plugins/task/build_sass.py
@@ -47,6 +47,7 @@ def gen_tasks(self):
         """Generate CSS out of Sass sources."""
         self.logger = utils.get_logger('build_sass', self.site.loghandlers)
         self.compiler_name = self.site.config['SASS_COMPILER']
+        self.compiler_options = self.site.config['SASS_OPTIONS']
 
         kw = {
             'cache_folder': self.site.config['CACHE_FOLDER'],
@@ -97,7 +98,7 @@ def compile_target(target, dst):
             run_in_shell = sys.platform == 'win32'
             src = os.path.join(kw['cache_folder'], self.sources_folder, target)
             try:
-                compiled = subprocess.check_output([self.compiler_name, src], shell=run_in_shell)
+                compiled = subprocess.check_output([self.compiler_name] + self.compiler_options + [src], shell=run_in_shell)
             except OSError:
                 utils.req_missing([self.compiler_name],
                                   'build Sass files (and use this theme)',


Test patch:
Repository: getnikola/nikola

Issue:
Use the setuptools’ “Extras” feature (optional dependencies)
This, to be exact: http://pythonhosted.org//setuptools/setuptools.html#declaring-extras-optional-features-with-their-own-dependencies



Discussion:

Resolving patch:
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -19,6 +19,16 @@
 with open('requirements.txt', 'r') as fh:
     dependencies = [l.strip() for l in fh]
 
+extras = {}
+
+with open('requirements-extras.txt', 'r') as fh:
+    extras['extras'] = [l.strip() for l in fh][1:]
+    # Alternative name.
+    extras['full'] = extras['extras']
+
+with open('requirements-tests.txt', 'r') as fh:
+    extras['tests'] = [l.strip() for l in fh][1:]
+
 ########### platform specific stuff #############
 import platform
 platform_system = platform.system()
@@ -177,6 +187,7 @@ def run(self):
                    'Topic :: Internet :: WWW/HTTP',
                    'Topic :: Text Processing :: Markup'),
       install_requires=dependencies,
+      extras_require = extras,
       include_package_data=True,
       cmdclass={'install': nikola_install},
       data_files=[


Test patch:
Repository: getnikola/nikola

Issue:
Fix #1098
This adds support for a "content" argument in create_post which will end up as the post's content.



Discussion:
We could make content a keyword argument, and just set the default outside the plugin, in new_post. Sounds better?

Yes.  Reshuffling kwargs may have bad effects, and even if it doesn’t, there will be less duplicated code throughout all the plugins.  (and let me tell you, it’s a pain to reliably change all the plugins…)

@Kwpolska can you take a look now? I _could_ kill is_page since it's only used by ipynb for a message but... too boring and no gain, really.

Looks well to me.  Waiting for Travis’ opinion.


Resolving patch:
diff --git a/nikola/nikola.py b/nikola/nikola.py
--- a/nikola/nikola.py
+++ b/nikola/nikola.py
@@ -967,10 +967,21 @@ def flatten(task):
             'task_dep': task_dep
         }
 
-    def scan_posts(self):
+    def scan_posts(self, really=False):
         """Scan all the posts."""
-        if self._scanned:
+        if self._scanned and not really:
             return
+
+        self.global_data = {}
+        self.posts = []
+        self.posts_per_year = defaultdict(list)
+        self.posts_per_month = defaultdict(list)
+        self.posts_per_tag = defaultdict(list)
+        self.posts_per_category = defaultdict(list)
+        self.post_per_file = {}
+        self.timeline = []
+        self.pages = []
+
         seen = set([])
         print("Scanning posts", end='', file=sys.stderr)
         slugged_tags = set([])


Test patch:
Repository: getnikola/nikola

Issue:
Files with non-ASCII characters in "files/*" get copied every time
If I create two files called i"files/vigo.ics" and "files/castellón.ics", they get copied to "output/" directory. Good.

If I run Nikola again, "files/castellón.ics" is copied again.  The file is copied again everytime Nikola is run.



Discussion:

Resolving patch:
diff --git a/nikola/utils.py b/nikola/utils.py
--- a/nikola/utils.py
+++ b/nikola/utils.py
@@ -517,12 +517,12 @@ def copy_tree(src, dst, link_cutoff=None):
                 continue
             dst_file = os.path.join(dst_dir, src_name)
             src_file = os.path.join(root, src_name)
-            if sys.version_info[0] == 2:
+            #if sys.version_info[0] == 2:
                 # Python2 prefers encoded str here
-                dst_file = sys_encode(dst_file)
-                src_file = sys_encode(src_file)
+                #dst_file = sys_encode(dst_file)
+                #src_file = sys_encode(src_file)
             yield {
-                'name': str(dst_file),
+                'name': dst_file,
                 'file_dep': [src_file],
                 'targets': [dst_file],
                 'actions': [(copy_file, (src_file, dst_file, link_cutoff))],


Test patch:
Repository: getnikola/nikola

Issue:
Never copy/list listings/*.py[co]
kinda related to invariance



Discussion:
also applies to `.pyo`.


Resolving patch:
diff --git a/nikola/plugins/task/listings.py b/nikola/plugins/task/listings.py
--- a/nikola/plugins/task/listings.py
+++ b/nikola/plugins/task/listings.py
@@ -92,6 +92,7 @@ def render_listing(in_name, out_name, folders=[], files=[]):
 
         template_deps = self.site.template_system.template_deps('listing.tmpl')
         for root, dirs, files in os.walk(kw['listings_folder']):
+            files = [f for f in files if os.path.splitext(f)[-1] not in ignored_extensions]
             # Render all files
             out_name = os.path.join(
                 kw['output_folder'],


Test patch:
Repository: getnikola/nikola

Issue:
freezing the dates of images makes their order unstable
So, that makes the "invariant" build actually less stable than the regular one, because they all have the same date, therefore they are practically unsorted.



Discussion:
It’s going to bite.  The dates change every `nikola init --demo`.  We need to devise a solution to give the images identical dates while copying in `init`.

The image dates are obtained from EXIF data first, so just adding missing EXIF data should do the trick.


Resolving patch:
diff --git a/nikola/plugins/task/galleries.py b/nikola/plugins/task/galleries.py
--- a/nikola/plugins/task/galleries.py
+++ b/nikola/plugins/task/galleries.py
@@ -567,7 +567,7 @@ def image_date(self, src):
             if exif is not None:
                 for tag, value in list(exif.items()):
                     decoded = ExifTags.TAGS.get(tag, tag)
-                    if decoded == 'DateTimeOriginal':
+                    if decoded in ('DateTimeOriginal', 'DateTimeDigitized'):
                         try:
                             self.dates[src] = datetime.datetime.strptime(
                                 value, r'%Y:%m:%d %H:%M:%S')


Test patch:
Repository: getnikola/nikola

Issue:
Bundles do not include Nikola-generated code.css
The `bundles` file:

```
assets/css/all.css=rst.css,code.css,theme.css
assets/css/all-nocdn.css=rst.css,code.css,theme.css
```

Yet, both files include only `rst.css` and `theme.css`, `code.css` (generated by Nikola) is not there.



Discussion:

Resolving patch:
diff --git a/nikola/plugins/task/bundles.py b/nikola/plugins/task/bundles.py
--- a/nikola/plugins/task/bundles.py
+++ b/nikola/plugins/task/bundles.py
@@ -91,10 +91,14 @@ def build_bundle(output, inputs):
                     files.append(os.path.join(dname, fname))
                 file_dep = [os.path.join(kw['output_folder'], fname)
                             for fname in files if
-                            utils.get_asset_path(fname, self.site.THEMES, self.site.config['FILES_FOLDERS'])]
+                            utils.get_asset_path(fname, self.site.THEMES, self.site.config['FILES_FOLDERS'])
+                            or fname == 'assets/css/code.css']
+                # code.css will be generated by us if it does not exist in
+                # FILES_FOLDERS or theme assets.  It is guaranteed that the
+                # generation will happen before this task.
                 task = {
                     'file_dep': list(file_dep),
-                    'task_dep': ['copy_assets'],
+                    'task_dep': ['copy_assets', 'copy_files'],
                     'basename': str(self.name),
                     'name': str(output_path),
                     'actions': [(build_bundle, (name, file_dep))],


Test patch: