internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: getnikola/nikola

Issue:
Replacing husl by hsluv?
<!--
Before creating an issue:
* make sure you are using an up-to-date version of Nikola
* search for existing issues that might be related

Describe your requested features as precisely as possible. -->

Nikola extras has husl. husl's home page <http://www.husl-colors.org/> is dead.

It seems that <https://www.hsluv.org/> and <https://github.com/hsluv/hsluv> is the currently active one. And its Python implementation is <https://github.com/hsluv/hsluv>.



Discussion:

Resolving patch:
diff --git a/nikola/utils.py b/nikola/utils.py
--- a/nikola/utils.py
+++ b/nikola/utils.py
@@ -80,9 +80,9 @@
     YAML = None
 
 try:
-    import husl
+    import hsluv
 except ImportError:
-    husl = None
+    hsluv = None
 
 __all__ = ('CustomEncoder', 'get_theme_path', 'get_theme_path_real',
            'get_theme_chain', 'load_messages', 'copy_tree', 'copy_file',
@@ -1752,7 +1752,7 @@ def colorize_str_from_base_color(string, base_color):
 
     Make up to 16 attempts (number of bytes returned by hashing) at picking a
     hue for our color at least 27 deg removed from the base color, leaving
-    lightness and saturation untouched using HUSL colorspace.
+    lightness and saturation untouched using HSLuv colorspace.
     """
     def hash_str(string, pos):
         return hashlib.md5(string.encode('utf-8')).digest()[pos]
@@ -1760,17 +1760,17 @@ def hash_str(string, pos):
     def degreediff(dega, degb):
         return min(abs(dega - degb), abs((degb - dega) + 360))
 
-    if husl is None:
-        req_missing(['husl'], 'Use color mixing (section colors)',
+    if hsluv is None:
+        req_missing(['hsluv'], 'Use color mixing (section colors)',
                     optional=True)
         return base_color
-    h, s, l = husl.hex_to_husl(base_color)
+    h, s, l = hsluv.hex_to_hsluv(base_color)
     old_h = h
     idx = 0
     while degreediff(old_h, h) < 27 and idx < 16:
         h = 360.0 * (float(hash_str(string, idx)) / 255)
         idx += 1
-    return husl.husl_to_hex(h, s, l)
+    return hsluv.hsluv_to_hex((h, s, l))
 
 
 def colorize_str(string: str, base_color: str, presets: dict):
@@ -1782,7 +1782,7 @@ def colorize_str(string: str, base_color: str, presets: dict):
 
 def color_hsl_adjust_hex(hexstr, adjust_h=None, adjust_s=None, adjust_l=None):
     """Adjust a hex color using HSL arguments, adjustments in percentages 1.0 to -1.0. Returns a hex color."""
-    h, s, l = husl.hex_to_husl(hexstr)
+    h, s, l = hsluv.hex_to_husl(hexstr)
 
     if adjust_h:
         h = h + (adjust_h * 360.0)
@@ -1793,7 +1793,7 @@ def color_hsl_adjust_hex(hexstr, adjust_h=None, adjust_s=None, adjust_l=None):
     if adjust_l:
         l = l + (adjust_l * 100.0)
 
-    return husl.husl_to_hex(h, s, l)
+    return hsluv.hsluv_to_hex((h, s, l))
 
 
 def dns_sd(port, inet6):


Test patch:
Repository: getnikola/nikola

Issue:
Incorrect time zone guessing on macOS
On macOS Big Sur 11.2, we guess the full path to the zoneinfo file (instead of just the zone name).

```
$ nikola init
…
Time zone [private/var/db/timezone/tz/2021a.1.0/zoneinfo/Europe/Warsaw]:
```


Discussion:

Resolving patch:
diff --git a/nikola/packages/tzlocal/unix.py b/nikola/packages/tzlocal/unix.py
--- a/nikola/packages/tzlocal/unix.py
+++ b/nikola/packages/tzlocal/unix.py
@@ -116,8 +116,9 @@ def _get_localzone(_root="/"):
         while start != 0:
             tzpath = tzpath[start:]
             try:
-                dateutil.tz.gettz(tzpath)
-                return tzpath
+                tested_tz = dateutil.tz.gettz(tzpath)
+                if tested_tz:
+                    return tzpath
             except Exception:
                 pass
             start = tzpath.find("/") + 1


Test patch:
Repository: getnikola/nikola

Issue:
Data files are not tracked as dependencies
<!--
Before creating an issue:
* make sure you are using an up-to-date version of Nikola
* search for existing issues that might be related

Make sure to:
* provide information about your environment (below)
* include all the output you get, and any other information related to your problem

Nikola v7.6.4, as provided by Ubuntu, is NOT SUPPORTED.
If you are using this version, you should upgrade: https://getnikola.com/getting-started.html
-->

### Environment

**Python Version:**
Python 3.6.9 (default, Mar 15 2022, 13:55:28) 
[GCC 8.4.0] on linux

**Nikola Version:**
Nikola v8.2.0

**Operating System:**
Ubuntu

### Description:
When using the `data` attribute in a post or page the included data file is not added to the dependency file of the post or page.
So a change to the data file does not trigger a rebuild of the including pages and posts.


Discussion:

Resolving patch:
diff --git a/nikola/post.py b/nikola/post.py
--- a/nikola/post.py
+++ b/nikola/post.py
@@ -324,6 +324,7 @@ def _load_data(self):
         for lang in self.translations:
             if self.meta[lang].get('data') is not None:
                 self.data[lang] = utils.load_data(self.meta[lang]['data'])
+                self.register_depfile(self.meta[lang]['data'], lang=lang)
 
     def _load_translated_metadata(self, default_metadata):
         """Load metadata from all translation sources."""


Test patch:
Repository: getnikola/nikola

Issue:
Cannot build, AttributeError error
<!--
Before creating an issue:
* make sure you are using an up-to-date version of Nikola
* search for existing issues that might be related

Make sure to:
* provide information about your environment (below)
* include all the output you get, and any other information related to your problem

Nikola v7.6.4, as provided by Ubuntu, is NOT SUPPORTED.
If you are using this version, you should upgrade: https://getnikola.com/getting-started.html
-->

### Environment

**Python Version:** 3.10

**Nikola Version:** 8.2.2

**Operating System:** MacOS

### Description:

I get the following error:

```python
nikola_find_formatter_class
    if alias.lower().contains('html'):
AttributeError: 'str' object has no attribute 'contains'
```
See gh action: https://github.com/Woile/woile.github.io/runs/6389001428?check_suite_focus=true

Python does not have a `contains` method for strings, I'm submitting a PR now


Discussion:

Resolving patch:
diff --git a/nikola/utils.py b/nikola/utils.py
--- a/nikola/utils.py
+++ b/nikola/utils.py
@@ -1714,7 +1714,7 @@ def wrap(self, source, *args):
 
 def nikola_find_formatter_class(alias):
     """Nikola-specific version of find_formatter_class."""
-    if alias.lower().contains('html'):
+    if "html" in alias.lower():
         return NikolaPygmentsHTML
     return _original_find_formatter_class(alias)
 


Test patch:
diff --git a/tests/test_utils.py b/tests/test_utils.py
--- a/tests/test_utils.py
+++ b/tests/test_utils.py
@@ -14,11 +14,13 @@
 from nikola.utils import (
     TemplateHookRegistry,
     TranslatableSetting,
+    NikolaPygmentsHTML,
     demote_headers,
     get_asset_path,
     get_crumbs,
     get_theme_chain,
     get_translation_candidate,
+    nikola_find_formatter_class,
     write_metadata,
     bool_from_meta,
     parselinenos
@@ -639,3 +641,7 @@ def test_parselinenos():
         parselinenos('-', 10)
     with pytest.raises(ValueError):
         parselinenos('3-1', 10)
+
+
+def test_nikola_find_formatter_class_returns_pygments_class():
+    assert NikolaPygmentsHTML == nikola_find_formatter_class("html")
Repository: getnikola/nikola

Issue:
nikola auto loops infinitely
### Environment

**Python Version:** 3.7.13

**Nikola Version:** 8.2.3

**Operating System:** Debian

### Description:

When I run `nikola auto` for my site and load the local url in the browser, sometimes after making some changes to a blog post the site will reload over and over again.

In the terminal where `nikola auto` is running I can see it refreshing a lot of the posts/taxonomies and site xml files. I end up having to interrupt the process, restart it, and reload my browser.

Let me know if there's anything I can do to help debug this!


Discussion:
I checked this problem since made practically un-usable `nikola` for me and I don't know if I'm crazy or what but the code for handling the filesystem events doesn't make sense:

```python
# Based on code from the 'hachiko' library by John Biesnecker — thanks!
# https://github.com/biesnecker/hachiko
class NikolaEventHandler:
    """A Nikola-specific event handler for Watchdog. Based on code from hachiko."""

    def __init__(self, function, loop):
        """Initialize the handler."""
        self.function = function
        self.loop = loop

    async def on_any_event(self, event):
        """Handle all file events."""
        await self.function(event)

    def dispatch(self, event):
        """Dispatch events to handler."""
        self.loop.call_soon_threadsafe(asyncio.ensure_future, self.on_any_event(event))

```

if I get this right, this code is putting in the queue any event, also opening a file. If I filter on `FileModifiedEvent` then it works.

Resolving patch:
diff --git a/nikola/plugins/command/auto/__init__.py b/nikola/plugins/command/auto/__init__.py
--- a/nikola/plugins/command/auto/__init__.py
+++ b/nikola/plugins/command/auto/__init__.py
@@ -587,13 +587,11 @@ def __init__(self, function, loop):
         self.function = function
         self.loop = loop
 
-    async def on_any_event(self, event):
-        """Handle all file events."""
-        await self.function(event)
-
     def dispatch(self, event):
         """Dispatch events to handler."""
-        self.loop.call_soon_threadsafe(asyncio.ensure_future, self.on_any_event(event))
+        if event.event_type in {"opened", "closed"}:
+            return
+        self.loop.call_soon_threadsafe(asyncio.ensure_future, self.function(event))
 
 
 class ConfigEventHandler(NikolaEventHandler):
@@ -601,11 +599,10 @@ class ConfigEventHandler(NikolaEventHandler):
 
     def __init__(self, configuration_filename, function, loop):
         """Initialize the handler."""
+        super().__init__(function, loop)
         self.configuration_filename = configuration_filename
-        self.function = function
-        self.loop = loop
 
-    async def on_any_event(self, event):
+    def dispatch(self, event):
         """Handle file events if they concern the configuration file."""
         if event._src_path == self.configuration_filename:
-            await self.function(event)
+            super().dispatch(event)


Test patch:
Repository: getnikola/nikola

Issue:
Listings for .DS_Store files are not generated, but .DS_Store appears in listing indexes
### Discussed in https://github.com/getnikola/nikola/discussions/3696

<div type='discussions-op-text'>

<sup>Originally posted by **allenbrunson** June 27, 2023</sup>
hey folks,

i just started using the nikola "check" command to find mistakes in my pages (great feature by the way), and it prints this:

[2023-06-26 19:07:45] WARNING: check: Broken link in output/listings/index.html: .DS_Store.html

this sure looks to me like a resurgence of issue 3099, from way back in 2018, which should be fixed?

https://github.com/getnikola/nikola/issues/3099

thanx
allen</div>


Discussion:

Resolving patch:
diff --git a/nikola/plugins/task/listings.py b/nikola/plugins/task/listings.py
--- a/nikola/plugins/task/listings.py
+++ b/nikola/plugins/task/listings.py
@@ -107,6 +107,7 @@ def gen_tasks(self):
         """Render pretty code listings."""
         # Things to ignore in listings
         ignored_extensions = (".pyc", ".pyo")
+        ignored_files = (".DS_Store",)
 
         def render_listing(in_name, out_name, input_folder, output_folder, folders=[], files=[]):
             needs_ipython_css = False
@@ -183,7 +184,9 @@ def render_listing(in_name, out_name, input_folder, output_folder, folders=[], f
 
         for input_folder, output_folder in self.kw['listings_folders'].items():
             for root, dirs, files in os.walk(input_folder, followlinks=True):
-                files = [f for f in files if os.path.splitext(f)[-1] not in ignored_extensions]
+                files = [f for f in files
+                         if os.path.splitext(f)[-1] not in ignored_extensions and
+                         f not in ignored_files]
 
                 uptodate = {'c': self.site.GLOBAL_CONTEXT}
 
@@ -224,7 +227,7 @@ def render_listing(in_name, out_name, input_folder, output_folder, folders=[], f
                     'clean': True,
                 }, self.kw["filters"])
                 for f in files:
-                    if f == '.DS_Store':
+                    if f in ignored_files:
                         continue
                     ext = os.path.splitext(f)[-1]
                     if ext in ignored_extensions:


Test patch:
Repository: getnikola/nikola

Issue:
New plugin manager does not support symlinks
### Environment

**Python Version:** 3.11

**Nikola Version:** 8.3.0

**Operating System:** openSUSE Tumbleweed

### Description:

The new plugin manager does not detect plugins if the plugin folder is a symlink (e.g. `ln -s ../nikola-plugins/v8/projectpages plugins/projectpages`).

Apparently, `pathlib.Path.rglob` ignores symlinks. We’ll need a custom thing to scan directories that does not ignore symlinks.


Discussion:

Resolving patch:
diff --git a/nikola/plugin_manager.py b/nikola/plugin_manager.py
--- a/nikola/plugin_manager.py
+++ b/nikola/plugin_manager.py
@@ -31,6 +31,8 @@
 import importlib.util
 import time
 import sys
+
+from collections import deque
 from dataclasses import dataclass
 from pathlib import Path
 from typing import Dict, List, Optional, Type, TYPE_CHECKING, Set
@@ -102,9 +104,13 @@ def locate_plugins(self) -> List[PluginCandidate]:
         """Locate plugins in plugin_places."""
         self.candidates = []
 
+        plugin_folders: deque = deque([place for place in self.plugin_places if place.exists() and place.is_dir()])
         plugin_files: List[Path] = []
-        for place in self.plugin_places:
-            plugin_files += place.rglob("*.plugin")
+        while plugin_folders:
+            base_folder = plugin_folders.popleft()
+            items = list(base_folder.iterdir())
+            plugin_folders.extend([item for item in items if item.is_dir() and item.name != "__pycache__"])
+            plugin_files.extend([item for item in items if item.suffix == ".plugin" and not item.is_dir()])
 
         for plugin_file in plugin_files:
             source_dir = plugin_file.parent


Test patch:
Repository: getnikola/nikola

Issue:
Directory traversal vulnerability
I can't find any information about security reporting, so I'll just put this here.
It appears that this project may have copied some code from aiohttp, which we've recently discovered is vulnerable to a directory traversal attack:
https://github.com/aio-libs/aiohttp/security/advisories/GHSA-5h86-8mv2-jq9f

The similar-looking code is at: https://github.com/getnikola/nikola/blob/2f5cf49a8da230e31c91c8a7ad2a2689cde1fe43/nikola/plugins/command/auto/__init__.py#L543

Please verify if your code is vulnerable and fix if appropriate. The patch we used for aiohttp is https://github.com/aio-libs/aiohttp/pull/8079/files


Discussion:

Resolving patch:
diff --git a/nikola/plugins/command/auto/__init__.py b/nikola/plugins/command/auto/__init__.py
--- a/nikola/plugins/command/auto/__init__.py
+++ b/nikola/plugins/command/auto/__init__.py
@@ -37,6 +37,7 @@
 import typing
 import urllib.parse
 import webbrowser
+from pathlib import Path
 
 import blinker
 import pkg_resources
@@ -521,8 +522,13 @@ async def _handle(self, request: 'web.Request') -> 'web.Response':
     async def handle_file(self, request: 'web.Request', filename: str, from_index=None) -> 'web.Response':
         """Handle file requests."""
         try:
-            filepath = self._directory.joinpath(filename).resolve()
-            if not self._follow_symlinks:
+            unresolved_path = self._directory.joinpath(filename)
+            if self._follow_symlinks:
+                normalized_path = Path(os.path.normpath(unresolved_path))
+                normalized_path.relative_to(self._directory)
+                filepath = normalized_path.resolve()
+            else:
+                filepath = unresolved_path.resolve()
                 filepath.relative_to(self._directory)
         except (ValueError, FileNotFoundError) as error:
             # relatively safe


Test patch:
Repository: getnikola/nikola

Issue:
nikola auto: "AttributeError: 'FileOpenedEvent' object has no attribute '_src_path'. Did you mean: 'src_path'?"
<!--
Before creating an issue:
* make sure you are using an up-to-date version of Nikola
* search for existing issues that might be related

Make sure to:
* provide information about your environment (below)
* include all the output you get, and any other information related to your problem

Nikola v7.6.4, as provided by Ubuntu, is NOT SUPPORTED.
If you are using this version, you should upgrade: https://getnikola.com/getting-started.html
-->

### Environment

**Python Version:** 3.12.2

**Nikola Version:** SHA #b46b1211128 on my branch

**Operating System:** Debian Bullseye

### Description:

`nikola auto -b` produces an error

```
Exception in thread Thread-1:
Traceback (most recent call last):
  File "/usr/local/python-3.12.2/lib/python3.12/threading.py", line 1073, in _bootstrap_inner
    self.run()
  File "/home/andreas/workspace-famsik/dj3ei/venv/lib/python3.12/site-packages/watchdog/observers/api.py", line 223, in run
    self.dispatch_events(self.event_queue)
  File "/home/andreas/workspace-famsik/dj3ei/venv/lib/python3.12/site-packages/watchdog/observers/api.py", line 402, in dispatch_events
    handler.dispatch(event)
  File "/home/andreas/comp/nikola/nikola/plugins/command/auto/__init__.py", line 623, in dispatch
    if event._src_path == self.configuration_filename:
       ^^^^^^^^^^^^^^^
AttributeError: 'FileOpenedEvent' object has no attribute '_src_path'. Did you mean: 'src_path'?
```

## Initial analysis:

Our dependency `watchdog` came out with a new major release 4.0.0 and 4.0.1. The private attribute `_src_path` has been promoted official to `src_path`.

Fix should be easy, forthcoming.


Discussion:

Resolving patch:
diff --git a/nikola/plugins/command/auto/__init__.py b/nikola/plugins/command/auto/__init__.py
--- a/nikola/plugins/command/auto/__init__.py
+++ b/nikola/plugins/command/auto/__init__.py
@@ -620,5 +620,5 @@ def __init__(self, configuration_filename, function, loop):
 
     def dispatch(self, event):
         """Handle file events if they concern the configuration file."""
-        if event._src_path == self.configuration_filename:
+        if event.src_path == self.configuration_filename:
             super().dispatch(event)


Test patch:
Repository: sparcs-kaist/otlplus

Issue:
포탈 연동 안되어 있을 때 연결 페이지 링크 text 수정



Discussion:

Resolving patch:
diff --git a/apps/review/views.py b/apps/review/views.py
--- a/apps/review/views.py
+++ b/apps/review/views.py
@@ -555,9 +555,9 @@ def ReviewInsertView(request,lecture_id=-1,semester=0):
     lec_year = (semester/10)+2000
     lec_sem = semester%10
     if len(user_profile.student_id) < 1:
-        ctx = {'errttle':'포탈 연동이 되어있지 않습니다!','errmsg':'<a href="/review/portal/">여기</a>를 눌러 연동주세요!'}
+        ctx = {'errttle':'포탈 연동이 되어있지 않습니다!','errmsg':'<a href="/review/portal/">포탈 연동하러가기!</a>'}
     else:
-        ctx = {'errttle':'수강한 과목이 없습니다!','errmsg':'만약 수강한 과목이 있다면, <a href="/review/refresh/">여기</a>를 눌러 갱신해주세요!'}
+        ctx = {'errttle':'수강한 과목이 없습니다!','errmsg':'만약 수강한 과목이 있다면, <a href="/review/refresh/">여기를 눌러 갱신해주세요!</a>'}
     if semester % 10 > 4 or semester < 0 or semester > 1000:
         return render(request, 'review/error.html',ctx)
     lecture_list = user_profile.take_lecture_list.all()


Test patch:
Repository: sparcs-kaist/otlplus

Issue:
django.http.Http404가 import되지 않아 500에러 발생
밤새 이런 메일이 자꾸 날아와서 죽을 것 같다.
이것 때문에 몇번 깼는지 모르겠다.

빠르게 import해주도록 한다.

![image](https://cloud.githubusercontent.com/assets/4944891/16860221/d6b11f24-4a70-11e6-8392-6682b7e18b30.png)

django.http.Http404가 import되지 않아 500에러 발생
밤새 이런 메일이 자꾸 날아와서 죽을 것 같다.
이것 때문에 몇번 깼는지 모르겠다.

빠르게 import해주도록 한다.

![image](https://cloud.githubusercontent.com/assets/4944891/16860221/d6b11f24-4a70-11e6-8392-6682b7e18b30.png)



Discussion:

Resolving patch:
diff --git a/apps/review/views.py b/apps/review/views.py
--- a/apps/review/views.py
+++ b/apps/review/views.py
@@ -5,7 +5,7 @@
 from apps.session.models import UserProfile
 from apps.subject.models import Course, Lecture, Department, CourseFiltered, Professor
 from apps.review.models import Comment,CommentVote, MajorBestComment, LiberalBestComment
-from django.http import HttpResponse, HttpResponseRedirect, JsonResponse
+from django.http import HttpResponse, HttpResponseRedirect, JsonResponse, Http404
 from django.db.models import Q
 from datetime import datetime, timedelta, time, date
 from django.utils import timezone


Test patch:
Repository: mindspore-lab/mindnlp

Issue:
"Trainer" doesn't take into account the case that "loss_fn" doesn't need to be passed in.
"Trainer" does not take into account the case where "loss" is already defined in the model, and there is no need to pass "loss_fn" to "Trainer".


Discussion:

Resolving patch:
diff --git a/mindnlp/engine/trainer.py b/mindnlp/engine/trainer.py
--- a/mindnlp/engine/trainer.py
+++ b/mindnlp/engine/trainer.py
@@ -167,7 +167,14 @@ def forward_fn(inputs, labels):
             return_list = (loss,) + logits_list
             return return_list
 
-        grad_fn = value_and_grad(forward_fn, None, optimizer.parameters, has_aux=True)
+        def forward_without_loss_fn(inputs, labels):
+            loss_and_logits = net(*inputs, *labels)
+            return loss_and_logits
+
+        if self.loss_fn is None:
+            grad_fn = value_and_grad(forward_without_loss_fn, None, optimizer.parameters, has_aux=True)
+        else:
+            grad_fn = value_and_grad(forward_fn, None, optimizer.parameters, has_aux=True)
 
         def _run_step(inputs, labels):
             """Core process of each step, including the forward propagation process and back propagation of data."""


Test patch:
Repository: mindspore-lab/mindnlp

Issue:
multi30k dataset url not avaliable



Discussion:

Resolving patch:
diff --git a/mindnlp/dataset/machine_translation/multi30k.py b/mindnlp/dataset/machine_translation/multi30k.py
--- a/mindnlp/dataset/machine_translation/multi30k.py
+++ b/mindnlp/dataset/machine_translation/multi30k.py
@@ -29,15 +29,15 @@
 from mindnlp.utils import untar
 
 URL = {
-    "train": "http://www.quest.dcs.shef.ac.uk/wmt16_files_mmt/training.tar.gz",
-    "valid": "http://www.quest.dcs.shef.ac.uk/wmt16_files_mmt/validation.tar.gz",
-    "test": "http://www.quest.dcs.shef.ac.uk/wmt16_files_mmt/mmt16_task1_test.tar.gz",
+    "train": "https://openi.pcl.ac.cn/lvyufeng/multi30k/raw/branch/master/training.tar.gz",
+    "valid": "https://openi.pcl.ac.cn/lvyufeng/multi30k/raw/branch/master/validation.tar.gz",
+    "test": "https://openi.pcl.ac.cn/lvyufeng/multi30k/raw/branch/master/mmt16_task1_test.tar.gz",
 }
 
 MD5 = {
     "train": "8ebce33f4ebeb71dcb617f62cba077b7",
     "valid": "2a46f18dbae0df0becc56e33d4e28e5d",
-    "test": "f63b12fc6f95beb3bfca2c393e861063",
+    "test": "1586ce11f70cba049e9ed3d64db08843",
 }
 
 


Test patch: