internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: sopel-irc/sopel

Issue:
find_updates: No error handling on JSON fetch
See this code:

https://github.com/sopel-irc/sopel/blob/b105fe4aaa6c1cd258337e60a4f17c1a0751ecb5/sopel/modules/find_updates.py#L49

There's no error-handling at all. If the JSON doesn't parse, Sopel will spit out an exception. If the JSON URL won't load for some reason (times out, connection gets reset, domain name expires, etc.), Sopel will spit out an exception. These are just examples.

This code really needs to be rewritten with a robust `try`/`catch` structure to catch as many possible error conditions as possible. It probably wouldn't have prevented e.g. #1433, but we still should gracefully handle failures in the update checker. At present the `latest.json` file Sopel checks is hosted on Netlify, with very good uptime, but the site is still (very) rarely unavailable.

Bonus points for implementing some kind of logic to detect several failed update checks in a row and also alert the bot's owner to that issue, in case there's a networking issue on Sopel's host machine, or a problem with the update endpoint.


Discussion:

Resolving patch:
diff --git a/sopel/modules/find_updates.py b/sopel/modules/find_updates.py
--- a/sopel/modules/find_updates.py
+++ b/sopel/modules/find_updates.py
@@ -39,13 +39,45 @@ def startup_version_check(bot, trigger):
         check_version(bot)
 
 
+def _check_succeeded(bot):
+    bot.memory['update_failures'] = 0
+
+
+def _check_failed(bot):
+    bot.memory['update_failures'] = 1 + bot.memory.get('update_failures', 0)
+
+
 @sopel.module.interval(wait_time)
 def check_version(bot):
     version = sopel.version_info
+    success = False
+
+    try:
+        r = requests.get(version_url, timeout=(5, 5))
+    except requests.exceptions.RequestException:
+        _check_failed(bot)
+    else:
+        success = True
+
+    try:
+        if success:
+            info = r.json()
+    except ValueError:
+        # TODO: use JSONDecodeError when dropping Pythons < 3.5
+        _check_failed(bot)
+
+    if not success and bot.memory.get('update_failures', 0) > 4:
+        bot.say("I haven't been able to check for updates in a while. "
+                "Please verify that {} is working and I can reach it."
+                .format(version_url), bot.config.core.owner)
+        bot.say("If this issue persists, please alert the Sopel dev team in "
+                "#sopel on freenode, or open a GitHub issue: "
+                "https://github.com/sopel-irc/sopel/issues",
+                bot.config.core.owner)
+        return
+
+    _check_succeeded(bot)
 
-    # TODO: Python3 specific. Disable urllib warning from config file.
-    # requests.packages.urllib3.disable_warnings()
-    info = requests.get(version_url).json()
     if version.releaselevel == 'final':
         latest = info['version']
         notes = info['release_notes']


Test patch:
Repository: sopel-irc/sopel

Issue:
ip: MaxMind GeoLite2 Changes
### Description
MaxMind appears to now require an account to download the GeoLite2 databases, which Sopel currently uses. The geolite.maxmind.com domain, where Sopel downloads the databases, no longer exists, which is causing tests to fail.

### Reproduction steps
1. Clone master
2. Run `./pytest_run.py -v sopel/modules/ip.py`
3. Observe `sopel/modules/ip.py::test_example_ip_0 FAILED`

### Expected behavior
Test passes

### Notes
There seems to be an API with a [python library](https://pypi.org/project/geoip2/), but that also requires an account.


Discussion:
[This](https://blog.maxmind.com/2019/12/18/significant-changes-to-accessing-and-using-geolite2-databases/) (CCPA nonsense) is why. Would have been nice of them to give a bit more than 12 days' notice.

I'll find time to look over the license agreement and decide if we can reasonably ship a built-in key like the reddit plugin does, or if `ip` will have to become an external package due to being unusable without the bot admin signing up for an account.

Resolving patch:
diff --git a/sopel/modules/ip.py b/sopel/modules/ip.py
--- a/sopel/modules/ip.py
+++ b/sopel/modules/ip.py
@@ -19,6 +19,7 @@
 
 from sopel.config.types import FilenameAttribute, StaticSection
 from sopel.module import commands, example
+from sopel.tools import web
 
 urlretrieve = None
 try:
@@ -89,10 +90,19 @@ def _find_geoip_db(bot):
     elif urlretrieve:
         LOGGER.info('Downloading GeoIP database')
         bot.say('Downloading GeoIP database, please wait...')
-        geolite_urls = [
-            'https://geolite.maxmind.com/download/geoip/database/GeoLite2-City.tar.gz',
-            'https://geolite.maxmind.com/download/geoip/database/GeoLite2-ASN.tar.gz'
-        ]
+
+        common_params = {'license_key': 'JXBEmLjOzislFnh4', 'suffix': 'tar.gz'}
+        base_url = 'https://download.maxmind.com/app/geoip_download'
+        geolite_urls = []
+
+        for edition in ['ASN', 'City']:
+            geolite_urls.append(
+                '{base}?{params}'.format(
+                    base=base_url,
+                    params=web.urlencode(dict(common_params, **{'edition_id': 'GeoLite2-%s' % edition})),
+                )
+            )
+
         for url in geolite_urls:
             LOGGER.debug('GeoIP Source URL: %s', url)
             full_path = os.path.join(config.core.homedir, url.split("/")[-1])


Test patch:
Repository: sopel-irc/sopel

Issue:
cli.utils: wrong example for `find_config()`
The "Example" in the documentation for this function wasn't updated when it got moved from `run_script` to `cli.utils`.

https://github.com/sopel-irc/sopel/blob/91cdff5a30dce03d4578445f7651378838ded2e6/sopel/cli/utils.py#L220-L232


Discussion:

Resolving patch:
diff --git a/sopel/cli/utils.py b/sopel/cli/utils.py
--- a/sopel/cli/utils.py
+++ b/sopel/cli/utils.py
@@ -217,18 +217,19 @@ def find_config(config_dir, name, extension='.cfg'):
 
     Example::
 
-        >>> from sopel import run_script
+        >>> from sopel.cli import utils
+        >>> from sopel import config
         >>> os.listdir()
         ['local.cfg', 'extra.ini']
         >>> os.listdir(config.DEFAULT_HOMEDIR)
         ['config.cfg', 'extra.ini', 'plugin.cfg', 'README']
-        >>> run_script.find_config(config.DEFAULT_HOMEDIR, 'local.cfg')
+        >>> utils.find_config(config.DEFAULT_HOMEDIR, 'local.cfg')
         'local.cfg'
-        >>> run_script.find_config(config.DEFAULT_HOMEDIR, 'local')
+        >>> utils.find_config(config.DEFAULT_HOMEDIR, 'local')
         '/home/username/.sopel/local'
-        >>> run_script.find_config(config.DEFAULT_HOMEDIR, 'config')
+        >>> utils.find_config(config.DEFAULT_HOMEDIR, 'config')
         '/home/username/.sopel/config.cfg'
-        >>> run_script.find_config(config.DEFAULT_HOMEDIR, 'extra', '.ini')
+        >>> utils.find_config(config.DEFAULT_HOMEDIR, 'extra', '.ini')
         '/home/username/.sopel/extra.ini'
 
     """


Test patch:
Repository: sopel-irc/sopel

Issue:
reddit: URL-encoded characters don't match in links
In the "post" pattern, for example, `\w` won't match the `%` in a URL-encoded character:
https://github.com/sopel-irc/sopel/blob/7cdefb91597cd7f995bd59eac3c7bbcc52da1964/sopel/modules/reddit.py#L42

Of course, anywhere a post's "slugified" title (not sure what else to call it) should allow `%`, too. We should be fine with other stuff. IDs only contain alphanumeric characters, subreddit names must be alphanumeric (plus `_`), and usernames are alphanumeric plus `-` and `_` (already covered).
reddit: URL-encoded characters don't match in links
In the "post" pattern, for example, `\w` won't match the `%` in a URL-encoded character:
https://github.com/sopel-irc/sopel/blob/7cdefb91597cd7f995bd59eac3c7bbcc52da1964/sopel/modules/reddit.py#L42

Of course, anywhere a post's "slugified" title (not sure what else to call it) should allow `%`, too. We should be fine with other stuff. IDs only contain alphanumeric characters, subreddit names must be alphanumeric (plus `_`), and usernames are alphanumeric plus `-` and `_` (already covered).


Discussion:

Resolving patch:
diff --git a/sopel/modules/reddit.py b/sopel/modules/reddit.py
--- a/sopel/modules/reddit.py
+++ b/sopel/modules/reddit.py
@@ -39,7 +39,7 @@
 
 domain = r'https?://(?:www\.|old\.|pay\.|ssl\.|[a-z]{2}\.)?reddit\.com'
 subreddit_url = r'%s/r/([\w-]+)/?$' % domain
-post_url = r'%s/r/\S+?/comments/([\w-]+)(?:/[\w]+)?/?$' % domain
+post_url = r'%s/r/\S+?/comments/([\w-]+)(?:/[\w%%]+)?/?$' % domain
 short_post_url = r'https?://redd\.it/([\w-]+)'
 user_url = r'%s/u(?:ser)?/([\w-]+)' % domain
 comment_url = r'%s/r/\S+?/comments/\S+?/\S+?/([\w-]+)' % domain


Test patch:
Repository: sopel-irc/sopel

Issue:
xkcd outputs link even if triggered by one
The `xkcd` plugin shouldn't output a link to the comic if it was triggered by a link.

See `wikipedia`, for example. This is an anti-cycling measure in case multiple bots handling the same link(s) are present in the channel.


Discussion:

Resolving patch:
diff --git a/sopel/modules/xkcd.py b/sopel/modules/xkcd.py
--- a/sopel/modules/xkcd.py
+++ b/sopel/modules/xkcd.py
@@ -96,7 +96,7 @@ def xkcd(bot, trigger):
     say_result(bot, requested)
 
 
-def numbered_result(bot, query, latest):
+def numbered_result(bot, query, latest, commanded=True):
     max_int = latest['num']
     if query > max_int:
         bot.say(("Sorry, comic #{} hasn't been posted yet. "
@@ -117,16 +117,18 @@ def numbered_result(bot, query, latest):
         # Negative: go back that many from current
         requested = get_info(max_int + query)
 
-    say_result(bot, requested)
+    say_result(bot, requested, commanded)
 
 
-def say_result(bot, result):
-    message = '{} | {} | Alt-text: {}'.format(result['url'], result['title'],
-                                              result['alt'])
+def say_result(bot, result, commanded=True):
+    message = '{}{} | Alt-text: {}'.format(
+        result['url'] + ' | ' if commanded else '',
+        result['title'], result['alt']
+    )
     bot.say(message)
 
 
 @url(r'xkcd.com/(\d+)')
 def get_url(bot, trigger, match):
     latest = get_info()
-    numbered_result(bot, int(match.group(1)), latest)
+    numbered_result(bot, int(match.group(1)), latest, commanded=False)


Test patch:
Repository: sopel-irc/sopel

Issue:
announce: Batch up to MAXTARGETS channels at a time if available
The simple loop in `announce.py` is kind of...meh. The number of lines it actually sends to the server could be reduced as much as 75% depending on how many channels Sopel is in when `.announce` is called.

Some IRC servers (Hybrid, for example) send a `MAXTARGETS` field in the `ISUPPORT` properties when a client connects. If this is present, it would be useful for `announce.py` to have that information and send its announcement messages to `MAXTARGETS` channels at a time.

Long-term thing, more of an optimization than anything.


Discussion:
ISUPPORT properties are set to become available in 7.0. This can wait a bit for that system (introduced in #1758) to stabilize, then get implemented in 7.1.

Resolving patch:
diff --git a/sopel/modules/announce.py b/sopel/modules/announce.py
--- a/sopel/modules/announce.py
+++ b/sopel/modules/announce.py
@@ -12,12 +12,30 @@
 from sopel import plugin
 
 
+def _chunks(items, size):
+    """Break a list of items into groups.
+
+    :param items: the collection of items to chunk
+    :type items: :term:`iterable`
+    :param int size: the size of each chunk
+    """
+    # from https://stackoverflow.com/a/312464/5991 with modified names for readability
+    for delim in range(0, len(items), size):
+        yield items[delim:delim + size]
+
+
 @plugin.command('announce')
 @plugin.example('.announce Some important message here')
 @plugin.require_admin('Sorry, I can\'t let you do that', reply=True)
 @plugin.output_prefix('[ANNOUNCEMENT] ')
 def announce(bot, trigger):
-    """Send an announcement to all channels the bot is in"""
-    for channel in bot.channels:
-        bot.say(trigger.group(2), channel)
+    """Send an announcement to all channels the bot is in."""
+    if trigger.group(2) is None:
+        bot.reply('Announce what? I need a message to say.')
+        return
+
+    size = bot.isupport.get('TARGMAX', {}).get('PRIVMSG', 1)
+    channels = _chunks(bot.channels.keys(), size)
+    for cgroup in channels:
+        bot.say(trigger.group(2), ','.join(cgroup))
     bot.reply('Announce complete.')


Test patch:
Repository: sopel-irc/sopel

Issue:
add ping/pong response to ping.py
### The problem
The default ping module doesn't include ping/pong despite it's name and I'd rather not replace it just for that (I could move it out of core hacks but why not upstream)!

### The solution
Add it, I will after I write this.

### Example

09:54:36 <+RhinosF1> .ping
09:54:37 <@ZppixBot-test> Pong!




Discussion:

Resolving patch:
diff --git a/sopel/modules/ping.py b/sopel/modules/ping.py
--- a/sopel/modules/ping.py
+++ b/sopel/modules/ping.py
@@ -9,7 +9,7 @@
 
 import random
 
-from sopel.module import priority, rule, thread
+from sopel.module import commands, priority, rule, thread
 
 
 @rule(r'(?i)(hi|hello|hey),? $nickname[ \t]*$')
@@ -29,3 +29,9 @@ def rude(bot, trigger):
 @thread(False)
 def interjection(bot, trigger):
     bot.say(trigger.nick + '!')
+
+
+@commands('ping')
+def ping(bot, trigger):
+    """Reply to ping command."""
+    bot.reply('Pong!')


Test patch:
Repository: sopel-irc/sopel

Issue:
rule: $nick in regex pattern is confusing
### The problem

When using `sopel.module.rule`, you can have the variable `$nick` in the regex pattern:

```
r'$nickhello' # will match "Bot: hello"
r'$nick hello' # won't match "Bot: hello", but "Bot:  hello" (two spaces)
```

The `$nick` variable is super confusing as it should probably accept `\s*` instead of `\s+` so a pattern like `r'$nick hello'` would be way easier to read and be less prone to error.

### The solution

I don't know yet. Is `\s*` an acceptable change? Do we need to wait until Sopel 8 for that?

### Notes

More documentation about these variables is required. Also this comes from a review by @dgw on #1873 where he points out that my comment should be more visible, and not just in a corner of the test suite.



Discussion:

Resolving patch:
diff --git a/sopel/tools/__init__.py b/sopel/tools/__init__.py
--- a/sopel/tools/__init__.py
+++ b/sopel/tools/__init__.py
@@ -214,9 +214,11 @@ def compile_rule(nick, pattern, alias_nicks):
         nick = re.escape(nick)
 
     pattern = pattern.replace('$nickname', nick)
-    pattern = pattern.replace('$nick', r'{}[,:]\s+'.format(nick))
+    pattern = pattern.replace('$nick ', r'{}[,:]\s*'.format(nick))  # @rule('$nick hi')
+    pattern = pattern.replace('$nick', r'{}[,:]\s+'.format(nick))  # @rule('$nickhi')
     flags = re.IGNORECASE
     if '\n' in pattern:
+        # See https://docs.python.org/3/library/re.html#re.VERBOSE
         flags |= re.VERBOSE
     return re.compile(pattern, flags)
 


Test patch:
Repository: sopel-irc/sopel

Issue:
settings: Bot assumes `+` for modes
The `core.modes` setting is assumed to be just a string of letters, and the leading `+` is assumed by `coretasks`: https://github.com/sopel-irc/sopel/blob/a33caf15090d61b90dc831f55cc195e56185dad3/sopel/coretasks.py#L155-L156

@cottongin rightly pointed out on IRC that sometimes it's desirable to _remove_ modes the IRC server sets by default. While some IRCds will happily accept `MODE nickname +-abc` (like freenode's), it's not a universal workaround.

I'm happy to add this to 7.1 because 1) it's a pretty trivial change to implement and 2) there's an obvious backward-compatible way to parse the setting.

Proposal: Add the leading `+` automatically only if the `core.modes` setting doesn't contain a prefix character (`+` or `-`).


Discussion:

Resolving patch:
diff --git a/sopel/coretasks.py b/sopel/coretasks.py
--- a/sopel/coretasks.py
+++ b/sopel/coretasks.py
@@ -153,7 +153,11 @@ def startup(bot, trigger):
     auth_after_register(bot)
 
     modes = bot.config.core.modes
-    bot.write(('MODE', '%s +%s' % (bot.nick, modes)))
+    if modes:
+        if not modes.startswith(('+', '-')):
+            # Assume "+" by default.
+            modes = '+' + modes
+        bot.write(('MODE', bot.nick, modes))
 
     bot.memory['retry_join'] = dict()
 


Test patch:
Repository: sopel-irc/sopel

Issue:
Invalid format of `AUTHENTICATE` message
## Description

Sopel uses a literal `\0` instead of the null byte in AUTHENTICATE payloads.

### Reproduction steps

1. use this config:

```
auth_username = jilles
auth_password = sesame
auth_method = sasl
```

Sopel will then send this to the server:

```
AUTHENTICATE amlsbGVzXDBqaWxsZXNcMHNlc2FtZQ==
```

### Expected behavior

It should send this:

```
AUTHENTICATE amlsbGVzAGppbGxlcwBzZXNhbWU=
```

base64-decoded, they can be represented as Python literals like this, respectively: `"jilles\\0jilles\\0sesame"` and `"jilles\0jilles\0sesame"`.

### Environment
- Sopel `.version`: master
- Sopel installed via: `setup.py install`

###

This is because of this change: https://github.com/sopel-irc/sopel/pull/1928/files#diff-a3d2e4c9269312d687f6dbab17168611fd71629ef36b099985b9ba4bd293c853R838

this is equivalent to replacing `"\0"` with `"\\0"`.

I'm guessing this was to fix some sort of lint warning, but it also changes the behavior. Instead, you should probably use a `bytes` object.


Discussion:
Nice catch!

Resolving patch:
diff --git a/sopel/coretasks.py b/sopel/coretasks.py
--- a/sopel/coretasks.py
+++ b/sopel/coretasks.py
@@ -835,7 +835,7 @@ def auth_proceed(bot, trigger):
     else:
         return
     sasl_username = sasl_username or bot.nick
-    sasl_token = r'\0'.join((sasl_username, sasl_username, sasl_password))
+    sasl_token = '\x00'.join((sasl_username, sasl_username, sasl_password))
     send_authenticate(bot, sasl_token)
 
 


Test patch:
Repository: sopel-irc/sopel

Issue:
Topic command expects that bot needs to be a channel operator
### Description
Topic command expects that bot needs to be channel operator, even when the channel does not have +t mode set.

### Reproduction steps
join a channel where +t is not set
```
<petteri> .topic test
<bot> petteri: I'm not a channel operator!
-- Mode #test [+o bot] by petteri
<petteri> .topic test
-- bot has changed topic for #test from "test" to "Welcome to: #test | Topic: test"
```

### Expected behavior
bot should be able to set the topic on a channel without +t even when it is not a channel operator


### Environment
- Sopel `.version`: [c705f450e56b9ddded10226c640a607249e33710]
- Sopel installed via: [source]
- Python version: [3.7.4]
- Operating system: [Ubuntu 19.04]
- IRCd `/version`: [oragono v2.0.0]




Discussion:
Rewriting our MODE tracking would be good to do for 7.1, now that we parse and expose ISUPPORT data. This would require adding real mode-tracking for channels, though; currently Sopel only pays attention to channel privileges.

Disclaimer: This could get bumped to a later release depending on how long it takes to design the interface for plugins to access channels' MODE info.
@Exirel Does this fruit seem low-hanging enough to get done for 7.1 yet, or do you wanna punt it?

Resolving patch:
diff --git a/sopel/modules/adminchannel.py b/sopel/modules/adminchannel.py
--- a/sopel/modules/adminchannel.py
+++ b/sopel/modules/adminchannel.py
@@ -287,16 +287,19 @@ def kickban(bot, trigger):
 
 
 @plugin.require_chanmsg
-@plugin.require_privilege(plugin.OP, ERROR_MESSAGE_NO_PRIV)
 @plugin.command('topic')
 def topic(bot, trigger):
     """Change the channel topic
 
-    The bot must be a channel operator for this command to work.
+    The bot must be a channel operator for this command to work in +t channels.
     """
-    if bot.channels[trigger.sender].privileges[bot.nick] < plugin.HALFOP:
+    mode_t = bot.channels[trigger.sender].modes.get("t", False)
+    if mode_t and not bot.has_channel_privilege(trigger.sender, plugin.HALFOP):
         bot.reply(ERROR_MESSAGE_NOT_OP)
         return
+    if mode_t and bot.channels[trigger.sender].privileges[trigger.nick] < plugin.HALFOP:
+        bot.reply(ERROR_MESSAGE_NO_PRIV)
+        return
     if not trigger.group(2):
         return
     channel = trigger.sender.lower()


Test patch:
Repository: sopel-irc/sopel

Issue:
reddit: floating point error in upvote ratio
https://github.com/sopel-irc/sopel/blob/d850844870ac62e4568b4743bd38f0f90d76af7d/sopel/modules/reddit.py#L183

Occasionally this code results in an output ratio like "57.99999999999999%".

Should be super easy for anyone who wants a quick PR.


Discussion:
Hey, I would like to work on this issue. I am a first-time contributor. Can you suggest relevant resources to be studied for working on this issue? It seems simple though.
@sanyam1997 using something like the Python built-in string formatting tools make this very simple e.g.:

``` python
percent = color("{:.2%}".format(s.upvote_ratio), point_color)
```

I have not tested this myself on my own bot instance but it should work
So how can I test the code after installing it following git repo?

Resolving patch:
diff --git a/sopel/modules/reddit.py b/sopel/modules/reddit.py
--- a/sopel/modules/reddit.py
+++ b/sopel/modules/reddit.py
@@ -192,7 +192,7 @@ def say_post_info(bot, trigger, id_, show_link=True, show_comments_link=False):
 
         points_text = 'point' if s.score == 1 else 'points'
 
-        percent = color(unicode(s.upvote_ratio * 100) + '%', point_color)
+        percent = color('{:.1%}'.format(s.upvote_ratio), point_color)
 
         comments_link = ''
         if show_comments_link:


Test patch: