internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: aio-libs-abandoned/aioredis-py

Issue:
Drop closed connections on Pool.acquire
Currently on redis server restart the poll is full of closed connections.

That causes assertion error on acquiring (https://github.com/aio-libs/aioredis/blob/master/aioredis/pool.py#L117)

I suggest dropping closed connections in `.fill_free` coroutine.



Discussion:

Resolving patch:
diff --git a/aioredis/pool.py b/aioredis/pool.py
--- a/aioredis/pool.py
+++ b/aioredis/pool.py
@@ -146,8 +146,18 @@ def release(self, conn):
         # FIXME: check event loop is not closed
         asyncio.async(self._wakeup(), loop=self._loop)
 
+    def _drop_closed(self):
+        for i in range(self.freesize):
+            conn = self._pool[0]
+            if conn.closed:
+                self._pool.popleft()
+            else:
+                self._pool.rotate(1)
+
     @asyncio.coroutine
     def _fill_free(self, *, override_min):
+        # drop closed connections first
+        self._drop_closed()
         while self.size < self.minsize:
             self._acquiring += 1
             try:
@@ -155,15 +165,20 @@ def _fill_free(self, *, override_min):
                 self._pool.append(conn)
             finally:
                 self._acquiring -= 1
+                # connection may be closed at yeild point
+                self._drop_closed()
         if self.freesize:
             return
-        if override_min and self.size < self.maxsize:
-            self._acquiring += 1
-            try:
-                conn = yield from self._create_new_connection()
-                self._pool.append(conn)
-            finally:
-                self._acquiring -= 1
+        if override_min:
+            while not self._pool and self.size < self.maxsize:
+                self._acquiring += 1
+                try:
+                    conn = yield from self._create_new_connection()
+                    self._pool.append(conn)
+                finally:
+                    self._acquiring -= 1
+                    # connection may be closed at yeild point
+                    self._drop_closed()
 
     def _create_new_connection(self):
         return create_redis(self._address,


Test patch:
diff --git a/tests/pool_test.py b/tests/pool_test.py
--- a/tests/pool_test.py
+++ b/tests/pool_test.py
@@ -302,3 +302,18 @@ def task2():
             tasks.append(asyncio.async(task1(_), loop=self.loop))
         tasks.append(asyncio.async(task2(), loop=self.loop))
         yield from asyncio.gather(*tasks, loop=self.loop)
+
+    @run_until_complete
+    def test_pool_with_closed_connections(self):
+        pool = yield from self.create_pool(
+            ('localhost', self.redis_port),
+            loop=self.loop,
+            minsize=1, maxsize=2)
+        self.assertEqual(1, pool.freesize)
+        conn1 = pool._pool[0]
+        conn1.close()
+        self.assertTrue(conn1.closed)
+        self.assertEqual(1, pool.freesize)
+        with (yield from pool) as conn2:
+            self.assertFalse(conn2.closed)
+            self.assertIsNot(conn1, conn2)
Repository: aio-libs-abandoned/aioredis-py

Issue:
Missing PyPI classifier for Python 3.6 support
Currently, the PyPI page shows the following as supported:

> Programming Language :: Python
> Programming Language :: Python :: 3
> Programming Language :: Python :: 3.3
> Programming Language :: Python :: 3.4
> Programming Language :: Python :: 3.5

However, Python 3.6 is part of the automated tests, and the README states it is supported,
so I'm presuming this is just an omission.


Discussion:

Resolving patch:
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -45,6 +45,7 @@ def read_version():
     'Programming Language :: Python :: 3.3',
     'Programming Language :: Python :: 3.4',
     'Programming Language :: Python :: 3.5',
+    'Programming Language :: Python :: 3.6',
     'Operating System :: POSIX',
     'Environment :: Web Environment',
     'Intended Audience :: Developers',


Test patch:
Repository: aio-libs-abandoned/aioredis-py

Issue:
Add a BUSYGROUP reply error
The XGROUP CREATE command can return a BUSYGROUP error when a group already exists: https://redis.io/commands/xgroup

I think the `ReplyError` subclass for matching it would look like this:

```py
class BusyGroupError(ReplyError):
    MATCH_REPLY = "BUSYGROUP Consumer Group name already exists"
```


Discussion:
@popravich hi
I would like to tackle this (if nobody else has started)

Resolving patch:
diff --git a/aioredis/errors.py b/aioredis/errors.py
--- a/aioredis/errors.py
+++ b/aioredis/errors.py
@@ -50,6 +50,12 @@ class AuthError(ReplyError):
     MATCH_REPLY = ("NOAUTH ", "ERR invalid password")
 
 
+class BusyGroupError(ReplyError):
+    """Raised if Consumer Group name already exists."""
+
+    MATCH_REPLY = "BUSYGROUP Consumer Group name already exists"
+
+
 class PipelineError(RedisError):
     """Raised if command within pipeline raised error."""
 


Test patch:
diff --git a/tests/stream_commands_test.py b/tests/stream_commands_test.py
--- a/tests/stream_commands_test.py
+++ b/tests/stream_commands_test.py
@@ -4,7 +4,7 @@
 from collections import OrderedDict
 from unittest import mock
 
-from aioredis import ReplyError
+from aioredis.errors import BusyGroupError
 from _testutils import redis_version
 
 pytestmark = redis_version(
@@ -314,7 +314,7 @@ async def test_xgroup_create_mkstream(redis, server_bin):
 async def test_xgroup_create_already_exists(redis, server_bin):
     await redis.xadd('test_stream', {'a': 1})
     await redis.xgroup_create('test_stream', 'test_group')
-    with pytest.raises(ReplyError):
+    with pytest.raises(BusyGroupError):
         await redis.xgroup_create('test_stream', 'test_group')
Repository: aio-libs-abandoned/aioredis-py

Issue:
Add the "MKSTREAM" option to xgroup_create
It's a not-yet-documented option for creating consumer groups before the stream exists: https://github.com/antirez/redis/issues/4824#issuecomment-438263421

Example signature for `StreamCommandsMixin`:
```py
    def xgroup_create(self, stream, group_name, latest_id='$', mkstream=False):
```


Discussion:
I would like to tackle this (if nobody else has started)

p.s. Hey @dcecile sorry if I interrupt and you wanted to make it yourself
@gyermolenko :) No problem, I haven't started it, so feel free to go ahead

Resolving patch:
diff --git a/aioredis/commands/streams.py b/aioredis/commands/streams.py
--- a/aioredis/commands/streams.py
+++ b/aioredis/commands/streams.py
@@ -79,8 +79,7 @@ def parse_lists_to_dicts(lists):
 class StreamCommandsMixin:
     """Stream commands mixin
 
-    Streams are under development in Redis and
-    not currently released.
+    Streams are available in Redis since v5.0
     """
 
     def xadd(self, stream, fields, message_id=b'*', max_len=None,
@@ -140,9 +139,12 @@ def xread_group(self, group_name, consumer_name, streams, timeout=0,
         )
         return wait_convert(fut, parse_messages_by_stream)
 
-    def xgroup_create(self, stream, group_name, latest_id='$'):
+    def xgroup_create(self, stream, group_name, latest_id='$', mkstream=False):
         """Create a consumer group"""
-        fut = self.execute(b'XGROUP', b'CREATE', stream, group_name, latest_id)
+        args = [b'CREATE', stream, group_name, latest_id]
+        if mkstream:
+            args.append(b'MKSTREAM')
+        fut = self.execute(b'XGROUP', *args)
         return wait_ok(fut)
 
     def xgroup_setid(self, stream, group_name, latest_id='$'):


Test patch:
diff --git a/tests/stream_commands_test.py b/tests/stream_commands_test.py
--- a/tests/stream_commands_test.py
+++ b/tests/stream_commands_test.py
@@ -287,8 +287,6 @@ async def test_xread_blocking(redis, create_redis, loop, server, server_bin):
 @pytest.mark.run_loop
 async def test_xgroup_create(redis, server_bin):
     # Also tests xinfo_groups()
-    # TODO: Remove xadd() if resolved:
-    #       https://github.com/antirez/redis/issues/4824
     await redis.xadd('test_stream', {'a': 1})
     await redis.xgroup_create('test_stream', 'test_group')
     info = await redis.xinfo_groups('test_stream')
@@ -300,6 +298,18 @@ async def test_xgroup_create(redis, server_bin):
     }]
 
 
+@pytest.mark.run_loop
+async def test_xgroup_create_mkstream(redis, server_bin):
+    await redis.xgroup_create('test_stream', 'test_group', mkstream=True)
+    info = await redis.xinfo_groups('test_stream')
+    assert info == [{
+        b'name': b'test_group',
+        b'last-delivered-id': mock.ANY,
+        b'pending': 0,
+        b'consumers': 0
+    }]
+
+
 @pytest.mark.run_loop
 async def test_xgroup_create_already_exists(redis, server_bin):
     await redis.xadd('test_stream', {'a': 1})
Repository: aio-libs-abandoned/aioredis-py

Issue:
Drop Python 3.5 support
Python 3.5 was a famous release (async/await syntax was introduced) but the version has reached end-of-life.
We can (and should) drop it.
Speaking as the maintainer of other libraries, I can say that it simplifies the code base a little


Discussion:
Agreed - I was surprised to see it still supported.

I do worry that dropping a language version likely means a major version bump, as that's definitely a breaking change, but it's a worthwhile one.

Resolving patch:
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -1,20 +1,8 @@
 import re
 import os.path
-import sys
-import platform
 from setuptools import setup, find_packages
 
 
-install_requires = ['async-timeout']
-if platform.python_implementation() == 'CPython':
-    install_requires.append('hiredis')
-
-PY_VER = sys.version_info
-
-if PY_VER < (3, 5):
-    raise RuntimeError("aioredis doesn't support Python version prior 3.5")
-
-
 def read(*parts):
     with open(os.path.join(*parts), 'rt') as f:
         return f.read().strip()
@@ -37,7 +25,6 @@ def read_version():
     'Development Status :: 4 - Beta',
     'Programming Language :: Python',
     'Programming Language :: Python :: 3',
-    'Programming Language :: Python :: 3.5',
     'Programming Language :: Python :: 3.6',
     'Programming Language :: Python :: 3.7',
     'Programming Language :: Python :: 3 :: Only',
@@ -60,6 +47,10 @@ def read_version():
       url="https://github.com/aio-libs/aioredis",
       license="MIT",
       packages=find_packages(exclude=["tests"]),
-      install_requires=install_requires,
+      install_requires=[
+          'async-timeout',
+          'hiredis; implementation_name=="cpython"'
+      ],
+      python_requires=">=3.6",
       include_package_data=True,
       )


Test patch:
Repository: aio-libs-abandoned/aioredis-py

Issue:
Explicit passing of coroutine objects to asyncio.wait() is deprecated since Python 3.8
Following warnings are reported while running the tests in Python 3.10

```
tests/sentinel_commands_test.py: 20 warnings
tests/sentinel_failover_test.py: 6 warnings
  /root/checked_repos/aioredis/aioredis/sentinel/pool.py:210: DeprecationWarning: The explicit passing of coroutine objects to asyncio.wait() is deprecated since Python 3.8, and scheduled for removal in Python 3.11.
    done, pending = await asyncio.wait(tasks,
```


Discussion:
@tirkarthi thanks for looking into this! 

Maintenance is currently underway to resolve these sorts of problems and make aioredis ready for the latest versions of Python. We've dropped Python3.5 support as of the last few days and will be looking for contributors who are willing to help us out. Would you be willing to contribute a potential fix for this issue?

Resolving patch:
diff --git a/aioredis/sentinel/pool.py b/aioredis/sentinel/pool.py
--- a/aioredis/sentinel/pool.py
+++ b/aioredis/sentinel/pool.py
@@ -1,7 +1,6 @@
 import asyncio
 import contextlib
 
-from concurrent.futures import ALL_COMPLETED
 from async_timeout import timeout as async_timeout
 
 from ..log import sentinel_logger
@@ -232,17 +231,10 @@ async def discover(self, timeout=None):  # TODO: better name?
         # TODO: discovery must be done with some customizable timeout.
         if timeout is None:
             timeout = self.discover_timeout
-        tasks = []
         pools = []
-        for addr in self._sentinels:  # iterate over unordered set
-            tasks.append(self._connect_sentinel(addr, timeout, pools))
-        done, pending = await asyncio.wait(tasks, return_when=ALL_COMPLETED)
-        assert not pending, ("Expected all tasks to complete", done, pending)
-
-        for task in done:
-            result = task.result()
-            if isinstance(result, Exception):
-                continue  # FIXME
+        tasks = [self._connect_sentinel(addr, timeout, pools) for addr in self._sentinels]
+        await asyncio.gather(*tasks, return_exceptions=True)
+
         if not pools:
             raise Exception("Could not connect to any sentinel")
         pools, self._pools[:] = self._pools[:], pools


Test patch:
Repository: aio-libs-abandoned/aioredis-py

Issue:
RuntimeWarning: coroutine 'Connection.disconnect' was never awaited
When running tests that need mocking an `aioredis` connection, the following Runtime Warning gets triggered each time, and is mumbled together with some random code, which makes me think it looks like a fairly low-level bug:

```
tests/foo/test_foo_bar.py ...                                    [100%]
/usr/local/lib/python3.9/site-packages/pkg_resources/_vendor/packaging/version.py:490:
RuntimeWarning: coroutine 'Connection.disconnect' was never awaited
  reversed(list(itertools.dropwhile(lambda x: x == 0, reversed(release))))
RuntimeWarning: Enable tracemalloc to get the object allocation traceback
```

I'm not sure where that comes from but not from my own code, as I never mention a `disconnect` method anywhere.

```python
import pytest
import aioredis

from foo import api


@pytest.fixture
async def aioredis_conn():
    conn = await aioredis.Redis(
        decode_responses=True,
        single_connection_client=True,
        **config.redis_test_credentials
    )
    yield conn
    await conn.flushdb()


@pytest.fixture
def mock_redis_conn(mocker, redis_conn, aioredis_conn):
    mocker.patch("foo.api.aioredis_conn", new=aioredis_conn)

def test_foobar(mock_redis_conn):
    do_something()
```

Am I doing this approximately right? Is there any specific reason why this warning occurs?


Discussion:
The same warning as it gets captured by pytest:

```
=============================== warnings summary ===============================
tests/foo/test_foo_bar.py::test_api_online_features
  /usr/local/lib/python3.9/site-packages/aioredis/connection.py:681: RuntimeWarning: coroutine 'Connection.disconnect' was never awaited
    reader, writer = await asyncio.open_connection(
```
This may be because async_timeout should be doing `async with` https://github.com/aio-libs/aioredis-py/blob/8ebc6e8c9c5ea293f1588769a89704fc9041d2f7/aioredis/connection.py#L680

Resolving patch:
diff --git a/aioredis/connection.py b/aioredis/connection.py
--- a/aioredis/connection.py
+++ b/aioredis/connection.py
@@ -677,7 +677,7 @@ async def connect(self):
 
     async def _connect(self):
         """Create a TCP socket connection"""
-        with async_timeout.timeout(self.socket_connect_timeout):
+        async with async_timeout.timeout(self.socket_connect_timeout):
             reader, writer = await asyncio.open_connection(
                 host=self.host, port=self.port, ssl=self.ssl_context, loop=self._loop
             )


Test patch:
Repository: aio-libs-abandoned/aioredis-py

Issue:
[2.0] Lock class uses blocking sleep
aioredis/lock.py calls `time.sleep` instead of `asyncio.sleep`. Pull request on its way...


Discussion:

Resolving patch:
diff --git a/aioredis/lock.py b/aioredis/lock.py
--- a/aioredis/lock.py
+++ b/aioredis/lock.py
@@ -1,3 +1,4 @@
+import asyncio
 import threading
 import time as mod_time
 import uuid
@@ -209,7 +210,7 @@ async def acquire(
             next_try_at = mod_time.monotonic() + sleep
             if stop_trying_at is not None and next_try_at > stop_trying_at:
                 return False
-            mod_time.sleep(sleep)
+            await asyncio.sleep(sleep)
 
     async def do_acquire(self, token: Union[str, bytes]) -> bool:
         if self.timeout:


Test patch:
Repository: aio-libs-abandoned/aioredis-py

Issue:
`BlockingConnectionPool` exception on `disconnect`
A user (@erewok) was trying to use `BlockingConnectionPool` from `v2.0.0.a1` in places where `ConnectionPool` is working fine and kept hitting an exception on `disconnect`:

```python
  File "/location/of/dependencies/lib/python3.9/site-packages/aioredis/connection.py", line 1596, in disconnect
    async with self._lock:
AttributeError: 'BlockingConnectionPool' object has no attribute '_lock'
```

This is in this chunk of code: https://github.com/aio-libs/aioredis-py/blob/c54fca02e216f0fb046b83e6403315fb00cd43e6/aioredis/connection.py#L1598


It looks like the _direct_ reason for this is that on `__init__` a regular `ConnectionPool` ends up calling `reset`, which sets the following attributes:

```
        self._lock = asyncio.Lock()
        self._created_connections = 0
        self._available_connections = []
        self._in_use_connections = set()
```

However, the `BlockingConnectionPool` does something different in its `reset` method: https://github.com/aio-libs/aioredis-py/blob/c54fca02e216f0fb046b83e6403315fb00cd43e6/aioredis/connection.py#L1491

I am probably going to use `ConnectionPool` for now: I think the `BlockingConnectionPool` isn't totally usable yet.

_Originally posted by @erewok in https://github.com/aio-libs/aioredis-py/issues/930#issuecomment-859620661_


Discussion:

Resolving patch:
diff --git a/aioredis/connection.py b/aioredis/connection.py
--- a/aioredis/connection.py
+++ b/aioredis/connection.py
@@ -1267,7 +1267,7 @@ def __init__(
         # will notice the first thread already did the work and simply
         # release the lock.
         self._fork_lock = threading.Lock()
-        self._lock: asyncio.Lock
+        self._lock = asyncio.Lock()
         self._created_connections: int
         self._available_connections: List[Connection]
         self._in_use_connections: Set[Connection]


Test patch:
diff --git a/tests/test_connection_pool.py b/tests/test_connection_pool.py
--- a/tests/test_connection_pool.py
+++ b/tests/test_connection_pool.py
@@ -24,6 +24,9 @@ def __init__(self, **kwargs):
     async def connect(self):
         pass
 
+    async def disconnect(self):
+        pass
+
     async def can_read(self, timeout: float = 0):
         return False
 
@@ -122,6 +125,13 @@ async def test_connection_creation(self, master_host):
         assert isinstance(connection, DummyConnection)
         assert connection.kwargs == connection_kwargs
 
+    async def test_disconnect(self, master_host):
+        """A regression test for #1047"""
+        connection_kwargs = {"foo": "bar", "biz": "baz", "host": master_host}
+        pool = self.get_pool(connection_kwargs=connection_kwargs)
+        await pool.get_connection("_")
+        await pool.disconnect()
+
     async def test_multiple_connections(self, master_host):
         connection_kwargs = {"host": master_host}
         pool = self.get_pool(connection_kwargs=connection_kwargs)
Repository: aio-libs-abandoned/aioredis-py

Issue:
Rename redis-py health check message
https://github.com/aio-libs/aioredis-py/blob/c82fe277a5a6521aa9baacd672dd131313fe1b45/aioredis/client.py#L3873

Should we change this to aioredis-py @seandstewart @abrookins in case someone uses both redis-py and aioredis?


Discussion:
Yeah, we should. Good catch!

Resolving patch:
diff --git a/aioredis/client.py b/aioredis/client.py
--- a/aioredis/client.py
+++ b/aioredis/client.py
@@ -3870,7 +3870,7 @@ class PubSub:
 
     PUBLISH_MESSAGE_TYPES = ("message", "pmessage")
     UNSUBSCRIBE_MESSAGE_TYPES = ("unsubscribe", "punsubscribe")
-    HEALTH_CHECK_MESSAGE = "redis-py-health-check"
+    HEALTH_CHECK_MESSAGE = "aioredis-py-health-check"
 
     def __init__(
         self,


Test patch:
Repository: aio-libs-abandoned/aioredis-py

Issue:
[2.0] Update setup.py Trove classifiers
The [classifiers](https://github.com/aio-libs/aioredis-py/blob/5a713fff3717094cca63e4a5f4b1cb7d6894a08f/setup.py#L25-L30) currently only list Python 3.6 and 3.7. This should be updated to include all versions that are tested. Or my personal preference is just to delete classifiers for minor versions, since probably no-one will remember to update them when new versions of Python are released.

It also indicates the status as 4 - Beta. That should probably change when we release the final 2.0.0.


Discussion:

Resolving patch:
diff --git a/aioredis/__init__.py b/aioredis/__init__.py
--- a/aioredis/__init__.py
+++ b/aioredis/__init__.py
@@ -31,7 +31,7 @@ def int_or_str(value):
         return value
 
 
-__version__ = "2.0.0b1"
+__version__ = "2.0.0"
 VERSION = tuple(map(int_or_str, __version__.split(".")))
 
 __all__ = [
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -22,11 +22,13 @@ def read_version():
 
 classifiers = [
     "License :: OSI Approved :: MIT License",
-    "Development Status :: 4 - Beta",
+    "Development Status :: 5 - Production/Stable",
     "Programming Language :: Python",
     "Programming Language :: Python :: 3",
     "Programming Language :: Python :: 3.6",
     "Programming Language :: Python :: 3.7",
+    "Programming Language :: Python :: 3.8",
+    "Programming Language :: Python :: 3.9",
     "Programming Language :: Python :: 3 :: Only",
     "Operating System :: POSIX",
     "Environment :: Web Environment",


Test patch: