Repository: lablup/backend.ai-common (language: python)

Issue:
Upgrade to aioredis v2
To stabilize our HA setup, we finally migrate to aioredis v2.

Goals:
* Work with a single Redis instance with restarts
* ~~Work with a Redis cluster accessed via a haproxy~~ (we will drop use of haproxy for Redis)
* Work with a Redis cluster accessed via Sentinel instances
* Migrate pub/sub + lists → streams with XREAD + streams with XREADGROUP

refs lablup/backend.ai#273



Discussion / hints:
# [Codecov](https://codecov.io/gh/lablup/backend.ai-common/pull/88?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=lablup) Report
> Merging [#88](https://codecov.io/gh/lablup/backend.ai-common/pull/88?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=lablup) (21fc960) into [main](https://codecov.io/gh/lablup/backend.ai-common/commit/9f4570215d2573ed318c98ab6ad0a5ad7249cf6b?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=lablup) (9f45702) will **increase** coverage by `0.44%`.
> The diff coverage is `72.13%`.

> :exclamation: Current head 21fc960 differs from pull request most recent head c5e3e05. Consider uploading reports for the commit c5e3e05 to get more accurate results
[![Impacted file tree graph](https://codecov.io/gh/lablup/backend.ai-common/pull/88/graphs/tree.svg?width=650&height=150&src=pr&token=Ko9szj2YUD&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=lablup)](https://codecov.io/gh/lablup/backend.ai-common/pull/88?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=lablup)

```diff
@@            Coverage Diff             @@
##             main      #88      +/-   ##
==========================================
+ Coverage   74.63%   75.08%   +0.44%     
==========================================
  Files          23       23              
  Lines        3036     3191     +155     
==========================================
+ Hits         2266     2396     +130     
- Misses        770      795      +25     
```


| [Impacted Files](https://codecov.io/gh/lablup/backend.ai-common/pull/88?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=lablup) | Coverage Δ | |
|---|---|---|
| [src/ai/backend/common/events.py](https://codecov.io/gh/lablup/backend.ai-common/pull/88/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=lablup#diff-c3JjL2FpL2JhY2tlbmQvY29tbW9uL2V2ZW50cy5weQ==) | `71.11% <24.32%> (+0.92%)` | :arrow_up: |
| [src/ai/backend/common/redis.py](https://codecov.io/gh/lablup/backend.ai-common/pull/88/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=lablup#diff-c3JjL2FpL2JhY2tlbmQvY29tbW9uL3JlZGlzLnB5) | `76.21% <78.77%> (+6.21%)` | :arrow_up: |
| [src/ai/backend/common/types.py](https://codecov.io/gh/lablup/backend.ai-common/pull/88/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=lablup#diff-c3JjL2FpL2JhY2tlbmQvY29tbW9uL3R5cGVzLnB5) | `91.68% <86.66%> (-0.17%)` | :arrow_down: |
| [src/ai/backend/common/logging.py](https://codecov.io/gh/lablup/backend.ai-common/pull/88/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=lablup#diff-c3JjL2FpL2JhY2tlbmQvY29tbW9uL2xvZ2dpbmcucHk=) | `61.77% <100.00%> (ø)` | |
| [src/ai/backend/common/validators.py](https://codecov.io/gh/lablup/backend.ai-common/pull/88/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=lablup#diff-c3JjL2FpL2JhY2tlbmQvY29tbW9uL3ZhbGlkYXRvcnMucHk=) | `64.06% <100.00%> (+0.76%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/lablup/backend.ai-common/pull/88?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=lablup).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=lablup)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/lablup/backend.ai-common/pull/88?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=lablup). Last update [9f45702...c5e3e05](https://codecov.io/gh/lablup/backend.ai-common/pull/88?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=lablup). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=lablup).

* Please fix all type errors as well.
* I'd like to accept both a single Redis server address or multiple addresses for Redis sentinel nodes for HA, depending on the etcd configuration. Could we update the code to handle both cases in `ai.backend.common.redis` and the etcd config scheme (as written as module-level docstring in `ai.backend.manager.config`)?

Fix patch:
diff --git a/changes/91.fix b/changes/91.fix
new file mode 100644
index 00000000..7bb69a94
--- /dev/null
+++ b/changes/91.fix
@@ -0,0 +1,1 @@
+Improve stability of test cases using a single Redis container as fixture (a follow-up to #88)


Test patch:
diff --git a/tests/redis/conftest.py b/tests/redis/conftest.py
index 5e2fd41d..48b31fbe 100644
--- a/tests/redis/conftest.py
+++ b/tests/redis/conftest.py
@@ -27,6 +27,7 @@ async def redis_container(test_ns, test_case_ns) -> AsyncIterator[str]:
     stdout = await p.stdout.read()
     await p.wait()
     cid = stdout.decode().strip()
+    await wait_redis_ready('127.0.0.1', 9379)
     try:
         yield cid
     finally: