Repository: internetarchive/openlibrary

Issue:
Linting for Python asyncio issues
### Question
<!-- What question needs to be answered to close this issue? This should be one sentence. -->
Is fixing these nine Python `asyncio` lint issues important?
`ruff --select=ASYNC .` --> `ASYNC100` * 2 and `ASYNC101` * 7

Pull requests:
* [ ] #7895 is `ASYNC100` * 2
* [ ] #7892 is `ASYNC101` * 1
* [ ] #8017 Six instances of `with open() as f:` is `ASYNC101` * 6

% `ruff rule ASYNC100`
# blocking-http-call-in-async-function (ASYNC100)

Derived from the **flake8-async** linter.

## What it does
Checks that async functions do not contain blocking HTTP calls.

## Why is this bad?
Blocking an async function via a blocking HTTP call will block the entire
event loop, preventing it from executing other tasks while waiting for the
HTTP response, negating the benefits of asynchronous programming.

Instead of making a blocking HTTP call, use an asynchronous HTTP client
library such as `aiohttp` or `httpx`.

## Example
```python
async def fetch():
    urllib.request.urlopen("https://example.com/foo/bar").read()
```

Use instead:
```python
async def fetch():
    async with aiohttp.ClientSession() as session:
        async with session.get("https://example.com/foo/bar") as resp:
            ...
```
---

% `ruff rule ASYNC101`
# open-sleep-or-subprocess-in-async-function (ASYNC101)

Derived from the **flake8-async** linter.

## What it does
Checks that async functions do not contain calls to `open`, `time.sleep`,
or `subprocess` methods.

## Why is this bad?
Blocking an async function via a blocking call will block the entire
event loop, preventing it from executing other tasks while waiting for the
call to complete, negating the benefits of asynchronous programming.

Instead of making a blocking call, use an equivalent asynchronous library
or function.

## Example
```python
async def foo():
    time.sleep(1000)
```

Use instead:
```python
async def foo():
    await asyncio.sleep(1000)
```
### Additional context
<!-- Add any other context or details here. -->

### Issue resolution criteria
<!-- When can this issue be closed? -->


### Stakeholders
<!-- @ tag stakeholders of this bug -->



Discussion:

Resolving patch:
diff --git a/openlibrary/solr/update_work.py b/openlibrary/solr/update_work.py
--- a/openlibrary/solr/update_work.py
+++ b/openlibrary/solr/update_work.py
@@ -9,6 +9,7 @@
 from typing import Literal, Optional, cast, Any, Union
 from collections.abc import Iterable
 
+import aiofiles
 import httpx
 import requests
 import sys
@@ -1495,11 +1496,10 @@ def _solr_update(requests: list[SolrUpdateRequest]):
             requests += [CommitRequest()]
 
         if output_file:
-            with open(output_file, "w") as f:
+            async with aiofiles.open(output_file, "w") as f:
                 for r in requests:
                     if isinstance(r, AddRequest):
-                        f.write(r.tojson())
-                        f.write("\n")
+                        await f.write(f"{r.tojson()}\n")
         else:
             _solr_update(requests)
 
@@ -1517,11 +1517,10 @@ def _solr_update(requests: list[SolrUpdateRequest]):
 
     if requests:
         if output_file:
-            with open(output_file, "w") as f:
+            async with aiofiles.open(output_file, "w") as f:
                 for r in requests:
                     if isinstance(r, AddRequest):
-                        f.write(r.tojson())
-                        f.write("\n")
+                        await f.write(f"{r.tojson()}\n")
         else:
             if commit:
                 requests += [CommitRequest()]
diff --git a/scripts/solr_builder/solr_builder/solr_builder.py b/scripts/solr_builder/solr_builder/solr_builder.py
--- a/scripts/solr_builder/solr_builder/solr_builder.py
+++ b/scripts/solr_builder/solr_builder/solr_builder.py
@@ -11,6 +11,7 @@
 import uuid
 from collections import namedtuple
 
+import aiofiles
 import psycopg2
 
 from openlibrary.core.bookshelves import Bookshelves
@@ -535,10 +536,10 @@ def fmt(self, k: str, val: Any) -> str:
 
         if progress:
             # Clear the file
-            with open(progress, 'w') as f:
-                f.write('')
-            with open(progress, 'a') as f:
-                f.write('Calculating total... ')
+            async with aiofiles.open(progress, 'w') as f:
+                await f.write('')
+            async with aiofiles.open(progress, 'a') as f:
+                await f.write('Calculating total... ')
 
         start = time.time()
         q_count = """SELECT COUNT(*) FROM(%s) AS foo""" % q
@@ -546,9 +547,9 @@ def fmt(self, k: str, val: Any) -> str:
         end = time.time()
 
         if progress:
-            with open(progress, 'a') as f:
-                f.write('%d (%.2fs)\n' % (count, end - start))
-                f.write('\t'.join(PLogEntry._fields) + '\n')
+            async with aiofiles.open(progress, 'a') as f:
+                await f.write('%d (%.2fs)\n' % (count, end - start))
+                await f.write('\t'.join(PLogEntry._fields) + '\n')
 
         plog.log(
             PLogEntry(0, count, '0.00%', 0, '?', '?', '?', '?', '?', start_at or '?')
diff --git a/scripts/solr_updater.py b/scripts/solr_updater.py
--- a/scripts/solr_updater.py
+++ b/scripts/solr_updater.py
@@ -20,6 +20,7 @@
 
 import _init_path  # noqa: F401  Imported for its side effect of setting PYTHONPATH
 
+import aiofiles
 import web
 
 from openlibrary.solr import update_work
@@ -304,8 +305,8 @@ async def main(
         if logfile.tell() != offset:
             offset = logfile.tell()
             logger.info("saving offset %s", offset)
-            with open(state_file, "w") as f:
-                f.write(offset)
+            async with aiofiles.open(state_file, "w") as f:
+                await f.write(offset)
 
         # don't sleep after committing some records.
         # While the commit was on, some more edits might have happened.


Test patch: