Repository: bcye/structured-wikivoyage-exports
Pull request #10: Add Bunny Storage Adapter
Author: login: bcye
Created: 2025-04-18T15:28:13Z | Merged: 2025-04-23T17:48:07Z

Closes #9 and depends on #8 

<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit

- **New Features**
  - Introduced pluggable, asynchronous output handlers for writing parsed entries to the filesystem or BunnyCDN storage.
  - Added support for configuring output handlers and their parameters via environment variables.
  - Enabled concurrent processing of multiple input files with configurable concurrency limits.
- **Chores**
  - Updated dependencies to include support for asynchronous file and HTTP operations.
  - Improved .gitignore to exclude additional files and directories.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

Linked issue #9: Bunny Storage Adapter
FTPS/SFTP doesn't seem to work well with it, so just consume their HTTP API directly. Closes #2 

Diff:
diff --git a/.gitignore b/.gitignore
index a1b941e..df47d38 100644
--- a/.gitignore
+++ b/.gitignore
@@ -11,3 +11,5 @@ wheels/
 
 .env
 node_modules
+output
+en-json
\ No newline at end of file
diff --git a/output_handlers/__init__.py b/output_handlers/__init__.py
new file mode 100644
index 0000000..856f4e8
--- /dev/null
+++ b/output_handlers/__init__.py
@@ -0,0 +1,3 @@
+from .base_handler import BaseHandler
+from .filesystem import FilesystemHandler
+from .bunny_storage import BunnyStorageHandler
\ No newline at end of file
diff --git a/output_handlers/base_handler.py b/output_handlers/base_handler.py
new file mode 100644
index 0000000..67d4886
--- /dev/null
+++ b/output_handlers/base_handler.py
@@ -0,0 +1,61 @@
+"""Reference handler for output handlers."""
+from abc import ABC, abstractmethod
+import logging
+
+
+
+class BaseHandler(ABC):
+    """
+    Abstract base class for output handlers. Defines the standardized interface that all output handlers must implement.
+    In particular, it requires the implementation of an asynchronous ("private") method `_write_entry` to write a single entry to the output.
+    """
+
+    logger = logging.getLogger(__name__)
+
+    def __init__(self, fail_on_error: bool = True, **kwargs):
+        """
+        Initializes the BaseHandler with optional parameters.
+
+        Args:
+            fail_on_error (bool): If True, the handler will raise an exception on error. Defaults to True.
+            **kwargs: Additional keyword arguments for specific handler implementations.
+        """
+        self.fail_on_error = fail_on_error
+
+
+    @abstractmethod
+    async def _write_entry(self, entry: dict, uid: str) -> bool:
+        """
+        Asynchronously writes a single entry to the output. This method should gracefully handle any exceptions that may occur during the writing process and simply return False if an error occurs.
+
+        Args:
+            entry (dict): The entry to write (will be JSON-encoded).
+            uid (str): The unique identifier for the entry. The default id provided by wikivoyage is recommended. 
+        Returns:
+            bool: True if the entry was written successfully, False otherwise.
+        """
+        pass
+
+
+    async def write_entry(self, entry: dict, uid: str):
+        """
+        Public method to write an entry to the output. It handles exceptions and logs errors.
+
+        Args:
+            entry (dict): The entry to write (will be JSON-encoded).
+            uid (str): The unique identifier for the entry. The default id provided by wikivoyage is recommended. 
+        """
+        success = await self._write_entry(entry, uid)
+        if success:
+            self.logger.debug(f"Successfully wrote entry with UID {uid}")
+        else:
+            self.logger.error(f"Failed to write entry with UID {uid}")
+            if self.fail_on_error:
+                raise Exception(f"Failed to write entry with UID {uid}")
+
+
+    async def close(self):
+        """
+        Closes the handler. This method should be overridden by subclasses if they need to perform any cleanup operations.
+        """
+        pass
diff --git a/output_handlers/bunny_storage.py b/output_handlers/bunny_storage.py
new file mode 100644
index 0000000..553fbbb
--- /dev/null
+++ b/output_handlers/bunny_storage.py
@@ -0,0 +1,54 @@
+import json
+import aiohttp
+from .base_handler import BaseHandler
+
+class BunnyStorageHandler(BaseHandler):
+    def __init__(
+        self,
+        region: str,
+        base_path: str,
+        api_key: str,
+        fail_on_error: bool = True,
+        keepalive_timeout: int = 75,
+    ):
+        super().__init__(fail_on_error=fail_on_error)
+        self.base_url = f"https://{region}.bunnycdn.com/{base_path}"
+        self.headers = {
+            "AccessKey": api_key,
+            "Content-Type": "application/json",
+            "accept": "application/json",
+        }
+
+        # initialized later, in a guaranteed async context
+        self._connector = None
+        self._session = None
+        self._keepalive_timeout = keepalive_timeout
+
+    async def setup_connector(self):
+        if self._session is None:
+            self._connector = aiohttp.TCPConnector(
+                # limit is implicitly set to 100
+                keepalive_timeout = self._keepalive_timeout,
+            )
+            self._session = aiohttp.ClientSession(connector=self._connector)
+
+    async def _write_entry(self, entry: dict, uid: str) -> bool:
+        await self.setup_connector()
+        payload = json.dumps(entry).encode("utf-8")
+        url = f"{self.base_url}/{uid}.json"
+
+        try:
+            async with self._session.put(url, data=payload, headers=self.headers) as resp:
+                if resp.status in (200, 201, 204):
+                    return True
+                body = await resp.text()
+                self.logger.error(f"Upload failed UID={uid} status={resp.status} body={body}")
+                return False
+
+        except Exception:
+            self.logger.exception(f"Exception while uploading UID={uid}")
+            return False
+
+    async def close(self):
+        await self._session.close()
+        await self._connector.close()
\ No newline at end of file
diff --git a/output_handlers/filesystem.py b/output_handlers/filesystem.py
new file mode 100644
index 0000000..855348b
--- /dev/null
+++ b/output_handlers/filesystem.py
@@ -0,0 +1,44 @@
+"""Handler that writes files to the filesystem."""
+from pathlib import Path
+import aiofiles
+from .base_handler import BaseHandler
+import json
+
+class FilesystemHandler(BaseHandler):
+    """
+    Handler that writes files to the filesystem.
+    """
+    def __init__(self, output_dir: str, **kwargs):
+        """
+        Initializes the FileSystemHandler with the specified output directory.
+
+        Args:
+            output_dir (str): The directory where files will be written.
+            **kwargs: Additional keyword arguments for the BaseHandler.
+        """
+        super().__init__(**kwargs)
+        self.output_dir = Path(output_dir)
+        # Ensure the target directory exists
+        self.output_dir.mkdir(parents=True, exist_ok=True)
+        self.logger.info(f"Output directory set to {self.output_dir}")
+
+
+    async def _write_entry(self, entry: dict, uid: str) -> bool:
+        """
+        Asynchronously writes a single entry to the filesystem.
+
+        Args:
+            entry (dict): The entry to write (will be JSON-encoded).
+            uid (str): The unique identifier for the entry.
+
+        Returns:
+            bool: True if the entry was written successfully, False otherwise.
+        """
+        try:
+            file_path = self.output_dir / f"{uid}.json"
+            async with aiofiles.open(file_path, 'w') as f:
+                await f.write(json.dumps(entry))
+            return True
+        except IOError as e:
+            self.logger.error(f"Error writing entry {uid}: {e}")
+            return False
diff --git a/pyproject.toml b/pyproject.toml
index b347f6c..60032fa 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -5,6 +5,10 @@ description = "Add your description here"
 readme = "README.md"
 requires-python = ">=3.12"
 dependencies = [
+    "aiofiles>=24.1.0",
+    "aiohttp>=3.11.16",
+    "asyncssh>=2.20.0",
     "mwparserfromhell>=0.6.6",
+    "python-dotenv>=1.1.0",
     "wikitextparser>=0.56.3",
 ]
diff --git a/transform-documents.py b/transform-documents.py
index 3cf4353..1a5a9bc 100644
--- a/transform-documents.py
+++ b/transform-documents.py
@@ -6,6 +6,11 @@
 import os
 from pathlib import Path
 import sys
+import asyncio
+import importlib
+import logging
+from dotenv import load_dotenv
+load_dotenv()
 
 class WikivoyageParser:
     def __init__(self):
@@ -342,41 +347,120 @@ def export_json(self, root=None, indent=2) -> str:
             
         return json.dumps(root, indent=indent)
 
-def process_file(input_file: Path, parser: WikivoyageParser) -> None:
-    """Process a single wiki file and save JSON output"""
-    # Create output path with .json extension
-    output_file = input_file.with_suffix('.json')
-    
-    # Ensure output directory exists
-    output_file.parent.mkdir(parents=True, exist_ok=True)
+async def process_file(
+    input_file: Path,
+    handler,
+) -> None:
+    """
+    Parse one wiki file and hand the resulting entry off to our handler.
+    Uses the filename (sans suffix) as the unique UID.
+    """
     
+    text = input_file.read_text(encoding="utf-8")
+    parser = WikivoyageParser()
+    entry = parser.parse(text)  # assume returns a dict
+    uid = input_file.stem
+
+    await handler.write_entry(entry, uid)
+
+def gather_handler_kwargs(handler_name: str) -> dict:
+    """
+    Find all ENV vars starting with HANDLER_<NAME>_ and turn them into kwargs.
+    E.g. HANDLER_SFTP_HOST=foo → {"host": "foo"}, HANDLER_SFTP_PORT=2222 → {"port": 2222}
+    """
+    prefix = f"HANDLER_{handler_name.upper()}_"
+    kwargs = {}
+
+    for env_key, val in os.environ.items():
+        if not env_key.startswith(prefix):
+            continue
+        param = env_key.replace(prefix, "").lower()
+        # cast ints
+        if val.isdigit():
+            val = int(val)
+        # cast bools
+        elif val.lower() in ("true", "false"):
+            val = val.lower() == "true"
+        kwargs[param] = val
+    print(f"Handler kwargs: {kwargs}")
+    return kwargs
+
+async def main():
+    logging.basicConfig(level=logging.DEBUG)
+
+    # 1. Which handler to load?
+    handler_name = os.getenv("HANDLER")
+    if not handler_name:
+        print("Error: set ENV HANDLER (e.g. 'filesystem')")
+        sys.exit(1)
+
+    # 2. Dynamic import
+    module_path = f"output_handlers.{handler_name}"
     try:
-        # Read and parse input file
-        with open(input_file, 'r', encoding='utf-8') as f:
-            wikitext = f.read()
-            
-        result = parser.parse(wikitext)
-        
-        # Write JSON output
-        with open(output_file, 'w', encoding='utf-8') as f:
-            f.write(parser.export_json())
-            
-    except Exception as e:
-        print(f"Error processing {input_file}: {e}")
+        mod = importlib.import_module(module_path)
+    except ImportError as e:
+        print(f"Error loading handler module {module_path}: {e}")
+        sys.exit(1)
+
+    # 3. Find the class: e.g. "sftp" → "SftpHandler"
+    class_name = handler_name.title().replace("_", "") + "Handler"
+    if not hasattr(mod, class_name):
+        print(f"{module_path} defines no class {class_name}")
+        sys.exit(1)
+    HandlerCls = getattr(mod, class_name)
+
+    # 4. Build kwargs from ENV
+    handler_kwargs = gather_handler_kwargs(handler_name)
+
+    # 5. Instantiate
+    handler = HandlerCls(**handler_kwargs)
+
+    # 6. Which dir to walk?
+    input_dir = Path(sys.argv[1]) if len(sys.argv) > 1 else Path(".")
+    txt_files = list(input_dir.rglob("*.txt"))
+
+    if not txt_files:
+        print(f"No .txt files found under {input_dir}")
+        sys.exit(1)
+
+    # 7. read concurrency setting
+    try:
+        max_conc = int(os.getenv("MAX_CONCURRENT", "0"))
+    except ValueError:
+        print("Error: MAX_CONCURRENT must be an integer")
+        sys.exit(1)
+
+    if max_conc < 0:
+        print("Error: MAX_CONCURRENT must be >= 0")
+        sys.exit(1)
+
+    # 8. schedule tasks
+    if max_conc == 0:
+        # unbounded
+        tasks = [
+            asyncio.create_task(process_file(txt, handler))
+            for txt in txt_files
+        ]
+    else:
+        # bounded by semaphore
+        sem = asyncio.Semaphore(max_conc)
+
+        async def bounded(txt):
+            async with sem:
+                return await process_file(txt, handler)
+
+        tasks = [
+            asyncio.create_task(bounded(txt))
+            for txt in txt_files
+        ]
+
+    # 9. run them all
+    await asyncio.gather(*tasks)
+    await handler.close()
+
+
+    print("All done.")
 
-def main():
-    # Initialize parser once for reuse
-    parser = WikivoyageParser()
-    
-    # Get input directory from command line or use current directory
-    input_dir = Path(sys.argv[1] if len(sys.argv) > 1 else '.')
-    
-    # Process all .txt files recursively
-    for txt_file in input_dir.rglob('*.txt'):
-        print(f"Processing {txt_file}")
-        process_file(txt_file, parser)
-        
-    print("Processing complete")
 
 if __name__ == "__main__":
-    main()
\ No newline at end of file
+    asyncio.run(main())
diff --git a/uv.lock b/uv.lock
index 6d3f2ac..7c107f1 100644
--- a/uv.lock
+++ b/uv.lock
@@ -1,21 +1,327 @@
 version = 1
 requires-python = ">=3.12"
 
+[[package]]
+name = "aiofiles"
+version = "24.1.0"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/0b/03/a88171e277e8caa88a4c77808c20ebb04ba74cc4681bf1e9416c862de237/aiofiles-24.1.0.tar.gz", hash = "sha256:22a075c9e5a3810f0c2e48f3008c94d68c65d763b9b03857924c99e57355166c", size = 30247 }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/a5/45/30bb92d442636f570cb5651bc661f52b610e2eec3f891a5dc3a4c3667db0/aiofiles-24.1.0-py3-none-any.whl", hash = "sha256:b4ec55f4195e3eb5d7abd1bf7e061763e864dd4954231fb8539a0ef8bb8260e5", size = 15896 },
+]
+
+[[package]]
+name = "aiohappyeyeballs"
+version = "2.6.1"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/26/30/f84a107a9c4331c14b2b586036f40965c128aa4fee4dda5d3d51cb14ad54/aiohappyeyeballs-2.6.1.tar.gz", hash = "sha256:c3f9d0113123803ccadfdf3f0faa505bc78e6a72d1cc4806cbd719826e943558", size = 22760 }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/0f/15/5bf3b99495fb160b63f95972b81750f18f7f4e02ad051373b669d17d44f2/aiohappyeyeballs-2.6.1-py3-none-any.whl", hash = "sha256:f349ba8f4b75cb25c99c5c2d84e997e485204d2902a9597802b0371f09331fb8", size = 15265 },
+]
+
+[[package]]
+name = "aiohttp"
+version = "3.11.16"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+    { name = "aiohappyeyeballs" },
+    { name = "aiosignal" },
+    { name = "attrs" },
+    { name = "frozenlist" },
+    { name = "multidict" },
+    { name = "propcache" },
+    { name = "yarl" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/f1/d9/1c4721d143e14af753f2bf5e3b681883e1f24b592c0482df6fa6e33597fa/aiohttp-3.11.16.tar.gz", hash = "sha256:16f8a2c9538c14a557b4d309ed4d0a7c60f0253e8ed7b6c9a2859a7582f8b1b8", size = 7676826 }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/db/38/100d01cbc60553743baf0fba658cb125f8ad674a8a771f765cdc155a890d/aiohttp-3.11.16-cp312-cp312-macosx_10_13_universal2.whl", hash = "sha256:911a6e91d08bb2c72938bc17f0a2d97864c531536b7832abee6429d5296e5b27", size = 704881 },
+    { url = "https://files.pythonhosted.org/packages/21/ed/b4102bb6245e36591209e29f03fe87e7956e54cb604ee12e20f7eb47f994/aiohttp-3.11.16-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:6ac13b71761e49d5f9e4d05d33683bbafef753e876e8e5a7ef26e937dd766713", size = 464564 },
+    { url = "https://files.pythonhosted.org/packages/3b/e1/a9ab6c47b62ecee080eeb33acd5352b40ecad08fb2d0779bcc6739271745/aiohttp-3.11.16-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:fd36c119c5d6551bce374fcb5c19269638f8d09862445f85a5a48596fd59f4bb", size = 456548 },
+    { url = "https://files.pythonhosted.org/packages/80/ad/216c6f71bdff2becce6c8776f0aa32cb0fa5d83008d13b49c3208d2e4016/aiohttp-3.11.16-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:d489d9778522fbd0f8d6a5c6e48e3514f11be81cb0a5954bdda06f7e1594b321", size = 1691749 },
+    { url = "https://files.pythonhosted.org/packages/bd/ea/7df7bcd3f4e734301605f686ffc87993f2d51b7acb6bcc9b980af223f297/aiohttp-3.11.16-cp312-cp312-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:69a2cbd61788d26f8f1e626e188044834f37f6ae3f937bd9f08b65fc9d7e514e", size = 1736874 },
+    { url = "https://files.pythonhosted.org/packages/51/41/c7724b9c87a29b7cfd1202ec6446bae8524a751473d25e2ff438bc9a02bf/aiohttp-3.11.16-cp312-cp312-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:cd464ba806e27ee24a91362ba3621bfc39dbbb8b79f2e1340201615197370f7c", size = 1786885 },
+    { url = "https://files.pythonhosted.org/packages/86/b3/f61f8492fa6569fa87927ad35a40c159408862f7e8e70deaaead349e2fba/aiohttp-3.11.16-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:1ce63ae04719513dd2651202352a2beb9f67f55cb8490c40f056cea3c5c355ce", size = 1698059 },
+    { url = "https://files.pythonhosted.org/packages/ce/be/7097cf860a9ce8bbb0e8960704e12869e111abcd3fbd245153373079ccec/aiohttp-3.11.16-cp312-cp312-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:09b00dd520d88eac9d1768439a59ab3d145065c91a8fab97f900d1b5f802895e", size = 1626527 },
+    { url = "https://files.pythonhosted.org/packages/1d/1d/aaa841c340e8c143a8d53a1f644c2a2961c58cfa26e7b398d6bf75cf5d23/aiohttp-3.11.16-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:7f6428fee52d2bcf96a8aa7b62095b190ee341ab0e6b1bcf50c615d7966fd45b", size = 1644036 },
+    { url = "https://files.pythonhosted.org/packages/2c/88/59d870f76e9345e2b149f158074e78db457985c2b4da713038d9da3020a8/aiohttp-3.11.16-cp312-cp312-musllinux_1_2_armv7l.whl", hash = "sha256:13ceac2c5cdcc3f64b9015710221ddf81c900c5febc505dbd8f810e770011540", size = 1685270 },
+    { url = "https://files.pythonhosted.org/packages/2b/b1/c6686948d4c79c3745595efc469a9f8a43cab3c7efc0b5991be65d9e8cb8/aiohttp-3.11.16-cp312-cp312-musllinux_1_2_i686.whl", hash = "sha256:fadbb8f1d4140825069db3fedbbb843290fd5f5bc0a5dbd7eaf81d91bf1b003b", size = 1650852 },
+    { url = "https://files.pythonhosted.org/packages/fe/94/3e42a6916fd3441721941e0f1b8438e1ce2a4c49af0e28e0d3c950c9b3c9/aiohttp-3.11.16-cp312-cp312-musllinux_1_2_ppc64le.whl", hash = "sha256:6a792ce34b999fbe04a7a71a90c74f10c57ae4c51f65461a411faa70e154154e", size = 1704481 },
+    { url = "https://files.pythonhosted.org/packages/b1/6d/6ab5854ff59b27075c7a8c610597d2b6c38945f9a1284ee8758bc3720ff6/aiohttp-3.11.16-cp312-cp312-musllinux_1_2_s390x.whl", hash = "sha256:f4065145bf69de124accdd17ea5f4dc770da0a6a6e440c53f6e0a8c27b3e635c", size = 1735370 },
+    { url = "https://files.pythonhosted.org/packages/73/2a/08a68eec3c99a6659067d271d7553e4d490a0828d588e1daa3970dc2b771/aiohttp-3.11.16-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:fa73e8c2656a3653ae6c307b3f4e878a21f87859a9afab228280ddccd7369d71", size = 1697619 },
+    { url = "https://files.pythonhosted.org/packages/61/d5/fea8dbbfb0cd68fbb56f0ae913270a79422d9a41da442a624febf72d2aaf/aiohttp-3.11.16-cp312-cp312-win32.whl", hash = "sha256:f244b8e541f414664889e2c87cac11a07b918cb4b540c36f7ada7bfa76571ea2", size = 411710 },
+    { url = "https://files.pythonhosted.org/packages/33/fb/41cde15fbe51365024550bf77b95a4fc84ef41365705c946da0421f0e1e0/aiohttp-3.11.16-cp312-cp312-win_amd64.whl", hash = "sha256:23a15727fbfccab973343b6d1b7181bfb0b4aa7ae280f36fd2f90f5476805682", size = 438012 },
+    { url = "https://files.pythonhosted.org/packages/52/52/7c712b2d9fb4d5e5fd6d12f9ab76e52baddfee71e3c8203ca7a7559d7f51/aiohttp-3.11.16-cp313-cp313-macosx_10_13_universal2.whl", hash = "sha256:a3814760a1a700f3cfd2f977249f1032301d0a12c92aba74605cfa6ce9f78489", size = 698005 },
+    { url = "https://files.pythonhosted.org/packages/51/3e/61057814f7247666d43ac538abcd6335b022869ade2602dab9bf33f607d2/aiohttp-3.11.16-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:9b751a6306f330801665ae69270a8a3993654a85569b3469662efaad6cf5cc50", size = 461106 },
+    { url = "https://files.pythonhosted.org/packages/4f/85/6b79fb0ea6e913d596d5b949edc2402b20803f51b1a59e1bbc5bb7ba7569/aiohttp-3.11.16-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:ad497f38a0d6c329cb621774788583ee12321863cd4bd9feee1effd60f2ad133", size = 453394 },
+    { url = "https://files.pythonhosted.org/packages/4b/04/e1bb3fcfbd2c26753932c759593a32299aff8625eaa0bf8ff7d9c0c34a36/aiohttp-3.11.16-cp313-cp313-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:ca37057625693d097543bd88076ceebeb248291df9d6ca8481349efc0b05dcd0", size = 1666643 },
+    { url = "https://files.pythonhosted.org/packages/0e/27/97bc0fdd1f439b8f060beb3ba8fb47b908dc170280090801158381ad7942/aiohttp-3.11.16-cp313-cp313-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:a5abcbba9f4b463a45c8ca8b7720891200658f6f46894f79517e6cd11f3405ca", size = 1721948 },
+    { url = "https://files.pythonhosted.org/packages/2c/4f/bc4c5119e75c05ef15c5670ef1563bbe25d4ed4893b76c57b0184d815e8b/aiohttp-3.11.16-cp313-cp313-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:f420bfe862fb357a6d76f2065447ef6f484bc489292ac91e29bc65d2d7a2c84d", size = 1774454 },
+    { url = "https://files.pythonhosted.org/packages/73/5b/54b42b2150bb26fdf795464aa55ceb1a49c85f84e98e6896d211eabc6670/aiohttp-3.11.16-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:58ede86453a6cf2d6ce40ef0ca15481677a66950e73b0a788917916f7e35a0bb", size = 1677785 },
+    { url = "https://files.pythonhosted.org/packages/10/ee/a0fe68916d3f82eae199b8535624cf07a9c0a0958c7a76e56dd21140487a/aiohttp-3.11.16-cp313-cp313-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:6fdec0213244c39973674ca2a7f5435bf74369e7d4e104d6c7473c81c9bcc8c4", size = 1608456 },
+    { url = "https://files.pythonhosted.org/packages/8b/48/83afd779242b7cf7e1ceed2ff624a86d3221e17798061cf9a79e0b246077/aiohttp-3.11.16-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:72b1b03fb4655c1960403c131740755ec19c5898c82abd3961c364c2afd59fe7", size = 1622424 },
+    { url = "https://files.pythonhosted.org/packages/6f/27/452f1d5fca1f516f9f731539b7f5faa9e9d3bf8a3a6c3cd7c4b031f20cbd/aiohttp-3.11.16-cp313-cp313-musllinux_1_2_armv7l.whl", hash = "sha256:780df0d837276276226a1ff803f8d0fa5f8996c479aeef52eb040179f3156cbd", size = 1660943 },
+    { url = "https://files.pythonhosted.org/packages/d6/e1/5c7d63143b8d00c83b958b9e78e7048c4a69903c760c1e329bf02bac57a1/aiohttp-3.11.16-cp313-cp313-musllinux_1_2_i686.whl", hash = "sha256:ecdb8173e6c7aa09eee342ac62e193e6904923bd232e76b4157ac0bfa670609f", size = 1622797 },
+    { url = "https://files.pythonhosted.org/packages/46/9e/2ac29cca2746ee8e449e73cd2fcb3d454467393ec03a269d50e49af743f1/aiohttp-3.11.16-cp313-cp313-musllinux_1_2_ppc64le.whl", hash = "sha256:a6db7458ab89c7d80bc1f4e930cc9df6edee2200127cfa6f6e080cf619eddfbd", size = 1687162 },
+    { url = "https://files.pythonhosted.org/packages/ad/6b/eaa6768e02edebaf37d77f4ffb74dd55f5cbcbb6a0dbf798ccec7b0ac23b/aiohttp-3.11.16-cp313-cp313-musllinux_1_2_s390x.whl", hash = "sha256:2540ddc83cc724b13d1838026f6a5ad178510953302a49e6d647f6e1de82bc34", size = 1718518 },
+    { url = "https://files.pythonhosted.org/packages/e5/18/dda87cbad29472a51fa058d6d8257dfce168289adaeb358b86bd93af3b20/aiohttp-3.11.16-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:3b4e6db8dc4879015b9955778cfb9881897339c8fab7b3676f8433f849425913", size = 1675254 },
+    { url = "https://files.pythonhosted.org/packages/32/d9/d2fb08c614df401d92c12fcbc60e6e879608d5e8909ef75c5ad8d4ad8aa7/aiohttp-3.11.16-cp313-cp313-win32.whl", hash = "sha256:493910ceb2764f792db4dc6e8e4b375dae1b08f72e18e8f10f18b34ca17d0979", size = 410698 },
+    { url = "https://files.pythonhosted.org/packages/ce/ed/853e36d5a33c24544cfa46585895547de152dfef0b5c79fa675f6e4b7b87/aiohttp-3.11.16-cp313-cp313-win_amd64.whl", hash = "sha256:42864e70a248f5f6a49fdaf417d9bc62d6e4d8ee9695b24c5916cb4bb666c802", size = 436395 },
+]
+
+[[package]]
+name = "aiosignal"
+version = "1.3.2"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+    { name = "frozenlist" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/ba/b5/6d55e80f6d8a08ce22b982eafa278d823b541c925f11ee774b0b9c43473d/aiosignal-1.3.2.tar.gz", hash = "sha256:a8c255c66fafb1e499c9351d0bf32ff2d8a0321595ebac3b93713656d2436f54", size = 19424 }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/ec/6a/bc7e17a3e87a2985d3e8f4da4cd0f481060eb78fb08596c42be62c90a4d9/aiosignal-1.3.2-py2.py3-none-any.whl", hash = "sha256:45cde58e409a301715980c2b01d0c28bdde3770d8290b5eb2173759d9acb31a5", size = 7597 },
+]
+
+[[package]]
+name = "asyncssh"
+version = "2.20.0"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+    { name = "cryptography" },
+    { name = "typing-extensions" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/43/f5/365489721f98bb1cd5e3d48f397598e1be5b2695301e2d685b64cfdab51d/asyncssh-2.20.0.tar.gz", hash = "sha256:020b6e384b2328ef8683908ad8e73de9ec2b9b62fd964571ea957bba98412983", size = 534719 }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/f2/c5/7a9d42c065f272f74792663194966a68810e851a34c0dd080246bd7308f5/asyncssh-2.20.0-py3-none-any.whl", hash = "sha256:af6888d937c07a4bf31293335a6166b4d87608cdb5957b49547da6ad87ecf174", size = 373109 },
+]
+
+[[package]]
+name = "attrs"
+version = "25.3.0"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/5a/b0/1367933a8532ee6ff8d63537de4f1177af4bff9f3e829baf7331f595bb24/attrs-25.3.0.tar.gz", hash = "sha256:75d7cefc7fb576747b2c81b4442d4d4a1ce0900973527c011d1030fd3bf4af1b", size = 812032 }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/77/06/bb80f5f86020c4551da315d78b3ab75e8228f89f0162f2c3a819e407941a/attrs-25.3.0-py3-none-any.whl", hash = "sha256:427318ce031701fea540783410126f03899a97ffc6f61596ad581ac2e40e3bc3", size = 63815 },
+]
+
+[[package]]
+name = "cffi"
+version = "1.17.1"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+    { name = "pycparser" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/fc/97/c783634659c2920c3fc70419e3af40972dbaf758daa229a7d6ea6135c90d/cffi-1.17.1.tar.gz", hash = "sha256:1c39c6016c32bc48dd54561950ebd6836e1670f2ae46128f67cf49e789c52824", size = 516621 }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/5a/84/e94227139ee5fb4d600a7a4927f322e1d4aea6fdc50bd3fca8493caba23f/cffi-1.17.1-cp312-cp312-macosx_10_9_x86_64.whl", hash = "sha256:805b4371bf7197c329fcb3ead37e710d1bca9da5d583f5073b799d5c5bd1eee4", size = 183178 },
+    { url = "https://files.pythonhosted.org/packages/da/ee/fb72c2b48656111c4ef27f0f91da355e130a923473bf5ee75c5643d00cca/cffi-1.17.1-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:733e99bc2df47476e3848417c5a4540522f234dfd4ef3ab7fafdf555b082ec0c", size = 178840 },
+    { url = "https://files.pythonhosted.org/packages/cc/b6/db007700f67d151abadf508cbfd6a1884f57eab90b1bb985c4c8c02b0f28/cffi-1.17.1-cp312-cp312-manylinux_2_12_i686.manylinux2010_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:1257bdabf294dceb59f5e70c64a3e2f462c30c7ad68092d01bbbfb1c16b1ba36", size = 454803 },
+    { url = "https://files.pythonhosted.org/packages/1a/df/f8d151540d8c200eb1c6fba8cd0dfd40904f1b0682ea705c36e6c2e97ab3/cffi-1.17.1-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:da95af8214998d77a98cc14e3a3bd00aa191526343078b530ceb0bd710fb48a5", size = 478850 },
+    { url = "https://files.pythonhosted.org/packages/28/c0/b31116332a547fd2677ae5b78a2ef662dfc8023d67f41b2a83f7c2aa78b1/cffi-1.17.1-cp312-cp312-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:d63afe322132c194cf832bfec0dc69a99fb9bb6bbd550f161a49e9e855cc78ff", size = 485729 },
+    { url = "https://files.pythonhosted.org/packages/91/2b/9a1ddfa5c7f13cab007a2c9cc295b70fbbda7cb10a286aa6810338e60ea1/cffi-1.17.1-cp312-cp312-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:f79fc4fc25f1c8698ff97788206bb3c2598949bfe0fef03d299eb1b5356ada99", size = 471256 },
+    { url = "https://files.pythonhosted.org/packages/b2/d5/da47df7004cb17e4955df6a43d14b3b4ae77737dff8bf7f8f333196717bf/cffi-1.17.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:b62ce867176a75d03a665bad002af8e6d54644fad99a3c70905c543130e39d93", size = 479424 },
+    { url = "https://files.pythonhosted.org/packages/0b/ac/2a28bcf513e93a219c8a4e8e125534f4f6db03e3179ba1c45e949b76212c/cffi-1.17.1-cp312-cp312-musllinux_1_1_aarch64.whl", hash = "sha256:386c8bf53c502fff58903061338ce4f4950cbdcb23e2902d86c0f722b786bbe3", size = 484568 },
+    { url = "https://files.pythonhosted.org/packages/d4/38/ca8a4f639065f14ae0f1d9751e70447a261f1a30fa7547a828ae08142465/cffi-1.17.1-cp312-cp312-musllinux_1_1_x86_64.whl", hash = "sha256:4ceb10419a9adf4460ea14cfd6bc43d08701f0835e979bf821052f1805850fe8", size = 488736 },
+    { url = "https://files.pythonhosted.org/packages/86/c5/28b2d6f799ec0bdecf44dced2ec5ed43e0eb63097b0f58c293583b406582/cffi-1.17.1-cp312-cp312-win32.whl", hash = "sha256:a08d7e755f8ed21095a310a693525137cfe756ce62d066e53f502a83dc550f65", size = 172448 },
+    { url = "https://files.pythonhosted.org/packages/50/b9/db34c4755a7bd1cb2d1603ac3863f22bcecbd1ba29e5ee841a4bc510b294/cffi-1.17.1-cp312-cp312-win_amd64.whl", hash = "sha256:51392eae71afec0d0c8fb1a53b204dbb3bcabcb3c9b807eedf3e1e6ccf2de903", size = 181976 },
+    { url = "https://files.pythonhosted.org/packages/8d/f8/dd6c246b148639254dad4d6803eb6a54e8c85c6e11ec9df2cffa87571dbe/cffi-1.17.1-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:f3a2b4222ce6b60e2e8b337bb9596923045681d71e5a082783484d845390938e", size = 182989 },
+    { url = "https://files.pythonhosted.org/packages/8b/f1/672d303ddf17c24fc83afd712316fda78dc6fce1cd53011b839483e1ecc8/cffi-1.17.1-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:0984a4925a435b1da406122d4d7968dd861c1385afe3b45ba82b750f229811e2", size = 178802 },
+    { url = "https://files.pythonhosted.org/packages/0e/2d/eab2e858a91fdff70533cab61dcff4a1f55ec60425832ddfdc9cd36bc8af/cffi-1.17.1-cp313-cp313-manylinux_2_12_i686.manylinux2010_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:d01b12eeeb4427d3110de311e1774046ad344f5b1a7403101878976ecd7a10f3", size = 454792 },
+    { url = "https://files.pythonhosted.org/packages/75/b2/fbaec7c4455c604e29388d55599b99ebcc250a60050610fadde58932b7ee/cffi-1.17.1-cp313-cp313-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:706510fe141c86a69c8ddc029c7910003a17353970cff3b904ff0686a5927683", size = 478893 },
+    { url = "https://files.pythonhosted.org/packages/4f/b7/6e4a2162178bf1935c336d4da8a9352cccab4d3a5d7914065490f08c0690/cffi-1.17.1-cp313-cp313-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:de55b766c7aa2e2a3092c51e0483d700341182f08e67c63630d5b6f200bb28e5", size = 485810 },
+    { url = "https://files.pythonhosted.org/packages/c7/8a/1d0e4a9c26e54746dc08c2c6c037889124d4f59dffd853a659fa545f1b40/cffi-1.17.1-cp313-cp313-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:c59d6e989d07460165cc5ad3c61f9fd8f1b4796eacbd81cee78957842b834af4", size = 471200 },
+    { url = "https://files.pythonhosted.org/packages/26/9f/1aab65a6c0db35f43c4d1b4f580e8df53914310afc10ae0397d29d697af4/cffi-1.17.1-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:dd398dbc6773384a17fe0d3e7eeb8d1a21c2200473ee6806bb5e6a8e62bb73dd", size = 479447 },
+    { url = "https://files.pythonhosted.org/packages/5f/e4/fb8b3dd8dc0e98edf1135ff067ae070bb32ef9d509d6cb0f538cd6f7483f/cffi-1.17.1-cp313-cp313-musllinux_1_1_aarch64.whl", hash = "sha256:3edc8d958eb099c634dace3c7e16560ae474aa3803a5df240542b305d14e14ed", size = 484358 },
+    { url = "https://files.pythonhosted.org/packages/f1/47/d7145bf2dc04684935d57d67dff9d6d795b2ba2796806bb109864be3a151/cffi-1.17.1-cp313-cp313-musllinux_1_1_x86_64.whl", hash = "sha256:72e72408cad3d5419375fc87d289076ee319835bdfa2caad331e377589aebba9", size = 488469 },
+    { url = "https://files.pythonhosted.org/packages/bf/ee/f94057fa6426481d663b88637a9a10e859e492c73d0384514a17d78ee205/cffi-1.17.1-cp313-cp313-win32.whl", hash = "sha256:e03eab0a8677fa80d646b5ddece1cbeaf556c313dcfac435ba11f107ba117b5d", size = 172475 },
+    { url = "https://files.pythonhosted.org/packages/7c/fc/6a8cb64e5f0324877d503c854da15d76c1e50eb722e320b15345c4d0c6de/cffi-1.17.1-cp313-cp313-win_amd64.whl", hash = "sha256:f6a16c31041f09ead72d69f583767292f750d24913dadacf5756b966aacb3f1a", size = 182009 },
+]
+
+[[package]]
+name = "cryptography"
+version = "44.0.2"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+    { name = "cffi", marker = "platform_python_implementation != 'PyPy'" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/cd/25/4ce80c78963834b8a9fd1cc1266be5ed8d1840785c0f2e1b73b8d128d505/cryptography-44.0.2.tar.gz", hash = "sha256:c63454aa261a0cf0c5b4718349629793e9e634993538db841165b3df74f37ec0", size = 710807 }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/92/ef/83e632cfa801b221570c5f58c0369db6fa6cef7d9ff859feab1aae1a8a0f/cryptography-44.0.2-cp37-abi3-macosx_10_9_universal2.whl", hash = "sha256:efcfe97d1b3c79e486554efddeb8f6f53a4cdd4cf6086642784fa31fc384e1d7", size = 6676361 },
+    { url = "https://files.pythonhosted.org/packages/30/ec/7ea7c1e4c8fc8329506b46c6c4a52e2f20318425d48e0fe597977c71dbce/cryptography-44.0.2-cp37-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:29ecec49f3ba3f3849362854b7253a9f59799e3763b0c9d0826259a88efa02f1", size = 3952350 },
+    { url = "https://files.pythonhosted.org/packages/27/61/72e3afdb3c5ac510330feba4fc1faa0fe62e070592d6ad00c40bb69165e5/cryptography-44.0.2-cp37-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:bc821e161ae88bfe8088d11bb39caf2916562e0a2dc7b6d56714a48b784ef0bb", size = 4166572 },
+    { url = "https://files.pythonhosted.org/packages/26/e4/ba680f0b35ed4a07d87f9e98f3ebccb05091f3bf6b5a478b943253b3bbd5/cryptography-44.0.2-cp37-abi3-manylinux_2_28_aarch64.whl", hash = "sha256:3c00b6b757b32ce0f62c574b78b939afab9eecaf597c4d624caca4f9e71e7843", size = 3958124 },
+    { url = "https://files.pythonhosted.org/packages/9c/e8/44ae3e68c8b6d1cbc59040288056df2ad7f7f03bbcaca6b503c737ab8e73/cryptography-44.0.2-cp37-abi3-manylinux_2_28_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:7bdcd82189759aba3816d1f729ce42ffded1ac304c151d0a8e89b9996ab863d5", size = 3678122 },
+    { url = "https://files.pythonhosted.org/packages/27/7b/664ea5e0d1eab511a10e480baf1c5d3e681c7d91718f60e149cec09edf01/cryptography-44.0.2-cp37-abi3-manylinux_2_28_x86_64.whl", hash = "sha256:4973da6ca3db4405c54cd0b26d328be54c7747e89e284fcff166132eb7bccc9c", size = 4191831 },
+    { url = "https://files.pythonhosted.org/packages/2a/07/79554a9c40eb11345e1861f46f845fa71c9e25bf66d132e123d9feb8e7f9/cryptography-44.0.2-cp37-abi3-manylinux_2_34_aarch64.whl", hash = "sha256:4e389622b6927d8133f314949a9812972711a111d577a5d1f4bee5e58736b80a", size = 3960583 },
+    { url = "https://files.pythonhosted.org/packages/bb/6d/858e356a49a4f0b591bd6789d821427de18432212e137290b6d8a817e9bf/cryptography-44.0.2-cp37-abi3-manylinux_2_34_x86_64.whl", hash = "sha256:f514ef4cd14bb6fb484b4a60203e912cfcb64f2ab139e88c2274511514bf7308", size = 4191753 },
+    { url = "https://files.pythonhosted.org/packages/b2/80/62df41ba4916067fa6b125aa8c14d7e9181773f0d5d0bd4dcef580d8b7c6/cryptography-44.0.2-cp37-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:1bc312dfb7a6e5d66082c87c34c8a62176e684b6fe3d90fcfe1568de675e6688", size = 4079550 },
+    { url = "https://files.pythonhosted.org/packages/f3/cd/2558cc08f7b1bb40683f99ff4327f8dcfc7de3affc669e9065e14824511b/cryptography-44.0.2-cp37-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:3b721b8b4d948b218c88cb8c45a01793483821e709afe5f622861fc6182b20a7", size = 4298367 },
+    { url = "https://files.pythonhosted.org/packages/71/59/94ccc74788945bc3bd4cf355d19867e8057ff5fdbcac781b1ff95b700fb1/cryptography-44.0.2-cp37-abi3-win32.whl", hash = "sha256:51e4de3af4ec3899d6d178a8c005226491c27c4ba84101bfb59c901e10ca9f79", size = 2772843 },
+    { url = "https://files.pythonhosted.org/packages/ca/2c/0d0bbaf61ba05acb32f0841853cfa33ebb7a9ab3d9ed8bb004bd39f2da6a/cryptography-44.0.2-cp37-abi3-win_amd64.whl", hash = "sha256:c505d61b6176aaf982c5717ce04e87da5abc9a36a5b39ac03905c4aafe8de7aa", size = 3209057 },
+    { url = "https://files.pythonhosted.org/packages/9e/be/7a26142e6d0f7683d8a382dd963745e65db895a79a280a30525ec92be890/cryptography-44.0.2-cp39-abi3-macosx_10_9_universal2.whl", hash = "sha256:8e0ddd63e6bf1161800592c71ac794d3fb8001f2caebe0966e77c5234fa9efc3", size = 6677789 },
+    { url = "https://files.pythonhosted.org/packages/06/88/638865be7198a84a7713950b1db7343391c6066a20e614f8fa286eb178ed/cryptography-44.0.2-cp39-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:81276f0ea79a208d961c433a947029e1a15948966658cf6710bbabb60fcc2639", size = 3951919 },
+    { url = "https://files.pythonhosted.org/packages/d7/fc/99fe639bcdf58561dfad1faa8a7369d1dc13f20acd78371bb97a01613585/cryptography-44.0.2-cp39-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:9a1e657c0f4ea2a23304ee3f964db058c9e9e635cc7019c4aa21c330755ef6fd", size = 4167812 },
+    { url = "https://files.pythonhosted.org/packages/53/7b/aafe60210ec93d5d7f552592a28192e51d3c6b6be449e7fd0a91399b5d07/cryptography-44.0.2-cp39-abi3-manylinux_2_28_aarch64.whl", hash = "sha256:6210c05941994290f3f7f175a4a57dbbb2afd9273657614c506d5976db061181", size = 3958571 },
+    { url = "https://files.pythonhosted.org/packages/16/32/051f7ce79ad5a6ef5e26a92b37f172ee2d6e1cce09931646eef8de1e9827/cryptography-44.0.2-cp39-abi3-manylinux_2_28_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:d1c3572526997b36f245a96a2b1713bf79ce99b271bbcf084beb6b9b075f29ea", size = 3679832 },
+    { url = "https://files.pythonhosted.org/packages/78/2b/999b2a1e1ba2206f2d3bca267d68f350beb2b048a41ea827e08ce7260098/cryptography-44.0.2-cp39-abi3-manylinux_2_28_x86_64.whl", hash = "sha256:b042d2a275c8cee83a4b7ae30c45a15e6a4baa65a179a0ec2d78ebb90e4f6699", size = 4193719 },
+    { url = "https://files.pythonhosted.org/packages/72/97/430e56e39a1356e8e8f10f723211a0e256e11895ef1a135f30d7d40f2540/cryptography-44.0.2-cp39-abi3-manylinux_2_34_aarch64.whl", hash = "sha256:d03806036b4f89e3b13b6218fefea8d5312e450935b1a2d55f0524e2ed7c59d9", size = 3960852 },
+    { url = "https://files.pythonhosted.org/packages/89/33/c1cf182c152e1d262cac56850939530c05ca6c8d149aa0dcee490b417e99/cryptography-44.0.2-cp39-abi3-manylinux_2_34_x86_64.whl", hash = "sha256:c7362add18b416b69d58c910caa217f980c5ef39b23a38a0880dfd87bdf8cd23", size = 4193906 },
+    { url = "https://files.pythonhosted.org/packages/e1/99/87cf26d4f125380dc674233971069bc28d19b07f7755b29861570e513650/cryptography-44.0.2-cp39-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:8cadc6e3b5a1f144a039ea08a0bdb03a2a92e19c46be3285123d32029f40a922", size = 4081572 },
+    { url = "https://files.pythonhosted.org/packages/b3/9f/6a3e0391957cc0c5f84aef9fbdd763035f2b52e998a53f99345e3ac69312/cryptography-44.0.2-cp39-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:6f101b1f780f7fc613d040ca4bdf835c6ef3b00e9bd7125a4255ec574c7916e4", size = 4298631 },
+    { url = "https://files.pythonhosted.org/packages/e2/a5/5bc097adb4b6d22a24dea53c51f37e480aaec3465285c253098642696423/cryptography-44.0.2-cp39-abi3-win32.whl", hash = "sha256:3dc62975e31617badc19a906481deacdeb80b4bb454394b4098e3f2525a488c5", size = 2773792 },
+    { url = "https://files.pythonhosted.org/packages/33/cf/1f7649b8b9a3543e042d3f348e398a061923ac05b507f3f4d95f11938aa9/cryptography-44.0.2-cp39-abi3-win_amd64.whl", hash = "sha256:5f6f90b72d8ccadb9c6e311c775c8305381db88374c65fa1a68250aa8a9cb3a6", size = 3210957 },
+]
+
+[[package]]
+name = "frozenlist"
+version = "1.6.0"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/ee/f4/d744cba2da59b5c1d88823cf9e8a6c74e4659e2b27604ed973be2a0bf5ab/frozenlist-1.6.0.tar.gz", hash = "sha256:b99655c32c1c8e06d111e7f41c06c29a5318cb1835df23a45518e02a47c63b68", size = 42831 }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/9c/8a/289b7d0de2fbac832ea80944d809759976f661557a38bb8e77db5d9f79b7/frozenlist-1.6.0-cp312-cp312-macosx_10_13_universal2.whl", hash = "sha256:c5b9e42ace7d95bf41e19b87cec8f262c41d3510d8ad7514ab3862ea2197bfb1", size = 160193 },
+    { url = "https://files.pythonhosted.org/packages/19/80/2fd17d322aec7f430549f0669f599997174f93ee17929ea5b92781ec902c/frozenlist-1.6.0-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:ca9973735ce9f770d24d5484dcb42f68f135351c2fc81a7a9369e48cf2998a29", size = 123831 },
+    { url = "https://files.pythonhosted.org/packages/99/06/f5812da431273f78c6543e0b2f7de67dfd65eb0a433978b2c9c63d2205e4/frozenlist-1.6.0-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:6ac40ec76041c67b928ca8aaffba15c2b2ee3f5ae8d0cb0617b5e63ec119ca25", size = 121862 },
+    { url = "https://files.pythonhosted.org/packages/d0/31/9e61c6b5fc493cf24d54881731204d27105234d09878be1a5983182cc4a5/frozenlist-1.6.0-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:95b7a8a3180dfb280eb044fdec562f9b461614c0ef21669aea6f1d3dac6ee576", size = 316361 },
+    { url = "https://files.pythonhosted.org/packages/9d/55/22ca9362d4f0222324981470fd50192be200154d51509ee6eb9baa148e96/frozenlist-1.6.0-cp312-cp312-manylinux_2_17_armv7l.manylinux2014_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:c444d824e22da6c9291886d80c7d00c444981a72686e2b59d38b285617cb52c8", size = 307115 },
+    { url = "https://files.pythonhosted.org/packages/ae/39/4fff42920a57794881e7bb3898dc7f5f539261711ea411b43bba3cde8b79/frozenlist-1.6.0-cp312-cp312-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:bb52c8166499a8150bfd38478248572c924c003cbb45fe3bcd348e5ac7c000f9", size = 322505 },
+    { url = "https://files.pythonhosted.org/packages/55/f2/88c41f374c1e4cf0092a5459e5f3d6a1e17ed274c98087a76487783df90c/frozenlist-1.6.0-cp312-cp312-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:b35298b2db9c2468106278537ee529719228950a5fdda686582f68f247d1dc6e", size = 322666 },
+    { url = "https://files.pythonhosted.org/packages/75/51/034eeb75afdf3fd03997856195b500722c0b1a50716664cde64e28299c4b/frozenlist-1.6.0-cp312-cp312-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:d108e2d070034f9d57210f22fefd22ea0d04609fc97c5f7f5a686b3471028590", size = 302119 },
+    { url = "https://files.pythonhosted.org/packages/2b/a6/564ecde55ee633270a793999ef4fd1d2c2b32b5a7eec903b1012cb7c5143/frozenlist-1.6.0-cp312-cp312-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:4e1be9111cb6756868ac242b3c2bd1f09d9aea09846e4f5c23715e7afb647103", size = 316226 },
+    { url = "https://files.pythonhosted.org/packages/f1/c8/6c0682c32377f402b8a6174fb16378b683cf6379ab4d2827c580892ab3c7/frozenlist-1.6.0-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:94bb451c664415f02f07eef4ece976a2c65dcbab9c2f1705b7031a3a75349d8c", size = 312788 },
+    { url = "https://files.pythonhosted.org/packages/b6/b8/10fbec38f82c5d163ca1750bfff4ede69713badf236a016781cf1f10a0f0/frozenlist-1.6.0-cp312-cp312-musllinux_1_2_armv7l.whl", hash = "sha256:d1a686d0b0949182b8faddea596f3fc11f44768d1f74d4cad70213b2e139d821", size = 325914 },
+    { url = "https://files.pythonhosted.org/packages/62/ca/2bf4f3a1bd40cdedd301e6ecfdbb291080d5afc5f9ce350c0739f773d6b9/frozenlist-1.6.0-cp312-cp312-musllinux_1_2_i686.whl", hash = "sha256:ea8e59105d802c5a38bdbe7362822c522230b3faba2aa35c0fa1765239b7dd70", size = 305283 },
+    { url = "https://files.pythonhosted.org/packages/09/64/20cc13ccf94abc2a1f482f74ad210703dc78a590d0b805af1c9aa67f76f9/frozenlist-1.6.0-cp312-cp312-musllinux_1_2_ppc64le.whl", hash = "sha256:abc4e880a9b920bc5020bf6a431a6bb40589d9bca3975c980495f63632e8382f", size = 319264 },
+    { url = "https://files.pythonhosted.org/packages/20/ff/86c6a2bbe98cfc231519f5e6d712a0898488ceac804a917ce014f32e68f6/frozenlist-1.6.0-cp312-cp312-musllinux_1_2_s390x.whl", hash = "sha256:9a79713adfe28830f27a3c62f6b5406c37376c892b05ae070906f07ae4487046", size = 326482 },
+    { url = "https://files.pythonhosted.org/packages/2f/da/8e381f66367d79adca245d1d71527aac774e30e291d41ef161ce2d80c38e/frozenlist-1.6.0-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:9a0318c2068e217a8f5e3b85e35899f5a19e97141a45bb925bb357cfe1daf770", size = 318248 },
+    { url = "https://files.pythonhosted.org/packages/39/24/1a1976563fb476ab6f0fa9fefaac7616a4361dbe0461324f9fd7bf425dbe/frozenlist-1.6.0-cp312-cp312-win32.whl", hash = "sha256:853ac025092a24bb3bf09ae87f9127de9fe6e0c345614ac92536577cf956dfcc", size = 115161 },
+    { url = "https://files.pythonhosted.org/packages/80/2e/fb4ed62a65f8cd66044706b1013f0010930d8cbb0729a2219561ea075434/frozenlist-1.6.0-cp312-cp312-win_amd64.whl", hash = "sha256:2bdfe2d7e6c9281c6e55523acd6c2bf77963cb422fdc7d142fb0cb6621b66878", size = 120548 },
+    { url = "https://files.pythonhosted.org/packages/6f/e5/04c7090c514d96ca00887932417f04343ab94904a56ab7f57861bf63652d/frozenlist-1.6.0-cp313-cp313-macosx_10_13_universal2.whl", hash = "sha256:1d7fb014fe0fbfee3efd6a94fc635aeaa68e5e1720fe9e57357f2e2c6e1a647e", size = 158182 },
+    { url = "https://files.pythonhosted.org/packages/e9/8f/60d0555c61eec855783a6356268314d204137f5e0c53b59ae2fc28938c99/frozenlist-1.6.0-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:01bcaa305a0fdad12745502bfd16a1c75b14558dabae226852f9159364573117", size = 122838 },
+    { url = "https://files.pythonhosted.org/packages/5a/a7/d0ec890e3665b4b3b7c05dc80e477ed8dc2e2e77719368e78e2cd9fec9c8/frozenlist-1.6.0-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:8b314faa3051a6d45da196a2c495e922f987dc848e967d8cfeaee8a0328b1cd4", size = 120980 },
+    { url = "https://files.pythonhosted.org/packages/cc/19/9b355a5e7a8eba903a008579964192c3e427444752f20b2144b10bb336df/frozenlist-1.6.0-cp313-cp313-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:da62fecac21a3ee10463d153549d8db87549a5e77eefb8c91ac84bb42bb1e4e3", size = 305463 },
+    { url = "https://files.pythonhosted.org/packages/9c/8d/5b4c758c2550131d66935ef2fa700ada2461c08866aef4229ae1554b93ca/frozenlist-1.6.0-cp313-cp313-manylinux_2_17_armv7l.manylinux2014_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:d1eb89bf3454e2132e046f9599fbcf0a4483ed43b40f545551a39316d0201cd1", size = 297985 },
+    { url = "https://files.pythonhosted.org/packages/48/2c/537ec09e032b5865715726b2d1d9813e6589b571d34d01550c7aeaad7e53/frozenlist-1.6.0-cp313-cp313-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:d18689b40cb3936acd971f663ccb8e2589c45db5e2c5f07e0ec6207664029a9c", size = 311188 },
+    { url = "https://files.pythonhosted.org/packages/31/2f/1aa74b33f74d54817055de9a4961eff798f066cdc6f67591905d4fc82a84/frozenlist-1.6.0-cp313-cp313-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:e67ddb0749ed066b1a03fba812e2dcae791dd50e5da03be50b6a14d0c1a9ee45", size = 311874 },
+    { url = "https://files.pythonhosted.org/packages/bf/f0/cfec18838f13ebf4b37cfebc8649db5ea71a1b25dacd691444a10729776c/frozenlist-1.6.0-cp313-cp313-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:fc5e64626e6682638d6e44398c9baf1d6ce6bc236d40b4b57255c9d3f9761f1f", size = 291897 },
+    { url = "https://files.pythonhosted.org/packages/ea/a5/deb39325cbbea6cd0a46db8ccd76150ae2fcbe60d63243d9df4a0b8c3205/frozenlist-1.6.0-cp313-cp313-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:437cfd39564744ae32ad5929e55b18ebd88817f9180e4cc05e7d53b75f79ce85", size = 305799 },
+    { url = "https://files.pythonhosted.org/packages/78/22/6ddec55c5243a59f605e4280f10cee8c95a449f81e40117163383829c241/frozenlist-1.6.0-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:62dd7df78e74d924952e2feb7357d826af8d2f307557a779d14ddf94d7311be8", size = 302804 },
+    { url = "https://files.pythonhosted.org/packages/5d/b7/d9ca9bab87f28855063c4d202936800219e39db9e46f9fb004d521152623/frozenlist-1.6.0-cp313-cp313-musllinux_1_2_armv7l.whl", hash = "sha256:a66781d7e4cddcbbcfd64de3d41a61d6bdde370fc2e38623f30b2bd539e84a9f", size = 316404 },
+    { url = "https://files.pythonhosted.org/packages/a6/3a/1255305db7874d0b9eddb4fe4a27469e1fb63720f1fc6d325a5118492d18/frozenlist-1.6.0-cp313-cp313-musllinux_1_2_i686.whl", hash = "sha256:482fe06e9a3fffbcd41950f9d890034b4a54395c60b5e61fae875d37a699813f", size = 295572 },
+    { url = "https://files.pythonhosted.org/packages/2a/f2/8d38eeee39a0e3a91b75867cc102159ecccf441deb6ddf67be96d3410b84/frozenlist-1.6.0-cp313-cp313-musllinux_1_2_ppc64le.whl", hash = "sha256:e4f9373c500dfc02feea39f7a56e4f543e670212102cc2eeb51d3a99c7ffbde6", size = 307601 },
+    { url = "https://files.pythonhosted.org/packages/38/04/80ec8e6b92f61ef085422d7b196822820404f940950dde5b2e367bede8bc/frozenlist-1.6.0-cp313-cp313-musllinux_1_2_s390x.whl", hash = "sha256:e69bb81de06827147b7bfbaeb284d85219fa92d9f097e32cc73675f279d70188", size = 314232 },
+    { url = "https://files.pythonhosted.org/packages/3a/58/93b41fb23e75f38f453ae92a2f987274c64637c450285577bd81c599b715/frozenlist-1.6.0-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:7613d9977d2ab4a9141dde4a149f4357e4065949674c5649f920fec86ecb393e", size = 308187 },
+    { url = "https://files.pythonhosted.org/packages/6a/a2/e64df5c5aa36ab3dee5a40d254f3e471bb0603c225f81664267281c46a2d/frozenlist-1.6.0-cp313-cp313-win32.whl", hash = "sha256:4def87ef6d90429f777c9d9de3961679abf938cb6b7b63d4a7eb8a268babfce4", size = 114772 },
+    { url = "https://files.pythonhosted.org/packages/a0/77/fead27441e749b2d574bb73d693530d59d520d4b9e9679b8e3cb779d37f2/frozenlist-1.6.0-cp313-cp313-win_amd64.whl", hash = "sha256:37a8a52c3dfff01515e9bbbee0e6063181362f9de3db2ccf9bc96189b557cbfd", size = 119847 },
+    { url = "https://files.pythonhosted.org/packages/df/bd/cc6d934991c1e5d9cafda83dfdc52f987c7b28343686aef2e58a9cf89f20/frozenlist-1.6.0-cp313-cp313t-macosx_10_13_universal2.whl", hash = "sha256:46138f5a0773d064ff663d273b309b696293d7a7c00a0994c5c13a5078134b64", size = 174937 },
+    { url = "https://files.pythonhosted.org/packages/f2/a2/daf945f335abdbfdd5993e9dc348ef4507436936ab3c26d7cfe72f4843bf/frozenlist-1.6.0-cp313-cp313t-macosx_10_13_x86_64.whl", hash = "sha256:f88bc0a2b9c2a835cb888b32246c27cdab5740059fb3688852bf91e915399b91", size = 136029 },
+    { url = "https://files.pythonhosted.org/packages/51/65/4c3145f237a31247c3429e1c94c384d053f69b52110a0d04bfc8afc55fb2/frozenlist-1.6.0-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:777704c1d7655b802c7850255639672e90e81ad6fa42b99ce5ed3fbf45e338dd", size = 134831 },
+    { url = "https://files.pythonhosted.org/packages/77/38/03d316507d8dea84dfb99bdd515ea245628af964b2bf57759e3c9205cc5e/frozenlist-1.6.0-cp313-cp313t-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:85ef8d41764c7de0dcdaf64f733a27352248493a85a80661f3c678acd27e31f2", size = 392981 },
+    { url = "https://files.pythonhosted.org/packages/37/02/46285ef9828f318ba400a51d5bb616ded38db8466836a9cfa39f3903260b/frozenlist-1.6.0-cp313-cp313t-manylinux_2_17_armv7l.manylinux2014_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:da5cb36623f2b846fb25009d9d9215322318ff1c63403075f812b3b2876c8506", size = 371999 },
+    { url = "https://files.pythonhosted.org/packages/0d/64/1212fea37a112c3c5c05bfb5f0a81af4836ce349e69be75af93f99644da9/frozenlist-1.6.0-cp313-cp313t-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:cbb56587a16cf0fb8acd19e90ff9924979ac1431baea8681712716a8337577b0", size = 392200 },
+    { url = "https://files.pythonhosted.org/packages/81/ce/9a6ea1763e3366e44a5208f76bf37c76c5da570772375e4d0be85180e588/frozenlist-1.6.0-cp313-cp313t-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:c6154c3ba59cda3f954c6333025369e42c3acd0c6e8b6ce31eb5c5b8116c07e0", size = 390134 },
+    { url = "https://files.pythonhosted.org/packages/bc/36/939738b0b495b2c6d0c39ba51563e453232813042a8d908b8f9544296c29/frozenlist-1.6.0-cp313-cp313t-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:2e8246877afa3f1ae5c979fe85f567d220f86a50dc6c493b9b7d8191181ae01e", size = 365208 },
+    { url = "https://files.pythonhosted.org/packages/b4/8b/939e62e93c63409949c25220d1ba8e88e3960f8ef6a8d9ede8f94b459d27/frozenlist-1.6.0-cp313-cp313t-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:7b0f6cce16306d2e117cf9db71ab3a9e8878a28176aeaf0dbe35248d97b28d0c", size = 385548 },
+    { url = "https://files.pythonhosted.org/packages/62/38/22d2873c90102e06a7c5a3a5b82ca47e393c6079413e8a75c72bff067fa8/frozenlist-1.6.0-cp313-cp313t-musllinux_1_2_aarch64.whl", hash = "sha256:1b8e8cd8032ba266f91136d7105706ad57770f3522eac4a111d77ac126a25a9b", size = 391123 },
+    { url = "https://files.pythonhosted.org/packages/44/78/63aaaf533ee0701549500f6d819be092c6065cb5c577edb70c09df74d5d0/frozenlist-1.6.0-cp313-cp313t-musllinux_1_2_armv7l.whl", hash = "sha256:e2ada1d8515d3ea5378c018a5f6d14b4994d4036591a52ceaf1a1549dec8e1ad", size = 394199 },
+    { url = "https://files.pythonhosted.org/packages/54/45/71a6b48981d429e8fbcc08454dc99c4c2639865a646d549812883e9c9dd3/frozenlist-1.6.0-cp313-cp313t-musllinux_1_2_i686.whl", hash = "sha256:cdb2c7f071e4026c19a3e32b93a09e59b12000751fc9b0b7758da899e657d215", size = 373854 },
+    { url = "https://files.pythonhosted.org/packages/3f/f3/dbf2a5e11736ea81a66e37288bf9f881143a7822b288a992579ba1b4204d/frozenlist-1.6.0-cp313-cp313t-musllinux_1_2_ppc64le.whl", hash = "sha256:03572933a1969a6d6ab509d509e5af82ef80d4a5d4e1e9f2e1cdd22c77a3f4d2", size = 395412 },
+    { url = "https://files.pythonhosted.org/packages/b3/f1/c63166806b331f05104d8ea385c4acd511598568b1f3e4e8297ca54f2676/frozenlist-1.6.0-cp313-cp313t-musllinux_1_2_s390x.whl", hash = "sha256:77effc978947548b676c54bbd6a08992759ea6f410d4987d69feea9cd0919911", size = 394936 },
+    { url = "https://files.pythonhosted.org/packages/ef/ea/4f3e69e179a430473eaa1a75ff986526571215fefc6b9281cdc1f09a4eb8/frozenlist-1.6.0-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:a2bda8be77660ad4089caf2223fdbd6db1858462c4b85b67fbfa22102021e497", size = 391459 },
+    { url = "https://files.pythonhosted.org/packages/d3/c3/0fc2c97dea550df9afd072a37c1e95421652e3206bbeaa02378b24c2b480/frozenlist-1.6.0-cp313-cp313t-win32.whl", hash = "sha256:a4d96dc5bcdbd834ec6b0f91027817214216b5b30316494d2b1aebffb87c534f", size = 128797 },
+    { url = "https://files.pythonhosted.org/packages/ae/f5/79c9320c5656b1965634fe4be9c82b12a3305bdbc58ad9cb941131107b20/frozenlist-1.6.0-cp313-cp313t-win_amd64.whl", hash = "sha256:e18036cb4caa17ea151fd5f3d70be9d354c99eb8cf817a3ccde8a7873b074348", size = 134709 },
+    { url = "https://files.pythonhosted.org/packages/71/3e/b04a0adda73bd52b390d730071c0d577073d3d26740ee1bad25c3ad0f37b/frozenlist-1.6.0-py3-none-any.whl", hash = "sha256:535eec9987adb04701266b92745d6cdcef2e77669299359c3009c3404dd5d191", size = 12404 },
+]
+
+[[package]]
+name = "idna"
+version = "3.10"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/f1/70/7703c29685631f5a7590aa73f1f1d3fa9a380e654b86af429e0934a32f7d/idna-3.10.tar.gz", hash = "sha256:12f65c9b470abda6dc35cf8e63cc574b1c52b11df2c86030af0ac09b01b13ea9", size = 190490 }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/76/c6/c88e154df9c4e1a2a66ccf0005a88dfb2650c1dffb6f5ce603dfbd452ce3/idna-3.10-py3-none-any.whl", hash = "sha256:946d195a0d259cbba61165e88e65941f16e9b36ea6ddb97f00452bae8b1287d3", size = 70442 },
+]
+
 [[package]]
 name = "mapvoyage-extract"
 version = "0.1.0"
 source = { virtual = "." }
 dependencies = [
+    { name = "aiofiles" },
+    { name = "aiohttp" },
+    { name = "asyncssh" },
     { name = "mwparserfromhell" },
+    { name = "python-dotenv" },
     { name = "wikitextparser" },
 ]
 
 [package.metadata]
 requires-dist = [
+    { name = "aiofiles", specifier = ">=24.1.0" },
+    { name = "aiohttp", specifier = ">=3.11.16" },
+    { name = "asyncssh", specifier = ">=2.20.0" },
     { name = "mwparserfromhell", specifier = ">=0.6.6" },
+    { name = "python-dotenv", specifier = ">=1.1.0" },
     { name = "wikitextparser", specifier = ">=0.56.3" },
 ]
 
+[[package]]
+name = "multidict"
+version = "6.4.3"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/da/2c/e367dfb4c6538614a0c9453e510d75d66099edf1c4e69da1b5ce691a1931/multidict-6.4.3.tar.gz", hash = "sha256:3ada0b058c9f213c5f95ba301f922d402ac234f1111a7d8fd70f1b99f3c281ec", size = 89372 }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/fc/bb/3abdaf8fe40e9226ce8a2ba5ecf332461f7beec478a455d6587159f1bf92/multidict-6.4.3-cp312-cp312-macosx_10_13_universal2.whl", hash = "sha256:1f1c2f58f08b36f8475f3ec6f5aeb95270921d418bf18f90dffd6be5c7b0e676", size = 64019 },
+    { url = "https://files.pythonhosted.org/packages/7e/b5/1b2e8de8217d2e89db156625aa0fe4a6faad98972bfe07a7b8c10ef5dd6b/multidict-6.4.3-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:26ae9ad364fc61b936fb7bf4c9d8bd53f3a5b4417142cd0be5c509d6f767e2f1", size = 37925 },
+    { url = "https://files.pythonhosted.org/packages/b4/e2/3ca91c112644a395c8eae017144c907d173ea910c913ff8b62549dcf0bbf/multidict-6.4.3-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:659318c6c8a85f6ecfc06b4e57529e5a78dfdd697260cc81f683492ad7e9435a", size = 37008 },
+    { url = "https://files.pythonhosted.org/packages/60/23/79bc78146c7ac8d1ac766b2770ca2e07c2816058b8a3d5da6caed8148637/multidict-6.4.3-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:e1eb72c741fd24d5a28242ce72bb61bc91f8451877131fa3fe930edb195f7054", size = 224374 },
+    { url = "https://files.pythonhosted.org/packages/86/35/77950ed9ebd09136003a85c1926ba42001ca5be14feb49710e4334ee199b/multidict-6.4.3-cp312-cp312-manylinux_2_17_armv7l.manylinux2014_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:3cd06d88cb7398252284ee75c8db8e680aa0d321451132d0dba12bc995f0adcc", size = 230869 },
+    { url = "https://files.pythonhosted.org/packages/49/97/2a33c6e7d90bc116c636c14b2abab93d6521c0c052d24bfcc231cbf7f0e7/multidict-6.4.3-cp312-cp312-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:4543d8dc6470a82fde92b035a92529317191ce993533c3c0c68f56811164ed07", size = 231949 },
+    { url = "https://files.pythonhosted.org/packages/56/ce/e9b5d9fcf854f61d6686ada7ff64893a7a5523b2a07da6f1265eaaea5151/multidict-6.4.3-cp312-cp312-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:30a3ebdc068c27e9d6081fca0e2c33fdf132ecea703a72ea216b81a66860adde", size = 231032 },
+    { url = "https://files.pythonhosted.org/packages/f0/ac/7ced59dcdfeddd03e601edb05adff0c66d81ed4a5160c443e44f2379eef0/multidict-6.4.3-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:b038f10e23f277153f86f95c777ba1958bcd5993194fda26a1d06fae98b2f00c", size = 223517 },
+    { url = "https://files.pythonhosted.org/packages/db/e6/325ed9055ae4e085315193a1b58bdb4d7fc38ffcc1f4975cfca97d015e17/multidict-6.4.3-cp312-cp312-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:c605a2b2dc14282b580454b9b5d14ebe0668381a3a26d0ac39daa0ca115eb2ae", size = 216291 },
+    { url = "https://files.pythonhosted.org/packages/fa/84/eeee6d477dd9dcb7691c3bb9d08df56017f5dd15c730bcc9383dcf201cf4/multidict-6.4.3-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:8bd2b875f4ca2bb527fe23e318ddd509b7df163407b0fb717df229041c6df5d3", size = 228982 },
+    { url = "https://files.pythonhosted.org/packages/82/94/4d1f3e74e7acf8b0c85db350e012dcc61701cd6668bc2440bb1ecb423c90/multidict-6.4.3-cp312-cp312-musllinux_1_2_armv7l.whl", hash = "sha256:c2e98c840c9c8e65c0e04b40c6c5066c8632678cd50c8721fdbcd2e09f21a507", size = 226823 },
+    { url = "https://files.pythonhosted.org/packages/09/f0/1e54b95bda7cd01080e5732f9abb7b76ab5cc795b66605877caeb2197476/multidict-6.4.3-cp312-cp312-musllinux_1_2_i686.whl", hash = "sha256:66eb80dd0ab36dbd559635e62fba3083a48a252633164857a1d1684f14326427", size = 222714 },
+    { url = "https://files.pythonhosted.org/packages/e7/a2/f6cbca875195bd65a3e53b37ab46486f3cc125bdeab20eefe5042afa31fb/multidict-6.4.3-cp312-cp312-musllinux_1_2_ppc64le.whl", hash = "sha256:c23831bdee0a2a3cf21be057b5e5326292f60472fb6c6f86392bbf0de70ba731", size = 233739 },
+    { url = "https://files.pythonhosted.org/packages/79/68/9891f4d2b8569554723ddd6154375295f789dc65809826c6fb96a06314fd/multidict-6.4.3-cp312-cp312-musllinux_1_2_s390x.whl", hash = "sha256:1535cec6443bfd80d028052e9d17ba6ff8a5a3534c51d285ba56c18af97e9713", size = 230809 },
+    { url = "https://files.pythonhosted.org/packages/e6/72/a7be29ba1e87e4fc5ceb44dabc7940b8005fd2436a332a23547709315f70/multidict-6.4.3-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:3b73e7227681f85d19dec46e5b881827cd354aabe46049e1a61d2f9aaa4e285a", size = 226934 },
+    { url = "https://files.pythonhosted.org/packages/12/c1/259386a9ad6840ff7afc686da96808b503d152ac4feb3a96c651dc4f5abf/multidict-6.4.3-cp312-cp312-win32.whl", hash = "sha256:8eac0c49df91b88bf91f818e0a24c1c46f3622978e2c27035bfdca98e0e18124", size = 35242 },
+    { url = "https://files.pythonhosted.org/packages/06/24/c8fdff4f924d37225dc0c56a28b1dca10728fc2233065fafeb27b4b125be/multidict-6.4.3-cp312-cp312-win_amd64.whl", hash = "sha256:11990b5c757d956cd1db7cb140be50a63216af32cd6506329c2c59d732d802db", size = 38635 },
+    { url = "https://files.pythonhosted.org/packages/6c/4b/86fd786d03915c6f49998cf10cd5fe6b6ac9e9a071cb40885d2e080fb90d/multidict-6.4.3-cp313-cp313-macosx_10_13_universal2.whl", hash = "sha256:7a76534263d03ae0cfa721fea40fd2b5b9d17a6f85e98025931d41dc49504474", size = 63831 },
+    { url = "https://files.pythonhosted.org/packages/45/05/9b51fdf7aef2563340a93be0a663acba2c428c4daeaf3960d92d53a4a930/multidict-6.4.3-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:805031c2f599eee62ac579843555ed1ce389ae00c7e9f74c2a1b45e0564a88dd", size = 37888 },
+    { url = "https://files.pythonhosted.org/packages/0b/43/53fc25394386c911822419b522181227ca450cf57fea76e6188772a1bd91/multidict-6.4.3-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:c56c179839d5dcf51d565132185409d1d5dd8e614ba501eb79023a6cab25576b", size = 36852 },
+    { url = "https://files.pythonhosted.org/packages/8a/68/7b99c751e822467c94a235b810a2fd4047d4ecb91caef6b5c60116991c4b/multidict-6.4.3-cp313-cp313-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:9c64f4ddb3886dd8ab71b68a7431ad4aa01a8fa5be5b11543b29674f29ca0ba3", size = 223644 },
+    { url = "https://files.pythonhosted.org/packages/80/1b/d458d791e4dd0f7e92596667784fbf99e5c8ba040affe1ca04f06b93ae92/multidict-6.4.3-cp313-cp313-manylinux_2_17_armv7l.manylinux2014_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:3002a856367c0b41cad6784f5b8d3ab008eda194ed7864aaa58f65312e2abcac", size = 230446 },
+    { url = "https://files.pythonhosted.org/packages/e2/46/9793378d988905491a7806d8987862dc5a0bae8a622dd896c4008c7b226b/multidict-6.4.3-cp313-cp313-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:3d75e621e7d887d539d6e1d789f0c64271c250276c333480a9e1de089611f790", size = 231070 },
+    { url = "https://files.pythonhosted.org/packages/a7/b8/b127d3e1f8dd2a5bf286b47b24567ae6363017292dc6dec44656e6246498/multidict-6.4.3-cp313-cp313-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:995015cf4a3c0d72cbf453b10a999b92c5629eaf3a0c3e1efb4b5c1f602253bb", size = 229956 },
+    { url = "https://files.pythonhosted.org/packages/0c/93/f70a4c35b103fcfe1443059a2bb7f66e5c35f2aea7804105ff214f566009/multidict-6.4.3-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:a2b0fabae7939d09d7d16a711468c385272fa1b9b7fb0d37e51143585d8e72e0", size = 222599 },
+    { url = "https://files.pythonhosted.org/packages/63/8c/e28e0eb2fe34921d6aa32bfc4ac75b09570b4d6818cc95d25499fe08dc1d/multidict-6.4.3-cp313-cp313-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:61ed4d82f8a1e67eb9eb04f8587970d78fe7cddb4e4d6230b77eda23d27938f9", size = 216136 },
+    { url = "https://files.pythonhosted.org/packages/72/f5/fbc81f866585b05f89f99d108be5d6ad170e3b6c4d0723d1a2f6ba5fa918/multidict-6.4.3-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:062428944a8dc69df9fdc5d5fc6279421e5f9c75a9ee3f586f274ba7b05ab3c8", size = 228139 },
+    { url = "https://files.pythonhosted.org/packages/bb/ba/7d196bad6b85af2307d81f6979c36ed9665f49626f66d883d6c64d156f78/multidict-6.4.3-cp313-cp313-musllinux_1_2_armv7l.whl", hash = "sha256:b90e27b4674e6c405ad6c64e515a505c6d113b832df52fdacb6b1ffd1fa9a1d1", size = 226251 },
+    { url = "https://files.pythonhosted.org/packages/cc/e2/fae46a370dce79d08b672422a33df721ec8b80105e0ea8d87215ff6b090d/multidict-6.4.3-cp313-cp313-musllinux_1_2_i686.whl", hash = "sha256:7d50d4abf6729921e9613d98344b74241572b751c6b37feed75fb0c37bd5a817", size = 221868 },
+    { url = "https://files.pythonhosted.org/packages/26/20/bbc9a3dec19d5492f54a167f08546656e7aef75d181d3d82541463450e88/multidict-6.4.3-cp313-cp313-musllinux_1_2_ppc64le.whl", hash = "sha256:43fe10524fb0a0514be3954be53258e61d87341008ce4914f8e8b92bee6f875d", size = 233106 },
+    { url = "https://files.pythonhosted.org/packages/ee/8d/f30ae8f5ff7a2461177f4d8eb0d8f69f27fb6cfe276b54ec4fd5a282d918/multidict-6.4.3-cp313-cp313-musllinux_1_2_s390x.whl", hash = "sha256:236966ca6c472ea4e2d3f02f6673ebfd36ba3f23159c323f5a496869bc8e47c9", size = 230163 },
+    { url = "https://files.pythonhosted.org/packages/15/e9/2833f3c218d3c2179f3093f766940ded6b81a49d2e2f9c46ab240d23dfec/multidict-6.4.3-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:422a5ec315018e606473ba1f5431e064cf8b2a7468019233dcf8082fabad64c8", size = 225906 },
+    { url = "https://files.pythonhosted.org/packages/f1/31/6edab296ac369fd286b845fa5dd4c409e63bc4655ed8c9510fcb477e9ae9/multidict-6.4.3-cp313-cp313-win32.whl", hash = "sha256:f901a5aace8e8c25d78960dcc24c870c8d356660d3b49b93a78bf38eb682aac3", size = 35238 },
+    { url = "https://files.pythonhosted.org/packages/23/57/2c0167a1bffa30d9a1383c3dab99d8caae985defc8636934b5668830d2ef/multidict-6.4.3-cp313-cp313-win_amd64.whl", hash = "sha256:1c152c49e42277bc9a2f7b78bd5fa10b13e88d1b0328221e7aef89d5c60a99a5", size = 38799 },
+    { url = "https://files.pythonhosted.org/packages/c9/13/2ead63b9ab0d2b3080819268acb297bd66e238070aa8d42af12b08cbee1c/multidict-6.4.3-cp313-cp313t-macosx_10_13_universal2.whl", hash = "sha256:be8751869e28b9c0d368d94f5afcb4234db66fe8496144547b4b6d6a0645cfc6", size = 68642 },
+    { url = "https://files.pythonhosted.org/packages/85/45/f1a751e1eede30c23951e2ae274ce8fad738e8a3d5714be73e0a41b27b16/multidict-6.4.3-cp313-cp313t-macosx_10_13_x86_64.whl", hash = "sha256:0d4b31f8a68dccbcd2c0ea04f0e014f1defc6b78f0eb8b35f2265e8716a6df0c", size = 40028 },
+    { url = "https://files.pythonhosted.org/packages/a7/29/fcc53e886a2cc5595cc4560df333cb9630257bda65003a7eb4e4e0d8f9c1/multidict-6.4.3-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:032efeab3049e37eef2ff91271884303becc9e54d740b492a93b7e7266e23756", size = 39424 },
+    { url = "https://files.pythonhosted.org/packages/f6/f0/056c81119d8b88703971f937b371795cab1407cd3c751482de5bfe1a04a9/multidict-6.4.3-cp313-cp313t-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:9e78006af1a7c8a8007e4f56629d7252668344442f66982368ac06522445e375", size = 226178 },
+    { url = "https://files.pythonhosted.org/packages/a3/79/3b7e5fea0aa80583d3a69c9d98b7913dfd4fbc341fb10bb2fb48d35a9c21/multidict-6.4.3-cp313-cp313t-manylinux_2_17_armv7l.manylinux2014_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:daeac9dd30cda8703c417e4fddccd7c4dc0c73421a0b54a7da2713be125846be", size = 222617 },
+    { url = "https://files.pythonhosted.org/packages/06/db/3ed012b163e376fc461e1d6a67de69b408339bc31dc83d39ae9ec3bf9578/multidict-6.4.3-cp313-cp313t-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:1f6f90700881438953eae443a9c6f8a509808bc3b185246992c4233ccee37fea", size = 227919 },
+    { url = "https://files.pythonhosted.org/packages/b1/db/0433c104bca380989bc04d3b841fc83e95ce0c89f680e9ea4251118b52b6/multidict-6.4.3-cp313-cp313t-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:f84627997008390dd15762128dcf73c3365f4ec0106739cde6c20a07ed198ec8", size = 226097 },
+    { url = "https://files.pythonhosted.org/packages/c2/95/910db2618175724dd254b7ae635b6cd8d2947a8b76b0376de7b96d814dab/multidict-6.4.3-cp313-cp313t-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:3307b48cd156153b117c0ea54890a3bdbf858a5b296ddd40dc3852e5f16e9b02", size = 220706 },
+    { url = "https://files.pythonhosted.org/packages/d1/af/aa176c6f5f1d901aac957d5258d5e22897fe13948d1e69063ae3d5d0ca01/multidict-6.4.3-cp313-cp313t-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:ead46b0fa1dcf5af503a46e9f1c2e80b5d95c6011526352fa5f42ea201526124", size = 211728 },
+    { url = "https://files.pythonhosted.org/packages/e7/42/d51cc5fc1527c3717d7f85137d6c79bb7a93cd214c26f1fc57523774dbb5/multidict-6.4.3-cp313-cp313t-musllinux_1_2_aarch64.whl", hash = "sha256:1748cb2743bedc339d63eb1bca314061568793acd603a6e37b09a326334c9f44", size = 226276 },
+    { url = "https://files.pythonhosted.org/packages/28/6b/d836dea45e0b8432343ba4acf9a8ecaa245da4c0960fb7ab45088a5e568a/multidict-6.4.3-cp313-cp313t-musllinux_1_2_armv7l.whl", hash = "sha256:acc9fa606f76fc111b4569348cc23a771cb52c61516dcc6bcef46d612edb483b", size = 212069 },
+    { url = "https://files.pythonhosted.org/packages/55/34/0ee1a7adb3560e18ee9289c6e5f7db54edc312b13e5c8263e88ea373d12c/multidict-6.4.3-cp313-cp313t-musllinux_1_2_i686.whl", hash = "sha256:31469d5832b5885adeb70982e531ce86f8c992334edd2f2254a10fa3182ac504", size = 217858 },
+    { url = "https://files.pythonhosted.org/packages/04/08/586d652c2f5acefe0cf4e658eedb4d71d4ba6dfd4f189bd81b400fc1bc6b/multidict-6.4.3-cp313-cp313t-musllinux_1_2_ppc64le.whl", hash = "sha256:ba46b51b6e51b4ef7bfb84b82f5db0dc5e300fb222a8a13b8cd4111898a869cf", size = 226988 },
+    { url = "https://files.pythonhosted.org/packages/82/e3/cc59c7e2bc49d7f906fb4ffb6d9c3a3cf21b9f2dd9c96d05bef89c2b1fd1/multidict-6.4.3-cp313-cp313t-musllinux_1_2_s390x.whl", hash = "sha256:389cfefb599edf3fcfd5f64c0410da686f90f5f5e2c4d84e14f6797a5a337af4", size = 220435 },
+    { url = "https://files.pythonhosted.org/packages/e0/32/5c3a556118aca9981d883f38c4b1bfae646f3627157f70f4068e5a648955/multidict-6.4.3-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:64bc2bbc5fba7b9db5c2c8d750824f41c6994e3882e6d73c903c2afa78d091e4", size = 221494 },
+    { url = "https://files.pythonhosted.org/packages/b9/3b/1599631f59024b75c4d6e3069f4502409970a336647502aaf6b62fb7ac98/multidict-6.4.3-cp313-cp313t-win32.whl", hash = "sha256:0ecdc12ea44bab2807d6b4a7e5eef25109ab1c82a8240d86d3c1fc9f3b72efd5", size = 41775 },
+    { url = "https://files.pythonhosted.org/packages/e8/4e/09301668d675d02ca8e8e1a3e6be046619e30403f5ada2ed5b080ae28d02/multidict-6.4.3-cp313-cp313t-win_amd64.whl", hash = "sha256:7146a8742ea71b5d7d955bffcef58a9e6e04efba704b52a460134fefd10a8208", size = 45946 },
+    { url = "https://files.pythonhosted.org/packages/96/10/7d526c8974f017f1e7ca584c71ee62a638e9334d8d33f27d7cdfc9ae79e4/multidict-6.4.3-py3-none-any.whl", hash = "sha256:59fe01ee8e2a1e8ceb3f6dbb216b09c8d9f4ef1c22c4fc825d045a147fa2ebc9", size = 10400 },
+]
+
 [[package]]
 name = "mwparserfromhell"
 version = "0.6.6"
@@ -29,6 +335,81 @@ wheels = [
     { url = "https://files.pythonhosted.org/packages/78/09/24c2f37524a3ebc3574975766748c7e4423ecefaa815c9fc4a324cbcf94a/mwparserfromhell-0.6.6-cp312-cp312-win_amd64.whl", hash = "sha256:cdc46c115b2495d4025920b7b30a6885a96d2b797ccc4009bf3cc02940ae55d3", size = 101071 },
 ]
 
+[[package]]
+name = "propcache"
+version = "0.3.1"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/07/c8/fdc6686a986feae3541ea23dcaa661bd93972d3940460646c6bb96e21c40/propcache-0.3.1.tar.gz", hash = "sha256:40d980c33765359098837527e18eddefc9a24cea5b45e078a7f3bb5b032c6ecf", size = 43651 }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/41/aa/ca78d9be314d1e15ff517b992bebbed3bdfef5b8919e85bf4940e57b6137/propcache-0.3.1-cp312-cp312-macosx_10_13_universal2.whl", hash = "sha256:f78eb8422acc93d7b69964012ad7048764bb45a54ba7a39bb9e146c72ea29723", size = 80430 },
+    { url = "https://files.pythonhosted.org/packages/1a/d8/f0c17c44d1cda0ad1979af2e593ea290defdde9eaeb89b08abbe02a5e8e1/propcache-0.3.1-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:89498dd49c2f9a026ee057965cdf8192e5ae070ce7d7a7bd4b66a8e257d0c976", size = 46637 },
+    { url = "https://files.pythonhosted.org/packages/ae/bd/c1e37265910752e6e5e8a4c1605d0129e5b7933c3dc3cf1b9b48ed83b364/propcache-0.3.1-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:09400e98545c998d57d10035ff623266927cb784d13dd2b31fd33b8a5316b85b", size = 46123 },
+    { url = "https://files.pythonhosted.org/packages/d4/b0/911eda0865f90c0c7e9f0415d40a5bf681204da5fd7ca089361a64c16b28/propcache-0.3.1-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:aa8efd8c5adc5a2c9d3b952815ff8f7710cefdcaf5f2c36d26aff51aeca2f12f", size = 243031 },
+    { url = "https://files.pythonhosted.org/packages/0a/06/0da53397c76a74271621807265b6eb61fb011451b1ddebf43213df763669/propcache-0.3.1-cp312-cp312-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:c2fe5c910f6007e716a06d269608d307b4f36e7babee5f36533722660e8c4a70", size = 249100 },
+    { url = "https://files.pythonhosted.org/packages/f1/eb/13090e05bf6b963fc1653cdc922133ced467cb4b8dab53158db5a37aa21e/propcache-0.3.1-cp312-cp312-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:a0ab8cf8cdd2194f8ff979a43ab43049b1df0b37aa64ab7eca04ac14429baeb7", size = 250170 },
+    { url = "https://files.pythonhosted.org/packages/3b/4c/f72c9e1022b3b043ec7dc475a0f405d4c3e10b9b1d378a7330fecf0652da/propcache-0.3.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:563f9d8c03ad645597b8d010ef4e9eab359faeb11a0a2ac9f7b4bc8c28ebef25", size = 245000 },
+    { url = "https://files.pythonhosted.org/packages/e8/fd/970ca0e22acc829f1adf5de3724085e778c1ad8a75bec010049502cb3a86/propcache-0.3.1-cp312-cp312-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:fb6e0faf8cb6b4beea5d6ed7b5a578254c6d7df54c36ccd3d8b3eb00d6770277", size = 230262 },
+    { url = "https://files.pythonhosted.org/packages/c4/42/817289120c6b9194a44f6c3e6b2c3277c5b70bbad39e7df648f177cc3634/propcache-0.3.1-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:1c5c7ab7f2bb3f573d1cb921993006ba2d39e8621019dffb1c5bc94cdbae81e8", size = 236772 },
+    { url = "https://files.pythonhosted.org/packages/7c/9c/3b3942b302badd589ad6b672da3ca7b660a6c2f505cafd058133ddc73918/propcache-0.3.1-cp312-cp312-musllinux_1_2_armv7l.whl", hash = "sha256:050b571b2e96ec942898f8eb46ea4bfbb19bd5502424747e83badc2d4a99a44e", size = 231133 },
+    { url = "https://files.pythonhosted.org/packages/98/a1/75f6355f9ad039108ff000dfc2e19962c8dea0430da9a1428e7975cf24b2/propcache-0.3.1-cp312-cp312-musllinux_1_2_i686.whl", hash = "sha256:e1c4d24b804b3a87e9350f79e2371a705a188d292fd310e663483af6ee6718ee", size = 230741 },
+    { url = "https://files.pythonhosted.org/packages/67/0c/3e82563af77d1f8731132166da69fdfd95e71210e31f18edce08a1eb11ea/propcache-0.3.1-cp312-cp312-musllinux_1_2_ppc64le.whl", hash = "sha256:e4fe2a6d5ce975c117a6bb1e8ccda772d1e7029c1cca1acd209f91d30fa72815", size = 244047 },
+    { url = "https://files.pythonhosted.org/packages/f7/50/9fb7cca01532a08c4d5186d7bb2da6c4c587825c0ae134b89b47c7d62628/propcache-0.3.1-cp312-cp312-musllinux_1_2_s390x.whl", hash = "sha256:feccd282de1f6322f56f6845bf1207a537227812f0a9bf5571df52bb418d79d5", size = 246467 },
+    { url = "https://files.pythonhosted.org/packages/a9/02/ccbcf3e1c604c16cc525309161d57412c23cf2351523aedbb280eb7c9094/propcache-0.3.1-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:ec314cde7314d2dd0510c6787326bbffcbdc317ecee6b7401ce218b3099075a7", size = 241022 },
+    { url = "https://files.pythonhosted.org/packages/db/19/e777227545e09ca1e77a6e21274ae9ec45de0f589f0ce3eca2a41f366220/propcache-0.3.1-cp312-cp312-win32.whl", hash = "sha256:7d2d5a0028d920738372630870e7d9644ce437142197f8c827194fca404bf03b", size = 40647 },
+    { url = "https://files.pythonhosted.org/packages/24/bb/3b1b01da5dd04c77a204c84e538ff11f624e31431cfde7201d9110b092b1/propcache-0.3.1-cp312-cp312-win_amd64.whl", hash = "sha256:88c423efef9d7a59dae0614eaed718449c09a5ac79a5f224a8b9664d603f04a3", size = 44784 },
+    { url = "https://files.pythonhosted.org/packages/58/60/f645cc8b570f99be3cf46714170c2de4b4c9d6b827b912811eff1eb8a412/propcache-0.3.1-cp313-cp313-macosx_10_13_universal2.whl", hash = "sha256:f1528ec4374617a7a753f90f20e2f551121bb558fcb35926f99e3c42367164b8", size = 77865 },
+    { url = "https://files.pythonhosted.org/packages/6f/d4/c1adbf3901537582e65cf90fd9c26fde1298fde5a2c593f987112c0d0798/propcache-0.3.1-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:dc1915ec523b3b494933b5424980831b636fe483d7d543f7afb7b3bf00f0c10f", size = 45452 },
+    { url = "https://files.pythonhosted.org/packages/d1/b5/fe752b2e63f49f727c6c1c224175d21b7d1727ce1d4873ef1c24c9216830/propcache-0.3.1-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:a110205022d077da24e60b3df8bcee73971be9575dec5573dd17ae5d81751111", size = 44800 },
+    { url = "https://files.pythonhosted.org/packages/62/37/fc357e345bc1971e21f76597028b059c3d795c5ca7690d7a8d9a03c9708a/propcache-0.3.1-cp313-cp313-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:d249609e547c04d190e820d0d4c8ca03ed4582bcf8e4e160a6969ddfb57b62e5", size = 225804 },
+    { url = "https://files.pythonhosted.org/packages/0d/f1/16e12c33e3dbe7f8b737809bad05719cff1dccb8df4dafbcff5575002c0e/propcache-0.3.1-cp313-cp313-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:5ced33d827625d0a589e831126ccb4f5c29dfdf6766cac441d23995a65825dcb", size = 230650 },
+    { url = "https://files.pythonhosted.org/packages/3e/a2/018b9f2ed876bf5091e60153f727e8f9073d97573f790ff7cdf6bc1d1fb8/propcache-0.3.1-cp313-cp313-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:4114c4ada8f3181af20808bedb250da6bae56660e4b8dfd9cd95d4549c0962f7", size = 234235 },
+    { url = "https://files.pythonhosted.org/packages/45/5f/3faee66fc930dfb5da509e34c6ac7128870631c0e3582987fad161fcb4b1/propcache-0.3.1-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:975af16f406ce48f1333ec5e912fe11064605d5c5b3f6746969077cc3adeb120", size = 228249 },
+    { url = "https://files.pythonhosted.org/packages/62/1e/a0d5ebda5da7ff34d2f5259a3e171a94be83c41eb1e7cd21a2105a84a02e/propcache-0.3.1-cp313-cp313-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:a34aa3a1abc50740be6ac0ab9d594e274f59960d3ad253cd318af76b996dd654", size = 214964 },
+    { url = "https://files.pythonhosted.org/packages/db/a0/d72da3f61ceab126e9be1f3bc7844b4e98c6e61c985097474668e7e52152/propcache-0.3.1-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:9cec3239c85ed15bfaded997773fdad9fb5662b0a7cbc854a43f291eb183179e", size = 222501 },
+    { url = "https://files.pythonhosted.org/packages/18/6d/a008e07ad7b905011253adbbd97e5b5375c33f0b961355ca0a30377504ac/propcache-0.3.1-cp313-cp313-musllinux_1_2_armv7l.whl", hash = "sha256:05543250deac8e61084234d5fc54f8ebd254e8f2b39a16b1dce48904f45b744b", size = 217917 },
+    { url = "https://files.pythonhosted.org/packages/98/37/02c9343ffe59e590e0e56dc5c97d0da2b8b19fa747ebacf158310f97a79a/propcache-0.3.1-cp313-cp313-musllinux_1_2_i686.whl", hash = "sha256:5cb5918253912e088edbf023788de539219718d3b10aef334476b62d2b53de53", size = 217089 },
+    { url = "https://files.pythonhosted.org/packages/53/1b/d3406629a2c8a5666d4674c50f757a77be119b113eedd47b0375afdf1b42/propcache-0.3.1-cp313-cp313-musllinux_1_2_ppc64le.whl", hash = "sha256:f3bbecd2f34d0e6d3c543fdb3b15d6b60dd69970c2b4c822379e5ec8f6f621d5", size = 228102 },
+    { url = "https://files.pythonhosted.org/packages/cd/a7/3664756cf50ce739e5f3abd48febc0be1a713b1f389a502ca819791a6b69/propcache-0.3.1-cp313-cp313-musllinux_1_2_s390x.whl", hash = "sha256:aca63103895c7d960a5b9b044a83f544b233c95e0dcff114389d64d762017af7", size = 230122 },
+    { url = "https://files.pythonhosted.org/packages/35/36/0bbabaacdcc26dac4f8139625e930f4311864251276033a52fd52ff2a274/propcache-0.3.1-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:5a0a9898fdb99bf11786265468571e628ba60af80dc3f6eb89a3545540c6b0ef", size = 226818 },
+    { url = "https://files.pythonhosted.org/packages/cc/27/4e0ef21084b53bd35d4dae1634b6d0bad35e9c58ed4f032511acca9d4d26/propcache-0.3.1-cp313-cp313-win32.whl", hash = "sha256:3a02a28095b5e63128bcae98eb59025924f121f048a62393db682f049bf4ac24", size = 40112 },
+    { url = "https://files.pythonhosted.org/packages/a6/2c/a54614d61895ba6dd7ac8f107e2b2a0347259ab29cbf2ecc7b94fa38c4dc/propcache-0.3.1-cp313-cp313-win_amd64.whl", hash = "sha256:813fbb8b6aea2fc9659815e585e548fe706d6f663fa73dff59a1677d4595a037", size = 44034 },
+    { url = "https://files.pythonhosted.org/packages/5a/a8/0a4fd2f664fc6acc66438370905124ce62e84e2e860f2557015ee4a61c7e/propcache-0.3.1-cp313-cp313t-macosx_10_13_universal2.whl", hash = "sha256:a444192f20f5ce8a5e52761a031b90f5ea6288b1eef42ad4c7e64fef33540b8f", size = 82613 },
+    { url = "https://files.pythonhosted.org/packages/4d/e5/5ef30eb2cd81576256d7b6caaa0ce33cd1d2c2c92c8903cccb1af1a4ff2f/propcache-0.3.1-cp313-cp313t-macosx_10_13_x86_64.whl", hash = "sha256:0fbe94666e62ebe36cd652f5fc012abfbc2342de99b523f8267a678e4dfdee3c", size = 47763 },
+    { url = "https://files.pythonhosted.org/packages/87/9a/87091ceb048efeba4d28e903c0b15bcc84b7c0bf27dc0261e62335d9b7b8/propcache-0.3.1-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:f011f104db880f4e2166bcdcf7f58250f7a465bc6b068dc84c824a3d4a5c94dc", size = 47175 },
+    { url = "https://files.pythonhosted.org/packages/3e/2f/854e653c96ad1161f96194c6678a41bbb38c7947d17768e8811a77635a08/propcache-0.3.1-cp313-cp313t-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:3e584b6d388aeb0001d6d5c2bd86b26304adde6d9bb9bfa9c4889805021b96de", size = 292265 },
+    { url = "https://files.pythonhosted.org/packages/40/8d/090955e13ed06bc3496ba4a9fb26c62e209ac41973cb0d6222de20c6868f/propcache-0.3.1-cp313-cp313t-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:8a17583515a04358b034e241f952f1715243482fc2c2945fd99a1b03a0bd77d6", size = 294412 },
+    { url = "https://files.pythonhosted.org/packages/39/e6/d51601342e53cc7582449e6a3c14a0479fab2f0750c1f4d22302e34219c6/propcache-0.3.1-cp313-cp313t-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:5aed8d8308215089c0734a2af4f2e95eeb360660184ad3912686c181e500b2e7", size = 294290 },
+    { url = "https://files.pythonhosted.org/packages/3b/4d/be5f1a90abc1881884aa5878989a1acdafd379a91d9c7e5e12cef37ec0d7/propcache-0.3.1-cp313-cp313t-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:6d8e309ff9a0503ef70dc9a0ebd3e69cf7b3894c9ae2ae81fc10943c37762458", size = 282926 },
+    { url = "https://files.pythonhosted.org/packages/57/2b/8f61b998c7ea93a2b7eca79e53f3e903db1787fca9373af9e2cf8dc22f9d/propcache-0.3.1-cp313-cp313t-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:b655032b202028a582d27aeedc2e813299f82cb232f969f87a4fde491a233f11", size = 267808 },
+    { url = "https://files.pythonhosted.org/packages/11/1c/311326c3dfce59c58a6098388ba984b0e5fb0381ef2279ec458ef99bd547/propcache-0.3.1-cp313-cp313t-musllinux_1_2_aarch64.whl", hash = "sha256:9f64d91b751df77931336b5ff7bafbe8845c5770b06630e27acd5dbb71e1931c", size = 290916 },
+    { url = "https://files.pythonhosted.org/packages/4b/74/91939924b0385e54dc48eb2e4edd1e4903ffd053cf1916ebc5347ac227f7/propcache-0.3.1-cp313-cp313t-musllinux_1_2_armv7l.whl", hash = "sha256:19a06db789a4bd896ee91ebc50d059e23b3639c25d58eb35be3ca1cbe967c3bf", size = 262661 },
+    { url = "https://files.pythonhosted.org/packages/c2/d7/e6079af45136ad325c5337f5dd9ef97ab5dc349e0ff362fe5c5db95e2454/propcache-0.3.1-cp313-cp313t-musllinux_1_2_i686.whl", hash = "sha256:bef100c88d8692864651b5f98e871fb090bd65c8a41a1cb0ff2322db39c96c27", size = 264384 },
+    { url = "https://files.pythonhosted.org/packages/b7/d5/ba91702207ac61ae6f1c2da81c5d0d6bf6ce89e08a2b4d44e411c0bbe867/propcache-0.3.1-cp313-cp313t-musllinux_1_2_ppc64le.whl", hash = "sha256:87380fb1f3089d2a0b8b00f006ed12bd41bd858fabfa7330c954c70f50ed8757", size = 291420 },
+    { url = "https://files.pythonhosted.org/packages/58/70/2117780ed7edcd7ba6b8134cb7802aada90b894a9810ec56b7bb6018bee7/propcache-0.3.1-cp313-cp313t-musllinux_1_2_s390x.whl", hash = "sha256:e474fc718e73ba5ec5180358aa07f6aded0ff5f2abe700e3115c37d75c947e18", size = 290880 },
+    { url = "https://files.pythonhosted.org/packages/4a/1f/ecd9ce27710021ae623631c0146719280a929d895a095f6d85efb6a0be2e/propcache-0.3.1-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:17d1c688a443355234f3c031349da69444be052613483f3e4158eef751abcd8a", size = 287407 },
+    { url = "https://files.pythonhosted.org/packages/3e/66/2e90547d6b60180fb29e23dc87bd8c116517d4255240ec6d3f7dc23d1926/propcache-0.3.1-cp313-cp313t-win32.whl", hash = "sha256:359e81a949a7619802eb601d66d37072b79b79c2505e6d3fd8b945538411400d", size = 42573 },
+    { url = "https://files.pythonhosted.org/packages/cb/8f/50ad8599399d1861b4d2b6b45271f0ef6af1b09b0a2386a46dbaf19c9535/propcache-0.3.1-cp313-cp313t-win_amd64.whl", hash = "sha256:e7fb9a84c9abbf2b2683fa3e7b0d7da4d8ecf139a1c635732a8bda29c5214b0e", size = 46757 },
+    { url = "https://files.pythonhosted.org/packages/b8/d3/c3cb8f1d6ae3b37f83e1de806713a9b3642c5895f0215a62e1a4bd6e5e34/propcache-0.3.1-py3-none-any.whl", hash = "sha256:9a8ecf38de50a7f518c21568c80f985e776397b902f1ce0b01f799aba1608b40", size = 12376 },
+]
+
+[[package]]
+name = "pycparser"
+version = "2.22"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/1d/b2/31537cf4b1ca988837256c910a668b553fceb8f069bedc4b1c826024b52c/pycparser-2.22.tar.gz", hash = "sha256:491c8be9c040f5390f5bf44a5b07752bd07f56edf992381b05c701439eec10f6", size = 172736 }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/13/a3/a812df4e2dd5696d1f351d58b8fe16a405b234ad2886a0dab9183fb78109/pycparser-2.22-py3-none-any.whl", hash = "sha256:c3702b6d3dd8c7abc1afa565d7e63d53a1d0bd86cdc24edd75470f4de499cfcc", size = 117552 },
+]
+
+[[package]]
+name = "python-dotenv"
+version = "1.1.0"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/88/2c/7bb1416c5620485aa793f2de31d3df393d3686aa8a8506d11e10e13c5baf/python_dotenv-1.1.0.tar.gz", hash = "sha256:41f90bc6f5f177fb41f53e87666db362025010eb28f60a01c9143bfa33a2b2d5", size = 39920 }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/1e/18/98a99ad95133c6a6e2005fe89faedf294a748bd5dc803008059409ac9b1e/python_dotenv-1.1.0-py3-none-any.whl", hash = "sha256:d7c01d9e2293916c18baf562d95698754b0dbbb5e74d457c45d4f6561fb9d55d", size = 20256 },
+]
+
 [[package]]
 name = "regex"
 version = "2024.11.6"
@@ -67,6 +448,15 @@ wheels = [
     { url = "https://files.pythonhosted.org/packages/45/94/bc295babb3062a731f52621cdc992d123111282e291abaf23faa413443ea/regex-2024.11.6-cp313-cp313-win_amd64.whl", hash = "sha256:2b3361af3198667e99927da8b84c1b010752fa4b1115ee30beaa332cabc3ef1a", size = 273545 },
 ]
 
+[[package]]
+name = "typing-extensions"
+version = "4.13.2"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/f6/37/23083fcd6e35492953e8d2aaaa68b860eb422b34627b13f2ce3eb6106061/typing_extensions-4.13.2.tar.gz", hash = "sha256:e6c81219bd689f51865d9e372991c540bda33a0379d5573cddb9a3a23f7caaef", size = 106967 }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/8b/54/b1ae86c0973cc6f0210b53d508ca3641fb6d0c56823f288d108bc7ab3cc8/typing_extensions-4.13.2-py3-none-any.whl", hash = "sha256:a439e7c04b49fec3e5d3e2beaa21755cadbbdc391694e28ccdd36ca4a1408f8c", size = 45806 },
+]
+
 [[package]]
 name = "wcwidth"
 version = "0.2.13"
@@ -88,3 +478,68 @@ sdist = { url = "https://files.pythonhosted.org/packages/68/8f/38ae3bb4d5b87a30f
 wheels = [
     { url = "https://files.pythonhosted.org/packages/81/5f/3109173deefaaf4a4d8f4086b20367a42fd2f77d6a096db04e835aa5dfe2/wikitextparser-0.56.3-py3-none-any.whl", hash = "sha256:49bcbe421f0c126fba254a8f2e41262e679a2a88f2010dda90198a287616b5e4", size = 66284 },
 ]
+
+[[package]]
+name = "yarl"
+version = "1.20.0"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+    { name = "idna" },
+    { name = "multidict" },
+    { name = "propcache" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/62/51/c0edba5219027f6eab262e139f73e2417b0f4efffa23bf562f6e18f76ca5/yarl-1.20.0.tar.gz", hash = "sha256:686d51e51ee5dfe62dec86e4866ee0e9ed66df700d55c828a615640adc885307", size = 185258 }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/c3/e8/3efdcb83073df978bb5b1a9cc0360ce596680e6c3fac01f2a994ccbb8939/yarl-1.20.0-cp312-cp312-macosx_10_13_universal2.whl", hash = "sha256:e06b9f6cdd772f9b665e5ba8161968e11e403774114420737f7884b5bd7bdf6f", size = 147089 },
+    { url = "https://files.pythonhosted.org/packages/60/c3/9e776e98ea350f76f94dd80b408eaa54e5092643dbf65fd9babcffb60509/yarl-1.20.0-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:b9ae2fbe54d859b3ade40290f60fe40e7f969d83d482e84d2c31b9bff03e359e", size = 97706 },
+    { url = "https://files.pythonhosted.org/packages/0c/5b/45cdfb64a3b855ce074ae607b9fc40bc82e7613b94e7612b030255c93a09/yarl-1.20.0-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:6d12b8945250d80c67688602c891237994d203d42427cb14e36d1a732eda480e", size = 95719 },
+    { url = "https://files.pythonhosted.org/packages/2d/4e/929633b249611eeed04e2f861a14ed001acca3ef9ec2a984a757b1515889/yarl-1.20.0-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:087e9731884621b162a3e06dc0d2d626e1542a617f65ba7cc7aeab279d55ad33", size = 343972 },
+    { url = "https://files.pythonhosted.org/packages/49/fd/047535d326c913f1a90407a3baf7ff535b10098611eaef2c527e32e81ca1/yarl-1.20.0-cp312-cp312-manylinux_2_17_armv7l.manylinux2014_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:69df35468b66c1a6e6556248e6443ef0ec5f11a7a4428cf1f6281f1879220f58", size = 339639 },
+    { url = "https://files.pythonhosted.org/packages/48/2f/11566f1176a78f4bafb0937c0072410b1b0d3640b297944a6a7a556e1d0b/yarl-1.20.0-cp312-cp312-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:3b2992fe29002fd0d4cbaea9428b09af9b8686a9024c840b8a2b8f4ea4abc16f", size = 353745 },
+    { url = "https://files.pythonhosted.org/packages/26/17/07dfcf034d6ae8837b33988be66045dd52f878dfb1c4e8f80a7343f677be/yarl-1.20.0-cp312-cp312-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:4c903e0b42aab48abfbac668b5a9d7b6938e721a6341751331bcd7553de2dcae", size = 354178 },
+    { url = "https://files.pythonhosted.org/packages/15/45/212604d3142d84b4065d5f8cab6582ed3d78e4cc250568ef2a36fe1cf0a5/yarl-1.20.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:bf099e2432131093cc611623e0b0bcc399b8cddd9a91eded8bfb50402ec35018", size = 349219 },
+    { url = "https://files.pythonhosted.org/packages/e6/e0/a10b30f294111c5f1c682461e9459935c17d467a760c21e1f7db400ff499/yarl-1.20.0-cp312-cp312-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:8a7f62f5dc70a6c763bec9ebf922be52aa22863d9496a9a30124d65b489ea672", size = 337266 },
+    { url = "https://files.pythonhosted.org/packages/33/a6/6efa1d85a675d25a46a167f9f3e80104cde317dfdf7f53f112ae6b16a60a/yarl-1.20.0-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:54ac15a8b60382b2bcefd9a289ee26dc0920cf59b05368c9b2b72450751c6eb8", size = 360873 },
+    { url = "https://files.pythonhosted.org/packages/77/67/c8ab718cb98dfa2ae9ba0f97bf3cbb7d45d37f13fe1fbad25ac92940954e/yarl-1.20.0-cp312-cp312-musllinux_1_2_armv7l.whl", hash = "sha256:25b3bc0763a7aca16a0f1b5e8ef0f23829df11fb539a1b70476dcab28bd83da7", size = 360524 },
+    { url = "https://files.pythonhosted.org/packages/bd/e8/c3f18660cea1bc73d9f8a2b3ef423def8dadbbae6c4afabdb920b73e0ead/yarl-1.20.0-cp312-cp312-musllinux_1_2_i686.whl", hash = "sha256:b2586e36dc070fc8fad6270f93242124df68b379c3a251af534030a4a33ef594", size = 365370 },
+    { url = "https://files.pythonhosted.org/packages/c9/99/33f3b97b065e62ff2d52817155a89cfa030a1a9b43fee7843ef560ad9603/yarl-1.20.0-cp312-cp312-musllinux_1_2_ppc64le.whl", hash = "sha256:866349da9d8c5290cfefb7fcc47721e94de3f315433613e01b435473be63daa6", size = 373297 },
+    { url = "https://files.pythonhosted.org/packages/3d/89/7519e79e264a5f08653d2446b26d4724b01198a93a74d2e259291d538ab1/yarl-1.20.0-cp312-cp312-musllinux_1_2_s390x.whl", hash = "sha256:33bb660b390a0554d41f8ebec5cd4475502d84104b27e9b42f5321c5192bfcd1", size = 378771 },
+    { url = "https://files.pythonhosted.org/packages/3a/58/6c460bbb884abd2917c3eef6f663a4a873f8dc6f498561fc0ad92231c113/yarl-1.20.0-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:737e9f171e5a07031cbee5e9180f6ce21a6c599b9d4b2c24d35df20a52fabf4b", size = 375000 },
+    { url = "https://files.pythonhosted.org/packages/3b/2a/dd7ed1aa23fea996834278d7ff178f215b24324ee527df53d45e34d21d28/yarl-1.20.0-cp312-cp312-win32.whl", hash = "sha256:839de4c574169b6598d47ad61534e6981979ca2c820ccb77bf70f4311dd2cc64", size = 86355 },
+    { url = "https://files.pythonhosted.org/packages/ca/c6/333fe0338305c0ac1c16d5aa7cc4841208d3252bbe62172e0051006b5445/yarl-1.20.0-cp312-cp312-win_amd64.whl", hash = "sha256:3d7dbbe44b443b0c4aa0971cb07dcb2c2060e4a9bf8d1301140a33a93c98e18c", size = 92904 },
+    { url = "https://files.pythonhosted.org/packages/0f/6f/514c9bff2900c22a4f10e06297714dbaf98707143b37ff0bcba65a956221/yarl-1.20.0-cp313-cp313-macosx_10_13_universal2.whl", hash = "sha256:2137810a20b933b1b1b7e5cf06a64c3ed3b4747b0e5d79c9447c00db0e2f752f", size = 145030 },
+    { url = "https://files.pythonhosted.org/packages/4e/9d/f88da3fa319b8c9c813389bfb3463e8d777c62654c7168e580a13fadff05/yarl-1.20.0-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:447c5eadd750db8389804030d15f43d30435ed47af1313303ed82a62388176d3", size = 96894 },
+    { url = "https://files.pythonhosted.org/packages/cd/57/92e83538580a6968b2451d6c89c5579938a7309d4785748e8ad42ddafdce/yarl-1.20.0-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:42fbe577272c203528d402eec8bf4b2d14fd49ecfec92272334270b850e9cd7d", size = 94457 },
+    { url = "https://files.pythonhosted.org/packages/e9/ee/7ee43bd4cf82dddd5da97fcaddb6fa541ab81f3ed564c42f146c83ae17ce/yarl-1.20.0-cp313-cp313-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:18e321617de4ab170226cd15006a565d0fa0d908f11f724a2c9142d6b2812ab0", size = 343070 },
+    { url = "https://files.pythonhosted.org/packages/4a/12/b5eccd1109e2097bcc494ba7dc5de156e41cf8309fab437ebb7c2b296ce3/yarl-1.20.0-cp313-cp313-manylinux_2_17_armv7l.manylinux2014_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:4345f58719825bba29895011e8e3b545e6e00257abb984f9f27fe923afca2501", size = 337739 },
+    { url = "https://files.pythonhosted.org/packages/7d/6b/0eade8e49af9fc2585552f63c76fa59ef469c724cc05b29519b19aa3a6d5/yarl-1.20.0-cp313-cp313-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:5d9b980d7234614bc4674468ab173ed77d678349c860c3af83b1fffb6a837ddc", size = 351338 },
+    { url = "https://files.pythonhosted.org/packages/45/cb/aaaa75d30087b5183c7b8a07b4fb16ae0682dd149a1719b3a28f54061754/yarl-1.20.0-cp313-cp313-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:af4baa8a445977831cbaa91a9a84cc09debb10bc8391f128da2f7bd070fc351d", size = 353636 },
+    { url = "https://files.pythonhosted.org/packages/98/9d/d9cb39ec68a91ba6e66fa86d97003f58570327d6713833edf7ad6ce9dde5/yarl-1.20.0-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:123393db7420e71d6ce40d24885a9e65eb1edefc7a5228db2d62bcab3386a5c0", size = 348061 },
+    { url = "https://files.pythonhosted.org/packages/72/6b/103940aae893d0cc770b4c36ce80e2ed86fcb863d48ea80a752b8bda9303/yarl-1.20.0-cp313-cp313-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:ab47acc9332f3de1b39e9b702d9c916af7f02656b2a86a474d9db4e53ef8fd7a", size = 334150 },
+    { url = "https://files.pythonhosted.org/packages/ef/b2/986bd82aa222c3e6b211a69c9081ba46484cffa9fab2a5235e8d18ca7a27/yarl-1.20.0-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:4a34c52ed158f89876cba9c600b2c964dfc1ca52ba7b3ab6deb722d1d8be6df2", size = 362207 },
+    { url = "https://files.pythonhosted.org/packages/14/7c/63f5922437b873795d9422cbe7eb2509d4b540c37ae5548a4bb68fd2c546/yarl-1.20.0-cp313-cp313-musllinux_1_2_armv7l.whl", hash = "sha256:04d8cfb12714158abf2618f792c77bc5c3d8c5f37353e79509608be4f18705c9", size = 361277 },
+    { url = "https://files.pythonhosted.org/packages/81/83/450938cccf732466953406570bdb42c62b5ffb0ac7ac75a1f267773ab5c8/yarl-1.20.0-cp313-cp313-musllinux_1_2_i686.whl", hash = "sha256:7dc63ad0d541c38b6ae2255aaa794434293964677d5c1ec5d0116b0e308031f5", size = 364990 },
+    { url = "https://files.pythonhosted.org/packages/b4/de/af47d3a47e4a833693b9ec8e87debb20f09d9fdc9139b207b09a3e6cbd5a/yarl-1.20.0-cp313-cp313-musllinux_1_2_ppc64le.whl", hash = "sha256:f9d02b591a64e4e6ca18c5e3d925f11b559c763b950184a64cf47d74d7e41877", size = 374684 },
+    { url = "https://files.pythonhosted.org/packages/62/0b/078bcc2d539f1faffdc7d32cb29a2d7caa65f1a6f7e40795d8485db21851/yarl-1.20.0-cp313-cp313-musllinux_1_2_s390x.whl", hash = "sha256:95fc9876f917cac7f757df80a5dda9de59d423568460fe75d128c813b9af558e", size = 382599 },
+    { url = "https://files.pythonhosted.org/packages/74/a9/4fdb1a7899f1fb47fd1371e7ba9e94bff73439ce87099d5dd26d285fffe0/yarl-1.20.0-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:bb769ae5760cd1c6a712135ee7915f9d43f11d9ef769cb3f75a23e398a92d384", size = 378573 },
+    { url = "https://files.pythonhosted.org/packages/fd/be/29f5156b7a319e4d2e5b51ce622b4dfb3aa8d8204cd2a8a339340fbfad40/yarl-1.20.0-cp313-cp313-win32.whl", hash = "sha256:70e0c580a0292c7414a1cead1e076c9786f685c1fc4757573d2967689b370e62", size = 86051 },
+    { url = "https://files.pythonhosted.org/packages/52/56/05fa52c32c301da77ec0b5f63d2d9605946fe29defacb2a7ebd473c23b81/yarl-1.20.0-cp313-cp313-win_amd64.whl", hash = "sha256:4c43030e4b0af775a85be1fa0433119b1565673266a70bf87ef68a9d5ba3174c", size = 92742 },
+    { url = "https://files.pythonhosted.org/packages/d4/2f/422546794196519152fc2e2f475f0e1d4d094a11995c81a465faf5673ffd/yarl-1.20.0-cp313-cp313t-macosx_10_13_universal2.whl", hash = "sha256:b6c4c3d0d6a0ae9b281e492b1465c72de433b782e6b5001c8e7249e085b69051", size = 163575 },
+    { url = "https://files.pythonhosted.org/packages/90/fc/67c64ddab6c0b4a169d03c637fb2d2a212b536e1989dec8e7e2c92211b7f/yarl-1.20.0-cp313-cp313t-macosx_10_13_x86_64.whl", hash = "sha256:8681700f4e4df891eafa4f69a439a6e7d480d64e52bf460918f58e443bd3da7d", size = 106121 },
+    { url = "https://files.pythonhosted.org/packages/6d/00/29366b9eba7b6f6baed7d749f12add209b987c4cfbfa418404dbadc0f97c/yarl-1.20.0-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:84aeb556cb06c00652dbf87c17838eb6d92cfd317799a8092cee0e570ee11229", size = 103815 },
+    { url = "https://files.pythonhosted.org/packages/28/f4/a2a4c967c8323c03689383dff73396281ced3b35d0ed140580825c826af7/yarl-1.20.0-cp313-cp313t-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:f166eafa78810ddb383e930d62e623d288fb04ec566d1b4790099ae0f31485f1", size = 408231 },
+    { url = "https://files.pythonhosted.org/packages/0f/a1/66f7ffc0915877d726b70cc7a896ac30b6ac5d1d2760613603b022173635/yarl-1.20.0-cp313-cp313t-manylinux_2_17_armv7l.manylinux2014_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:5d3d6d14754aefc7a458261027a562f024d4f6b8a798adb472277f675857b1eb", size = 390221 },
+    { url = "https://files.pythonhosted.org/packages/41/15/cc248f0504610283271615e85bf38bc014224122498c2016d13a3a1b8426/yarl-1.20.0-cp313-cp313t-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:2a8f64df8ed5d04c51260dbae3cc82e5649834eebea9eadfd829837b8093eb00", size = 411400 },
+    { url = "https://files.pythonhosted.org/packages/5c/af/f0823d7e092bfb97d24fce6c7269d67fcd1aefade97d0a8189c4452e4d5e/yarl-1.20.0-cp313-cp313t-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:4d9949eaf05b4d30e93e4034a7790634bbb41b8be2d07edd26754f2e38e491de", size = 411714 },
+    { url = "https://files.pythonhosted.org/packages/83/70/be418329eae64b9f1b20ecdaac75d53aef098797d4c2299d82ae6f8e4663/yarl-1.20.0-cp313-cp313t-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:9c366b254082d21cc4f08f522ac201d0d83a8b8447ab562732931d31d80eb2a5", size = 404279 },
+    { url = "https://files.pythonhosted.org/packages/19/f5/52e02f0075f65b4914eb890eea1ba97e6fd91dd821cc33a623aa707b2f67/yarl-1.20.0-cp313-cp313t-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:91bc450c80a2e9685b10e34e41aef3d44ddf99b3a498717938926d05ca493f6a", size = 384044 },
+    { url = "https://files.pythonhosted.org/packages/6a/36/b0fa25226b03d3f769c68d46170b3e92b00ab3853d73127273ba22474697/yarl-1.20.0-cp313-cp313t-musllinux_1_2_aarch64.whl", hash = "sha256:9c2aa4387de4bc3a5fe158080757748d16567119bef215bec643716b4fbf53f9", size = 416236 },
+    { url = "https://files.pythonhosted.org/packages/cb/3a/54c828dd35f6831dfdd5a79e6c6b4302ae2c5feca24232a83cb75132b205/yarl-1.20.0-cp313-cp313t-musllinux_1_2_armv7l.whl", hash = "sha256:d2cbca6760a541189cf87ee54ff891e1d9ea6406079c66341008f7ef6ab61145", size = 402034 },
+    { url = "https://files.pythonhosted.org/packages/10/97/c7bf5fba488f7e049f9ad69c1b8fdfe3daa2e8916b3d321aa049e361a55a/yarl-1.20.0-cp313-cp313t-musllinux_1_2_i686.whl", hash = "sha256:798a5074e656f06b9fad1a162be5a32da45237ce19d07884d0b67a0aa9d5fdda", size = 407943 },
+    { url = "https://files.pythonhosted.org/packages/fd/a4/022d2555c1e8fcff08ad7f0f43e4df3aba34f135bff04dd35d5526ce54ab/yarl-1.20.0-cp313-cp313t-musllinux_1_2_ppc64le.whl", hash = "sha256:f106e75c454288472dbe615accef8248c686958c2e7dd3b8d8ee2669770d020f", size = 423058 },
+    { url = "https://files.pythonhosted.org/packages/4c/f6/0873a05563e5df29ccf35345a6ae0ac9e66588b41fdb7043a65848f03139/yarl-1.20.0-cp313-cp313t-musllinux_1_2_s390x.whl", hash = "sha256:3b60a86551669c23dc5445010534d2c5d8a4e012163218fc9114e857c0586fdd", size = 423792 },
+    { url = "https://files.pythonhosted.org/packages/9e/35/43fbbd082708fa42e923f314c24f8277a28483d219e049552e5007a9aaca/yarl-1.20.0-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:3e429857e341d5e8e15806118e0294f8073ba9c4580637e59ab7b238afca836f", size = 422242 },
+    { url = "https://files.pythonhosted.org/packages/ed/f7/f0f2500cf0c469beb2050b522c7815c575811627e6d3eb9ec7550ddd0bfe/yarl-1.20.0-cp313-cp313t-win32.whl", hash = "sha256:65a4053580fe88a63e8e4056b427224cd01edfb5f951498bfefca4052f0ce0ac", size = 93816 },
+    { url = "https://files.pythonhosted.org/packages/3f/93/f73b61353b2a699d489e782c3f5998b59f974ec3156a2050a52dfd7e8946/yarl-1.20.0-cp313-cp313t-win_amd64.whl", hash = "sha256:53b2da3a6ca0a541c1ae799c349788d480e5144cac47dba0266c7cb6c76151fe", size = 101093 },
+    { url = "https://files.pythonhosted.org/packages/ea/1f/70c57b3d7278e94ed22d85e09685d3f0a38ebdd8c5c73b65ba4c0d0fe002/yarl-1.20.0-py3-none-any.whl", hash = "sha256:5d0fe6af927a47a230f31e6004621fd0959eaa915fc62acfafa67ff7229a3124", size = 46124 },
+]


Reviews:
user: login: coderabbitai[bot]
body: **Actionable comments posted: 3**

<details>
<summary>🧹 Nitpick comments (6)</summary><blockquote>

<details>
<summary>output_handlers/__init__.py (1)</summary><blockquote>

`1-3`: **Add `__all__` declaration for explicit exports**

The imports are correctly set up for package-level access, but the static analysis tool flags them as unused. Consider adding an `__all__` declaration to explicitly define what the package exports.


```diff
from .base_handler import BaseHandler
from .filesystem import FilesystemHandler
from .bunny_storage import BunnyStorageHandler
+
+__all__ = ["BaseHandler", "FilesystemHandler", "BunnyStorageHandler"]
```

<details>
<summary>🧰 Tools</summary>

<details>
<summary>🪛 Ruff (0.8.2)</summary>

1-1: `.base_handler.BaseHandler` imported but unused; consider removing, adding to `__all__`, or using a redundant alias

(F401)

---

2-2: `.filesystem.FilesystemHandler` imported but unused; consider removing, adding to `__all__`, or using a redundant alias

(F401)

---

3-3: `.bunny_storage.BunnyStorageHandler` imported but unused; consider removing, adding to `__all__`, or using a redundant alias

(F401)

</details>

</details>

</blockquote></details>
<details>
<summary>output_handlers/filesystem.py (1)</summary><blockquote>

`26-44`: **Enhance error handling with more specific exceptions**

The implementation of `_write_entry` is correct and follows the contract from `BaseHandler`. However, the exception handling could be improved by catching more specific exceptions rather than the broad `IOError`.

Consider:
1. Handling `json.JSONEncodeError` separately for serialization issues
2. Distinguishing between file access vs. write errors


```diff
async def _write_entry(self, entry: dict, uid: str) -> bool:
    """
    Asynchronously writes a single entry to the filesystem.

    Args:
        entry (dict): The entry to write (will be JSON-encoded).
        uid (str): The unique identifier for the entry.

    Returns:
        bool: True if the entry was written successfully, False otherwise.
    """
    try:
        file_path = self.output_dir / f"{uid}.json"
+       json_data = json.dumps(entry)
+    except json.JSONEncodeError as e:
+        self.logger.error(f"Error serializing entry {uid}: {e}")
+        return False
+            
+    try:
        async with aiofiles.open(file_path, 'w') as f:
-            await f.write(json.dumps(entry))
+            await f.write(json_data)
        return True
-    except IOError as e:
+    except (IOError, OSError) as e:
        self.logger.error(f"Error writing entry {uid}: {e}")
        return False
```

</blockquote></details>
<details>
<summary>output_handlers/bunny_storage.py (1)</summary><blockquote>

`25-44`: **Minor robustness & observability tweaks**

1. The constructed URL can end up with a double slash if `base_path` starts with `/`.  
   `url = f"https://{self.region}.bunnycdn.com/{self.base_path.lstrip('/')}/{uid}.json"` avoids that.

2. Catch‑all `except Exception` will also hide `asyncio.CancelledError`, making task cancellation harder.  Consider re‑raising `CancelledError`:

```python
except asyncio.CancelledError:
    raise
except Exception:
    ...
```

3. When a non‑2xx status is returned you log the body but still swallow the response object.  Adding `await resp.release()` (or using `resp.text()` as you do ensures release) is good; you already call `.text()`, so ✅.

Taken together these changes make the handler safer and easier to debug at scale.

</blockquote></details>
<details>
<summary>transform-documents.py (3)</summary><blockquote>

`360-363`: **Blocking I/O inside async task**

`Path.read_text()` performs synchronous disk I/O, blocking the event loop while the whole file is read.  With many concurrent tasks this negates the benefits of asyncio.

Consider using `aiofiles`:

```python
import aiofiles

async with aiofiles.open(input_file, "r", encoding="utf-8") as f:
    text = await f.read()
```

This keeps the loop responsive, especially on slower disks or when reading large corpora.

---

`374-382`: **Environment variable casting is too naive**

`val.isdigit()` only matches positive integers; negative numbers and booleans (`true/false`) are ignored.  A small helper improves DX:

```python
def _auto_cast(val: str):
    if val.lower() in {"true", "false"}:
        return val.lower() == "true"
    try:
        return int(val)
    except ValueError:
        try:
            return float(val)
        except ValueError:
            return val
```

and replace the current logic with `val = _auto_cast(val)`.  
This allows users to pass richer configuration without code changes.

---

`441-456`: **Unbounded task creation may exhaust memory**

When `MAX_CONCURRENT=0`, the code fires off one task per file. On a large dataset this can overwhelm RAM and the BunnyCDN connection pool.

Default to a sensible limit (e.g. CPU count * 5) when the env var is absent or zero, or warn users about the risk.

```python
if max_conc == 0:
    max_conc = (os.cpu_count() or 4) * 5
    logging.warning("MAX_CONCURRENT not set – defaulting to %s", max_conc)
```

</blockquote></details>

</blockquote></details>

<details>
<summary>📜 Review details</summary>

**Configuration used: CodeRabbit UI**
**Review profile: CHILL**
**Plan: Pro**

<details>
<summary>📥 Commits</summary>

Reviewing files that changed from the base of the PR and between 55841f1d9e6322ffae79563ef6ad255de7bb384e and 40984a06f85f9fabc4823db6fe630a22d0b3ebc4.

</details>

<details>
<summary>⛔ Files ignored due to path filters (1)</summary>

* `uv.lock` is excluded by `!**/*.lock`

</details>

<details>
<summary>📒 Files selected for processing (7)</summary>

* `.gitignore` (1 hunks)
* `output_handlers/__init__.py` (1 hunks)
* `output_handlers/base_handler.py` (1 hunks)
* `output_handlers/bunny_storage.py` (1 hunks)
* `output_handlers/filesystem.py` (1 hunks)
* `pyproject.toml` (1 hunks)
* `transform-documents.py` (2 hunks)

</details>

<details>
<summary>🧰 Additional context used</summary>

<details>
<summary>🧬 Code Graph Analysis (3)</summary>

<details>
<summary>output_handlers/filesystem.py (1)</summary><blockquote>

<details>
<summary>output_handlers/base_handler.py (1)</summary>

* `BaseHandler` (7-54)

</details>

</blockquote></details>
<details>
<summary>output_handlers/bunny_storage.py (1)</summary><blockquote>

<details>
<summary>output_handlers/base_handler.py (1)</summary>

* `BaseHandler` (7-54)

</details>

</blockquote></details>
<details>
<summary>transform-documents.py (2)</summary><blockquote>

<details>
<summary>output_handlers/base_handler.py (1)</summary>

* `write_entry` (40-54)

</details>
<details>
<summary>output_handlers/bunny_storage.py (1)</summary>

* `close` (46-48)

</details>

</blockquote></details>

</details><details>
<summary>🪛 Ruff (0.8.2)</summary>

<details>
<summary>output_handlers/__init__.py</summary>

1-1: `.base_handler.BaseHandler` imported but unused; consider removing, adding to `__all__`, or using a redundant alias

(F401)

---

2-2: `.filesystem.FilesystemHandler` imported but unused; consider removing, adding to `__all__`, or using a redundant alias

(F401)

---

3-3: `.bunny_storage.BunnyStorageHandler` imported but unused; consider removing, adding to `__all__`, or using a redundant alias

(F401)

</details>

</details>

</details>

<details>
<summary>🔇 Additional comments (5)</summary><blockquote>

<details>
<summary>.gitignore (1)</summary>

`14-15`: **Appropriate additions to .gitignore**

Adding `output` and `en-json` to .gitignore makes sense for excluding generated data and export directories from version control. This aligns well with the new output handling system being implemented.

</details>
<details>
<summary>pyproject.toml (1)</summary>

`8-13`: **LGTM: Dependencies added for async operations**

The new dependencies appropriately support the asynchronous functionality:
- `aiofiles` for async file operations
- `aiohttp` for async HTTP requests (presumably for BunnyCDN)
- `asyncssh` for possible SSH transfers
- `python-dotenv` for environment variable configuration

All versions specified are recent and use the recommended '>=' format.

</details>
<details>
<summary>output_handlers/filesystem.py (2)</summary>

`1-6`: **LGTM: Clean module documentation and imports**

The module has clear documentation and organizes imports appropriately.

---

`7-24`: **LGTM: Well-structured initialization logic**

The `FilesystemHandler` class is well-documented and correctly inherits from `BaseHandler`. The constructor properly:
1. Calls the parent constructor
2. Converts the output directory to a Path object
3. Creates the directory if it doesn't exist
4. Logs the output location

This approach ensures the handler is ready to use immediately after initialization.

</details>
<details>
<summary>transform-documents.py (1)</summary>

`462-464`: **`hasattr(handler, "close")` should also check for `asyncio.iscoroutinefunction`**

If a handler provides a synchronous `close()` the `await` will raise.  Safer:

```python
close_fn = getattr(handler, "close", None)
if close_fn:
    if asyncio.iscoroutinefunction(close_fn):
        await close_fn()
    else:
        close_fn()
```

</details>

</blockquote></details>

</details>

<!-- This is an auto-generated comment by CodeRabbit for review status -->
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-18T15:30:46Z
user: login: bcye
state: COMMENTED
author_association: OWNER
submitted_at: 2025-04-18T16:18:34Z
user: login: coderabbitai[bot]
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-18T16:19:03Z
user: login: moll-re
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-04-19T14:28:49Z
user: login: coderabbitai[bot]
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-19T14:29:09Z
user: login: moll-re
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-04-23T09:57:10Z
user: login: coderabbitai[bot]
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-23T09:57:37Z
user: login: coderabbitai[bot]
body: **Actionable comments posted: 1**

<details>
<summary>🧹 Nitpick comments (2)</summary><blockquote>

<details>
<summary>output_handlers/base_handler.py (2)</summary><blockquote>

`40-54`: **Consider adding return value to `write_entry`.**

The `write_entry` method awaits `_write_entry` and captures its success status, but doesn't return this value to callers. While the current error handling works with `fail_on_error=True`, callers might benefit from knowing if a write succeeded when `fail_on_error=False`.

```diff
     async def write_entry(self, entry: dict, uid: str):
         """
         Public method to write an entry to the output. It handles exceptions and logs errors.
 
         Args:
             entry (dict): The entry to write (will be JSON-encoded).
             uid (str): The unique identifier for the entry. The default id provided by wikivoyage is recommended. 
+        Returns:
+            bool: True if the entry was written successfully, False otherwise.
         """
         success = await self._write_entry(entry, uid)
         if success:
             self.logger.debug(f"Successfully wrote entry with UID {uid}")
         else:
             self.logger.error(f"Failed to write entry with UID {uid}")
             if self.fail_on_error:
                 raise Exception(f"Failed to write entry with UID {uid}")
+        return success
```

---

`57-61`: **Consider marking `close` as abstract or using a docstring that clarifies its optional nature.**

The `close` method is provided as a placeholder for cleanup operations, but the static analysis tool flags it as an empty method without an abstract decorator. Since some handlers (like `BunnyStorageHandler`) need to override this for resource cleanup while others might not, you could either:

1. Mark it as abstract to make implementers explicitly opt-out if no cleanup is needed
2. Add a clearer docstring explaining its optional nature

```diff
-    async def close(self):
+    @abstractmethod
+    async def close(self):
         """
-        Closes the handler. This method should be overridden by subclasses if they need to perform any cleanup operations.
+        Closes the handler and releases any resources. Implement this method in subclasses that need to perform cleanup operations.
+        If no cleanup is needed, simply implement with 'pass'.
         """
         pass
```

<details>
<summary>🧰 Tools</summary>

<details>
<summary>🪛 Ruff (0.8.2)</summary>

57-61: `BaseHandler.close` is an empty method in an abstract base class, but has no abstract decorator

(B027)

</details>

</details>

</blockquote></details>

</blockquote></details>

<details>
<summary>📜 Review details</summary>

**Configuration used: CodeRabbit UI**
**Review profile: CHILL**
**Plan: Pro**

<details>
<summary>📥 Commits</summary>

Reviewing files that changed from the base of the PR and between 40984a06f85f9fabc4823db6fe630a22d0b3ebc4 and 0ada91a8ce7734bfccf71ebbe563ac024aae22e8.

</details>

<details>
<summary>📒 Files selected for processing (3)</summary>

* `output_handlers/base_handler.py` (1 hunks)
* `output_handlers/bunny_storage.py` (1 hunks)
* `transform-documents.py` (2 hunks)

</details>

<details>
<summary>🚧 Files skipped from review as they are similar to previous changes (1)</summary>

* output_handlers/bunny_storage.py

</details>

<details>
<summary>🧰 Additional context used</summary>

<details>
<summary>🧠 Learnings (1)</summary>

<details>
<summary>transform-documents.py (1)</summary>

```
Learnt from: moll-re
PR: bcye/structured-wikivoyage-exports#10
File: transform-documents.py:350-365
Timestamp: 2025-04-23T09:57:35.531Z
Learning: In the WikivoyageParser module, create separate parser instances for each concurrent task to avoid race conditions, as the parser maintains mutable state (self.root, self.current_section) that could be corrupted during concurrent access.
```

</details>

</details><details>
<summary>🧬 Code Graph Analysis (2)</summary>

<details>
<summary>output_handlers/base_handler.py (1)</summary><blockquote>

<details>
<summary>output_handlers/bunny_storage.py (2)</summary>

* `_write_entry` (35-50)
* `close` (52-54)

</details>

</blockquote></details>
<details>
<summary>transform-documents.py (2)</summary><blockquote>

<details>
<summary>output_handlers/base_handler.py (2)</summary>

* `write_entry` (40-54)
* `close` (57-61)

</details>
<details>
<summary>output_handlers/bunny_storage.py (1)</summary>

* `close` (52-54)

</details>

</blockquote></details>

</details><details>
<summary>🪛 Ruff (0.8.2)</summary>

<details>
<summary>output_handlers/base_handler.py</summary>

57-61: `BaseHandler.close` is an empty method in an abstract base class, but has no abstract decorator

(B027)

</details>

</details>

</details>

<details>
<summary>🔇 Additional comments (8)</summary><blockquote>

<details>
<summary>output_handlers/base_handler.py (2)</summary>

`7-11`: **Well-designed abstract base class.**

The `BaseHandler` class provides a clean, standardized interface that all output handlers must implement. The design follows good object-oriented principles by defining a common contract while allowing specialized implementations.

---

`27-37`: **Good method documentation with clear error handling expectations.**

The `_write_entry` abstract method is well-documented, clearly stating that implementations should handle exceptions internally and return a boolean success indicator rather than raising exceptions directly. This pattern ensures consistent error handling across different handler implementations.

</details>
<details>
<summary>transform-documents.py (6)</summary>

`9-13`: **Good use of asyncio and environment configuration.**

The addition of asyncio and dotenv support provides a solid foundation for concurrent processing with configurable behavior. Loading environment variables at import time is a common pattern and works well here.

---

`350-364`: **Correctly addressed potential data race with parser instances.**

You've correctly implemented the previous suggestion to create a separate parser instance for each file to avoid data races. This ensures that the mutable state in each parser (`self.root`, `self.current_section`) remains isolated during concurrent processing.


Using the learning from past reviews: Creating separate parser instances for each concurrent task avoids race conditions, as the parser maintains mutable state that could be corrupted during concurrent access.

---

`366-386`: **Well-implemented parameter handling from environment variables.**

The `gather_handler_kwargs` function provides a flexible way to configure handlers through environment variables with automatic type conversion. This approach cleanly separates configuration from code and follows the 12-factor app methodology.

---

`391-417`: **Robust handler loading with good error handling.**

The dynamic import mechanism with appropriate error checking provides a flexible and extendable system for loading different output handlers. The code gracefully handles missing environment variables, import errors, and missing handler classes with clear error messages.

---

`426-456`: **Good concurrency control with both bounded and unbounded options.**

The implementation offers flexible concurrency control by supporting both unbounded processing and semaphore-bounded concurrency. Using a semaphore to limit concurrent tasks is a well-established pattern for controlling resource usage in asyncio applications.

---

`465-466`: **Good use of `asyncio.run()` for managing the async event loop.**

Using `asyncio.run(main())` is the recommended way to run an async application as it properly handles starting and shutting down the event loop, including cleanup of resources.

</details>

</blockquote></details>

</details>

<!-- This is an auto-generated comment by CodeRabbit for review status -->
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-23T12:03:36Z

Review comments:
diff_hunk: @@ -0,0 +1,54 @@
+"""Reference handler for output handlers."""
+from abc import ABC, abstractmethod
+import logging
+
+
+
+class BaseHandler(ABC):
+    """
+    Abstract base class for output handlers. Defines the standardized interface that all output handlers must implement.
+    In particular, it requires the implementation of an asynchronous ("private") method `_write_entry` to write a single entry to the output.
+    """
+
+    logger = logging.getLogger(__name__)
+
+    def __init__(self, fail_on_error: bool = True, **kwargs):
+        """
+        Initializes the BaseHandler with optional parameters.
+
+        Args:
+            fail_on_error (bool): If True, the handler will raise an exception on error. Defaults to True.
+            **kwargs: Additional keyword arguments for specific handler implementations.
+        """
+        self.fail_on_error = fail_on_error
+
+
+    @abstractmethod
+    async def _write_entry(self, entry: dict, uid: str) -> bool:
+        """
+        Asynchronously writes a single entry to the output. This method should gracefully handle any exceptions that may occur during the writing process and simply return False if an error occurs.
+
+        Args:
+            entry (dict): The entry to write (will be JSON-encoded).
+            uid (str): The unique identifier for the entry. The default id provided by wikivoyage is recommended. 
+        Returns:
+            bool: True if the entry was written successfully, False otherwise.
+        """
+        pass
+
+
+    async def write_entry(self, entry: dict, uid: str):
+        """
+        Public method to write an entry to the output. It handles exceptions and logs errors.
+
+        Args:
+            entry (dict): The entry to write (will be JSON-encoded).
+            uid (str): The unique identifier for the entry. The default id provided by wikivoyage is recommended. 
+        """
+        success = await self._write_entry(entry, uid)
+        if success:
+            self.logger.debug(f"Successfully wrote entry with UID {uid}")
+        else:
+            self.logger.error(f"Failed to write entry with UID {uid}")
+            if self.fail_on_error:
+                raise Exception(f"Failed to write entry with UID {uid}")
path: output_handlers/base_handler.py
user: login: coderabbitai[bot]
body: _🛠️ Refactor suggestion_

**Return value of `write_entry` is silently discarded**

`write_entry()` awaits `_write_entry()` but intentionally drops the boolean it receives.  
Call‑sites (see `transform-documents.py`) may want to know whether a write actually succeeded, especially when `fail_on_error=False`.  Consider propagating the result:

```diff
-        success = await self._write_entry(entry, uid)
-        if success:
+        success = await self._write_entry(entry, uid)
+        if success:
             self.logger.debug(f"Successfully wrote entry with UID {uid}")
         else:
             self.logger.error(f"Failed to write entry with UID {uid}")
             if self.fail_on_error:
                 raise Exception(f"Failed to write entry with UID {uid}")
+        return success
```

This simple change makes the public API more useful without breaking existing behaviour.  
Optionally replace the broad `Exception` with a dedicated subclass (e.g. `HandlerWriteError`) for clearer error handling.



<!-- suggestion_start -->

<details>
<summary>📝 Committable suggestion</summary>

> ‼️ **IMPORTANT**
> Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

`````suggestion
    async def write_entry(self, entry: dict, uid: str):
        """
        Public method to write an entry to the output. It handles exceptions and logs errors.

        Args:
            entry (dict): The entry to write (will be JSON-encoded).
            uid (str): The unique identifier for the entry. The default id provided by wikivoyage is recommended. 
        """
        success = await self._write_entry(entry, uid)
        if success:
            self.logger.debug(f"Successfully wrote entry with UID {uid}")
        else:
            self.logger.error(f"Failed to write entry with UID {uid}")
            if self.fail_on_error:
                raise Exception(f"Failed to write entry with UID {uid}")
        return success
`````

</details>

<!-- suggestion_end -->

<!-- This is an auto-generated reply by CodeRabbit -->
created_at: 2025-04-18T15:30:45Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,48 @@
+import json
+import aiohttp
+from .base_handler import BaseHandler
+
+class BunnyStorageHandler(BaseHandler):
+    def __init__(
+        self,
+        region: str,
+        base_path: str,
+        api_key: str,
+        fail_on_error: bool = True,
+        keepalive_timeout: int = 75,
+    ):
+        super().__init__(fail_on_error=fail_on_error)
+        self.region = region
+        self.base_path = base_path
+        self.api_key = api_key
+
+        # no explicit 'limit'; use the default (100)
+        self._connector = aiohttp.TCPConnector(
+            keepalive_timeout=keepalive_timeout
+        )
+        self._session = aiohttp.ClientSession(connector=self._connector)
+
path: output_handlers/bunny_storage.py
user: login: coderabbitai[bot]
body: _🛠️ Refactor suggestion_

**`ClientSession` created outside an event‑loop context**

`aiohttp.ClientSession()` is instantiated during `__init__`, which may run before an event loop is running (e.g. when the handler is created in synchronous code).  Recent aiohttp versions discourage this and emit deprecation warnings; future versions may even error.

A safer pattern is lazy, in‑loop construction:

```diff
-        self._connector = aiohttp.TCPConnector(
-            keepalive_timeout=keepalive_timeout
-        )
-        self._session = aiohttp.ClientSession(connector=self._connector)
+        self._connector = None
+        self._session = None
+        self._keepalive = keepalive_timeout
```

and in `_write_entry`:

```diff
+        if self._session is None:
+            # first use – create inside running loop
+            self._connector = aiohttp.TCPConnector(keepalive_timeout=self._keepalive)
+            self._session = aiohttp.ClientSession(connector=self._connector)
```

This guarantees compatibility with both current and upcoming aiohttp 4.x behaviour.



> Committable suggestion skipped: line range outside the PR's diff.

<!-- This is an auto-generated reply by CodeRabbit -->

✅ Addressed in commit 0ada91a

<!-- This is an auto-generated comment by CodeRabbit -->
created_at: 2025-04-18T15:30:45Z
author_association: CONTRIBUTOR
diff_hunk: @@ -342,41 +347,122 @@ def export_json(self, root=None, indent=2) -> str:
             
         return json.dumps(root, indent=indent)
 
-def process_file(input_file: Path, parser: WikivoyageParser) -> None:
-    """Process a single wiki file and save JSON output"""
-    # Create output path with .json extension
-    output_file = input_file.with_suffix('.json')
-    
-    # Ensure output directory exists
-    output_file.parent.mkdir(parents=True, exist_ok=True)
+async def process_file(
+    input_file: Path,
+    parser: WikivoyageParser,
+    handler,
+) -> None:
+    """
+    Parse one wiki file and hand the resulting entry off to our handler.
+    Uses the filename (sans suffix) as the unique UID.
+    """
     
+    text = input_file.read_text(encoding="utf-8")
+    entry = parser.parse(text)  # assume returns a dict
+    uid = input_file.stem
+
+    await handler.write_entry(entry, uid)
+
path: transform-documents.py
user: login: coderabbitai[bot]
body: _⚠️ Potential issue_

**Data race: one `WikivoyageParser` shared by all tasks**

`process_file()` mutates the parser instance (`self.root`, `self.current_section`) while tasks run concurrently. Passing a single `parser` object to many tasks leads to interleaved state and corrupted JSON.

Refactor so every task gets a fresh parser – simplest is to build it inside `process_file`:

```diff
-async def process_file(
-    input_file: Path,
-    parser: WikivoyageParser,
-    handler,
-) -> None:
+async def process_file(
+    input_file: Path,
+    handler,
+) -> None:
@@
-    text = input_file.read_text(encoding="utf-8")
-    entry = parser.parse(text)  # assume returns a dict
+    text = input_file.read_text(encoding="utf-8")
+    parser = WikivoyageParser()
+    entry = parser.parse(text)
```

and adjust the call site (`create_task(...)`) accordingly.

Without this, results will be nondeterministic under concurrency.  



<!-- suggestion_start -->

<details>
<summary>📝 Committable suggestion</summary>

> ‼️ **IMPORTANT**
> Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

`````suggestion
async def process_file(
    input_file: Path,
    handler,
) -> None:
    """
    Parse one wiki file and hand the resulting entry off to our handler.
    Uses the filename (sans suffix) as the unique UID.
    """
    
    text = input_file.read_text(encoding="utf-8")
    parser = WikivoyageParser()
    entry = parser.parse(text)
    uid = input_file.stem

    await handler.write_entry(entry, uid)
`````

</details>

<!-- suggestion_end -->

<!-- This is an auto-generated reply by CodeRabbit -->

✅ Addressed in commit 0ada91a

<!-- This is an auto-generated comment by CodeRabbit -->
created_at: 2025-04-18T15:30:45Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,54 @@
+"""Reference handler for output handlers."""
+from abc import ABC, abstractmethod
+import logging
+
+
+
+class BaseHandler(ABC):
+    """
+    Abstract base class for output handlers. Defines the standardized interface that all output handlers must implement.
+    In particular, it requires the implementation of an asynchronous ("private") method `_write_entry` to write a single entry to the output.
+    """
+
+    logger = logging.getLogger(__name__)
+
+    def __init__(self, fail_on_error: bool = True, **kwargs):
+        """
+        Initializes the BaseHandler with optional parameters.
+
+        Args:
+            fail_on_error (bool): If True, the handler will raise an exception on error. Defaults to True.
+            **kwargs: Additional keyword arguments for specific handler implementations.
+        """
+        self.fail_on_error = fail_on_error
+
+
+    @abstractmethod
+    async def _write_entry(self, entry: dict, uid: str) -> bool:
+        """
+        Asynchronously writes a single entry to the output. This method should gracefully handle any exceptions that may occur during the writing process and simply return False if an error occurs.
+
+        Args:
+            entry (dict): The entry to write (will be JSON-encoded).
+            uid (str): The unique identifier for the entry. The default id provided by wikivoyage is recommended. 
+        Returns:
+            bool: True if the entry was written successfully, False otherwise.
+        """
+        pass
+
+
+    async def write_entry(self, entry: dict, uid: str):
+        """
+        Public method to write an entry to the output. It handles exceptions and logs errors.
+
+        Args:
+            entry (dict): The entry to write (will be JSON-encoded).
+            uid (str): The unique identifier for the entry. The default id provided by wikivoyage is recommended. 
+        """
+        success = await self._write_entry(entry, uid)
+        if success:
+            self.logger.debug(f"Successfully wrote entry with UID {uid}")
+        else:
+            self.logger.error(f"Failed to write entry with UID {uid}")
+            if self.fail_on_error:
+                raise Exception(f"Failed to write entry with UID {uid}")
path: output_handlers/base_handler.py
user: login: bcye
body: fail_on_error to true so return doesnt matter
created_at: 2025-04-18T16:18:34Z
author_association: OWNER
diff_hunk: @@ -0,0 +1,54 @@
+"""Reference handler for output handlers."""
+from abc import ABC, abstractmethod
+import logging
+
+
+
+class BaseHandler(ABC):
+    """
+    Abstract base class for output handlers. Defines the standardized interface that all output handlers must implement.
+    In particular, it requires the implementation of an asynchronous ("private") method `_write_entry` to write a single entry to the output.
+    """
+
+    logger = logging.getLogger(__name__)
+
+    def __init__(self, fail_on_error: bool = True, **kwargs):
+        """
+        Initializes the BaseHandler with optional parameters.
+
+        Args:
+            fail_on_error (bool): If True, the handler will raise an exception on error. Defaults to True.
+            **kwargs: Additional keyword arguments for specific handler implementations.
+        """
+        self.fail_on_error = fail_on_error
+
+
+    @abstractmethod
+    async def _write_entry(self, entry: dict, uid: str) -> bool:
+        """
+        Asynchronously writes a single entry to the output. This method should gracefully handle any exceptions that may occur during the writing process and simply return False if an error occurs.
+
+        Args:
+            entry (dict): The entry to write (will be JSON-encoded).
+            uid (str): The unique identifier for the entry. The default id provided by wikivoyage is recommended. 
+        Returns:
+            bool: True if the entry was written successfully, False otherwise.
+        """
+        pass
+
+
+    async def write_entry(self, entry: dict, uid: str):
+        """
+        Public method to write an entry to the output. It handles exceptions and logs errors.
+
+        Args:
+            entry (dict): The entry to write (will be JSON-encoded).
+            uid (str): The unique identifier for the entry. The default id provided by wikivoyage is recommended. 
+        """
+        success = await self._write_entry(entry, uid)
+        if success:
+            self.logger.debug(f"Successfully wrote entry with UID {uid}")
+        else:
+            self.logger.error(f"Failed to write entry with UID {uid}")
+            if self.fail_on_error:
+                raise Exception(f"Failed to write entry with UID {uid}")
path: output_handlers/base_handler.py
user: login: coderabbitai[bot]
body: @bcye I understand your point. Since you're using `fail_on_error=True`, the method will throw an exception on failure, making the return value unnecessary in your current implementation. I'll drop this suggestion.


<!-- <agentComponentType>chat-answer</agentComponentType> -->

<!-- <rawResChunk><answer>@bcye I understand your point. Since you're using `fail_on_error=True`, the method will throw an exception on failure, making the return value unnecessary in your current implementation. I'll drop this suggestion.</answer></rawResChunk> -->

<!-- This is an auto-generated reply by CodeRabbit -->
created_at: 2025-04-18T16:19:03Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,48 @@
+import json
+import aiohttp
+from .base_handler import BaseHandler
+
+class BunnyStorageHandler(BaseHandler):
+    def __init__(
+        self,
+        region: str,
+        base_path: str,
+        api_key: str,
+        fail_on_error: bool = True,
+        keepalive_timeout: int = 75,
+    ):
+        super().__init__(fail_on_error=fail_on_error)
+        self.region = region
+        self.base_path = base_path
+        self.api_key = api_key
+
+        # no explicit 'limit'; use the default (100)
+        self._connector = aiohttp.TCPConnector(
+            keepalive_timeout=keepalive_timeout
+        )
+        self._session = aiohttp.ClientSession(connector=self._connector)
+
path: output_handlers/bunny_storage.py
user: login: moll-re
body: This makes sense to me :+1: 

created_at: 2025-04-19T14:28:49Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,48 @@
+import json
+import aiohttp
+from .base_handler import BaseHandler
+
+class BunnyStorageHandler(BaseHandler):
+    def __init__(
+        self,
+        region: str,
+        base_path: str,
+        api_key: str,
+        fail_on_error: bool = True,
+        keepalive_timeout: int = 75,
+    ):
+        super().__init__(fail_on_error=fail_on_error)
+        self.region = region
+        self.base_path = base_path
+        self.api_key = api_key
+
+        # no explicit 'limit'; use the default (100)
+        self._connector = aiohttp.TCPConnector(
+            keepalive_timeout=keepalive_timeout
+        )
+        self._session = aiohttp.ClientSession(connector=self._connector)
+
path: output_handlers/bunny_storage.py
user: login: coderabbitai[bot]
body: @moll-re Great! I'm glad the suggestion makes sense to you. Would you like me to help implement the complete solution for this refactor? I could provide the full implementation code for the lazy initialization pattern in both `__init__` and `_write_entry` methods, making sure it works correctly with the rest of your code.


<!-- <agentComponentType>chat-answer</agentComponentType> -->

<!-- <rawResChunk><answer>@moll-re Great! I'm glad the suggestion makes sense to you. Would you like me to help implement the complete solution for this refactor? I could provide the full implementation code for the lazy initialization pattern in both `__init__` and `_write_entry` methods, making sure it works correctly with the rest of your code.</answer></rawResChunk> -->

<!-- This is an auto-generated reply by CodeRabbit -->
created_at: 2025-04-19T14:29:09Z
author_association: CONTRIBUTOR
diff_hunk: @@ -342,41 +347,122 @@ def export_json(self, root=None, indent=2) -> str:
             
         return json.dumps(root, indent=indent)
 
-def process_file(input_file: Path, parser: WikivoyageParser) -> None:
-    """Process a single wiki file and save JSON output"""
-    # Create output path with .json extension
-    output_file = input_file.with_suffix('.json')
-    
-    # Ensure output directory exists
-    output_file.parent.mkdir(parents=True, exist_ok=True)
+async def process_file(
+    input_file: Path,
+    parser: WikivoyageParser,
+    handler,
+) -> None:
+    """
+    Parse one wiki file and hand the resulting entry off to our handler.
+    Uses the filename (sans suffix) as the unique UID.
+    """
     
+    text = input_file.read_text(encoding="utf-8")
+    entry = parser.parse(text)  # assume returns a dict
+    uid = input_file.stem
+
+    await handler.write_entry(entry, uid)
+
path: transform-documents.py
user: login: moll-re
body: I see how this might be a problem, however in my testing I was unable to reproduce an error. Anyway, having individual parsers probably makes more sense.
created_at: 2025-04-23T09:57:10Z
author_association: COLLABORATOR
diff_hunk: @@ -342,41 +347,122 @@ def export_json(self, root=None, indent=2) -> str:
             
         return json.dumps(root, indent=indent)
 
-def process_file(input_file: Path, parser: WikivoyageParser) -> None:
-    """Process a single wiki file and save JSON output"""
-    # Create output path with .json extension
-    output_file = input_file.with_suffix('.json')
-    
-    # Ensure output directory exists
-    output_file.parent.mkdir(parents=True, exist_ok=True)
+async def process_file(
+    input_file: Path,
+    parser: WikivoyageParser,
+    handler,
+) -> None:
+    """
+    Parse one wiki file and hand the resulting entry off to our handler.
+    Uses the filename (sans suffix) as the unique UID.
+    """
     
+    text = input_file.read_text(encoding="utf-8")
+    entry = parser.parse(text)  # assume returns a dict
+    uid = input_file.stem
+
+    await handler.write_entry(entry, uid)
+
path: transform-documents.py
user: login: coderabbitai[bot]
body: @moll-re You're right that race conditions can be tricky to reproduce, especially during testing with small datasets or limited concurrency. They tend to appear unpredictably under higher load or specific timing conditions.

Creating individual parser instances per task is definitely the safer approach since it guarantees each task operates with its own isolated state, removing any possibility of one task's operations interfering with another's internal parser state. This pattern is particularly important in async code where execution order isn't deterministic.

Thanks for considering the suggestion and agreeing to this more robust design pattern!



🐰 *thoughtful rabbit noises*

---

<details>
<summary>✏️ Learnings added</summary>

```
Learnt from: moll-re
PR: bcye/structured-wikivoyage-exports#10
File: transform-documents.py:350-365
Timestamp: 2025-04-23T09:57:35.531Z
Learning: In the WikivoyageParser module, create separate parser instances for each concurrent task to avoid race conditions, as the parser maintains mutable state (self.root, self.current_section) that could be corrupted during concurrent access.
```

> Note: Learnings are effective only in the context of similar code segments. To apply general coding guidelines, please configure [review instructions](https://docs.coderabbit.ai/guides/review-instructions). You can manage existing learnings in the [app](https://app.coderabbit.ai/learnings).

</details>




<!-- <agentComponentType>chat-answer</agentComponentType> -->

<!-- <rawResChunk><answer>@moll-re You're right that race conditions can be tricky to reproduce, especially during testing with small datasets or limited concurrency. They tend to appear unpredictably under higher load or specific timing conditions.

Creating individual parser instances per task is definitely the safer approach since it guarantees each task operates with its own isolated state, removing any possibility of one task's operations interfering with another's internal parser state. This pattern is particularly important in async code where execution order isn't deterministic.

Thanks for considering the suggestion and agreeing to this more robust design pattern!



🐰 *thoughtful rabbit noises*

---

<details>
<summary>✏️ Learnings added</summary>

```
Learnt from: moll-re
PR: bcye/structured-wikivoyage-exports#10
File: transform-documents.py:350-365
Timestamp: 2025-04-23T09:57:35.531Z
Learning: In the WikivoyageParser module, create separate parser instances for each concurrent task to avoid race conditions, as the parser maintains mutable state (self.root, self.current_section) that could be corrupted during concurrent access.
```

> Note: Learnings are effective only in the context of similar code segments. To apply general coding guidelines, please configure [review instructions](https://docs.coderabbit.ai/guides/review-instructions). You can manage existing learnings in the [app](https://app.coderabbit.ai/learnings).

</details>

</answer></rawResChunk> -->

<!-- This is an auto-generated reply by CodeRabbit -->
created_at: 2025-04-23T09:57:37Z
author_association: CONTRIBUTOR
diff_hunk: @@ -342,41 +347,120 @@ def export_json(self, root=None, indent=2) -> str:
             
         return json.dumps(root, indent=indent)
 
-def process_file(input_file: Path, parser: WikivoyageParser) -> None:
-    """Process a single wiki file and save JSON output"""
-    # Create output path with .json extension
-    output_file = input_file.with_suffix('.json')
-    
-    # Ensure output directory exists
-    output_file.parent.mkdir(parents=True, exist_ok=True)
+async def process_file(
+    input_file: Path,
+    handler,
+) -> None:
+    """
+    Parse one wiki file and hand the resulting entry off to our handler.
+    Uses the filename (sans suffix) as the unique UID.
+    """
     
+    text = input_file.read_text(encoding="utf-8")
+    parser = WikivoyageParser()
+    entry = parser.parse(text)  # assume returns a dict
+    uid = input_file.stem
+
+    await handler.write_entry(entry, uid)
+
+def gather_handler_kwargs(handler_name: str) -> dict:
+    """
+    Find all ENV vars starting with HANDLER_<NAME>_ and turn them into kwargs.
+    E.g. HANDLER_SFTP_HOST=foo → {"host": "foo"}, HANDLER_SFTP_PORT=2222 → {"port": 2222}
+    """
+    prefix = f"HANDLER_{handler_name.upper()}_"
+    kwargs = {}
+
+    for env_key, val in os.environ.items():
+        if not env_key.startswith(prefix):
+            continue
+        param = env_key.replace(prefix, "").lower()
+        # cast ints
+        if val.isdigit():
+            val = int(val)
+        # cast bools
+        elif val.lower() in ("true", "false"):
+            val = val.lower() == "true"
+        kwargs[param] = val
+    print(f"Handler kwargs: {kwargs}")
+    return kwargs
+
+async def main():
+    logging.basicConfig(level=logging.DEBUG)
+
+    # 1. Which handler to load?
+    handler_name = os.getenv("HANDLER")
+    if not handler_name:
+        print("Error: set ENV HANDLER (e.g. 'filesystem')")
+        sys.exit(1)
+
+    # 2. Dynamic import
+    module_path = f"output_handlers.{handler_name}"
     try:
-        # Read and parse input file
-        with open(input_file, 'r', encoding='utf-8') as f:
-            wikitext = f.read()
-            
-        result = parser.parse(wikitext)
-        
-        # Write JSON output
-        with open(output_file, 'w', encoding='utf-8') as f:
-            f.write(parser.export_json())
-            
-    except Exception as e:
-        print(f"Error processing {input_file}: {e}")
+        mod = importlib.import_module(module_path)
+    except ImportError as e:
+        print(f"Error loading handler module {module_path}: {e}")
+        sys.exit(1)
+
+    # 3. Find the class: e.g. "sftp" → "SftpHandler"
+    class_name = handler_name.title().replace("_", "") + "Handler"
+    if not hasattr(mod, class_name):
+        print(f"{module_path} defines no class {class_name}")
+        sys.exit(1)
+    HandlerCls = getattr(mod, class_name)
+
+    # 4. Build kwargs from ENV
+    handler_kwargs = gather_handler_kwargs(handler_name)
+
+    # 5. Instantiate
+    handler = HandlerCls(**handler_kwargs)
+
+    # 6. Which dir to walk?
+    input_dir = Path(sys.argv[1]) if len(sys.argv) > 1 else Path(".")
+    txt_files = list(input_dir.rglob("*.txt"))
+
+    if not txt_files:
+        print(f"No .txt files found under {input_dir}")
+        sys.exit(1)
+
+    # 7. read concurrency setting
+    try:
+        max_conc = int(os.getenv("MAX_CONCURRENT", "0"))
+    except ValueError:
+        print("Error: MAX_CONCURRENT must be an integer")
+        sys.exit(1)
+
+    if max_conc < 0:
+        print("Error: MAX_CONCURRENT must be >= 0")
+        sys.exit(1)
+
+    # 8. schedule tasks
+    if max_conc == 0:
+        # unbounded
+        tasks = [
+            asyncio.create_task(process_file(txt, handler))
+            for txt in txt_files
+        ]
+    else:
+        # bounded by semaphore
+        sem = asyncio.Semaphore(max_conc)
+
+        async def bounded(txt):
+            async with sem:
+                return await process_file(txt, handler)
+
+        tasks = [
+            asyncio.create_task(bounded(txt))
+            for txt in txt_files
+        ]
+
+    # 9. run them all
+    await asyncio.gather(*tasks)
+    await handler.close()
path: transform-documents.py
user: login: coderabbitai[bot]
body: _🛠️ Refactor suggestion_

**Don't forget to handle potential exceptions in `handler.close()`.**

When calling `handler.close()`, there's no error handling. Some handlers (like `BunnyStorageHandler`) might perform network operations during close that could fail.

```diff
-    await asyncio.gather(*tasks)
-    await handler.close()
+    await asyncio.gather(*tasks)
+    try:
+        await handler.close()
+    except Exception as e:
+        logging.error(f"Error closing handler: {e}")
+        # Consider whether this should be fatal or not
```


<!-- suggestion_start -->

<details>
<summary>📝 Committable suggestion</summary>

> ‼️ **IMPORTANT**
> Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

`````suggestion
    await asyncio.gather(*tasks)
    try:
        await handler.close()
    except Exception as e:
        logging.error(f"Error closing handler: {e}")
        # Consider whether this should be fatal or not
`````

</details>

<!-- suggestion_end -->

<!-- This is an auto-generated comment by CodeRabbit -->
created_at: 2025-04-23T12:03:35Z
author_association: CONTRIBUTOR

PR comments:
user: login: coderabbitai[bot]
created_at: 2025-04-18T15:28:19Z
body: <!-- This is an auto-generated comment: summarize by coderabbit.ai -->
<!-- walkthrough_start -->

## Walkthrough

This update introduces a modular, asynchronous output handling system for processing and storing parsed documents. Three new output handlers are implemented: a base abstract handler, a filesystem-based handler, and a BunnyCDN storage handler that uploads entries via HTTP PUT requests. The main document transformation script is refactored to support asynchronous, concurrent processing of files using asyncio, with dynamic selection and configuration of output handlers via environment variables. Project dependencies are updated to include required async libraries. The `.gitignore` is expanded to exclude output artifacts.

## Changes

| File(s)                                                      | Change Summary                                                                                      |
|--------------------------------------------------------------|---------------------------------------------------------------------------------------------------|
| .gitignore                                                   | Added `output` and `en-json` to ignore list.                                                      |
| output_handlers/__init__.py                                  | Added package initializer to expose handler classes (`BaseHandler`, `FilesystemHandler`, `BunnyStorageHandler`). |
| output_handlers/base_handler.py                              | Introduced `BaseHandler` abstract class for async output handling with standardized interface.    |
| output_handlers/filesystem.py                                | Added `FilesystemHandler` for async JSON writing to filesystem with directory creation and error handling. |
| output_handlers/bunny_storage.py                             | Added `BunnyStorageHandler` for async HTTP PUT uploads to BunnyCDN storage with connection management and error logging. |
| pyproject.toml                                               | Added dependencies: `aiofiles`, `aiohttp`, `asyncssh`, `python-dotenv`.                           |
| transform-documents.py                                       | Refactored to async, concurrent file processing; dynamic output handler loading and configuration via environment variables; added error handling and concurrency control. |

## Sequence Diagram(s)

```mermaid
sequenceDiagram
    participant User
    participant Script
    participant Handler
    participant Parser
    participant Output (Filesystem/BunnyCDN)

    User->>Script: Run transform-documents.py with input dir
    Script->>Script: Load env vars, configure logging
    Script->>Handler: Dynamically load and instantiate handler
    Script->>Parser: Initialize WikivoyageParser
    Script->>Script: Collect .txt files
    loop For each file (concurrent)
        Script->>Parser: Parse file
        Parser-->>Script: Return parsed entry
        Script->>Handler: write_entry(entry, uid)
        Handler->>Output: Store entry (Filesystem or BunnyCDN)
    end
    Script->>Handler: Call close() (if exists)
```

## Assessment against linked issues

| Objective                                                                 | Addressed | Explanation                  |
|---------------------------------------------------------------------------|-----------|------------------------------|
| Implement Bunny Storage Adapter using HTTP API (Closes #2, #9)            | ✅        |                              |

## Possibly related PRs

- bcye/structured-wikivoyage-exports#8: Introduced base and filesystem output handlers, forming the foundation extended by the BunnyStorageHandler in this PR.

## Suggested reviewers

- bcye

## Poem

> A hop, a jump, async we go,  
> Files to parse and entries to stow.  
> Bunny hops to storage in the cloud,  
> Filesystem burrows, silent, proud.  
> With handlers new and code so spry,  
> We process, store, and multiply!  
> —🐇✨

<!-- walkthrough_end -->
<!-- internal state start -->


<!-- DwQgtGAEAqAWCWBnSTIEMB26CuAXA9mAOYCmGJATmriQCaQDG+Ats2bgFyQAOFk+AIwBWJBrngA3EsgEBPRvlqU0AgfFwA6NPEgQAfACgjoCEYDEZyAAUASpETZWaCrKNwSPbABsvkCiQBHbGlcSHFcLzpIACIAQVp6ACFsDAx5AGUCKlJIeLRuGgpoyAB3NGQHAWZ1Gno5MNgPbERKSAEGWQ9yzx8/QODEUIxHAVaARgAGFCxcRr7ufER1fBcNGARkW0YvRekURAcPYeZRvgBOdAx6JW4yWmR8GbmmZm5I8Uf+ADMe338gkKQY6nSAADjWADl8OgEup4I80L4lIgGBR4AV4VgVpAlLhtF5kCp8HgGh54K9ImwMHiPlhnB5ePgJPAlLQ1u5rHYGJg2h5mSQSlFqJBYLhcNxEBwAPRSojqWDYAQaF5S9qdKWDCjYMTYfy0MAleAAa0k+FkaFIYBIAA8FhRcIgpdxvF4pZMNEZ9MZwFA7t8cARiGRlLUFKx2FxePxhKJxFIZPImEoqKp1FodF6TFA4KhUDy0HhCKRyFRQy8qZw/GgSvZHMxnPJ6knlKnNNpdGBDN7TAYNHLxEQMCsSBwDNFxwYLLkAJJBkvUKIOJwuf0MWCYUiINxzAAGfbhg+HO8gX3gkVK3Ww3FoC/oBGmDC82CUYRK0PINfYaOkXB3xPFeDHpg9A7mQYBCIgjx7us0hdLCtLIPAGCatqoQAOLqGE0LwIe/iXPIp6RA8fC0PA/hiCs8B7BgaBsCB/7OrgQFXJAoEYOBkEYMeSGkvMizLC4AA0PD+FI1JIUQpLMCeFAsLyElhFQDBGlE2LljUdAekYk6WLEXiFNQmLIPesweEoj7OIZjwPD8Np2qG2LOgIXjwAwkDsHC0hGFC5BjhOBgQGARgMXgAD665XJEFCOqFoVIeosUaNwsijuO0Q6TOc4houdYNquEWbtuqBrhuZLUrJtDansPI7rF8W4IlyXHoRZW8X+eCMeFwFRYgx7cGgykWh4szCuS9nGbA/geBZBw/qxiTlCQAAS3WUM1snSaZrECItXWRWtkDMIo3gkMJO4AGJntIsiDCQzArftFDrXJW07i1iA3TQzDHkdlWRMJwHzSkaSZCsQ0PbQUXPZtu4CMDsihYMYOkD9x2ROyGz2CQuBXieA1nuoC7ICQ5RUXwA0MNIxnQqZLQiqtfAzS0yAlI0WBjSs4gYJJXwbW1IUNRFkOUL1PADUaQ1aeYun6SGRlYbx5leJZCH+nZnOqXwTkuW5HniF5AWQD5HgABRbSV3MeI8XjyGgCSIa8nP2DSd3sIhTwMuLQ3THCiLwAAXq0hqzP+h2KPAXyyApmDyOr9qa54zmue54m4PISsq0ZACUHppZ6PYC3twvRaqu1C1FSUpX56VTrEs7Ftl9BLvWK74D8FuFQYsRAgK6ACJqA2hDtdNM/Ni0Q1DJ7YgL9OPYh7u4BVVW0MJSinhg0fO8BzikYH9BIYUXwDR4XzYiUaJc5JX5UYSH0YGuslDs0NtrNOoT/NgZF7JUTN7PeY2UnYG1UK591AkFCl+WQx4mCyTwEhDwbAQ4r1KAgNc6AxR3QKNTUoF8uj2AkueSBkATZoBxK5WkDYs4oCUOJU8UR6ikJSPAAEkBpwABFLj0H8DjCgyF0BtHwPgSIPIkKkW5JfWsDBKYHH4HwI+Z5dQeGDrAUOVAkAKRtJTDE1kMYMkVDrdAd8H6PGJMgRBKiQKgJoBA8qUDGCIgJKxEBuCbGLygcJHYRAKjamkcRPGCj/AAxYmo5mlx3LWi0bSfgWB5FPjwhHNqsTQqPAgRQWST0TzK0kqgFooR7zQC1CQXR2xyjICYMhReqFsQUxIFg/hb18TJIwKk9Jx4BCCOEVgfqVBEGtHvOUxeQj3JpOxOXBSox1zMmqSxBw3BxowlIhQ3wKlZBvgoPQZwRBHBuynnwG0NBkLwDUC5NOaxu5vGPiorwyZDGyHvpNExzRWKPl2D9bGFj9giSZCyKIp8+A/2VrNbBTJKBohfAkx8JNhjcE+eQOgmlIAABl8BEDlNzT5B8SBEFLL87ESg4aoujixUF2J/D2QklLDKekDKqxMnMDOpZ5Zt3CfZBO2tk56xvp6XICQhT90XoPEpMidwLRaBPA6PF2oAUFgzR0w9wHl0oJXY8yjRyQF0JAAAwtZSpFE+C1TihvBqoUTYtC8F8YSSSUkkooFwdpQyAC8MBCnCQAFSuqNGUCgXis47gMOqqAsR+VKVCOUO5blzGKCcVY8BkDTUkHNcJSBXAxG4GEp/WgXBNRUM7AIoRfqA25CMYdd5UadwxtcS4eNiaU5uJTeQ9NLIs2L19f6jVsRi2RpAi8lo1avitrzgFAuHUwqKpLnDVICMkbZCKclVKE5a712DDipuuVW7twKgbbuH4hXIBFfDUGM7xUZNzOVY6lNkFIUaBfBSvM5IivHgzaCb9PnIhwuQO80Irw7DtpAAAUukAA8hCWt35sHJEnZqthIHp3e2ZKQpa0BoBWGsAAVWgH0AEgxEDFPKShPV6ApG1IdGLHp2MRa7N4hBtIUGQP+DlI8YS8qxazGErEKw05IArKCcM9Js9Ia3qyUEjZXGSC1L9lIMI5ISChz+bxRDyGFCpFjJiV+oR6rwD9oHCaHhmOoZsIizhIoSbJhkCSNeFHUQkyicyra2h8CinFDATVKHynkAI4DR8VFqRYwOJiSAKRxC+HzEYh5T8ymPBoNadThIJD4hUOjIwsQ6ShcfqYktSDWK5KvKFNzsYVjcSNZply2neL2cczC6ALmlPuayKUeU/Dymni2Smc8KlxMuUk+INgodPP+CJrxLzQDmZLE+AkocoavD9doPIDTWmEUchgQg0tIFnFgMrXYsN9y0vP3kN+/AdtCRkLEJiPKRDuikIA8Bk8V0FakLw7q0M1HZC0cgPpwz8r6CfC2vRzETHFosdgMJ/hTCWE/NoWTYhqH2E51YaEXJhI5myV4Jpmgxm7YUaQo+Z8Cktrsc4ysozAygFp1uMUrtWMrg6f/UBkD/VZA/voPBsJCmUNWHQ5hgYobgnY11HwncBTgjcR+D96QCxkIeEGNQJ5ojXIDYcERmRJsABMEwJjCVV2MYS2JVcABZYfThF3Mfbv7YmIGEphTxhIsA2vq7MXi0O2HCSlzjCLShgf+EQOLum7SZvCVMlgbhfO93nURC0aCyXY4ROI/LSqaJ0VbRNxsvC3JsBEFFB4lFpB6DKId+wj30hvDqbpKxUPBISDHiD7wrS3cdw9or7c7bjyzErc+YyZkrIFaMluBQG22xdjyaQ65lyw2qb+c848WrKwk3IV1ApSFmBcbMs98SCg0jKW11loy6yCstoMqsnw5lccHJa30Ry1OXLDaasBYSXlmagaTsPeDJ90xWKFzHXK+GiMsiSyaobAAsq3nbKyL+HVEarFH2sJL9o8M2hQP9i0KFP1LMLAQDNwPAKFCsigf4l4E0i0isHah0pAE6oLqdKJh1pIOAt1jJngFwAfEQZAAAOwACsvqr+o8+6j+P+pAx6fqUAgBmWwBdAv4W2bka8WMOM3AuWk++WFAfarBPE7Bz2T+3BT6ABQBd+whxaYha21icaZqFqoGKUx2aaAWTazsFAOaegeaXghWu6D+IMXBy0qhfB6hIBrEIhOIJA7cOwva+h8hWAihB6jhPB1c+cQUBg7+sqUo70n0d0lc86NcukS684oYzceUzKHcW6PcNYLUrEMRt0zAyq0wgylU0ibUl0REsR90L+TMAMtY7QN+/oD6YqT6q80g76UQ94MaNOwGVo98igUQ181UqWzefedKJ8V0H0BRxSY6GKRW82OeDWPIM8pE5EWQ8gSBQOKCrksAKAoQZADgnu4SSAJGTY/WEiMcPcvieU3SQCqxMhN8amkAVuisX8eqGx1AuxV4Y28xJWB+uGw4J4KQJ2CIJys2yA7ePydQ8gdmIx4WGWHytUFakCO4wkrMOxYQaAKkR2qap2K452LEjCG8YONC4gdC/yi8EkzulAxWAcf8cwRC94V2EIwOMaiEeS0IpCuRNEdE6AXwhQvE4OZJkOuepCe4EEUE4SByo27MHs/AI6oQ9x7x5OreiqyA04UogGvGKwCYzxWeuOjQ0kgMVefCpC9qnS0wYihk6KCuvisi2Bii/oW03R+APe/x2kVKW+B+2Ce+ogys2+h+tktoGsX2p+ScusF+Bsgad+dhF0kxVRPBr+UqnUH+0R8Z0xf+0ZneFOBq9UEB+h2uCpoUqxWB7qnqmyiA/hsZFR10BRIRWZQonareSJLiehCaBhyaxhja9+2augVh5pth7BNZUxX0IRg6gURgyUjIIgYgGgBAzAXgCRVKyRjctYy48gGRm6W4BgHI05sYnhtwVwZADAN8SZU5skM5mg85Nh9MkmowZALKwE9CtssI6Kp8uo2RYsg0m4wh8I70x4JsUg0U/myuuuGgYwGgUwoyOE16WcZ05WYo3AAFQFMpkAAAzOBRBWMAAGz2kIDp6UBwXuFGIHCwDIUiwgUaCq6QV4UwWEXA47jJQhzsS0D4AHISDkXAWfBYU0XQUEUWG6J0zAFwg770jTAtDxx1BeGAmAy0SPCSRbQ2gnEKQ3B3DHmnk7jMAlDdISV3rMCNA+DMSWLGhgLRY6VrSQjQhsXXqMBbm75zB7liAZZ2zUCkJqSPDNa6gH7oB4T1hKAb4yw0ryzjGeEWQBk2Qsohn2nsoRnhCX7Gy5z+QTkGACrIR/LMBgCsUMDbLUg4ZzqhGLpZQrprktwbkbqlTbkcgohogFAXjID+BHx6q/J8ykJwmmLO4EIkBgCzDTZRCMi+IKS/QJp3YpbhphbtVKbZVpLHmyBgADKyQ+B9XogJrwIBZLDooiHwjFKURoqIisT9VUyhQtTNTAm0homoK7FtW7Z9B2zCTmXIKAznxsUeDMnynSqUatWdX1YmhhD7I3b/R1XzDKwXptC2yjX3z7WyS+JHVXTHgjTvwmbW70DmXU4tT+5syN7GLhZ95KCRBEADZbT3WGF3Y4h3K0Ry4+DyCM6qQKn8ZRQADke6yJtibySC5O2gMSp1/mZQEWGAQFoY94HhgMQ4JQo4BgEFSKB29wKczIj8FYkAcWaICWewzQCkO4rF7Fe4BgyuawNgiNvEsxPJJ8fMr0S0sQEIbCiKAAojYMeGQLLY8PLYrZps5F0CxDNjyRTX3hzPaNTjAp7uLqROir9CdEZrMUzB6BhZAFbdFiGsgGOmAF7qIBHMnCsmssnlshWMgHpTLWRI7UAs7crRCQ1fANaFELnjuGbRbdbTYKFMABCLEP/lbXoKFKifYoMApHFk+H/NhD5tiOaSgpQF0Ejq6UrTQB6GBawhUpgOIATXMLMbnrTF0N6jlQ6B6EwbraILqEsFIH3kwItWIISL0HuLgNFs1JMTJHJKQkqesfYLcCeXQtCbxKeNFKEOWMBGAC5OQD5ZnUAnil4QWPpD6c8LqP4D5jfSsLIB6NhZvYdo1o8FNWAx0M8eSJhDna9P/rEAABqhSarAaar6Y2BW0QjQB218150YBO3OAu3ngmxTBsCYDIApDtIpCd7lKIMzU5wGAMFrDpBrh0AnS3xjU7bGTlBGjYIHUyIkxoJo3uTyitDMPEhHmhkCKsPPn8ItD1jcAqJ4T3guTVBv0IOgMzUejgi5BlDqARYUjYz+bMoOKYmIDiMehnBrCaoOLW6Y3PI+EN4U7fbz0MwejujWDx4kYvBvA2OfBsAHCSxGBW0jJ8BjLoo80LK4p8DVB+bor20UNUNK2u3m7TBd0siTXGP3zyBd0DDCTpPrWSSzEh1ET2m/z5PC292MT/V7DvlXDFLVXohDD4A1hah8KwnhqHQc1An3xRLM47ibX4AaADMmy+rFK8AkBTJPJXXZ23bTZUmkbVPA6vVWIKSeLJzJP+BHRSBsgwRKaDK+BfA7A1iZH0A52fXWP2DCPN56mun8BAVtPYJckui2xrPCTsMlMdBzWRYLWRDI3LVf1KINZvBp742u1vWtMf5KaeV6gK2aa51y0F3UNF0BW5Bem0o0z0p+mZw75H7BmSXRVn6xWeTblQCaqboPNc2fCMVQ2HUtQmxISdQtRcBWCfF3XOASVcAADqJlEgZoQ0/L0UhFfZRsjwDegtxabL+A0NnL3LYUvL1gArdNsruaxsthO4qViA6VmVqrq9eVUCHoDZ9AO6XwLLWAO4+NpkFARcUUGBXqXiJsY6oURtsBlhxhhrxrprWVFryq1rWqTLYzIJjr9YSE8zcNnJyrcbGACbSZwbKwGVobWd4boRQ6XYWYKcX2PwBYgYDcxV5YEYVYNYaRK4TYAxKYagbYGYnYBghb6kDULIiAoUokVEgotA3+zgoQXobbPokAuuEwZwoIuuaAEw2FXwoITBXwZwR87QuuoIyuaFtAAg87JA2FaFEwaAyuyutAEwAgaFJAa7HYhghbh7N4ZwYwaAoIlMDBDBaFuuAgXwUiXwDBYwl7owTB+7A0EwoFaAaAJAx7JAoI173YUAHbcU9wPbyzfbdANi9A+gQAA== -->

<!-- internal state end -->
<!-- finishing_touch_checkbox_start -->

<details>
<summary>✨ Finishing Touches</summary>

- [ ] <!-- {"checkboxId": "7962f53c-55bc-4827-bfbf-6a18da830691"} --> 📝 Generate Docstrings

</details>

<!-- finishing_touch_checkbox_end -->
<!-- tips_start -->

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

<details>
<summary>❤️ Share</summary>

- [X](https://twitter.com/intent/tweet?text=I%20just%20used%20%40coderabbitai%20for%20my%20code%20review%2C%20and%20it%27s%20fantastic%21%20It%27s%20free%20for%20OSS%20and%20offers%20a%20free%20trial%20for%20the%20proprietary%20code.%20Check%20it%20out%3A&url=https%3A//coderabbit.ai)
- [Mastodon](https://mastodon.social/share?text=I%20just%20used%20%40coderabbitai%20for%20my%20code%20review%2C%20and%20it%27s%20fantastic%21%20It%27s%20free%20for%20OSS%20and%20offers%20a%20free%20trial%20for%20the%20proprietary%20code.%20Check%20it%20out%3A%20https%3A%2F%2Fcoderabbit.ai)
- [Reddit](https://www.reddit.com/submit?title=Great%20tool%20for%20code%20review%20-%20CodeRabbit&text=I%20just%20used%20CodeRabbit%20for%20my%20code%20review%2C%20and%20it%27s%20fantastic%21%20It%27s%20free%20for%20OSS%20and%20offers%20a%20free%20trial%20for%20proprietary%20code.%20Check%20it%20out%3A%20https%3A//coderabbit.ai)
- [LinkedIn](https://www.linkedin.com/sharing/share-offsite/?url=https%3A%2F%2Fcoderabbit.ai&mini=true&title=Great%20tool%20for%20code%20review%20-%20CodeRabbit&summary=I%20just%20used%20CodeRabbit%20for%20my%20code%20review%2C%20and%20it%27s%20fantastic%21%20It%27s%20free%20for%20OSS%20and%20offers%20a%20free%20trial%20for%20proprietary%20code)

</details>

<details>
<summary>🪧 Tips</summary>

### Chat

There are 3 ways to chat with [CodeRabbit](https://coderabbit.ai?utm_source=oss&utm_medium=github&utm_campaign=bcye/structured-wikivoyage-exports&utm_content=10):

- Review comments: Directly reply to a review comment made by CodeRabbit. Example:
  - `I pushed a fix in commit <commit_id>, please review it.`
  - `Generate unit testing code for this file.`
  - `Open a follow-up GitHub issue for this discussion.`
- Files and specific lines of code (under the "Files changed" tab): Tag `@coderabbitai` in a new review comment at the desired location with your query. Examples:
  - `@coderabbitai generate unit testing code for this file.`
  -	`@coderabbitai modularize this function.`
- PR comments: Tag `@coderabbitai` in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
  - `@coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.`
  - `@coderabbitai read src/utils.ts and generate unit testing code.`
  - `@coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.`
  - `@coderabbitai help me debug CodeRabbit configuration file.`

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

### CodeRabbit Commands (Invoked using PR comments)

- `@coderabbitai pause` to pause the reviews on a PR.
- `@coderabbitai resume` to resume the paused reviews.
- `@coderabbitai review` to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
- `@coderabbitai full review` to do a full review from scratch and review all the files again.
- `@coderabbitai summary` to regenerate the summary of the PR.
- `@coderabbitai generate docstrings` to [generate docstrings](https://docs.coderabbit.ai/finishing-touches/docstrings) for this PR.
- `@coderabbitai generate sequence diagram` to generate a sequence diagram of the changes in this PR.
- `@coderabbitai resolve` resolve all the CodeRabbit review comments.
- `@coderabbitai configuration` to show the current CodeRabbit configuration for the repository.
- `@coderabbitai help` to get help.

### Other keywords and placeholders

- Add `@coderabbitai ignore` anywhere in the PR description to prevent this PR from being reviewed.
- Add `@coderabbitai summary` to generate the high-level summary at a specific location in the PR description.
- Add `@coderabbitai` anywhere in the PR title to generate the title automatically.

### CodeRabbit Configuration File (`.coderabbit.yaml`)

- You can programmatically configure CodeRabbit by adding a `.coderabbit.yaml` file to the root of your repository.
- Please see the [configuration documentation](https://docs.coderabbit.ai/guides/configure-coderabbit) for more information.
- If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: `# yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json`

### Documentation and Community

- Visit our [Documentation](https://docs.coderabbit.ai) for detailed information on how to use CodeRabbit.
- Join our [Discord Community](http://discord.gg/coderabbit) to get help, request features, and share feedback.
- Follow us on [X/Twitter](https://twitter.com/coderabbitai) for updates and announcements.

</details>

<!-- tips_end -->
author_association: CONTRIBUTOR
user: login: moll-re
created_at: 2025-04-23T12:07:23Z
body: I have added the changes suggested by coderabbit along with a tiny bit of cleanup, as well as a fix for reading boolean type env vars. I couldn't really test this for the upload to bunny, so maybe try that before merging.

@bcye  Let me know what you think.
author_association: COLLABORATOR
user: login: bcye
created_at: 2025-04-23T17:47:59Z
body: works great!
author_association: OWNER