Repository: bcye/structured-wikivoyage-exports
Pull request #21: Dockerize
Author: login: bcye
Created: 2025-04-26T21:05:45Z | Merged: 2025-04-30T20:11:31Z

Depends on and includes code from #18. Closes #1. Closes #5. On my machine docker seems to have quite low ulimit settings, so I included running the image with high ones in the README (More than there are pages), there doesn't seem to be a default (only applicable for FS handler). @coderabbitai review Dockerfile README.md

<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit

## Summary by CodeRabbit

- **New Features**
  - Introduced a GitHub Actions workflow to automatically build and push Docker images on tag pushes.
  - Enhanced usage instructions in the documentation, including Docker usage and TypeScript type information.

- **Refactor**
  - Migrated the project environment from Node.js to Python, updating the Dockerfile and removing Node.js dependencies.
  - Rewrote the main data processing script to stream and parse Wikimedia dumps directly from the web, improving performance and scalability.

- **Chores**
  - Updated the `.dockerignore` file to ignore additional files and directories.
  - Removed obsolete files and dependencies related to the previous Node.js implementation.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

Linked issue #1: Dockerize Python
The Dockerfile is still for the old script. We need to adapt it to run the python script instead

Diff:
diff --git a/.dockerignore b/.dockerignore
index 6288a4d..db7e340 100644
--- a/.dockerignore
+++ b/.dockerignore
@@ -1,8 +1,8 @@
 .env
-
-# NODE
-node_modules
-
-# PYTHON
 __pycache__
 .venv
+.pytest_cache
+docs
+node_modules
+output
+sketching
diff --git a/.github/workflows/build-image.yaml b/.github/workflows/build-image.yaml
new file mode 100644
index 0000000..1e667cf
--- /dev/null
+++ b/.github/workflows/build-image.yaml
@@ -0,0 +1,33 @@
+name: Build Docker Image
+
+on:
+  push:
+    tags:
+      - "docker/*"
+
+jobs:
+  docker:
+    runs-on: ubuntu-latest
+    permissions:
+      packages: write
+      contents: read
+      id-token: write
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v4
+
+      - name: Login to GitHub Container Registry
+        uses: docker/login-action@v3
+        with:
+          registry: ghcr.io
+          username: ${{ github.actor }}
+          password: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Set up Docker Buildx
+        uses: docker/setup-buildx-action@v3
+
+      - name: Build and push
+        uses: docker/build-push-action@v6
+        with:
+          push: true
+          tags: ghcr.io/bcye/structured-wikivoyage-exports:latest
diff --git a/Dockerfile b/Dockerfile
index 4dfe203..161a820 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -1,12 +1,10 @@
-FROM node:22
+FROM ghcr.io/astral-sh/uv:0.6-python3.12-bookworm
 
 WORKDIR /app
 
-COPY package.json .
-COPY package-lock.json .
+COPY pyproject.toml uv.lock ./
+RUN uv sync --frozen
 
-RUN npm install
+COPY . .
 
-COPY index.ts .
-
-CMD [ "node", "--max-old-space-size=4096", "--experimental-strip-types", "index.ts" ]
+CMD ["uv", "run", "main.py"]
\ No newline at end of file
diff --git a/README.md b/README.md
index 02cb2a5..69a1aaa 100644
--- a/README.md
+++ b/README.md
@@ -2,7 +2,15 @@
 
 Small utility to convert the wikitext data from the Wikivoyage dumps into a structured format. The goal is to make it easier to work with the data and extract useful information programmatically.
 
-## Installation
+## Usage
+
+### Docker
+
+This script is intended to be run with docker. A docker image is [available from the GitHub registry](https://github.com/bcye/structured-wikivoyage-exports/pkgs/container/structured-wikivoyage-exports). For example, you may run it using the filesystem handler with `docker run -e HANDLER=filesystem -e HANDLER_FILESYSTEM_OUTPUT_DIR=/output -v ./output:/output --ulimit nofile=65536:65536 ghcr.io/bcye/structured-wikivoyage-exports`. For all the different options, refer to [the docs](docs).
+
+### Types
+
+TypeScript types for consuming the json output are available, you may install them from the [@bcye/structured-wikivoyage-types](https://www.npmjs.com/package/@bcye/structured-wikivoyage-types) npm package. Refer to the included docstrings in [types/index.d.ts](types/index.d.ts) for reference.
 
 ## Documentation
 
@@ -22,4 +30,4 @@ All code and documentation unless otherwise stated is licensed under the AGPLv3
 
 ### Examples
 
-Files in the `docs/example` and `tests/fixtures` are copies (.txt) or derivatives (.json) of the Boston Article on Wikivoyage and licensed under CC BY-SA 4.0. A [list of contributors is available on the original article](https://en.wikivoyage.org/w/index.php?title=Boston&action=history).
+Files in the `docs/example` and `tests/fixtures` are copies (.txt) or derivatives (.json) of the Boston Article on Wikivoyage and licensed under CC BY-SA 4.0. A [list of contributors is available on the original article](https://en.wikivoyage.org/w/index.php?title=Boston&action=history).
\ No newline at end of file


Reviews:
user: login: coderabbitai[bot]
body: **Actionable comments posted: 1**

<details>
<summary>🧹 Nitpick comments (6)</summary><blockquote>

<details>
<summary>README.md (1)</summary><blockquote>

`9-14`: **Fill in the placeholder links to avoid broken documentation references.**

Two Markdown links are currently empty (`[]()`), triggering MD042 and confusing readers. Either supply real URLs or convert to inline code/text.

```diff
-This script is intended to be run with docker. A docker image is [available from the GitHub registry]().
+This script is intended to be run with Docker. A pre-built image is 
+[available from the GitHub Container Registry](https://github.com/bcye/structured-wikivoyage-exports/pkgs/container/structured-wikivoyage-exports).

…
-TypeScript types for consuming the json output are available, you may install them from the [@bcye/structured-wikivoyage-types]() npm package.
+TypeScript types for consuming the JSON output are available via 
+[`@bcye/structured-wikivoyage-types`](https://www.npmjs.com/package/@bcye/structured-wikivoyage-types).
```

<details>
<summary>🧰 Tools</summary>

<details>
<summary>🪛 markdownlint-cli2 (0.17.2)</summary>

9-9: No empty links
null

(MD042, no-empty-links)

---

13-13: No empty links
null

(MD042, no-empty-links)

</details>

</details>

</blockquote></details>
<details>
<summary>Dockerfile (1)</summary><blockquote>

`1-10`: **Consider a two-stage build to keep the final image minimal and non-root.**

The current image ships the full `uv` build toolchain and retains build caches, which inflates size.  
A slim runtime stage (e.g. `python:3.12-slim`) copying only `site-packages/` and source code typically cuts 40-50 % off the image and drops unnecessary compilers.

Additionally, switching to a non-root user (e.g. `useradd -m app && USER app`) improves security out-of-the-box.

No blocker, but worth considering before publishing v1 images.

</blockquote></details>
<details>
<summary>.github/workflows/build-image.yaml (1)</summary><blockquote>

`25-29`: **Tag the image with the git tag for traceability.**

Only `:latest` is pushed; consumers lose the ability to pin to a specific release.

```diff
-          tags: ghcr.io/bcye/structured-wikivoyage-exports:latest
+          tags: |
+            ghcr.io/bcye/structured-wikivoyage-exports:latest
+            ghcr.io/bcye/structured-wikivoyage-exports:${{ github.ref_name }}
```
( `${{ github.ref_name }}` resolves to `docker/<version>`; consider stripping the `docker/` prefix if undesired.)

</blockquote></details>
<details>
<summary>transform-documents.py (3)</summary><blockquote>

`11-12`: **Remove unused import to keep the module clean.**

`Path` is never referenced.

```diff
-from pathlib import Path
```

<details>
<summary>🧰 Tools</summary>

<details>
<summary>🪛 Ruff (0.8.2)</summary>

11-11: `pathlib.Path` imported but unused

Remove unused import: `pathlib.Path`

(F401)

</details>

</details>

---

`57-60`: **Combine nested `async with` statements to satisfy Ruff SIM117 and reduce indentation.**

```diff
-    async with aiohttp.ClientSession() as session:
-        async with session.get(sql_url) as resp:
+    async with aiohttp.ClientSession() as session, session.get(sql_url) as resp:
```
The same pattern occurs in `process_dump()` (lines 170-172).

<details>
<summary>🧰 Tools</summary>

<details>
<summary>🪛 Ruff (0.8.2)</summary>

57-58: Use a single `with` statement with multiple contexts instead of nested `with` statements

Combine `with` statements

(SIM117)

</details>

</details>

---

`18-37`: **`isdigit()` misses negative or decimal numbers; broaden ENV casting logic.**

`"-1".isdigit()` → `False`, likewise `"3.14"`.  
Use `int()` / `float()` detection instead.

```diff
-        # cast ints
-        if val.isdigit():
-            val = int(val)
+        try:
+            val = int(val)
+        except ValueError:
+            try:
+                val = float(val)
+            except ValueError:
+                if val.lower() in ("true", "false"):
+                    val = val.lower() == "true"
```

</blockquote></details>

</blockquote></details>

<details>
<summary>📜 Review details</summary>

**Configuration used: CodeRabbit UI**
**Review profile: CHILL**
**Plan: Pro**

<details>
<summary>📥 Commits</summary>

Reviewing files that changed from the base of the PR and between e578473c9ff494b1670dbacb7acf2b3f99a41a04 and 6f1b9cec19edcd3df984012c12e110e60ebbd86b.

</details>

<details>
<summary>⛔ Files ignored due to path filters (1)</summary>

* `package-lock.json` is excluded by `!**/package-lock.json`

</details>

<details>
<summary>📒 Files selected for processing (7)</summary>

* `.dockerignore` (1 hunks)
* `.github/workflows/build-image.yaml` (1 hunks)
* `Dockerfile` (1 hunks)
* `README.md` (1 hunks)
* `package.json` (0 hunks)
* `split-dump.ts` (0 hunks)
* `transform-documents.py` (3 hunks)

</details>

<details>
<summary>💤 Files with no reviewable changes (2)</summary>

* package.json
* split-dump.ts

</details>

<details>
<summary>🧰 Additional context used</summary>

<details>
<summary>🪛 markdownlint-cli2 (0.17.2)</summary>

<details>
<summary>README.md</summary>

9-9: No empty links
null

(MD042, no-empty-links)

---

13-13: No empty links
null

(MD042, no-empty-links)

</details>

</details>
<details>
<summary>🪛 Ruff (0.8.2)</summary>

<details>
<summary>transform-documents.py</summary>

11-11: `pathlib.Path` imported but unused

Remove unused import: `pathlib.Path`

(F401)

---

57-58: Use a single `with` statement with multiple contexts instead of nested `with` statements

Combine `with` statements

(SIM117)

---

170-171: Use a single `with` statement with multiple contexts instead of nested `with` statements

Combine `with` statements

(SIM117)

</details>

</details>

</details>

<details>
<summary>🔇 Additional comments (2)</summary><blockquote>

<details>
<summary>.dockerignore (1)</summary>

`4-8`: **Looks good – pragmatic additions to keep the build context lean.**

Adding `.pytest_cache`, `docs`, `output`, and `sketching` will noticeably shrink the Docker build context and speed-up the layer cache.  
No further action required.

</details>
<details>
<summary>transform-documents.py (1)</summary>

`213-221`: **Guard against unconstrained concurrency defaulting to “infinite”.**

When `MAX_CONCURRENT` is absent or `0` the semaphore is disabled, potentially spawning one task per matching page (memory issue above).  
Recommend:

```python
DEFAULT_CONCURRENCY = os.cpu_count() or 4
max_conc = int(os.getenv("MAX_CONCURRENT", str(DEFAULT_CONCURRENCY)))
```

and warn if the user explicitly passes `0`.

<details>
<summary>🧰 Tools</summary>

<details>
<summary>🪛 Ruff (0.8.2)</summary>

217-217: Within an `except` clause, raise exceptions with `raise ... from err` or `raise ... from None` to distinguish them from errors in exception handling

(B904)

</details>

</details>

</details>

</blockquote></details>

</details>

<!-- This is an auto-generated comment by CodeRabbit for review status -->
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-26T21:08:51Z
user: login: coderabbitai[bot]
body: **Actionable comments posted: 0**

<details>
<summary>🔭 Outside diff range comments (1)</summary><blockquote>

<details>
<summary>transform-documents.py (1)</summary><blockquote>

`1-246`: _💡 Verification agent_

<details>
<summary>🧩 Analysis chain</summary>

**Verify memory management for large dumps**

The overall approach of streaming processing is excellent for handling large XML dumps without loading everything into memory. However, there are a few areas where memory usage could potentially grow unbounded:

1. The task list in `WikiDumpHandler` accumulates all tasks
2. No limit on the number of mappings loaded from the SQL dump

For very large Wikivoyage dumps, these could become memory concerns.

---

🏁 Script executed:

```shell
#!/bin/bash
# Check how large the Wikivoyage dump files typically are
curl -s -I "https://dumps.wikimedia.org/enwikivoyage/latest/enwikivoyage-latest-pages-articles.xml.bz2" | grep Content-Length
curl -s -I "https://dumps.wikimedia.org/enwikivoyage/latest/enwikivoyage-latest-page_props.sql.gz" | grep Content-Length

# Search for any mention of expected page counts in documentation or other files
grep -r "wiki.*pages" --include="*.md" .
```

Length of output: 341

---



</details>

**Control asyncio.Task accumulation to prevent memory spikes**

We measured the current dumps: the page_props SQL dump is ~1.5 MB (compressed), and the XML articles dump is ~120 MB. While streaming avoids loading the full files into memory, you still:

- Build a `mappings` dict (~a few MB in memory)—this is modest given the dump size.
- Accumulate **one `asyncio.Task` per page** in `dump_handler.tasks` until the very end. For tens of thousands of pages, this can cause peak memory growth.

Consider refactoring to release task references as they complete, for example:

- Batch your `await asyncio.gather(*tasks)` calls every _N_ tasks, then clear the list.
- Use `asyncio.as_completed(tasks)` to process and drop tasks as they finish.
- Or integrate semaphore-based concurrency with immediate awaiting to avoid unbounded task buildup.

These changes will bound your in-flight tasks and keep overall memory usage predictable.

<details>
<summary>🧰 Tools</summary>

<details>
<summary>🪛 Ruff (0.8.2)</summary>

11-11: `pathlib.Path` imported but unused

Remove unused import: `pathlib.Path`

(F401)

---

57-58: Use a single `with` statement with multiple contexts instead of nested `with` statements

Combine `with` statements

(SIM117)

---

172-173: Use a single `with` statement with multiple contexts instead of nested `with` statements

Combine `with` statements

(SIM117)

---

219-219: Within an `except` clause, raise exceptions with `raise ... from err` or `raise ... from None` to distinguish them from errors in exception handling

(B904)

</details>

</details>

</blockquote></details>

</blockquote></details>

<details>
<summary>🧹 Nitpick comments (5)</summary><blockquote>

<details>
<summary>transform-documents.py (5)</summary><blockquote>

`4-14`: **Update imports for better organization**

There are several new imports to support remote streaming and XML parsing, but not all are used. Specifically, `pathlib.Path` is imported but never used in the code.

```diff
- from pathlib import Path
```

<details>
<summary>🧰 Tools</summary>

<details>
<summary>🪛 Ruff (0.8.2)</summary>

11-11: `pathlib.Path` imported but unused

Remove unused import: `pathlib.Path`

(F401)

</details>

</details>

---

`40-74`: **Well-implemented streaming decompression logic**

The `fetch_mappings` function effectively handles streaming decompression of gzipped SQL content, with good memory management through chunked downloading and buffer management. One suggestion for improvement would be to add logging for progress monitoring during this potentially long-running operation.


```diff
async def fetch_mappings() -> dict[str, str]:
    """
    Download and gunzip the page_props SQL dump, extract
    page→wikibase_item mappings.
    """
    sql_url = (
        "https://dumps.wikimedia.org/"
        "enwikivoyage/latest/"
        "enwikivoyage-latest-page_props.sql.gz"
    )
    # decompress gzip
    decomp = zlib.decompressobj(16 + zlib.MAX_WBITS)
    # regex for tuples: (page,'prop','value',NULL_or_number)
    tuple_re = re.compile(r"\((\d+),'([^']+)','([^']+)',(NULL|[\d\.]+)\)")
    buffer = ""
    mappings: dict[str, str] = {}
+    logger.info("Starting download of SQL mappings file")
+    mapping_count = 0

    async with aiohttp.ClientSession() as session:
        async with session.get(sql_url) as resp:
            resp.raise_for_status()
            async for chunk in resp.content.iter_chunked(1024 * 1024):
                data = decomp.decompress(chunk)
                if not data:
                    continue
                text = data.decode("utf-8", errors="ignore")
                buffer += text
                for m in tuple_re.finditer(buffer):
                    page_id, prop, value = m.group(1), m.group(2), m.group(3)
                    if prop == "wikibase_item":
                        mappings[page_id] = value
+                        mapping_count += 1
+                        if mapping_count % 10000 == 0:
+                            logger.debug(f"Processed {mapping_count} mappings so far")
                # keep tail to handle split tuples
                if len(buffer) > 1000:
                    buffer = buffer[-1000:]
    return mappings
```

<details>
<summary>🧰 Tools</summary>

<details>
<summary>🪛 Ruff (0.8.2)</summary>

57-58: Use a single `with` statement with multiple contexts instead of nested `with` statements

Combine `with` statements

(SIM117)

</details>

</details>

---

`57-59`: **Simplify nested context managers**

The code uses nested `with` statements for the HTTP session and request handling. Modern Python allows combining these into a single statement for better readability.

```diff
-    async with aiohttp.ClientSession() as session:
-        async with session.get(sql_url) as resp:
+    async with aiohttp.ClientSession() as session, session.get(sql_url) as resp:
```

And similarly:

```diff
-    async with aiohttp.ClientSession() as session:
-        async with session.get(xml_url) as resp:
+    async with aiohttp.ClientSession() as session, session.get(xml_url) as resp:
```


Also applies to: 172-174

<details>
<summary>🧰 Tools</summary>

<details>
<summary>🪛 Ruff (0.8.2)</summary>

57-58: Use a single `with` statement with multiple contexts instead of nested `with` statements

Combine `with` statements

(SIM117)

</details>

</details>

---

`75-155`: **Well-designed SAX handler with proper state management**

The `WikiDumpHandler` class is well implemented with clear state transitions and efficient text accumulation. The concurrency control using a semaphore is a good approach to limit resource usage. Two small suggestions:

1. Consider adding a progress counter to log the number of processed pages
2. Add handling for SAX parsing exceptions in the processing methods


```diff
class WikiDumpHandler(xml.sax.ContentHandler):
    """
    SAX handler that, for each <page> whose <id> is in mappings,
    collects the <text> and schedules an async task to parse
    and write via the user‐supplied handler.
    """

    def __init__(self, mappings, handler, max_concurrent):
        super().__init__()
        self.mappings = mappings
        self.handler = handler
        self.sem = (
            asyncio.Semaphore(max_concurrent) if max_concurrent > 0 else None
        )
        self.tasks: list[asyncio.Task] = []
+        self.processed_count = 0
+        self.scheduled_count = 0

        self.currentTag: str | None = None
        self.inPage = False
        self.inRevision = False
        self.inText = False
        self.currentPageId: str | None = None
        self.currentText: list[str] = []

    # ... [other methods remain unchanged] ...

    async def _process(self, text: str, uid: str):
+        try:
            parser = WikivoyageParser()
            entry = parser.parse(text)
            await self.handler.write_entry(entry, uid)
+            self.processed_count += 1
+            if self.processed_count % 100 == 0:
+                logger.info(f"Processed {self.processed_count} pages")
+        except Exception as e:
+            logger.error(f"Error processing page {uid}: {e}")
```

---

`156-184`: **Efficient streaming XML processing implementation**

The `process_dump` function effectively combines streaming download, decompression, and SAX parsing. The chunked approach minimizes memory usage while maximizing throughput. The error handling for HTTP responses is good, but there's no explicit timeout handling for network operations.


Consider adding timeout configuration for HTTP requests:

```diff
    async with aiohttp.ClientSession() as session:
-        async with session.get(xml_url) as resp:
+        async with session.get(xml_url, timeout=aiohttp.ClientTimeout(total=3600)) as resp:
```

<details>
<summary>🧰 Tools</summary>

<details>
<summary>🪛 Ruff (0.8.2)</summary>

172-173: Use a single `with` statement with multiple contexts instead of nested `with` statements

Combine `with` statements

(SIM117)

</details>

</details>

</blockquote></details>

</blockquote></details>

<details>
<summary>📜 Review details</summary>

**Configuration used: CodeRabbit UI**
**Review profile: CHILL**
**Plan: Pro**

<details>
<summary>📥 Commits</summary>

Reviewing files that changed from the base of the PR and between ff662c3ac5e89af5f3b9a5618ca252af52e1a6c6 and 789da95e49883b771d3d620b7467fc0c9bc21d78.

</details>

<details>
<summary>📒 Files selected for processing (1)</summary>

* `transform-documents.py` (3 hunks)

</details>

<details>
<summary>🧰 Additional context used</summary>

<details>
<summary>🪛 Ruff (0.8.2)</summary>

<details>
<summary>transform-documents.py</summary>

11-11: `pathlib.Path` imported but unused

Remove unused import: `pathlib.Path`

(F401)

---

57-58: Use a single `with` statement with multiple contexts instead of nested `with` statements

Combine `with` statements

(SIM117)

---

172-173: Use a single `with` statement with multiple contexts instead of nested `with` statements

Combine `with` statements

(SIM117)

</details>

</details>

</details>

<details>
<summary>🔇 Additional comments (1)</summary><blockquote>

<details>
<summary>transform-documents.py (1)</summary>

`185-233`: **Well-structured main function with good error handling**

The main function is well-organized with a clear step-by-step approach. The refactoring to use asynchronous processing is a good improvement for handling large data efficiently. The error handling for module loading and parameter validation is thorough.

<details>
<summary>🧰 Tools</summary>

<details>
<summary>🪛 Ruff (0.8.2)</summary>

219-219: Within an `except` clause, raise exceptions with `raise ... from err` or `raise ... from None` to distinguish them from errors in exception handling

(B904)

</details>

</details>

</details>

</blockquote></details>

</details>

<!-- This is an auto-generated comment by CodeRabbit for review status -->
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-30T11:36:08Z
user: login: moll-re
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-04-30T19:25:27Z
user: login: moll-re
body: Yeah, straightforward. In the long run I see two potential improvements:
- we could migrate the code to a `src/` directory and selectively copy files instead of excluding them in the `.dockerignore`. That would be a little cleaner as the repo grows.
- we could try to have the final container be a distroless container and relegate the package installation to a separate step. It's obviously overkill but I think I'd like to try that when I have a little time. 
state: APPROVED
author_association: COLLABORATOR
submitted_at: 2025-04-30T19:31:07Z
user: login: bcye
state: COMMENTED
author_association: OWNER
submitted_at: 2025-04-30T20:00:13Z

Review comments:
diff_hunk: @@ -0,0 +1,132 @@
+#!/usr/bin/env python3
path: main.py
user: login: moll-re
body: Does this make sense? Did it make sense in a non-docker environment?
created_at: 2025-04-30T19:25:27Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,132 @@
+#!/usr/bin/env python3
path: main.py
user: login: bcye
body: I think it's probably good practice, it'should enable you to just ./main.py in the console instead of typing python main.py
created_at: 2025-04-30T20:00:13Z
author_association: OWNER

PR comments:
user: login: coderabbitai[bot]
created_at: 2025-04-26T21:05:52Z
body: <!-- This is an auto-generated comment: summarize by coderabbit.ai -->
<!-- This is an auto-generated comment: rate limited by coderabbit.ai -->

> [!WARNING]
> ## Rate limit exceeded
> 
> @bcye has exceeded the limit for the number of commits or files that can be reviewed per hour. Please wait **23 minutes and 47 seconds** before requesting another review.
> 
> <details>
> <summary>⌛ How to resolve this issue?</summary>
> 
> After the wait time has elapsed, a review can be triggered using the `@coderabbitai review` command as a PR comment. Alternatively, push new commits to this PR.
> 
> We recommend that you space out your commits to avoid hitting the rate limit.
> 
> </details>
> 
> 
> <details>
> <summary>🚦 How do rate limits work?</summary>
> 
> CodeRabbit enforces hourly rate limits for each developer per organization.
> 
> Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.
> 
> Please see our [FAQ](https://docs.coderabbit.ai/faq) for further information.
> 
> </details>
> 
> <details>
> <summary>📥 Commits</summary>
> 
> Reviewing files that changed from the base of the PR and between 789da95e49883b771d3d620b7467fc0c9bc21d78 and 1a5b9b44e085ffd0db18139ea8792ebb611a6870.
> 
> </details>
> 
> <details>
> <summary>📒 Files selected for processing (2)</summary>
> 
> * `Dockerfile` (1 hunks)
> * `README.md` (2 hunks)
> 
> </details>

<!-- end of auto-generated comment: rate limited by coderabbit.ai -->
<!-- walkthrough_start -->

## Walkthrough

This set of changes transitions the project from a Node.js-based implementation to a Python-based one, updating the Dockerfile to use a Python environment and modifying the main script to process Wikimedia dumps directly from remote sources. The build process is automated with a new GitHub Actions workflow for Docker image creation and publishing. The `.dockerignore` and README are updated to reflect the new structure and usage. Node.js-specific files and logic are removed, while the Python script is refactored for asynchronous, streaming XML parsing with concurrency control.

## Changes

| File(s)                                      | Change Summary |
|----------------------------------------------|----------------|
| `.dockerignore`                              | Expanded and reordered ignore list; added new entries; removed language section comments. |
| `.github/workflows/build-image.yaml`         | Added new GitHub Actions workflow to build and push Docker images on tag push. |
| `Dockerfile`                                | Switched from Node.js to Python base image; updated dependency installation and entrypoint to use Python script. |
| `README.md`                                 | Updated instructions for Docker-based usage; added section on TypeScript types for JSON output. |
| `package.json`, `split-dump.ts`             | Deleted Node.js package/dependency file and the main TypeScript processing script. |
| `transform-documents.py`                    | Refactored to process remote Wikimedia dumps asynchronously with SAX parsing and concurrency control; removed local file processing logic. |

## Sequence Diagram(s)

```mermaid
sequenceDiagram
    participant User
    participant Docker Container
    participant Python Script
    participant Wikimedia Server

    User->>Docker Container: Run with environment/volumes
    Docker Container->>Python Script: Start transform-documents.py
    Python Script->>Wikimedia Server: Fetch SQL dump (gzipped)
    Wikimedia Server-->>Python Script: SQL dump stream
    Python Script->>Python Script: Extract page ID mappings
    Python Script->>Wikimedia Server: Fetch XML dump (bz2)
    Wikimedia Server-->>Python Script: XML dump stream
    Python Script->>Python Script: Stream, decompress, and parse XML
    Python Script->>Python Script: For each relevant page, process text with handler (concurrency limited)
    Python Script->>User: Write output files/streams as configured
```

## Assessment against linked issues

| Objective                                                    | Addressed | Explanation |
|--------------------------------------------------------------|-----------|-------------|
| Adapt Dockerfile to run the Python script instead of Node.js (#1) | ✅        |             |

## Poem

> Hoppity hop, the code takes a leap,  
> From Node to Python, no secrets to keep.  
> The Docker now hums with a Pythonic tune,  
> Streaming big dumps by the light of the moon.  
> With workflows and docs all shiny and new,  
> This bunny says thanks—hip hip, yahoo! 🐇✨

<!-- walkthrough_end -->
<!-- internal state start -->


<!-- DwQgtGAEAqAWCWBnSTIEMB26CuAXA9mAOYCmGJATmriQCaQDG+Ats2bgFyQAOFk+AIwBWJBrngA3EsgEBPRvlqU0AgfFwA6NPEgQAfACgjoCEYDEZyAAUASpETZWaCrKNwSPbABsvkCiQBHbGlcSHFcLzpIACIAEXwGAGtKeAAvEgAaaPtsAWZ1Gno5SGxESkgBBlkPNGRbSAxHAXKAJgBGDJQMXApFbAZpSHik8odubnwKUIAzSbDYD158ETENGAQ6uyVuMlpkfCxMengMBi9sJWQmJUhp3uZ0Hn8JeHxS6zsACjM2gA4ASnQGHo2mYyAIjC8+DKKEQDkGPyB9DMAFY1u5IJcGBR4NxxAcGvgaODYNQhglknwCbgFgpujiBHhJgByZDMNAMBDkSCk5BQgDuJS88HyoTKuHEGCIMnkSmmaG8uE6/IQnIU3hB02molCNJq42FDGorwwrJ4lFmFHZpw8c2m2FO+IwaGFuFk6Pw6FotH8cPmSE6esgNgAogBBWIAWRDPNqFRIZBK3Fo1CiEJOZwuHhOiB6/SdyEtfgdGBORHmHmGlJQ7NIkH56lgPPgRAWfG8IvU9hIErLiE6iB2DHg03gRp88nFzdb5RpmArDSa5Xw0x4aFIyCWAzhdDWAElQqgXYhPWxuia06TdW3s8gMJ65QqvKEO6Lu72pegDWOVJFICQAA8BjxW45iDUdIkQWRcxIB5SWBSIKHRWl6iNLBmj8EgXhIfkohOTEe20XwySDJYXiUegAFUbAAGS4WAJW4RAOAAehYohG1yDQmGYFjKmqFjcwofNsH8WgwAbRJJHwWR1xIMBAImKZEBY7hvC8Fj2g0cxLAAYRYM9cGQBwnBcNxaVKcp+I8KgkCiNA6QGChDgEN5rw8eoM3OWgy0YeCN1ue5+D1PgdmBXzbGQI50EQZJ6CLNBaBeAZ+CwWB8EFCFrTk/g+HSrx6BXVceMM3zqXSmEXX8RL5GeeAcKiTlMA3fcsF9CYMDKQNaQVAh2UKGLECQXBMFCa5lFUdRtCefByKiMopCoXx8DxE1CzmeDaGFT8HDVIbcGwY0DmYow2jWAAqc6rGoJsADF4GfShmMuyAw29XyHIAA24xQJrUTRZOYLxPrpUciFEo6sAgjwISDfwJn2yZ5AhQDMxuQdRBHMdbgewY5h8/wxEmerCyCuqcM6F0BV8spIjESRbXyOETX4VdodJlhgrbD5EG0gwWguq67FiaRsVxJ0hngQnxCkDgXr3U5vN8z6AAFxqoSaRp0Ft738EH8KDeosRxVbqU9WLcQXcnBRXBd2CljybD5gBmQXrsspFg2kRwPH0pxgWe87IEoobP1V9WVH+6buAVMp9cOD4FFYdgwk9GhmCU5wHvkUln0w7D+Si4FIHD36NajnRfR9kGIV9EapgrZg+YAFkFgBlEg6ZljwbCw+rBT961aDloPe6CEJfOt1L7CHLGGBxyCKnkEr7c/BsaRLtWy8jqbK77nDIGADHh1HBgwGhvRPs6WYGFKXztToAQOUSaeY6mMdvGcNdlNZhdbD5/SShy671ZtqMSg1hqYBSvhcUkpyzYG4I8Y+c9QYtghhLIsNJUD1CfmUQqWA9QwhWgWPmbdchlHHt0LwshOjWUwuPGCIJ879xnpjU+VN5C2yDJ9KsFpcYg2ip9UMEZowaGYLQEG7M1iAL+iA62URXIb3Zp7Mi8AKIES1pEeKCZaBPySBwIwUBLq8IoNDL2BcR7rErBSPhf5UC4R8GAISIlwHRVmD4DKMgQhPA5OIbc+4XxlBJB4SiAA1NcSQcrZVIHwdeTYrCyBpASV2bQWjTxFmoTAnRiGdnSDIbAD16BGk5L5YoZQ37Gk/NsXYZAqhdFzFTSGgVObjQUNwWQZZOiwW4FCaCC5PoQDuPgdIGBJFeHXKnf8nVRI2RIEsWg/R4ACD/IyQp/ZPbshOFrTqjwzgkHnNFIS2hWy4EtPyZwRSDjbNGD2BBfMjHnWEVGEMYj6C9wsS9DETzoxdDRoMIM5BtyIGcPIExJRgV1loAkH23RIYDn6E2OMOY8z02OqBPgwkMClk/EGMFTBYUnEoF0gCaAM5/hKkcZAcSpkvF6BgQykAJBZ1/IMaKEh8DnDYJAZgbxzxSkpsXe8xJp5BhFEpEa3QhSdjFD2OBUU2T2wOHhAhKEtiixNk6AB29NbTWPJ6IVl4SK0m+TGfF2zkDdLdJAbaiR1p8FxTYvgIocrRSDNAWQOw25ixAm6HYm5n5yWVKqJsiB0oanjAvLRXR4yWg8GwCgHEcWejIA4fw4KnqMHnPsoa1D0AMCBRWGEvo3gUH8ZAe6zoJydAjjqnQRVKBpnNtgIgG4xSz1PkmFMwqUaNnKGYkcBEhrgOKNZe5YBDAGBMFAXYv9eqEFIOQKgA0V7dC4LwfgwgdQMxlAoIBO9NDTX0JO8AUA4CoCPIcJkxAyDKGXQZdgXAqCChMuyFwS9d2yIPToI9xgT1GA0FCkYOIiC6xIAY6IEGDAWFenua9i7Uz0BfSC3+TUpTSHMh4b6gHKQ60mCQSRuN6xxgQV2xtmFJhAM9opV1tJhS5l/rh8ByjooEx1MTaQyFMP3iUAAfR5QsyCIN2BvvOWyWaUQhkPAciugaZRUUEM9A5VjYgpk9HdK9d6ToXSYgdkTHErLi7KNwmmxK6jYa0kYx4OjnAS4aHacSXAPHikLCviXQDiBXOfTcmpXArnBGxR7CUqU30rFPD7m8K497uhXFTEQYmqRfLCZJu+8ZUpDp1k+J9SwAA5AA8rEEMnnLBWAAJrQAABK5ey59QExnZk8qkLQAV9AgxJdZWm+8gprNRDiW5bs8meT7KAbzIwOlXqPSXWtSZQYlBnGcJDfYq5FKTAGnMNSyzsb23EOhgw2WlUGAg9EQxpgDAaA4jSXILF+STESNMAUKlVkFTAM60gGhAZeHA5B6DYZYMLtvfNRwr7OHFX8jtsMDQD4AHF1DldyK9eTVKbt3Yyg0UlURogACECkFXJEByAe5awkGyOe+kfQBi0ACWEYDMT9hYDUqG/5ad1xsmoEF8spFqA0GcjEbDlAWLnWiJx+sSOBQEVHOQKK9gyx/iEIITNPh0e84oNkOcoRMW04XOMhzwdGTdGwDSqWBxDJC9lwIWMm5KBMyGmiqcEIqYo/5DiOBkyY6RI3J0aqPlPzw2hOoJGdIaDRea/WJ3H0sB7liKnZIGATdy/xUNXMi3QKiXsDQJiXBOSiCkp+PrcM5m+4IG+8aABua1+AW2+S2WnWk0PcCw7N/pAli6vYcSEvIO+OLaS3woP4SVtf695sL2abEdB7bHk9g5Ag0fU94eOFgOT/gjKl9gb5BBuPqxY8KQBdFXomU2noGCx7RS0AxzUK6EmpfopH7D/QenXJywOTBS9mG65SAgmQNEVs2INCvD4lUEggkKKB0YkEk8AUk7KskpACkAE4qzEWuIQKu1eHg/ecOje1yfAvcreamQuR+YAd+g+LMqA+KYMMyLWno+BoqhONYbAPkqYuaaA0wXOFQ2OXuRAfMY2YYE2C202tIs24yk2aKtsy2UwUQa2uQhoqm6gJMhikAe2/4MBK2ohoU4hm25422UUaafBzgSh3KigWMdApeWCVwoOXoewjw5AgoKBZuYYCOwuFAt2ouxBaCAhUMhG3052sAl2129hyOhcfELBz2hOb2pKwM2kh2x2YARgJi0Mn2R232v2N6S6AOpkwOfkzUO2GIuC2YVBomaRaG8UQUDke2SgGgQgyALQaSz+kyDkCSSSWAKSLQYAWRxwORjY/S2AEgIMmK4gXKnwi01uWAAADBoAAGyAgEgZLwDzgY74D4CJDeFNyhau6JBRKYByQMqhojgDSSayG/SlH+pu4eDKKZbLFyT7EHACLFyfSnFQFQhJDnGjKAiCIYDcAPDIpUw1yei1HpR04Bp1jHHXEeq9ArCaB9TAyeyfQdEaB3GJA1b1htFfQdH2CyCnC6Dny9AjIgwUrFzpidQjQ+AERhRKCnAkxC4niiQpRMDtK+S5Hx4cpqIIZdB25S5Sh/ifS6S5alaQBrAaBYkGTRTIpDYoYrQdKfgnA+TkSHS+BSKhZmraCLpmgNJTBr7YlFKg4FGcyYrYoP67ElFlH2DeqhCZbimAQaBGRwnUrIIdrFFHHjLSiTJakfTWCJI/EGkaqQCZY9CYCICWjMBgCAYwpGR2ayBwkd7s60iQkSDFijJJxDzsFQaWCcFc7cHmYeBaEuFJ7CGrbKEbbzxbbSEGBQByGoYBQowKEiGFQ5kSH5nSDMgERzYZn1gNrcqJTZgqqoDRG4xhGQaFknYmqvKxEJkwZwb/aIaA7Ia2wlkZHGrhjPJEbIAkaMm1xzLjIpRBjRAKwNI+CQzZByYSzUoOQ8qaGES4z0DRAhxyS7lboHDISoAWH9YSxeRZjIDIrCS2FFiOmd4eCICGngq+QmIDjtqinlhkC0pG4pxMo4gspFz0Dsqcpxq8pGQ77eZ4CxgISdKez+BgK1L/K8HQqGRNIJSaYmjaY5LHRrBvQ+RaZVrmEHzRDup+q7nkLXlYC5GmZ0CdAnC+mr5lAUBRSuSoVBhoBMoPQqAPTqCpEMUkBerum+qDCYK0gABSbcVW/AeAPmnFisFwleeJjSj5eJb5BYO+AKrxESKxdY0UDkEwWys4SBvy3kaYHqaZJA4uUhaK1K7mPQfYQuNqPBHgf2S0mI+F7ATS/gmyOyDoU5FOo2Q5SZt6U2qZ9Z/B3BQh5Z2ZnguZkh6hRge25AB2PZEAkRBgNxJADxg58RI5SRY5KRKGoOiAGGkhaa1xfxpVZRFxC8HguRSgkQhQt5hYhGvA4WpQuacohKLWtISwIJMUTwkgqYnsVp9U41iwLVLZzoMSjKT0LMnw3ALxDwAAnBoG0K7GkgeawifNjKGmgCiKkubrAP8JTt1vQEoFIFCK8SnNUsCLUslk+awWEE5WLicG5TskWDafsQOGgABMHg6AIAltwI0UJPsswJTFSp3MRJuIvttnwK6k5T+RqpTnqkldofQN0SKM5USV9XjCqt+RDZ7NDbDfDT0IjWZXJLzKFmFbNKRauEYR1f+MKPkM6MKlTNyoRF2g5CxnMjUqcLKKIMlUZQpYsMCTqPGd9lwUZYlemSlUtmlToettWWoQWblaVflUdr2UVYOK6P6Y4NwGaSdOERVQFYyUhm+pOXVQ1d1T2BJoRmKpEIZPZNyoqPAE4mnjNAWhCNqLgJyJ0G/F1PNcJWmdQA5DsQAOpgGk20GYhW2s0Hg4x8UzCBYLBMJECw07D0BtwACKNEGdGc/Ai0kA5W0A0AVgbcA4jNaOIIxcs2LAg1O4xwSoVGAEXpYgUQ7I4wfYzSDwMcdYEe4InoKdUkWRKA6c+OsQO60dvk5dldOYlAYoI06c7AVKCJmEpAAElOeoWAo15ATCCwXgOwfA9ojoLMEIwKUgYQgEoQotkydx2mRmjY+EDkC1oh6lqF7iQC1a1Uzu4EHKlGA6gK0gwKLgFFUuZK35sEJ+6Uaac2fouRP1ZGfNXY+K3eveVQ3UbwrYi985hIoQviDMualkc+C44VtwAofM2Wb9nFedEdBdjwMNjRPE3deCkAAAGpGJXQstXeJnwPXY3c3antVDQZ7J3RnL6AI+oMHtHXhIEk6W3GGII7I4jb5MI5Xeo0hEsc4LxVTs/JeB4IQynIYwHm/ZMm/VQCppPR4NPZ7HqFLINAkFMQNDQAPfY5KjsUGHY84mIDMmsLdHMPsmqK45xauIeSftwIlgBMNBJkFK1s4MKOUBvVXdwOw/YHHUtWuHWP47qIpmXuODzc6PIwwUwUGEwD3tIB1L9XPUsrUNmEvRHvCRvF9BoJJPAGaQPUJgPSmiaELsY2hVtFEBLn48zvNbvTDFQJ1EDTFlhepMjIpvmo4J/DQK/QEwcLmsE7SP4JEHvqEHY53LBPvXzBiIw/fQNpMJnkcn4wsDCDBExN2JQuIBwlwOHWzi2aPVKOsmfSHXA75CEyI3k8HlCBXp+EsEQMo1Rj3pMKzRiAi1QA8IBOoPZIweUHwz1eM5ACGCi3lEcNtOWNg8g77fFHMOQLgN4S/P4AwkZC3XI/+KS+sm4oRitPFcdMqAieML0LwL47MiCSzHMLC4mmwaFhI0LWar0NKaLmxVBKcLAHShFiovcHZE0R001lMqmjTK3fzQ/sXBmGFSFb4FuOCznquPwZClbR1QfUkqhVCIlBCwsFi+eKc/IFXsLUee6DFSrcmWrXZRrUZaleKjrSoXmfrTtlACGN60cW4ebeoJbRnDbSDO7YUB7rBLNB9ASVmdG5lQ80ZdFJg0EgYpALoCXP87AJGEk32J8P8FwFYNq2UMAL3I07QEfF5fyrI2WHoJfAYNW1AJ9M/SQNAPgDRAkC6PdJEJ8NDDU2BgO/26LVwEJGWC29YO2yQMAOymosO6OyXB3CPeg/hpCLUM602PHkAXMIIhyEEA7M2627u8AM25AAALx6CMr4CHufQjs1vXG9BAqGOfAj3JPAtcBdsUa9s4j9ubtSh6DbttssB2T7t/u0BHtAfhUvs7tocdsHtYcAfhGm1GBemdS+n+nBXRbBnlWJkJHwaybjnO0g7pH1UGAYitK5FYW+Kz4OkrkchWazvSkDUgc2uUttEI2ko37fyhw6ns17NtM0FTFCNQtiPW1LF1SavAcJBwM8bQySKRVOhBpjhNjqNz4SlqJSmVPaZlNOvKhxjs2NaIP3m1AomcgavvClssyfR1t8aNvAsgxsXehkZQr8gYBuvt1PWiBd1IsORF24gl2QC5MafEqD3O6uPL2TJtML04sPAQdj1hlH2mmvQQ6CjaO6N2ObSISXt+ifRtOxBW3lbkuUDBdIqk4LLk4u5mMwy0gw24i8NxdwNRB2Mad6OkqdBKA0AuMs09NNjuMGy0iFfAuUzbPMC7N9ckBeP2dmrsDB4/kF3eA1CquecHCatr2fjRSO5SE4q1C2rzeOSiREPLxXKKuMqqf/2oPcDntC66dAo8YadGcP0EjSdggLh8PKOjfqdW1TexdKMjcvmHinAWuwo0VuI6JI9+Upc6NyeUB/fhXA8DY8c4RO6B6TK0Aokyfji5pRcLgoWhA1dEq3D50MOBfSjdSJjWt+iQuiOOvUoEPPe1LyB4OsuYQ9IcgesK1DV8gic6bSz+7nJeAvzct2IhssywtjhrAztwvljwyrlRAiuSoNJ73RY76UC9BY1msYCzDC1wgs1C6C9NNS3vjO6oB0Of3SpIOKiYAkAXdmMQv3dou0hysEkKscpMMo6bF4sakPBf2id/jc/yeTJg+Tx5t7PjdJ80kImnfqvnfvCVdyeycjSxQGlHcUvK2Jmq0JVhsy3zYRta1RuVkZV63hAFlQC9wNYSbGcmhcCfTueolyhgtwgGe4yfAnA+aj+RCts3SdBM8UCAjjq7HkCSJBSfQUc+mTB+kBmGS8ztIAdQCUXd8g8YB98D/zxD/+crfSgftL8+RiAADaQkbLAAuvHCXBv1Rzvzc/v4WRpuohWz74VtIAjXZrq1woCfAAIQMDQMChPpoF2ALXBCJQH+Dv91+yzTflaGo63xd+wZYLm0TYBJI9gVbIDjxh4yA1HMPGT4LTGmCdBr+6yefrQIhpOYDgNjboCgMA5jslSuAEMD7XYBUDO4NA1HGwEpgSg+K7A49p9F2A8Drm3QfgV4EEHLtxBQHJqM4y5yIA5BggvbmwIA4SDz+YuSADxiT4aDAwb9Ddj0E6AFJh4sjJQWOz0FD8eMAlT6rQEMHic4Qxg/ZjZmf4lA1EZghfgf3/7H95MZ/U7voKT6A8ra4HdnsxB0yP8vBQkF/nP3AGMCAIzA04EL1XRMkUB0aNAd6S/40cgyv/KAJGD0KjhAhToYIR530G4csh+EHIZRy35YDAye/EMo9yipl4OI88CEHTwYEs9OGgLSDhzy1Z6c/QinDwGN355tEneL3APO90FKJRf48fHmlnylDdkTaP6KdFMkKgJMr0DtO9MnAyFPocgNVYoDWn+haBv046Y9BAFjLqAyBewHjPImcFcC0SE6DYSMWmBtABAe1AYAwDaB7U6ADAWgM7FoDTA9qvwZuIMVSS/CWgJANoG0EGIkARiCI1QLQF+AjEzcR6DYbQAADsKIQEb8A+HNxfgaAZ2OiL+DYjBitAOESMSNDOxm42InEQSPzRGhm4FRGES8KuFQAtQIxEYi0AYDOwOQKIEgL8D2oMEUQ0wZ2F8KuojE/gRoFoCiBaBiiYRbQNADSJGIciNh2IkUSmD2pCjm4YI34JKOxHYi2gQI2gLyMGICBsRzcEYtiOmAMBBiDAPapUHaCMiORnIm4Y5jUSIAHh+8XCM4JnT6AgAA -->

<!-- internal state end -->
<!-- tips_start -->

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

<details>
<summary>❤️ Share</summary>

- [X](https://twitter.com/intent/tweet?text=I%20just%20used%20%40coderabbitai%20for%20my%20code%20review%2C%20and%20it%27s%20fantastic%21%20It%27s%20free%20for%20OSS%20and%20offers%20a%20free%20trial%20for%20the%20proprietary%20code.%20Check%20it%20out%3A&url=https%3A//coderabbit.ai)
- [Mastodon](https://mastodon.social/share?text=I%20just%20used%20%40coderabbitai%20for%20my%20code%20review%2C%20and%20it%27s%20fantastic%21%20It%27s%20free%20for%20OSS%20and%20offers%20a%20free%20trial%20for%20the%20proprietary%20code.%20Check%20it%20out%3A%20https%3A%2F%2Fcoderabbit.ai)
- [Reddit](https://www.reddit.com/submit?title=Great%20tool%20for%20code%20review%20-%20CodeRabbit&text=I%20just%20used%20CodeRabbit%20for%20my%20code%20review%2C%20and%20it%27s%20fantastic%21%20It%27s%20free%20for%20OSS%20and%20offers%20a%20free%20trial%20for%20proprietary%20code.%20Check%20it%20out%3A%20https%3A//coderabbit.ai)
- [LinkedIn](https://www.linkedin.com/sharing/share-offsite/?url=https%3A%2F%2Fcoderabbit.ai&mini=true&title=Great%20tool%20for%20code%20review%20-%20CodeRabbit&summary=I%20just%20used%20CodeRabbit%20for%20my%20code%20review%2C%20and%20it%27s%20fantastic%21%20It%27s%20free%20for%20OSS%20and%20offers%20a%20free%20trial%20for%20proprietary%20code)

</details>

<details>
<summary>🪧 Tips</summary>

### Chat

There are 3 ways to chat with [CodeRabbit](https://coderabbit.ai?utm_source=oss&utm_medium=github&utm_campaign=bcye/structured-wikivoyage-exports&utm_content=21):

- Review comments: Directly reply to a review comment made by CodeRabbit. Example:
  - `I pushed a fix in commit <commit_id>, please review it.`
  - `Generate unit testing code for this file.`
  - `Open a follow-up GitHub issue for this discussion.`
- Files and specific lines of code (under the "Files changed" tab): Tag `@coderabbitai` in a new review comment at the desired location with your query. Examples:
  - `@coderabbitai generate unit testing code for this file.`
  -	`@coderabbitai modularize this function.`
- PR comments: Tag `@coderabbitai` in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
  - `@coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.`
  - `@coderabbitai read src/utils.ts and generate unit testing code.`
  - `@coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.`
  - `@coderabbitai help me debug CodeRabbit configuration file.`

### Support

Need help? Create a ticket on our [support page](https://www.coderabbit.ai/contact-us/support) for assistance with any issues or questions.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

### CodeRabbit Commands (Invoked using PR comments)

- `@coderabbitai pause` to pause the reviews on a PR.
- `@coderabbitai resume` to resume the paused reviews.
- `@coderabbitai review` to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
- `@coderabbitai full review` to do a full review from scratch and review all the files again.
- `@coderabbitai summary` to regenerate the summary of the PR.
- `@coderabbitai generate docstrings` to [generate docstrings](https://docs.coderabbit.ai/finishing-touches/docstrings) for this PR.
- `@coderabbitai generate sequence diagram` to generate a sequence diagram of the changes in this PR.
- `@coderabbitai resolve` resolve all the CodeRabbit review comments.
- `@coderabbitai configuration` to show the current CodeRabbit configuration for the repository.
- `@coderabbitai help` to get help.

### Other keywords and placeholders

- Add `@coderabbitai ignore` anywhere in the PR description to prevent this PR from being reviewed.
- Add `@coderabbitai summary` to generate the high-level summary at a specific location in the PR description.
- Add `@coderabbitai` anywhere in the PR title to generate the title automatically.

### CodeRabbit Configuration File (`.coderabbit.yaml`)

- You can programmatically configure CodeRabbit by adding a `.coderabbit.yaml` file to the root of your repository.
- Please see the [configuration documentation](https://docs.coderabbit.ai/guides/configure-coderabbit) for more information.
- If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: `# yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json`

### Documentation and Community

- Visit our [Documentation](https://docs.coderabbit.ai) for detailed information on how to use CodeRabbit.
- Join our [Discord Community](http://discord.gg/coderabbit) to get help, request features, and share feedback.
- Follow us on [X/Twitter](https://twitter.com/coderabbitai) for updates and announcements.

</details>

<!-- tips_end -->
author_association: CONTRIBUTOR
user: login: bcye
created_at: 2025-04-26T21:11:38Z
body: @coderabbitai this includes changes from other PRs, is there a good way to hold off on commenting on those/managing this? as those are already reviewed in the respective other pending PRs.
author_association: OWNER
user: login: coderabbitai[bot]
created_at: 2025-04-26T21:11:58Z
body: <!-- This is an auto-generated reply by CodeRabbit -->
> [!TIP]
> For best results, initiate chat on the files or code changes.

@bcye Yes, there are a few ways to handle PRs that include changes from other pending PRs:

1. **Path Filters**: You can add a `.coderabbit.yaml` configuration file to your repository that excludes specific files or directories from review:

```yaml
# .coderabbit.yaml
reviews:
  path_filters:
    exclude:
      - "path/to/files/from/other/PRs/**"
```

2. **PR Description**: Add `@coderabbitai ignore` to the PR description to completely skip this PR's review.

3. **Commands**: Use `@coderabbitai pause` in a PR comment to pause reviews temporarily, and `@coderabbitai resume` when you're ready.

4. **Selective Review**: Comment with `@coderabbitai review <specific-file>` to only review particular files you want feedback on.

Would you like me to help you set up a specific configuration for this PR?


<!-- <agentComponentType>chat-answer</agentComponentType> -->

<!-- <rawResChunk><answer>@bcye Yes, there are a few ways to handle PRs that include changes from other pending PRs:

1. **Path Filters**: You can add a `.coderabbit.yaml` configuration file to your repository that excludes specific files or directories from review:

```yaml
# .coderabbit.yaml
reviews:
  path_filters:
    exclude:
      - "path/to/files/from/other/PRs/**"
```

2. **PR Description**: Add `@coderabbitai ignore` to the PR description to completely skip this PR's review.

3. **Commands**: Use `@coderabbitai pause` in a PR comment to pause reviews temporarily, and `@coderabbitai resume` when you're ready.

4. **Selective Review**: Comment with `@coderabbitai review <specific-file>` to only review particular files you want feedback on.

Would you like me to help you set up a specific configuration for this PR?</answer></rawResChunk> -->
author_association: CONTRIBUTOR
user: login: bcye
created_at: 2025-04-26T21:13:28Z
body: @coderabbitai review Dockerfile README.md 
author_association: OWNER
user: login: coderabbitai[bot]
created_at: 2025-04-26T21:14:27Z
body: <!-- This is an auto-generated reply by CodeRabbit -->
> [!TIP]
> For best results, initiate chat on the files or code changes.

<details>
<summary>🧩 Analysis chain</summary>

🏁 Script executed:
```
#!/bin/bash
# Get the content of the Dockerfile
cat Dockerfile
```

Length of output: 188

---

🏁 Script executed:
```
#!/bin/bash
# Get the content of the README.md
cat README.md
```

Length of output: 988

---



</details>

@bcye I've reviewed the Dockerfile and README.md per your request.

**Dockerfile Review:**
The Dockerfile is well-structured and follows best practices:
- Uses UV package manager with Python 3.12 on Debian
- Optimizes build caching by separating dependency installation from code copying
- Uses the `--frozen` flag to ensure reproducible builds
- Keeps the container setup clean and straightforward

**README.md Review:**
The Docker usage documentation has been added as required, including:
- Instructions for running with Docker
- Example command with environment variables and volume mounting
- The important ulimit settings (mentioned in your PR description)

I noticed the links are empty for both the Docker image (`[]()`) and the TypeScript types package (`[]()`) in the README. These should be filled in before merging the PR to ensure users can easily find these resources.

Would you like me to suggest specific updates to either of these files?


<!-- <agentComponentType>chat-answer</agentComponentType> -->

<!-- <rawResChunk><answer><details>
<summary>🧩 Analysis chain</summary>

🏁 Script executed:
```
#!/bin/bash
# Get the content of the Dockerfile
cat Dockerfile
```

Length of output: 188

---

🏁 Script executed:
```
#!/bin/bash
# Get the content of the README.md
cat README.md
```

Length of output: 988

---



</details>

@bcye I've reviewed the Dockerfile and README.md per your request.

**Dockerfile Review:**
The Dockerfile is well-structured and follows best practices:
- Uses UV package manager with Python 3.12 on Debian
- Optimizes build caching by separating dependency installation from code copying
- Uses the `--frozen` flag to ensure reproducible builds
- Keeps the container setup clean and straightforward

**README.md Review:**
The Docker usage documentation has been added as required, including:
- Instructions for running with Docker
- Example command with environment variables and volume mounting
- The important ulimit settings (mentioned in your PR description)

I noticed the links are empty for both the Docker image (`[]()`) and the TypeScript types package (`[]()`) in the README. These should be filled in before merging the PR to ensure users can easily find these resources.

Would you like me to suggest specific updates to either of these files?</answer></rawResChunk> -->
author_association: CONTRIBUTOR
user: login: bcye
created_at: 2025-04-30T19:59:17Z
body: .dockerignore can ignore globs/directories afaik I think whitelisting files would probably get very tedious, but yeah having the python code be an actual module is probably a good idea, I tried to move in this direction with the main.py rename (i think its the only remaining python file at root now).

To be honest I'm not yet familiar with distroless containers so can't comment on that . The package installation would still be part of the Dockerfile though?
author_association: OWNER