Repository: tldr-pages/tldr-python-client

Issue:
Fetch individual translation archives for cache based on the environment variable configuration
As specified in [Client Specification 2.0](https://github.com/tldr-pages/tldr/blob/v2.0/CLIENT-SPECIFICATION.md)


_If appropriate, it is RECOMMENDED that clients implement a cache of pages. If implemented, clients MUST download the entire archive either as a whole from https://tldr.sh/assets/tldr.zip (Which redirects to https://raw.githubusercontent.com/tldr-pages/tldr-pages.github.io/main/assets/tldr.zip) or download language-specific translation archives in the format https://tldr.sh/assets/tldr-pages.{{language-code}}.zip (Which redirects to [https://raw.githubusercontent.com/tldr-pages/tldr-pages.github.io/main/assets/tldr-pages.{{language-code}}.zip](https://raw.githubusercontent.com/tldr-pages/tldr-pages.github.io/main/assets)), along with the archive for English from https://tldr.sh/assets/tldr-pages.zip (It redirects to https://raw.githubusercontent.com/tldr-pages/tldr-pages.github.io/main/assets/tldr-pages.zip)._

_Caching SHOULD be done according to the user's language configuration (if any), to not waste unneeded space for unused languages. Additionally, clients MAY automatically update the cache regularly._

So, we need to update the client behaviour to work with fetching only the specified language's archive when fetching based on configuration or via the `tldr <page> --language <language-code>` command, we can include English by default as a fallback.

Discussion:
@SaurabhDRao: Hello! Can I take this up ?

Below are the changes that needs to be done right ?
- if a language is set, then fetch only the data for that particular language from the respective language archive location.
- if no language is set, then fetch the english archive as default.
@kbdharun: > Hello! Can I take this up ?

Sure

> Below are the changes that needs to be done right ?
> 
> * if a language is set, then fetch only the data for that particular language from the respective language archive location.
> * if no language is set, then fetch the english archive as default.

Yes, if a language is set (in the environment variable) only the language and English translation archive must be fetched and when a page isn't translated the client needs to fall back to the English page.@SaurabhDRao: I have added change for pulling the language specific archives into cache. (#218)

But I noticed that the fallback on en is already present. In the `get_page` function, there is a call to get a list of all languages where it is attaching the default language (en) at the end. Let me know if I am missing something.

Solution patch:
diff --git a/.github/dependabot.yml b/.github/dependabot.yml
new file mode 100644
index 0000000..8ac6b8c
--- /dev/null
+++ b/.github/dependabot.yml
@@ -0,0 +1,6 @@
+version: 2
+updates:
+  - package-ecosystem: "github-actions"
+    directory: "/"
+    schedule:
+      interval: "monthly"
diff --git a/.github/workflows/publish.yml b/.github/workflows/publish.yml
index 9453db8..be62097 100644
--- a/.github/workflows/publish.yml
+++ b/.github/workflows/publish.yml
@@ -1,4 +1,4 @@
-# This workflows will upload a Python Package using Twine when a release is created
+# This workflow will upload a Python Package using Twine when a release is created
 # For more information see: https://help.github.com/en/actions/language-and-framework-guides/using-python-with-github-actions#publishing-to-package-registries
 
 name: Upload Python Package
@@ -10,13 +10,12 @@ on:
 jobs:
   deploy:
 
-    runs-on: ubuntu-20.04
-
+    runs-on: ubuntu-latest
     steps:
     - uses: actions/checkout@v4
 
     - name: Set up Python
-      uses: actions/setup-python@v4
+      uses: actions/setup-python@v5
       with:
         python-version: '3.8'
 
diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml
index 1c2c192..bd194d9 100644
--- a/.github/workflows/test.yml
+++ b/.github/workflows/test.yml
@@ -4,17 +4,17 @@ on: ['push', 'pull_request']
 
 jobs:
   build:
-    runs-on: ubuntu-20.04
+    runs-on: ubuntu-latest
 
     strategy:
       matrix:
-        python-version: ['3.8', '3.9', '3.10', '3.11', 'pypy3.9', 'pypy3.10']
+        python-version: ['3.8', '3.9', '3.10', '3.11', '3.12', 'pypy3.9', 'pypy3.10']
 
     steps:
     - uses: actions/checkout@v4
 
     - name: Set up Python ${{ matrix.python-version }}
-      uses: actions/setup-python@v4
+      uses: actions/setup-python@v5
       with:
         python-version: ${{ matrix.python-version }}
 
@@ -46,7 +46,7 @@ jobs:
     - name: Lint codebase
       run: python3 -m flake8
 
-    - name: Run testsuite
+    - name: Run test suite
       run: python3 setup.py pytest
 
     - name: Test tldr cli
@@ -58,12 +58,12 @@ jobs:
     runs-on: ubuntu-latest
 
     steps:
-     - uses: actions/checkout@v3
+     - uses: actions/checkout@v4
      - uses: snapcore/action-build@v1
        id: snapcraft-build
        with:
          snapcraft-args: "-v"
-     - uses: actions/upload-artifact@v3
+     - uses: actions/upload-artifact@v4
        with:
          name: ${{ steps.snapcraft-build.outputs.snap }}
          path: ${{ steps.snapcraft-build.outputs.snap }}
diff --git a/README.md b/README.md
index 5e4d066..8853d63 100644
--- a/README.md
+++ b/README.md
@@ -5,10 +5,12 @@
 
 Python command-line client for [tldr pages](https://github.com/tldr-pages/tldr).
 
-![tldr pages example](https://raw.github.com/tldr-pages/tldr/main/images/tldr.svg)
+![tldr pages example](https://raw.github.com/tldr-pages/tldr/main/images/tldr-dark.png)
 
 ## Installation
 
+[![Packaging status](https://repology.org/badge/vertical-allrepos/tldr-python-client.svg)](https://repology.org/project/python-tldr-client/versions)
+
 ### from PyPI
 
 ```bash
diff --git a/tldr.py b/tldr.py
index bfae548..c267379 100755
--- a/tldr.py
+++ b/tldr.py
@@ -43,6 +43,9 @@ if int(os.environ.get('TLDR_ALLOW_INSECURE', '0')) == 1:
 OS_DIRECTORIES = {
     "linux": "linux",
     "darwin": "osx",
+    "freebsd": "freebsd",
+    "openbsd": "openbsd",
+    "netbsd": "netbsd",
     "sunos": "sunos",
     "win32": "windows"
 }
@@ -374,35 +377,38 @@ def output(page: str, plain: bool = False) -> None:
 
 
 def update_cache(language: Optional[List[str]] = None) -> None:
-    if language is None:
-        tldr_language = os.environ.get("TLDR_LANGUAGE", get_default_language())
-        language = tldr_language if tldr_language else 'en'
-    elif isinstance(language, list):
-        language = language[0]
-    try:
-        pages_dir = "pages"
-        if language and language != 'en':
-            pages_dir += "." + language
-        req = urlopen(Request(
-            DOWNLOAD_CACHE_LOCATION,
-            headers=REQUEST_HEADERS
-        ), context=URLOPEN_CONTEXT)
-        zipfile = ZipFile(BytesIO(req.read()))
-        pattern = re.compile(r"{}/(.+)/(.+)\.md".format(pages_dir))
-        cached = 0
-        for entry in zipfile.namelist():
-            match = pattern.match(entry)
-            if match:
-                store_page_to_cache(
-                    zipfile.read(entry),
-                    match.group(2),
-                    match.group(1),
-                    language
-                )
-                cached += 1
-        print("Updated cache for {:d} entries".format(cached))
-    except Exception:
-        sys.exit("Error: Unable to update cache from " + DOWNLOAD_CACHE_LOCATION)
+    languages = get_language_list()
+    if language and language[0] not in languages:
+        languages.append(language[0])
+    for language in languages:
+        try:
+            cache_location = f"{DOWNLOAD_CACHE_LOCATION[:-4]}-pages.{language}.zip"
+            req = urlopen(Request(
+                cache_location,
+                headers=REQUEST_HEADERS
+            ), context=URLOPEN_CONTEXT)
+            zipfile = ZipFile(BytesIO(req.read()))
+            pattern = re.compile(r"(.+)/(.+)\.md")
+            cached = 0
+            for entry in zipfile.namelist():
+                match = pattern.match(entry)
+                if match:
+                    store_page_to_cache(
+                        zipfile.read(entry),
+                        match.group(2),
+                        match.group(1),
+                        language
+                    )
+                    cached += 1
+            print(
+                "Updated cache for language "
+                f"{language}: {cached} entries"
+            )
+        except Exception:
+            print(
+                "Error: Unable to update cache for language "
+                f"{language} from {cache_location}"
+            )
 
 
 def create_parser() -> ArgumentParser:
@@ -425,7 +431,7 @@ def create_parser() -> ArgumentParser:
                         type=str,
                         help="Search for a specific command from a query")
 
-    parser.add_argument('-u', '--update_cache',
+    parser.add_argument('-u', '--update', '--update_cache',
                         action='store_true',
                         help="Update the local cache of pages and exit")
 
@@ -497,14 +503,14 @@ def main() -> None:
     if options.color is False:
         os.environ["FORCE_COLOR"] = "true"
 
-    if options.update_cache:
+    if options.update:
         update_cache(language=options.language)
         return
     elif len(sys.argv) == 1:
         parser.print_help(sys.stderr)
         sys.exit(1)
     if options.list:
-        print(get_commands(options.platform))
+        print('\n'.join(get_commands(options.platform)))
     elif options.render:
         for command in options.command:
             if os.path.exists(command):


Test patch:
diff --git a/tests/test_tldr.py b/tests/test_tldr.py
index b49c3ed..3826e68 100644
--- a/tests/test_tldr.py
+++ b/tests/test_tldr.py
@@ -108,7 +108,9 @@ def test_tldr_language(tldr_language, language, lang, expected, monkeypatch):
     ("win32", "windows"),
     ("darwin", "osx"),
     ("sunos", "sunos"),
-    ("freebsd", "linux"),
+    ("freebsd", "freebsd"),
+    ("openbsd", "openbsd"),
+    ("netbsd", "netbsd"),
     ("aix", "linux"),
 ])
 def test_get_platform(platform, expected):