Repository: valkey-io/valkey-release-automation
Pull request #11: Automated updating the dockerhub-description whenever a new version is released
Author: login: Nikhil-Manglore
Created: 2025-03-21T09:11:56Z | Merged: 2025-03-27T22:36:22Z

This python script will update the docker hub description whenever a new version of valkey is released. To test this I ran the script with the version as 8.1.0-rc2 and environment as "prod".

As displayed below a pull request was created on `Valkey-Container`:  

<img width="1373" alt="Screenshot 2025-03-21 at 2 04 47 AM" src="https://github.com/user-attachments/assets/79872054-3865-4a65-abe5-b7494a2ee9d5" />_______
   

Here is the updated diff for `Valkey-Container`: https://github.com/Nikhil-Manglore/valkey-container/commit/78a35842fda6be7cfbb9499dcdc59417a2f10f09

We're mainly focusing on the diff for the `valkey-container/dockerhub-description.md` file. The image below shows the automation after releasing 8.1.0-rc2.



<img width="1067" alt="Screenshot 2025-03-21 at 2 06 50 AM" src="https://github.com/user-attachments/assets/b104bec8-201b-484d-808c-53df65e67e79" />_______

Here is the updated workflow:

<img width="958" alt="Screenshot 2025-03-21 at 3 35 46 PM" src="https://github.com/user-attachments/assets/4d2b7ae6-1d3d-4546-9b33-9ec19ada22ed" />

Resolves #5 


Linked issue #5: Automate updating the dockerhub description
Automate the process of updating [dockerhub-description.md](https://github.com/valkey-io/valkey-container/blob/mainline/dockerhub-description.md) in the valkey-container repository.

Diff:
diff --git a/.github/workflows/update-docker.yml b/.github/workflows/update-docker.yml
index ce651b5f..46f03785 100644
--- a/.github/workflows/update-docker.yml
+++ b/.github/workflows/update-docker.yml
@@ -25,14 +25,15 @@ jobs:
       - name: Set up bashbrew
         uses: docker-library/bashbrew@v0.1.12
       
-      - name: Check if bashbrew in installed
+      - name: Check if bashbrew is installed
         run: |
           bashbrew --version
 
-      - name: Checkout code
+      - name: Checkout valkey-container
         uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4.1.1
         with:
           repository: ${{ github.repository_owner }}/valkey-container
+          path: valkey-container
           token: ${{ secrets.PAT_TOKEN }}
       
       - name: Strip patch version
@@ -53,6 +54,7 @@ jobs:
       
       - name: Update new version
         run: |
+            cd valkey-container
             # Download the scripts for bashbrew in local
             jqt='.jq-template.awk'
             wget -qO "$jqt" 'https://github.com/docker-library/bashbrew/raw/9f6a35772ac863a0241f147c820354e4008edf38/scripts/jq-template.awk'
@@ -61,20 +63,45 @@ jobs:
 
             # Run update script
             ./update.sh ${{ steps.strip_version.outputs.stripped_version }}
+            git config user.name "github-actions[bot]"
+            git config user.email "github-actions[bot]@users.noreply.github.com"
+            git add .
+            git commit -m "Update versions.json and Dockerfiles for version ${{ inputs.version }}" -s
 
-      - name: Check for Changes
+      - name: Setup Python
+        uses: actions/setup-python@v4
+        with:
+          python-version: '3.x'
+
+      - name: Checkout valkey-release-automation
+        uses: actions/checkout@v4
+        with:
+          repository: ${{ github.repository_owner }}/valkey-release-automation
+          path: valkey-release-automation
+
+      - name: Generate bashbrew output
+        id: generate-bashbrew
         run: |
-          if git diff --exit-code; then
-            echo "No changes detected. Skipping PR creation."
-            exit 0
-          fi
+          cd valkey-container
+          BASHBREW_SCRIPTS=/home/runner/work/_actions/docker-library/bashbrew/v0.1.12/scripts
+          "$BASHBREW_SCRIPTS/github-actions/generate.sh" > bashbrew_output.json
+
+      - name: Update Docker Description File
+        run: |
+          python valkey-release-automation/scripts/automate_docker_description.py valkey-container/bashbrew_output.json valkey-container/dockerhub-description-template.md valkey-container/dockerhub-description.md
+          rm valkey-container/bashbrew_output.json
+          cd valkey-container
+          git add .
+          git commit -m "Update tags in the docker description for version ${{ inputs.version }}" -s
 
       - name: Create Pull Request
+        id: create-pr
         uses: peter-evans/create-pull-request@v7
         with:
           token: ${{ secrets.PAT_TOKEN }}
+          path: valkey-container
           branch: update-${{ inputs.version }}
-          commit-message: "Update for version ${{ inputs.version }}"
+          commit-message: "Preparing changes for version ${{ inputs.version }}"
           committer: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
           delete-branch: true
           signoff: true
@@ -82,5 +109,4 @@ jobs:
           title: "Update to version ${{ inputs.version }}"
           body: |
             This pull request updates the repository to version `${{ inputs.version }}`.
-            Please review the changes and merge when appropriate.
-          
\ No newline at end of file
+            Please review the changes and merge when appropriate.
\ No newline at end of file
diff --git a/scripts/automate_docker_description.py b/scripts/automate_docker_description.py
new file mode 100644
index 00000000..7e33df01
--- /dev/null
+++ b/scripts/automate_docker_description.py
@@ -0,0 +1,99 @@
+import json
+import sys
+import logging
+from datetime import datetime
+
+logging.basicConfig(level=logging.ERROR, format="%(asctime)s - %(levelname)s - %(message)s")
+
+def clean_tag(tag: str) -> str:
+    if ":" in tag:
+        return tag.split(":", 1)[1]
+    return tag
+
+def format_tag_line(entry: dict) -> str:
+    try:
+        meta_entries = entry.get("meta", {}).get("entries", [])
+        if not meta_entries:
+            raise KeyError("Missing or empty 'entries' list in 'meta'.")
+        
+        first_entry = meta_entries[0]
+        raw_tags = first_entry.get("tags", [])
+        if not raw_tags:
+            raise KeyError("Missing or empty 'tags' list in entry.")
+        
+        directory = first_entry.get("directory", None)
+        if not directory:
+            raise KeyError("Missing 'directory' field in entry.")
+
+        formatted_tags = [f'`{clean_tag(tag)}`' for tag in raw_tags]
+        tags = ", ".join(formatted_tags)
+
+        return f"- [{tags}](https://github.com/valkey-io/valkey-container/blob/master/{directory}/Dockerfile)"
+    
+    except KeyError as e:
+        logging.error(f"JSON structure error: {e}")
+        raise
+    
+    except Exception as e:
+        logging.error(f"Unexpected error in format_tag_line: {e}")
+        raise
+
+def update_docker_description(json_file: str, template_file: str, output_file: str) -> None:
+    try:
+        with open(json_file, 'r') as f:
+            data = json.load(f)
+        
+        with open(template_file, 'r') as f:
+            template = f.read()
+
+        official_releases = []
+        release_candidates = []
+        latest_unstable = []
+
+        for entry in data["matrix"]["include"]:
+            line = format_tag_line(entry)
+            if "rc" in entry["name"]:
+                release_candidates.append(line)
+            elif "unstable" in entry["name"]:
+                latest_unstable.append(line)
+            else:
+                official_releases.append(line)
+
+        official_releases_section = "\n".join(official_releases)
+        rc_section = "" if not release_candidates else f"\n## Release candidates\n{'\n'.join(release_candidates)}"
+        unstable_section = "" if not latest_unstable else f"\n## Latest unstable\n{'\n'.join(latest_unstable)}"
+
+        content = template.format(
+            update_date=datetime.now().strftime("%Y-%m-%d"),
+            official_releases=official_releases_section,
+            release_candidates_section=rc_section,
+            unstable_section=unstable_section
+        )
+
+        with open(output_file, 'w') as f:
+            f.write(content)
+
+    except FileNotFoundError as e:
+        logging.error(f"File not found: {e}")
+        sys.exit(1)
+    except json.JSONDecodeError:
+        logging.error(f"Failed to parse JSON file '{json_file}'. Please check its syntax.")
+        sys.exit(1)
+    except KeyError as e:
+        logging.error(f"Invalid JSON structure: {e}")
+        sys.exit(1)
+    except Exception as e:
+        logging.error(f"Error processing data: {e}")
+        sys.exit(1)
+
+if __name__ == "__main__":
+    if len(sys.argv) != 4:
+        logging.error("Invalid number of arguments.")
+        logging.error("Usage: python automate_docker_description.py <json_file> <template_file> <output_file>")
+        sys.exit(1)
+
+    try:
+        update_docker_description(sys.argv[1], sys.argv[2], sys.argv[3])
+    except Exception as e:
+        logging.error(f"An unexpected error occurred: {e}")
+        sys.exit(1)
\ No newline at end of file


Reviews:
user: login: vudiep411
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-21T19:44:15Z
user: login: roshkhatri
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-21T21:06:53Z
user: login: roshkhatri
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-21T21:46:48Z
user: login: Nikhil-Manglore
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-21T23:00:54Z
user: login: roshkhatri
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-24T21:21:07Z
user: login: roshkhatri
body: We might want to take a different approach for creating the description file.
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-24T21:24:41Z
user: login: vudiep411
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-26T23:03:08Z
user: login: Nikhil-Manglore
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-27T00:10:46Z
user: login: Nikhil-Manglore
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-27T00:16:26Z
user: login: vudiep411
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-27T02:21:10Z
user: login: vudiep411
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-27T02:44:14Z
user: login: roshkhatri
body: I would ideally move this file to valkey-container. Please open a PR for valkey-container and you you can use the template when we checkout the repository.
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-27T06:59:09Z
user: login: Nikhil-Manglore
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-27T19:10:25Z
user: login: roshkhatri
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-27T21:34:54Z
user: login: roshkhatri
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-27T21:38:20Z
user: login: roshkhatri
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-27T21:40:06Z
user: login: roshkhatri
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-27T21:46:25Z
user: login: roshkhatri
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-27T21:47:32Z
user: login: roshkhatri
body: LGTM
state: APPROVED
author_association: MEMBER
submitted_at: 2025-03-27T22:35:33Z

Review comments:
diff_hunk: @@ -0,0 +1,88 @@
+import json
+import sys
+import re
+
+def clean_tag(tag: str) -> str:
+    if ":" in tag:
+        return tag.split(":", 1)[1]
+    return tag
+
+def format_tag_line(entry: dict) -> str:
+    raw_tags = entry['meta']['entries'][0]['tags']
+    
+    formatted_tags = []
+    for tag in raw_tags:
+        clean = clean_tag(tag)
+        formatted_tags.append(f'`{clean}`')
+    
+    tags = ', '.join(formatted_tags)
+    directory = entry['meta']['entries'][0]['directory']
+    
+    return f"- [{tags}](https://github.com/valkey-io/valkey-container/blob/master/{directory}/Dockerfile)"
+
+def update_section(content: str, section_header: str, new_content: str) -> str:
+    # Find the section
+    section_start = content.find(section_header)
+    if section_start == -1:
+        return content
+    
+    search_start = section_start + len(section_header)
+    
+    next_section = content.find("\n## ", search_start)
+    what_is_section = content.find('\nWhat is [Valkey](https://github.com/valkey-io/valkey)?', search_start)
+    next_header = content.find("\n# ", search_start)
+    
+    # Find the next closest section
+    section_end = float('inf')
+    for pos in [next_section, what_is_section, next_header]:
+        if pos != -1 and pos < section_end:
+            section_end = pos
+    
+    if section_end == float('inf'):
+        section_end = len(content)
+
+    # Get the content until the section header
+    section_content_start = content.find("\n", section_start) + 1
+    
+    # Replace only the content between current section and next section
+    return (content[:section_content_start] + "\n" + new_content + "\n\n" + content[section_end:])
+
+def update_docker_description(json_file: str, docker_description_file: str) -> None:
path: scripts/automate_docker_description.py
user: login: vudiep411
body: What is this `json_file`? Can u make the name a bit more descriptive so it's easier to understand?
created_at: 2025-03-21T19:36:24Z
author_association: CONTRIBUTOR
diff_hunk: @@ -83,4 +125,13 @@ jobs:
           body: |
             This pull request updates the repository to version `${{ inputs.version }}`.
             Please review the changes and merge when appropriate.
+
+      - name: Update Docker Hub Description
path: .github/workflows/update-docker.yml
user: login: vudiep411
body: We've decided to move this step to this repo?
created_at: 2025-03-21T19:42:26Z
author_association: CONTRIBUTOR
diff_hunk: @@ -83,4 +125,13 @@ jobs:
           body: |
             This pull request updates the repository to version `${{ inputs.version }}`.
             Please review the changes and merge when appropriate.
+
+      - name: Update Docker Hub Description
path: .github/workflows/update-docker.yml
user: login: roshkhatri
body: Just adding the description file changes to the PR.
This is not required

created_at: 2025-03-21T21:06:53Z
author_association: MEMBER
diff_hunk: @@ -62,17 +68,53 @@ jobs:
             # Run update script
             ./update.sh ${{ steps.strip_version.outputs.stripped_version }}
 
+      - name: Setup Python
+        uses: actions/setup-python@v4
+        with:
+          python-version: '3.x'
+
+      - name: Checkout automation scripts
+        uses: actions/checkout@v4
+        with:
+          repository: ${{ github.repository_owner }}/valkey-release-automation
+          path: valkey-release-automation
+          token: ${{ secrets.PAT_TOKEN }}
+
+      - name: Generate bashbrew output
+        id: generate-bashbrew
+        run: |
+          cd valkey-container
+          BASHBREW_SCRIPTS=/home/runner/work/_actions/docker-library/bashbrew/v0.1.12/scripts
+          "$BASHBREW_SCRIPTS/github-actions/generate.sh" > bashbrew_output.json
+
+      - name: Update Docker Description
path: .github/workflows/update-docker.yml
user: login: roshkhatri
body: ```suggestion
      - name: Update Docker Description File
```
created_at: 2025-03-21T21:46:48Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,88 @@
+import json
+import sys
+import re
+
+def clean_tag(tag: str) -> str:
+    if ":" in tag:
+        return tag.split(":", 1)[1]
+    return tag
+
+def format_tag_line(entry: dict) -> str:
+    raw_tags = entry['meta']['entries'][0]['tags']
+    
+    formatted_tags = []
+    for tag in raw_tags:
+        clean = clean_tag(tag)
+        formatted_tags.append(f'`{clean}`')
+    
+    tags = ', '.join(formatted_tags)
+    directory = entry['meta']['entries'][0]['directory']
+    
+    return f"- [{tags}](https://github.com/valkey-io/valkey-container/blob/master/{directory}/Dockerfile)"
+
+def update_section(content: str, section_header: str, new_content: str) -> str:
+    # Find the section
+    section_start = content.find(section_header)
+    if section_start == -1:
+        return content
+    
+    search_start = section_start + len(section_header)
+    
+    next_section = content.find("\n## ", search_start)
+    what_is_section = content.find('\nWhat is [Valkey](https://github.com/valkey-io/valkey)?', search_start)
+    next_header = content.find("\n# ", search_start)
+    
+    # Find the next closest section
+    section_end = float('inf')
+    for pos in [next_section, what_is_section, next_header]:
+        if pos != -1 and pos < section_end:
+            section_end = pos
+    
+    if section_end == float('inf'):
+        section_end = len(content)
+
+    # Get the content until the section header
+    section_content_start = content.find("\n", section_start) + 1
+    
+    # Replace only the content between current section and next section
+    return (content[:section_content_start] + "\n" + new_content + "\n\n" + content[section_end:])
+
+def update_docker_description(json_file: str, docker_description_file: str) -> None:
path: scripts/automate_docker_description.py
user: login: Nikhil-Manglore
body: the json_file contains the [bashbrew output](https://github.com/valkey-io/valkey-container/actions/runs/13888909666/job/38857506155). We use the bashbrew output to extract the necessary information such as the directory and the tags.
created_at: 2025-03-21T23:00:54Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,87 @@
+import json
+import sys
+
+def clean_tag(tag: str) -> str:
+    if ":" in tag:
+        return tag.split(":", 1)[1]
+    return tag
+
+def format_tag_line(entry: dict) -> str:
+    raw_tags = entry['meta']['entries'][0]['tags']
+    
+    formatted_tags = []
+    for tag in raw_tags:
+        clean = clean_tag(tag)
+        formatted_tags.append(f'`{clean}`')
+    
+    tags = ', '.join(formatted_tags)
+    directory = entry['meta']['entries'][0]['directory']
+    
+    return f"- [{tags}](https://github.com/valkey-io/valkey-container/blob/master/{directory}/Dockerfile)"
+
+def update_section(content: str, section_header: str, new_content: str) -> str:
+    # Find the section
+    section_start = content.find(section_header)
+    if section_start == -1:
+        return content
+    
+    search_start = section_start + len(section_header)
+    
+    next_section = content.find("\n## ", search_start)
+    what_is_section = content.find('\nWhat is [Valkey](https://github.com/valkey-io/valkey)?', search_start)
+    next_header = content.find("\n# ", search_start)
+    
+    # Find the next closest section
+    section_end = float('inf')
+    for pos in [next_section, what_is_section, next_header]:
+        if pos != -1 and pos < section_end:
+            section_end = pos
+    
+    if section_end == float('inf'):
+        section_end = len(content)
+
+    # Get the content until the section header
+    section_content_start = content.find("\n", section_start) + 1
+    
+    # Replace only the content between current section and next section
+    return (content[:section_content_start] + "\n" + new_content + "\n\n" + content[section_end:])
+
path: scripts/automate_docker_description.py
user: login: roshkhatri
body: This looks quite complex, can we create a template file for the description and just fill in the blanks?
dockerhub-description-template.md and create a new dockerhub-description.m?
I think would be easier to follow.

We could also add a comment in dockerhub-description.md saying this is auto generated and do not edit it manually.
created_at: 2025-03-24T21:21:07Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,87 @@
+import json
+import sys
+
+def clean_tag(tag: str) -> str:
+    if ":" in tag:
+        return tag.split(":", 1)[1]
+    return tag
+
+def format_tag_line(entry: dict) -> str:
+    raw_tags = entry['meta']['entries'][0]['tags']
+    
+    formatted_tags = []
+    for tag in raw_tags:
+        clean = clean_tag(tag)
+        formatted_tags.append(f'`{clean}`')
+    
+    tags = ', '.join(formatted_tags)
+    directory = entry['meta']['entries'][0]['directory']
+    
+    return f"- [{tags}](https://github.com/valkey-io/valkey-container/blob/master/{directory}/Dockerfile)"
+
+def update_section(content: str, section_header: str, new_content: str) -> str:
+    # Find the section
+    section_start = content.find(section_header)
+    if section_start == -1:
+        return content
+    
+    search_start = section_start + len(section_header)
+    
+    next_section = content.find("\n## ", search_start)
+    what_is_section = content.find('\nWhat is [Valkey](https://github.com/valkey-io/valkey)?', search_start)
+    next_header = content.find("\n# ", search_start)
+    
+    # Find the next closest section
+    section_end = float('inf')
+    for pos in [next_section, what_is_section, next_header]:
+        if pos != -1 and pos < section_end:
+            section_end = pos
+    
+    if section_end == float('inf'):
+        section_end = len(content)
+
+    # Get the content until the section header
+    section_content_start = content.find("\n", section_start) + 1
+    
+    # Replace only the content between current section and next section
+    return (content[:section_content_start] + "\n" + new_content + "\n\n" + content[section_end:])
+
+def update_docker_description(json_file: str, docker_description_file: str) -> None:
+    with open(json_file) as f:
+        data = json.load(f)
+
+    with open(docker_description_file, 'r') as f:
+        content = f.read()
+
+    # Generate all the content for each section
+    official_lines = []
+    for entry in data["matrix"]["include"]:
+        if "rc" not in entry["name"] and "unstable" not in entry["name"]:
+            line = format_tag_line(entry)
+            official_lines.append(line)
+    official_releases = "\n".join(official_lines)
+
+    rc_lines = []
+    for entry in data["matrix"]["include"]:
+        if "rc" in entry["name"]:
+            line = format_tag_line(entry)
+            rc_lines.append(line)
+    release_candidates = "\n".join(rc_lines)
+
+    unstable_lines = []
+    for entry in data["matrix"]["include"]:
+        if "unstable" in entry["name"]:
+            line = format_tag_line(entry)
+            unstable_lines.append(line)
+    latest_unstable = "\n".join(unstable_lines)
+
+    # Update each section separately
+    content = update_section(content, "## Official releases", official_releases)
+    content = update_section(content, "## Release candidates", release_candidates)
+    content = update_section(content, "## Latest unstable", latest_unstable)
+
+    with open(docker_description_file, 'w') as f:
+        f.write(content)
+
+if __name__ == "__main__":
+    update_docker_description(sys.argv[1], sys.argv[2])
path: scripts/automate_docker_description.py
user: login: roshkhatri
body: Can you add Error handling for number of arguments? and corresponding error logs?
created_at: 2025-03-24T21:21:59Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,87 @@
+import json
+import sys
+
+def clean_tag(tag: str) -> str:
+    if ":" in tag:
+        return tag.split(":", 1)[1]
+    return tag
+
+def format_tag_line(entry: dict) -> str:
+    raw_tags = entry['meta']['entries'][0]['tags']
+    
+    formatted_tags = []
+    for tag in raw_tags:
+        clean = clean_tag(tag)
+        formatted_tags.append(f'`{clean}`')
+    
+    tags = ', '.join(formatted_tags)
+    directory = entry['meta']['entries'][0]['directory']
+    
+    return f"- [{tags}](https://github.com/valkey-io/valkey-container/blob/master/{directory}/Dockerfile)"
+
+def update_section(content: str, section_header: str, new_content: str) -> str:
+    # Find the section
+    section_start = content.find(section_header)
+    if section_start == -1:
+        return content
+    
+    search_start = section_start + len(section_header)
+    
+    next_section = content.find("\n## ", search_start)
+    what_is_section = content.find('\nWhat is [Valkey](https://github.com/valkey-io/valkey)?', search_start)
+    next_header = content.find("\n# ", search_start)
+    
+    # Find the next closest section
+    section_end = float('inf')
+    for pos in [next_section, what_is_section, next_header]:
+        if pos != -1 and pos < section_end:
+            section_end = pos
+    
+    if section_end == float('inf'):
+        section_end = len(content)
+
+    # Get the content until the section header
+    section_content_start = content.find("\n", section_start) + 1
+    
+    # Replace only the content between current section and next section
+    return (content[:section_content_start] + "\n" + new_content + "\n\n" + content[section_end:])
+
+def update_docker_description(json_file: str, docker_description_file: str) -> None:
+    with open(json_file) as f:
+        data = json.load(f)
+
+    with open(docker_description_file, 'r') as f:
+        content = f.read()
path: scripts/automate_docker_description.py
user: login: roshkhatri
body: Can you add Error handling for number of opening files? and corresponding error logs?
created_at: 2025-03-24T21:22:21Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,87 @@
+import json
+import sys
+
+def clean_tag(tag: str) -> str:
+    if ":" in tag:
+        return tag.split(":", 1)[1]
+    return tag
+
+def format_tag_line(entry: dict) -> str:
+    raw_tags = entry['meta']['entries'][0]['tags']
+    
path: scripts/automate_docker_description.py
user: login: roshkhatri
body: Can you add Error handling for change in json structure.? and corresponding error logs?
something like a try blcok?
created_at: 2025-03-24T21:23:48Z
author_association: MEMBER
diff_hunk: @@ -62,17 +64,54 @@ jobs:
             # Run update script
             ./update.sh ${{ steps.strip_version.outputs.stripped_version }}
 
+      - name: Setup Python
+        uses: actions/setup-python@v4
+        with:
+          python-version: '3.x'
+
+      - name: Checkout automation scripts
+        uses: actions/checkout@v4
+        with:
+          repository: ${{ github.repository_owner }}/valkey-release-automation
+          path: valkey-release-automation
+          token: ${{ secrets.PAT_TOKEN }}
+
+      - name: Generate bashbrew output
+        id: generate-bashbrew
+        run: |
+          cd valkey-container
+          BASHBREW_SCRIPTS=/home/runner/work/_actions/docker-library/bashbrew/v0.1.12/scripts
+          "$BASHBREW_SCRIPTS/github-actions/generate.sh" > bashbrew_output.json
+
+      - name: Update Docker Description File
+        run: |
+          python valkey-release-automation/scripts/automate_docker_description.py valkey-container/bashbrew_output.json valkey-release-automation/documents/dockerhub-description-template.md valkey-container/dockerhub-description.md
+          rm valkey-container/bashbrew_output.json
+
+      - name: Ensure Up-to-Date Branch
+        run: |
+          cd valkey-container
+          git config --global user.email "github-actions[bot]@users.noreply.github.com"
+          git config --global user.name "github-actions[bot]"
+          git add .
+          git commit -m "Preparing changes for version ${{ inputs.version }}"
+          git checkout mainline
+          git pull origin mainline --rebase
path: .github/workflows/update-docker.yml
user: login: vudiep411
body: Not sure if we really need this step?
created_at: 2025-03-26T22:59:20Z
author_association: CONTRIBUTOR
diff_hunk: @@ -62,17 +64,54 @@ jobs:
             # Run update script
             ./update.sh ${{ steps.strip_version.outputs.stripped_version }}
 
+      - name: Setup Python
+        uses: actions/setup-python@v4
+        with:
+          python-version: '3.x'
+
+      - name: Checkout automation scripts
+        uses: actions/checkout@v4
+        with:
+          repository: ${{ github.repository_owner }}/valkey-release-automation
+          path: valkey-release-automation
+          token: ${{ secrets.PAT_TOKEN }}
+
+      - name: Generate bashbrew output
+        id: generate-bashbrew
+        run: |
+          cd valkey-container
+          BASHBREW_SCRIPTS=/home/runner/work/_actions/docker-library/bashbrew/v0.1.12/scripts
+          "$BASHBREW_SCRIPTS/github-actions/generate.sh" > bashbrew_output.json
+
+      - name: Update Docker Description File
+        run: |
+          python valkey-release-automation/scripts/automate_docker_description.py valkey-container/bashbrew_output.json valkey-release-automation/documents/dockerhub-description-template.md valkey-container/dockerhub-description.md
+          rm valkey-container/bashbrew_output.json
+
+      - name: Ensure Up-to-Date Branch
+        run: |
+          cd valkey-container
+          git config --global user.email "github-actions[bot]@users.noreply.github.com"
+          git config --global user.name "github-actions[bot]"
+          git add .
+          git commit -m "Preparing changes for version ${{ inputs.version }}"
+          git checkout mainline
+          git pull origin mainline --rebase
+
       - name: Check for Changes
         run: |
-          if git diff --exit-code; then
+          cd valkey-container
+          if git diff --exit-code origin/mainline; then
             echo "No changes detected. Skipping PR creation."
             exit 0
           fi
path: .github/workflows/update-docker.yml
user: login: vudiep411
body: if there are no changes, would `peter-evans/create-pull-request@v7` just skip by itself or we still need this step?
created_at: 2025-03-26T23:02:18Z
author_association: CONTRIBUTOR
diff_hunk: @@ -62,17 +64,54 @@ jobs:
             # Run update script
             ./update.sh ${{ steps.strip_version.outputs.stripped_version }}
 
+      - name: Setup Python
+        uses: actions/setup-python@v4
+        with:
+          python-version: '3.x'
+
+      - name: Checkout automation scripts
+        uses: actions/checkout@v4
+        with:
+          repository: ${{ github.repository_owner }}/valkey-release-automation
+          path: valkey-release-automation
+          token: ${{ secrets.PAT_TOKEN }}
+
+      - name: Generate bashbrew output
+        id: generate-bashbrew
+        run: |
+          cd valkey-container
+          BASHBREW_SCRIPTS=/home/runner/work/_actions/docker-library/bashbrew/v0.1.12/scripts
+          "$BASHBREW_SCRIPTS/github-actions/generate.sh" > bashbrew_output.json
+
+      - name: Update Docker Description File
+        run: |
+          python valkey-release-automation/scripts/automate_docker_description.py valkey-container/bashbrew_output.json valkey-release-automation/documents/dockerhub-description-template.md valkey-container/dockerhub-description.md
+          rm valkey-container/bashbrew_output.json
+
+      - name: Ensure Up-to-Date Branch
+        run: |
+          cd valkey-container
+          git config --global user.email "github-actions[bot]@users.noreply.github.com"
+          git config --global user.name "github-actions[bot]"
+          git add .
+          git commit -m "Preparing changes for version ${{ inputs.version }}"
+          git checkout mainline
+          git pull origin mainline --rebase
+
       - name: Check for Changes
         run: |
-          if git diff --exit-code; then
+          cd valkey-container
+          if git diff --exit-code origin/mainline; then
             echo "No changes detected. Skipping PR creation."
             exit 0
           fi
path: .github/workflows/update-docker.yml
user: login: Nikhil-Manglore
body: Good catch, according to the documentation in `peter-evans/create-pull-request@v7` the action will not create a PR if there are no changes. This step is redundant, will remove it.
created_at: 2025-03-27T00:10:46Z
author_association: MEMBER
diff_hunk: @@ -62,17 +64,54 @@ jobs:
             # Run update script
             ./update.sh ${{ steps.strip_version.outputs.stripped_version }}
 
+      - name: Setup Python
+        uses: actions/setup-python@v4
+        with:
+          python-version: '3.x'
+
+      - name: Checkout automation scripts
+        uses: actions/checkout@v4
+        with:
+          repository: ${{ github.repository_owner }}/valkey-release-automation
+          path: valkey-release-automation
+          token: ${{ secrets.PAT_TOKEN }}
+
+      - name: Generate bashbrew output
+        id: generate-bashbrew
+        run: |
+          cd valkey-container
+          BASHBREW_SCRIPTS=/home/runner/work/_actions/docker-library/bashbrew/v0.1.12/scripts
+          "$BASHBREW_SCRIPTS/github-actions/generate.sh" > bashbrew_output.json
+
+      - name: Update Docker Description File
+        run: |
+          python valkey-release-automation/scripts/automate_docker_description.py valkey-container/bashbrew_output.json valkey-release-automation/documents/dockerhub-description-template.md valkey-container/dockerhub-description.md
+          rm valkey-container/bashbrew_output.json
+
+      - name: Ensure Up-to-Date Branch
+        run: |
+          cd valkey-container
+          git config --global user.email "github-actions[bot]@users.noreply.github.com"
+          git config --global user.name "github-actions[bot]"
+          git add .
+          git commit -m "Preparing changes for version ${{ inputs.version }}"
+          git checkout mainline
+          git pull origin mainline --rebase
path: .github/workflows/update-docker.yml
user: login: Nikhil-Manglore
body: You're correct, `peter-evans/create-pull-request@v7` will already handle all the necessary git operations, will remove this step entirely. 
created_at: 2025-03-27T00:16:26Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,103 @@
+import json
+import sys
+import logging
+from datetime import datetime
+
+logging.basicConfig(level=logging.ERROR, format="%(asctime)s - %(levelname)s - %(message)s")
+
+def clean_tag(tag: str) -> str:
+    if ":" in tag:
+        return tag.split(":", 1)[1]
+    return tag
+
+def format_tag_line(entry: dict) -> str:
+    try:
+        meta_entries = entry.get("meta", {}).get("entries", [])
+        if not meta_entries:
+            raise KeyError("Missing or empty 'entries' list in 'meta'.")
+        
+        first_entry = meta_entries[0]
+        raw_tags = first_entry.get("tags", [])
+        if not raw_tags:
+            raise KeyError("Missing or empty 'tags' list in entry.")
+        
+        directory = first_entry.get("directory", None)
+        if not directory:
+            raise KeyError("Missing 'directory' field in entry.")
+
+        formatted_tags = [f'`{clean_tag(tag)}`' for tag in raw_tags]
+        tags = ", ".join(formatted_tags)
+
+        return f"- [{tags}](https://github.com/valkey-io/valkey-container/blob/master/{directory}/Dockerfile)"
+    
+    except KeyError as e:
+        logging.error(f"JSON structure error: {e}")
+        raise
+    
+    except Exception as e:
+        logging.error(f"Unexpected error in format_tag_line: {e}")
+        raise
+
+def update_docker_description(json_file: str, template_file: str, output_file: str) -> None:
+    try:
+        with open(json_file, 'r') as f:
+            data = json.load(f)
+        
+        with open(template_file, 'r') as f:
+            template = f.read()
+    except FileNotFoundError as e:
+        logging.error(f"File not found: {e}")
+        sys.exit(1)
+    except json.JSONDecodeError:
+        logging.error(f"Failed to parse JSON file '{json_file}'. Please check its syntax.")
+        sys.exit(1)
+    except Exception as e:
+        logging.error(f"Error reading input files: {e}")
+        sys.exit(1)
+
+    try:
+        official_releases = []
+        release_candidates = []
+        latest_unstable = []
+
+        for entry in data["matrix"]["include"]:
+            line = format_tag_line(entry)
+            if "rc" in entry["name"]:
+                release_candidates.append(line)
+            elif "unstable" in entry["name"]:
+                latest_unstable.append(line)
+            else:
+                official_releases.append(line)
+
+        official_releases_section = "\n".join(official_releases)
+        rc_section = "" if not release_candidates else f"\n## Release candidates\n{'\n'.join(release_candidates)}"
+        unstable_section = "" if not latest_unstable else f"\n## Latest unstable\n{'\n'.join(latest_unstable)}"
+
+        content = template.format(
+            update_date=datetime.now().strftime("%Y-%m-%d"),
+            official_releases=official_releases_section,
+            release_candidates_section=rc_section,
+            unstable_section=unstable_section
+        )
+
+        with open(output_file, 'w') as f:
+            f.write(content)
+
+    except KeyError as e:
+        logging.error(f"Invalid JSON structure: {e}")
+        sys.exit(1)
+    except Exception as e:
+        logging.error(f"Error processing data: {e}")
+        sys.exit(1)
path: scripts/automate_docker_description.py
user: login: vudiep411
body: You could merge these into 1 try catch block instead of splitting them.
```python
def update_docker_description(json_file: str, template_file: str, output_file: str) -> None:
    try:
    # Your logic
    ...
    except FileNotFoundError as e:
        logging.error(f"File not found: {e}")
        sys.exit(1)
    except json.JSONDecodeError:
        logging.error(f"Failed to parse JSON file '{json_file}'. Please check its syntax.")
        sys.exit(1)
    except KeyError as e:
        logging.error(f"Invalid JSON structure: {e}")
        sys.exit(1)
    except Exception as e:
        logging.error(f"Error processing data: {e}")
        sys.exit(1)
created_at: 2025-03-27T02:21:09Z
author_association: CONTRIBUTOR
diff_hunk: @@ -61,26 +63,50 @@ jobs:
 
             # Run update script
             ./update.sh ${{ steps.strip_version.outputs.stripped_version }}
+            git config user.name "github-actions[bot]"
+            git config user.email "github-actions[bot]@users.noreply.github.com"
+            git add .
+            git commit -m "Update to version ${{ inputs.version }}"
path: .github/workflows/update-docker.yml
user: login: vudiep411
body: nit: If u want, u could probably create a new step for the commit step here like this
```yml
- name: Commit changes to generate bashbrew output
   run: |
          ...
```
it might be easier for other to read
created_at: 2025-03-27T02:32:44Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,105 @@
+## THIS FILE IS AUTO-GENERATED - DO NOT EDIT MANUALLY
path: documents/dockerhub-description-template.md
user: login: roshkhatri
body: I would also add which file generates this file.
created_at: 2025-03-27T06:56:38Z
author_association: MEMBER
diff_hunk: @@ -61,26 +63,50 @@ jobs:
 
             # Run update script
             ./update.sh ${{ steps.strip_version.outputs.stripped_version }}
+            git config user.name "github-actions[bot]"
+            git config user.email "github-actions[bot]@users.noreply.github.com"
+            git add .
+            git commit -m "Update to version ${{ inputs.version }}"
path: .github/workflows/update-docker.yml
user: login: Nikhil-Manglore
body: I think it's fine how it is
created_at: 2025-03-27T19:10:25Z
author_association: MEMBER
diff_hunk: @@ -61,26 +63,50 @@ jobs:
 
             # Run update script
             ./update.sh ${{ steps.strip_version.outputs.stripped_version }}
+            git config user.name "github-actions[bot]"
+            git config user.email "github-actions[bot]@users.noreply.github.com"
+            git add .
+            git commit -m "Update to version ${{ inputs.version }}"
path: .github/workflows/update-docker.yml
user: login: roshkhatri
body: do we need need to sign off here?

created_at: 2025-03-27T21:34:54Z
author_association: MEMBER
diff_hunk: @@ -61,26 +63,50 @@ jobs:
 
             # Run update script
             ./update.sh ${{ steps.strip_version.outputs.stripped_version }}
+            git config user.name "github-actions[bot]"
+            git config user.email "github-actions[bot]@users.noreply.github.com"
+            git add .
+            git commit -m "Update to version ${{ inputs.version }}"
 
-      - name: Check for Changes
+      - name: Setup Python
+        uses: actions/setup-python@v4
+        with:
+          python-version: '3.x'
+
+      - name: Checkout valkey-release-automation
+        uses: actions/checkout@v4
+        with:
+          repository: ${{ github.repository_owner }}/valkey-release-automation
+          path: valkey-release-automation
+          token: ${{ secrets.PAT_TOKEN }}
+
+      - name: Generate bashbrew output
+        id: generate-bashbrew
         run: |
-          if git diff --exit-code; then
-            echo "No changes detected. Skipping PR creation."
-            exit 0
-          fi
+          cd valkey-container
+          BASHBREW_SCRIPTS=/home/runner/work/_actions/docker-library/bashbrew/v0.1.12/scripts
+          "$BASHBREW_SCRIPTS/github-actions/generate.sh" > bashbrew_output.json
+
+      - name: Update Docker Description File
+        run: |
+          python valkey-release-automation/scripts/automate_docker_description.py valkey-container/bashbrew_output.json valkey-container/dockerhub-description-template.md valkey-container/dockerhub-description.md
+          rm valkey-container/bashbrew_output.json
+          cd valkey-container
+          git reset HEAD~1
path: .github/workflows/update-docker.yml
user: login: roshkhatri
body: We dont need to squash the commits

created_at: 2025-03-27T21:38:20Z
author_association: MEMBER
diff_hunk: @@ -61,26 +63,50 @@ jobs:
 
             # Run update script
             ./update.sh ${{ steps.strip_version.outputs.stripped_version }}
+            git config user.name "github-actions[bot]"
+            git config user.email "github-actions[bot]@users.noreply.github.com"
+            git add .
+            git commit -m "Update to version ${{ inputs.version }}"
 
-      - name: Check for Changes
+      - name: Setup Python
+        uses: actions/setup-python@v4
+        with:
+          python-version: '3.x'
+
+      - name: Checkout valkey-release-automation
+        uses: actions/checkout@v4
+        with:
+          repository: ${{ github.repository_owner }}/valkey-release-automation
+          path: valkey-release-automation
+          token: ${{ secrets.PAT_TOKEN }}
path: .github/workflows/update-docker.yml
user: login: roshkhatri
body: we can remove this
created_at: 2025-03-27T21:40:06Z
author_association: MEMBER
diff_hunk: @@ -61,26 +63,50 @@ jobs:
 
             # Run update script
             ./update.sh ${{ steps.strip_version.outputs.stripped_version }}
+            git config user.name "github-actions[bot]"
+            git config user.email "github-actions[bot]@users.noreply.github.com"
+            git add .
+            git commit -m "Update to version ${{ inputs.version }}"
path: .github/workflows/update-docker.yml
user: login: roshkhatri
body: ```suggestion
            git commit -m "Update versions.json and Dockerfiles for version ${{ inputs.version }}" -s
```
created_at: 2025-03-27T21:46:25Z
author_association: MEMBER
diff_hunk: @@ -61,26 +63,50 @@ jobs:
 
             # Run update script
             ./update.sh ${{ steps.strip_version.outputs.stripped_version }}
+            git config user.name "github-actions[bot]"
+            git config user.email "github-actions[bot]@users.noreply.github.com"
+            git add .
+            git commit -m "Update to version ${{ inputs.version }}"
 
-      - name: Check for Changes
+      - name: Setup Python
+        uses: actions/setup-python@v4
+        with:
+          python-version: '3.x'
+
+      - name: Checkout valkey-release-automation
+        uses: actions/checkout@v4
+        with:
+          repository: ${{ github.repository_owner }}/valkey-release-automation
+          path: valkey-release-automation
+          token: ${{ secrets.PAT_TOKEN }}
+
+      - name: Generate bashbrew output
+        id: generate-bashbrew
         run: |
-          if git diff --exit-code; then
-            echo "No changes detected. Skipping PR creation."
-            exit 0
-          fi
+          cd valkey-container
+          BASHBREW_SCRIPTS=/home/runner/work/_actions/docker-library/bashbrew/v0.1.12/scripts
+          "$BASHBREW_SCRIPTS/github-actions/generate.sh" > bashbrew_output.json
+
+      - name: Update Docker Description File
+        run: |
+          python valkey-release-automation/scripts/automate_docker_description.py valkey-container/bashbrew_output.json valkey-container/dockerhub-description-template.md valkey-container/dockerhub-description.md
+          rm valkey-container/bashbrew_output.json
+          cd valkey-container
+          git reset HEAD~1
path: .github/workflows/update-docker.yml
user: login: roshkhatri
body: ```suggestion
         git add .
         git commit -m "Update tags in the docker description for version ${{ inputs.version }}" -s
```
created_at: 2025-03-27T21:47:31Z
author_association: MEMBER

PR comments:
user: login: vudiep411
created_at: 2025-03-21T19:46:42Z
body: I would recommend creating unit testing for the python script since this can have many edge cases. One possible way is to set up test cases and create another step that would run pytest in github action. This unit test should be part of the CI when someone raises a PR.
author_association: CONTRIBUTOR
user: login: roshkhatri
created_at: 2025-03-21T21:45:27Z
body: Good callout @vudiep411 

@Nikhil-Manglore We do not want to update the docker hub from here, just update the PR
author_association: MEMBER
user: login: roshkhatri
created_at: 2025-03-24T21:31:13Z
body: Can you also explore to edit the aliases here for the latest and the major version?
https://github.com/roshkhatri/valkey-container/blob/58c923048945c7b532914396508cbf061bd43ac7/generate-stackbrew-library.sh#L4

RC of new version we should not add it or edit it.

Also, this would need to be done before running the update.sh script


author_association: MEMBER
user: login: Nikhil-Manglore
created_at: 2025-03-25T22:15:42Z
body: > Can you also explore to edit the aliases here for the latest and the major version? https://github.com/roshkhatri/valkey-container/blob/58c923048945c7b532914396508cbf061bd43ac7/generate-stackbrew-library.sh#L4
> 
> RC of new version we should not add it or edit it.
> 
> Also, this would need to be done before running the update.sh script

Will make a separate issue for this
author_association: MEMBER