Repository: mandiant/VM-Packages
Pull request #1320: Update wiki that lists packages by category
Author: login: sara-rn
Created: 2025-02-28T16:22:35Z | Merged: 2025-03-12T17:18:08Z

Update the script that generates the wiki with the list of packages by category. The new nuspec packages now contain the category even if it is not used to prevent packages from being uncategorized.

Resolves #1107 

Linked issue #1107: Move the categories to the nuspec & Add wiki page with tools by categories
### Details

Use GH actions to have two Wiki page that documents the tools and categories:
- One page should be for users with a header and a table with package name, tool name and tools description per category. This should make it easier to find the tools that we have available.
- The second page should be for script usage to use it in FLARE-VM/Commando-VM to sort the packages by category in the installer.
Linked issue comments: user: login: Ana06
created_at: 2025-01-09T11:03:50Z
body: I have discussed with @sara-rn the following implementation proposal, which is more robust than the original proposal:
- Add a **single _Packages_ Wiki page with a table with package name, tool name and tools description per category**. This should make it easier to find the tools that we have available. This is addressed in https://github.com/mandiant/VM-Packages/pull/1197.
- **Move the categories to the nuspec**. This way we can use the XML https://www.myget.org/F/vm-packages/Packages directly in the FLARE-VM/Commando-VM installers to sort the packages by category (tracked in FLARE-VM in https://github.com/mandiant/flare-vm/issues/578) and we do NOT need and extra wiki page generated by GH actions. @sara-rn is working on this part. The move of the categories to the nuspec involves:
  - <ins>**Add the category to the nuspec of all packages</ins> (for example in the [`tags` field](https://learn.microsoft.com/en-us/nuget/reference/nuspec#tags))**. 
  - <ins>**Adapt installers and linter**</ins>. **In the packages that use the category to install a shortcut in the `Tools` directory, take it from the nuspec instead of hard-coding it**. I think this should be easy to do, but in case this is not possible, we should add a linter to ensure they are on-sync. In any case, we should **adapt the [category linter](https://github.com/mandiant/VM-Packages/blob/main/scripts/test/lint.py#L316) to ensure the nuspec uses a valid category and that the category is accessed in the correct way**. We may want to split this into two linters. The changes described in this bullet point need to be in the same PR.
  - <ins>**Adapt the _Packages_ wiki page**</ins> introduced in https://github.com/mandiant/VM-Packages/pull/1197 to use the category from the nuspec. I think this is a nice improvement, as it removes the `Not Categorized` section making this page more useful.

As moving the categories to the nuspec and changing how they are accesses in the install scripts touches all packages. So I recommend we try to merge all PRs that would cause conflicts before sending this change and try to send small atomic PR if possible. For example, adding the categories to the nuspec could be an independent PR.


author_association: MEMBER
user: login: Ana06
created_at: 2025-01-29T11:13:29Z
body: Part of this issue (moving the categories to the nuspec) is still not addressed.
author_association: MEMBER
user: login: Ana06
created_at: 2025-01-31T10:03:21Z
body: ### More detailed proposal to move categories to the nuspec
1. **Add the category to the nuspec of ALL packages in the [`tags` field](https://learn.microsoft.com/en-us/nuget/reference/nuspec#tags)**. Note you may need to introduce new categories for some packages that don't have a category ATM like `IDA plugins`, as we plan to display this category to the user in the installer. We should try to keep the number of categories small and use existent categories if possible. For example `libraries.python3.vm` can have the existent `Python` category. 
2. **Add a new linter to [lint.py](https://github.com/mandiant/VM-Packages/blob/main/scripts/test/lint.py), which ensures the category in the nuspec is a valid one**. This linter implementation should be similar to the current `UsesInvalidCategory`, but the exceptions should be only the packages that we do not plan to display to the user: `installer.vm` and `debloat.vm`.
3. **Implement a `VM-Get-Category` function in common**. To implement this function you first need to look for the nuspec file using `$MyInvocation.MyCommand.Definition` and them get the `tags` field from it. `$MyInvocation.MyCommand.Definition` gives you the path of the current `chocolateyinstall.ps1` file, for example `C:\ProgramData\chocolatey\lib\capa.vm\tools\chocolateyinstall.ps1`. So you can easily get the package folder (that contains the nuspec) with something like `Split-Path -parent (Split-Path -parent $MyInvocation.MyCommand.Definition)`. Note we use already `$MyInvocation.MyCommand.Definition` in many packages, such as [libraries.python3.vm](https://github.com/mandiant/VM-Packages/blob/93ce8d3f2f7abcaa3d800c1fa2be8767c7c5d471/packages/libraries.python3.vm/tools/chocolateyuninstall.ps1#L5)
4. **Replace the `$category = '{category}'` line in the `chocolateyinstall.ps1` and `chocolateyuninstall.ps1` by `$category = VM-Get-Category` and adapt the linter** `UsesInvalidCategory` to look for the line `$category = VM-Get-Category` for all not excluded packages.
5. **Update `create_package_template.py`** to add the category in the nuspec and use `$category = VM-Get-Category` in `chocolateyinstall.ps1` and `chocolateyuninstall.ps1`
6. **Adapt the _Packages_ wiki page**</ins> introduced in https://github.com/mandiant/VM-Packages/pull/1197 to use the category from the nuspec. Only include packages with a category in the nuspec (ALL with the exception of `installer.vm` and `debloat.vm`).
@sara-rn please follow this implementation in order and send a separate PR for every of the number items.
author_association: MEMBER
user: login: sara-rn
created_at: 2025-01-31T10:21:38Z
body: Thanks @Ana06  for the suggestions, this makes it easier to implement as there were different ways to implement it and also  it would imply changing many files, therefore this order of implementation with separate PRs help.
I just miss another step, the last one, which would be 6. Change the ISSUE templates and the create package script to include this implementation. Let me know if this should be implemented as the last PR
author_association: CONTRIBUTOR
user: login: Ana06
created_at: 2025-01-31T10:43:09Z
body: @sara-rn you only need to update the categories in the ISSUE TEMPLATES. This should be done together with the categories linter as they are supposed to be on sync. See https://github.com/mandiant/VM-Packages/issues/298.
author_association: MEMBER
user: login: Ana06
created_at: 2025-01-31T10:47:21Z
body: @sara-rn you are right that the script to create packages also needs to be updated. I have update the list above to explicitly include it.
author_association: MEMBER
user: login: Ana06
created_at: 2025-02-19T15:38:23Z
body: This PR was closed automatically by #1272  as @sara had added `Closes #1107` to the description. But that PR does not close this issue, as it is just one step, all the items above need to be addressed. Until we have enabled back the linter (WIP in https://github.com/mandiant/VM-Packages/pull/1287) we can't consider this done. In addition, we still need to update the action for the wiki page.
author_association: MEMBER
user: login: Ana06
created_at: 2025-02-20T12:02:00Z
body: This issue has been closed again! 😠  @sara-rn all the following keywords close automatically the referenced issue:
```
close
closes
closed
fix
fixes
fixed
resolve
resolves
resolved
```

You can use `<KEYWORD> (partially) #issue` to reference the issue without closing it, but if you use `<KEYWORD> #issue` it is closed automatically:

![Image](https://github.com/user-attachments/assets/1a62f2d6-f0c8-4bbb-b888-08accc01c68b)

Check GitHub documentation about [Linking a pull request to an issue using a keyword](https://docs.github.com/en/issues/tracking-your-work-with-issues/using-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword) for more details.
author_association: MEMBER
user: login: Ana06
created_at: 2025-02-28T10:24:00Z
body: Only adapting the GH workflow that generates the Wiki page missing here.
author_association: MEMBER

Diff:
diff --git a/.github/workflows/generate_package_wiki.yml b/.github/workflows/generate_package_wiki.yml
index 34cb56659..6ee08cbcb 100644
--- a/.github/workflows/generate_package_wiki.yml
+++ b/.github/workflows/generate_package_wiki.yml
@@ -24,7 +24,7 @@ jobs:
 
       - name: Generate Wiki Content
         id: generate-content
-        run: scripts/utils/generate_package_wiki.ps1
+        run: python scripts/utils/generate_package_wiki.py
 
       - name: Commit changes
         working-directory: wiki
diff --git a/scripts/utils/generate_package_wiki.py b/scripts/utils/generate_package_wiki.py
new file mode 100644
index 000000000..457cef803
--- /dev/null
+++ b/scripts/utils/generate_package_wiki.py
@@ -0,0 +1,88 @@
+import argparse
+import os
+import pathlib
+import xml.etree.ElementTree as ET
+from collections import defaultdict
+
+# Dict[str (category), list (packages information)]
+packages_by_category = defaultdict(str)
+
+
+def sort_write_wiki_content(file_path):
+    """ Writes package information sorted by category to a Markdown wiki file.
+
+    This function iterates through the `packages_by_category` dictionary, which
+    contains package information organized by category. For each category, it
+    generates a Markdown header and a table containing package names and
+    descriptions. The resulting Markdown content is then written to the specified
+    file.
+
+    Args:
+        file_path (str): The path to the output Markdown file.
+    """
+    wikiContent = ""
+    for category in packages_by_category.keys():
+        wikiContent += "## " + category + "\n\n"
+        wikiContent += "| Package | Description |\n"
+        wikiContent += "| ------- | ----------- |\n"
+        wikiContent += packages_by_category[category] + "\n\n"
+    with open(file_path, "w", encoding="utf-8") as f:
+        f.write(wikiContent)
+
+
+def find_element_text(parent, tag_name):
+    """Retrieves the text content of a specified XML element.
+
+    This function searches for a child element with the given tag name within the
+    provided parent element. It handles XML namespaces by using a wildcard.
+
+    Args:
+        parent: The parent XML element.
+        tag_name: The name of the child element to find.
+
+    Returns:
+        The text content of the found element, or None if the element is not found.
+    """
+    tag = parent.find(f"{{*}}{tag_name}")
+    if tag is None:
+        return None
+    return tag.text
+
+
+def process_packages_directory(packages_dir):
+    """ Obtains the package name, description and category from a directory
+    
+    This function parses all the nuspec files in the specified packages directory.
+    It saves into packages_by_category[category] a line containing the package
+    name and the description separated by "|".
+    
+    Args:
+        packages: directory where the packages reside.
+    """
+    if not os.path.isdir(packages_dir):
+        raise FileNotFoundError(f"Packages directory not found: {packages_dir}")
+
+    for nuspec_path in pathlib.Path(packages_dir).glob("**/*.nuspec"):
+        nuspec_tree = ET.parse(nuspec_path)
+        nuspec_metadata = nuspec_tree.find("{*}metadata")
+        if nuspec_metadata is not None:
+            category = find_element_text(nuspec_metadata, "tags")
+            # We are only interested in the packages with a category.
+            # Some packages that assist with the istallation (such as common.vm) do
+            # not contain a category
+            if not category:
+                continue
+            description = find_element_text(nuspec_metadata, "description")
+            package = find_element_text(nuspec_metadata, "id")
+            """packages_by_category is a dictionary of str where each package and description
+            are appended formatted in Markdown"""
+            packages_by_category[category] += "| " + package + " | " + description + " |\n"
+
+
+if __name__ == "__main__":
+    parser = argparse.ArgumentParser(description="Create wiki of packages.")
+    parser.add_argument("--packages", type=str, required=False, default="packages", help="Path to packages")
+    parser.add_argument("--wiki", type=str, required=False, default="wiki/Packages.md", help="Path to the wiki")
+    args = parser.parse_args()
+    process_packages_directory(args.packages)
+    sort_write_wiki_content(args.wiki)


Reviews:
user: login: Ana06
body: @sara-rn the current script contains code to handle non categorized packages and handle errors that can't happen now that we have moved the category to the nuspec. This should be removed to simplify the code making it easier to understand. @sara-rn note that @emtuls first version of the script was a proof of concept and there may be issues with it. Finishing this GH action is part of https://github.com/mandiant/VM-Packages/issues/1107

I recomend you rewrite the script to generate the wiki in Python as the rest of our scripts, allowing us to keep them consistent and to reuse code. The code would be IMO also simpler and easier to modify and review. See for example https://github.com/mandiant/VM-Packages/blob/main/scripts/utils/generate_daily_results.py for another script that creates the wiki page. The linter is a good example to parse nuspec files.



state: CHANGES_REQUESTED
author_association: MEMBER
submitted_at: 2025-03-03T12:07:32Z
user: login: sara-rn
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-03T12:16:40Z
user: login: sara-rn
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-03T12:17:12Z
user: login: Ana06
body: can you please run the following Python linters/formatters so that we do not have to discuss format/style in the PR review:
```
black --line-length=120
flake8 --max-line-length=150
isort --profile black --line-length=120
```

I plan to enable these linters in our CI as I have done in FLARE-VM: https://github.com/mandiant/flare-vm/blob/main/.github/workflows/linter.yml I was waiting for https://github.com/mandiant/VM-Packages/pull/1261, but we may want to enable it already if that PR gets long to complete.

You can remove the PS script, the Python script goes in the right direction.
state: CHANGES_REQUESTED
author_association: MEMBER
submitted_at: 2025-03-05T10:59:39Z
user: login: sara-rn
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-05T11:30:50Z
user: login: sara-rn
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-05T11:31:29Z
user: login: Ana06
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-06T10:31:15Z
user: login: Ana06
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-06T10:36:38Z
user: login: sara-rn
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-06T11:18:30Z
user: login: sara-rn
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-06T11:19:09Z
user: login: sara-rn
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-06T12:01:19Z
user: login: Ana06
state: CHANGES_REQUESTED
author_association: MEMBER
submitted_at: 2025-03-07T14:18:03Z
user: login: Ana06
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-07T14:20:04Z
user: login: sara-rn
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-07T17:00:33Z
user: login: Ana06
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-07T20:14:04Z
user: login: Ana06
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-07T20:29:02Z
user: login: Ana06
body: This looks good @sara-rn, just some minor improvement suggestion :wink: 
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-12T08:46:43Z
user: login: Ana06
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-12T08:47:12Z
user: login: Ana06
body: Nice code @sara-rn, simple and well documented, easy to understand and maintain! :bouquet: Looking forward triggering the action! :rocket: 

state: APPROVED
author_association: MEMBER
submitted_at: 2025-03-12T17:17:56Z

Review comments:
diff_hunk: @@ -4,29 +4,23 @@ $wikiContent = ""
 $packagesByCategory = @{}
 
 foreach ($package in $packages) {
-    $categoryFile = "$packagesDir/$package/tools/chocolateyinstall.ps1"
-    if (Test-Path $categoryFile) {
-        $categoryMatch = Select-String -Path $categoryFile -Pattern '\$category\s*=\s*["''](.+?)["'']' -AllMatches
-        if ($categoryMatch) {
-            $category = $categoryMatch.Matches.Groups[1].Value.Trim("'""")
-        } else {
-            $category = "Not Categorized"
-        }
-    } else {
-        $category = "Not Categorized"
-    }
-
-    if (-not ($packagesByCategory.ContainsKey($category))) {
-        $packagesByCategory[$category] = ""
-    }
 
     try {
         $nuspecFile = "$packagesDir\$package\$package.nuspec"
         if (Test-Path $nuspecFile) {
             $xml = [xml](Get-Content $nuspecFile)
             $description = $xml.package.metadata.description
+            $category = $xml.package.metadata.tags
+            if (-not $category){
+                $category = "Not Categorized"
path: scripts/utils/generate_package_wiki.ps1
user: login: Ana06
body: **All packages that should be in this page must have a category**. Only the packages common.vm, debloat.vm, and installer.vm  that assist with the installation and allow to share code between packages are missing a category.
created_at: 2025-03-03T11:56:04Z
author_association: MEMBER
diff_hunk: @@ -4,29 +4,23 @@ $wikiContent = ""
 $packagesByCategory = @{}
 
 foreach ($package in $packages) {
-    $categoryFile = "$packagesDir/$package/tools/chocolateyinstall.ps1"
-    if (Test-Path $categoryFile) {
-        $categoryMatch = Select-String -Path $categoryFile -Pattern '\$category\s*=\s*["''](.+?)["'']' -AllMatches
-        if ($categoryMatch) {
-            $category = $categoryMatch.Matches.Groups[1].Value.Trim("'""")
-        } else {
-            $category = "Not Categorized"
-        }
-    } else {
-        $category = "Not Categorized"
-    }
-
-    if (-not ($packagesByCategory.ContainsKey($category))) {
-        $packagesByCategory[$category] = ""
-    }
 
     try {
         $nuspecFile = "$packagesDir\$package\$package.nuspec"
         if (Test-Path $nuspecFile) {
             $xml = [xml](Get-Content $nuspecFile)
             $description = $xml.package.metadata.description
+            $category = $xml.package.metadata.tags
+            if (-not $category){
+                $category = "Not Categorized"
+            } else {
+                if (-not ($packagesByCategory.ContainsKey($category))) {
+                    $packagesByCategory[$category] = ""
+                }
+            }
         } else {
             $description = "Nuspec file not found."
+            $category = "Nuspec file not found."
path: scripts/utils/generate_package_wiki.ps1
user: login: Ana06
body: What is this code doing? If the nuspec file is not found, something strange has happened and I think the script should fail. 


created_at: 2025-03-03T12:01:39Z
author_association: MEMBER
diff_hunk: @@ -4,29 +4,23 @@ $wikiContent = ""
 $packagesByCategory = @{}
 
 foreach ($package in $packages) {
-    $categoryFile = "$packagesDir/$package/tools/chocolateyinstall.ps1"
-    if (Test-Path $categoryFile) {
-        $categoryMatch = Select-String -Path $categoryFile -Pattern '\$category\s*=\s*["''](.+?)["'']' -AllMatches
-        if ($categoryMatch) {
-            $category = $categoryMatch.Matches.Groups[1].Value.Trim("'""")
-        } else {
-            $category = "Not Categorized"
-        }
-    } else {
-        $category = "Not Categorized"
-    }
-
-    if (-not ($packagesByCategory.ContainsKey($category))) {
-        $packagesByCategory[$category] = ""
-    }
 
     try {
         $nuspecFile = "$packagesDir\$package\$package.nuspec"
         if (Test-Path $nuspecFile) {
             $xml = [xml](Get-Content $nuspecFile)
             $description = $xml.package.metadata.description
+            $category = $xml.package.metadata.tags
+            if (-not $category){
+                $category = "Not Categorized"
path: scripts/utils/generate_package_wiki.ps1
user: login: sara-rn
body: understood, I simply adapted the existent code, I will write a new Python script from the scratch
created_at: 2025-03-03T12:16:40Z
author_association: CONTRIBUTOR
diff_hunk: @@ -4,29 +4,23 @@ $wikiContent = ""
 $packagesByCategory = @{}
 
 foreach ($package in $packages) {
-    $categoryFile = "$packagesDir/$package/tools/chocolateyinstall.ps1"
-    if (Test-Path $categoryFile) {
-        $categoryMatch = Select-String -Path $categoryFile -Pattern '\$category\s*=\s*["''](.+?)["'']' -AllMatches
-        if ($categoryMatch) {
-            $category = $categoryMatch.Matches.Groups[1].Value.Trim("'""")
-        } else {
-            $category = "Not Categorized"
-        }
-    } else {
-        $category = "Not Categorized"
-    }
-
-    if (-not ($packagesByCategory.ContainsKey($category))) {
-        $packagesByCategory[$category] = ""
-    }
 
     try {
         $nuspecFile = "$packagesDir\$package\$package.nuspec"
         if (Test-Path $nuspecFile) {
             $xml = [xml](Get-Content $nuspecFile)
             $description = $xml.package.metadata.description
+            $category = $xml.package.metadata.tags
+            if (-not $category){
+                $category = "Not Categorized"
+            } else {
+                if (-not ($packagesByCategory.ContainsKey($category))) {
+                    $packagesByCategory[$category] = ""
+                }
+            }
         } else {
             $description = "Nuspec file not found."
+            $category = "Nuspec file not found."
path: scripts/utils/generate_package_wiki.ps1
user: login: sara-rn
body: this was existent code that printed that in case the nuspec file was not found or able to parse
created_at: 2025-03-03T12:17:12Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,62 @@
+import os
+import sys
+import re
+import datetime
+import argparse
+
+
+packagesByCategory={}
+
+
+def write_wiki_content(file_path):
+    wikiContent = ""
+    for category in packagesByCategory.keys():
+        wikiContent += "## " + category + "\n\n"
+        wikiContent += "| Package | Description |\n"
+        wikiContent += "|---|---|\n"
+        wikiContent += packagesByCategory[category] + "\n\n"
+    try:   
+        with open(file_path, 'w', encoding='utf-8') as f:
+           f.write(wikiContent)
+    except FileNotFoundError:
path: scripts/utils/generate_package_wiki.py
user: login: Ana06
body: how can this error happen?
created_at: 2025-03-05T09:41:36Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,62 @@
+import os
+import sys
+import re
+import datetime
+import argparse
+
+
+packagesByCategory={}
+
+
+def write_wiki_content(file_path):
+    wikiContent = ""
+    for category in packagesByCategory.keys():
+        wikiContent += "## " + category + "\n\n"
+        wikiContent += "| Package | Description |\n"
+        wikiContent += "|---|---|\n"
+        wikiContent += packagesByCategory[category] + "\n\n"
+    try:   
+        with open(file_path, 'w', encoding='utf-8') as f:
+           f.write(wikiContent)
+    except FileNotFoundError:
+        print(f"Error: File '{file_path}' not found.")
+
+  
+
+
+def process_packages_directory(packages_dir):
+    """Processes all.nuspec files in the 'packages' directory structure."""
+    if not os.path.isdir(packages_dir):
path: scripts/utils/generate_package_wiki.py
user: login: Ana06
body: If the packages directory does not exist, the script should exit with error to ensure the action fails as something has gone wrong. This `if` causes the script succeeds in this case.
created_at: 2025-03-05T09:43:52Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,62 @@
+import os
+import sys
+import re
+import datetime
+import argparse
+
+
+packagesByCategory={}
+
+
+def write_wiki_content(file_path):
+    wikiContent = ""
+    for category in packagesByCategory.keys():
+        wikiContent += "## " + category + "\n\n"
+        wikiContent += "| Package | Description |\n"
+        wikiContent += "|---|---|\n"
+        wikiContent += packagesByCategory[category] + "\n\n"
+    try:   
+        with open(file_path, 'w', encoding='utf-8') as f:
+           f.write(wikiContent)
+    except FileNotFoundError:
+        print(f"Error: File '{file_path}' not found.")
+
+  
+
+
+def process_packages_directory(packages_dir):
+    """Processes all.nuspec files in the 'packages' directory structure."""
+    if not os.path.isdir(packages_dir):
+        print(f"Error: Directory '{packages_dir}' not found.")
+        return
+
+    
+    for package_name in os.listdir(packages_dir):
+        package = os.path.splitext(package_name)[0]  
+        package_path = os.path.join(packages_dir, package_name)
+        nuspec_file = os.path.join(package_path, f"{package_name}.nuspec")
path: scripts/utils/generate_package_wiki.py
user: login: Ana06
body: As you want to iterate the nuspecs, I think using `glob` from `pathlib.Path` makes the code more readable. I would also rename `nuspc_file` by `nuspec_path`:

```suggestion
    for nuspec_path in pathlib.Path(packages_dir).glob('**/*.nuspec'):
```

You need to import `pathlib`.
created_at: 2025-03-05T09:51:16Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,62 @@
+import os
+import sys
+import re
+import datetime
+import argparse
+
+
+packagesByCategory={}
+
+
+def write_wiki_content(file_path):
+    wikiContent = ""
+    for category in packagesByCategory.keys():
+        wikiContent += "## " + category + "\n\n"
+        wikiContent += "| Package | Description |\n"
+        wikiContent += "|---|---|\n"
+        wikiContent += packagesByCategory[category] + "\n\n"
+    try:   
+        with open(file_path, 'w', encoding='utf-8') as f:
+           f.write(wikiContent)
+    except FileNotFoundError:
+        print(f"Error: File '{file_path}' not found.")
+
+  
+
+
+def process_packages_directory(packages_dir):
+    """Processes all.nuspec files in the 'packages' directory structure."""
+    if not os.path.isdir(packages_dir):
+        print(f"Error: Directory '{packages_dir}' not found.")
+        return
+
+    
+    for package_name in os.listdir(packages_dir):
+        package = os.path.splitext(package_name)[0]  
+        package_path = os.path.join(packages_dir, package_name)
+        nuspec_file = os.path.join(package_path, f"{package_name}.nuspec")
+        try:
+            with open(nuspec_file, 'r', encoding='utf-8') as f:
+                content = f.read()
+            category_match = re.search(r'<tags>([^<]*)</tags>', content)
+            if category_match:
+                category = category_match.group(1)
+                description_match = re.search (r'<description>([^<]*)</description>', content)
+                description = description_match.group(1)
+                if not category in packagesByCategory:
+            	    packagesByCategory[category]=""
+                packagesByCategory[category]+="|" + package + "|" + description + "|\n"
+                print (f"| {package} | {description} |")
+        except FileNotFoundError:
path: scripts/utils/generate_package_wiki.py
user: login: Ana06
body: how can this error happen? If this error happens something has gone wrong and the script should exit with an error to ensure the GH action fails.
created_at: 2025-03-05T09:59:12Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,62 @@
+import os
+import sys
+import re
+import datetime
+import argparse
+
+
+packagesByCategory={}
+
+
+def write_wiki_content(file_path):
+    wikiContent = ""
+    for category in packagesByCategory.keys():
+        wikiContent += "## " + category + "\n\n"
+        wikiContent += "| Package | Description |\n"
+        wikiContent += "|---|---|\n"
+        wikiContent += packagesByCategory[category] + "\n\n"
+    try:   
+        with open(file_path, 'w', encoding='utf-8') as f:
+           f.write(wikiContent)
+    except FileNotFoundError:
+        print(f"Error: File '{file_path}' not found.")
+
+  
+
+
+def process_packages_directory(packages_dir):
+    """Processes all.nuspec files in the 'packages' directory structure."""
+    if not os.path.isdir(packages_dir):
+        print(f"Error: Directory '{packages_dir}' not found.")
+        return
+
+    
+    for package_name in os.listdir(packages_dir):
+        package = os.path.splitext(package_name)[0]  
+        package_path = os.path.join(packages_dir, package_name)
+        nuspec_file = os.path.join(package_path, f"{package_name}.nuspec")
+        try:
+            with open(nuspec_file, 'r', encoding='utf-8') as f:
+                content = f.read()
+            category_match = re.search(r'<tags>([^<]*)</tags>', content)
+            if category_match:
+                category = category_match.group(1)
+                description_match = re.search (r'<description>([^<]*)</description>', content)
+                description = description_match.group(1)
path: scripts/utils/generate_package_wiki.py
user: login: Ana06
body: I think it is more readable and a more robust solution to parse the XML using Python builtin `xml.etree.ElementTree` than doing it regular expressions as you need to access several fields. This will also allow us to add more fields to the Wiki page in the future.
```suggestion
            nuspec_tree = ET.parse(nuspec_path)
            nuspec_metadata = nuspec_tree.find("{*}metadata")
            package = nuspec_metadata.find("{*}id").text
            category = nuspec_metadata.find("{*}tags").text
            description = nuspec_metadata.find("{*}description").text
```

To avoid code duplication this code can be simplified with the following helper function:
```Python
get_text(element, field):
    return element.find("{*}field").text
````

created_at: 2025-03-05T10:41:57Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,62 @@
+import os
+import sys
+import re
+import datetime
+import argparse
+
+
+packagesByCategory={}
+
+
+def write_wiki_content(file_path):
+    wikiContent = ""
+    for category in packagesByCategory.keys():
+        wikiContent += "## " + category + "\n\n"
+        wikiContent += "| Package | Description |\n"
+        wikiContent += "|---|---|\n"
+        wikiContent += packagesByCategory[category] + "\n\n"
+    try:   
+        with open(file_path, 'w', encoding='utf-8') as f:
+           f.write(wikiContent)
+    except FileNotFoundError:
+        print(f"Error: File '{file_path}' not found.")
+
+  
+
+
+def process_packages_directory(packages_dir):
+    """Processes all.nuspec files in the 'packages' directory structure."""
+    if not os.path.isdir(packages_dir):
+        print(f"Error: Directory '{packages_dir}' not found.")
+        return
+
+    
+    for package_name in os.listdir(packages_dir):
+        package = os.path.splitext(package_name)[0]  
+        package_path = os.path.join(packages_dir, package_name)
+        nuspec_file = os.path.join(package_path, f"{package_name}.nuspec")
+        try:
+            with open(nuspec_file, 'r', encoding='utf-8') as f:
+                content = f.read()
+            category_match = re.search(r'<tags>([^<]*)</tags>', content)
+            if category_match:
+                category = category_match.group(1)
+                description_match = re.search (r'<description>([^<]*)</description>', content)
+                description = description_match.group(1)
+                if not category in packagesByCategory:
+            	    packagesByCategory[category]=""
path: scripts/utils/generate_package_wiki.py
user: login: Ana06
body: Use [defaultdict](https://docs.python.org/2/library/collections.html#collections.defaultdict) to initialize this variable to remove this `if`

```suggestion
```
created_at: 2025-03-05T10:43:32Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,62 @@
+import os
+import sys
+import re
+import datetime
+import argparse
+
+
+packagesByCategory={}
path: scripts/utils/generate_package_wiki.py
user: login: Ana06
body: Follow [variables naming convention in Google Python Style Guide](https://google.github.io/styleguide/pyguide.html#316-naming)
```suggestion
packages_by_category = {}
```
created_at: 2025-03-05T10:46:26Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,62 @@
+import os
+import sys
+import re
+import datetime
+import argparse
+
+
+packagesByCategory={}
+
+
+def write_wiki_content(file_path):
+    wikiContent = ""
+    for category in packagesByCategory.keys():
+        wikiContent += "## " + category + "\n\n"
+        wikiContent += "| Package | Description |\n"
+        wikiContent += "|---|---|\n"
+        wikiContent += packagesByCategory[category] + "\n\n"
+    try:   
+        with open(file_path, 'w', encoding='utf-8') as f:
+           f.write(wikiContent)
+    except FileNotFoundError:
+        print(f"Error: File '{file_path}' not found.")
+
+  
+
+
+def process_packages_directory(packages_dir):
+    """Processes all.nuspec files in the 'packages' directory structure."""
+    if not os.path.isdir(packages_dir):
+        print(f"Error: Directory '{packages_dir}' not found.")
+        return
+
+    
+    for package_name in os.listdir(packages_dir):
+        package = os.path.splitext(package_name)[0]  
+        package_path = os.path.join(packages_dir, package_name)
+        nuspec_file = os.path.join(package_path, f"{package_name}.nuspec")
+        try:
+            with open(nuspec_file, 'r', encoding='utf-8') as f:
+                content = f.read()
+            category_match = re.search(r'<tags>([^<]*)</tags>', content)
+            if category_match:
+                category = category_match.group(1)
+                description_match = re.search (r'<description>([^<]*)</description>', content)
+                description = description_match.group(1)
+                if not category in packagesByCategory:
+            	    packagesByCategory[category]=""
+                packagesByCategory[category]+="|" + package + "|" + description + "|\n"
+                print (f"| {package} | {description} |")
path: scripts/utils/generate_package_wiki.py
user: login: Ana06
body: what is the purpose of this `print`?
created_at: 2025-03-05T10:47:23Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,62 @@
+import os
+import sys
+import re
+import datetime
+import argparse
+
+
+packagesByCategory={}
+
+
+def write_wiki_content(file_path):
+    wikiContent = ""
+    for category in packagesByCategory.keys():
+        wikiContent += "## " + category + "\n\n"
+        wikiContent += "| Package | Description |\n"
+        wikiContent += "|---|---|\n"
+        wikiContent += packagesByCategory[category] + "\n\n"
+    try:   
+        with open(file_path, 'w', encoding='utf-8') as f:
+           f.write(wikiContent)
+    except FileNotFoundError:
+        print(f"Error: File '{file_path}' not found.")
+
+  
+
+
+def process_packages_directory(packages_dir):
+    """Processes all.nuspec files in the 'packages' directory structure."""
+    if not os.path.isdir(packages_dir):
+        print(f"Error: Directory '{packages_dir}' not found.")
+        return
+
+    
+    for package_name in os.listdir(packages_dir):
+        package = os.path.splitext(package_name)[0]  
+        package_path = os.path.join(packages_dir, package_name)
+        nuspec_file = os.path.join(package_path, f"{package_name}.nuspec")
+        try:
+            with open(nuspec_file, 'r', encoding='utf-8') as f:
+                content = f.read()
+            category_match = re.search(r'<tags>([^<]*)</tags>', content)
+            if category_match:
+                category = category_match.group(1)
+                description_match = re.search (r'<description>([^<]*)</description>', content)
+                description = description_match.group(1)
+                if not category in packagesByCategory:
+            	    packagesByCategory[category]=""
+                packagesByCategory[category]+="|" + package + "|" + description + "|\n"
+                print (f"| {package} | {description} |")
+        except FileNotFoundError:
+            print(f"Error: File '{nuspec_file}' not found.")
+        
+
+	    
+if __name__ == "__main__":
+
+
+    parser = argparse.ArgumentParser(description="Create wiki of packages.")
+    parser.add_argument("packages", type=str, help="Path to packages")
+    args = parser.parse_args()
+    process_packages_directory(args.packages)
+    write_wiki_content("wiki/Packages.md")
path: scripts/utils/generate_package_wiki.py
user: login: Ana06
body: I find it inconsistent to have an argument for the path for the packages directory and not one for the wiki page path. I suggest to have either both as arguments or have both as global constant variables. The only reason to provide them as argument is to allow to call the script from a different folder locally, in the action they will always be the same. If you use them as argument, please provide a default value, as in the action they will be always the same, so that we do not need to provide them there.
created_at: 2025-03-05T10:49:30Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,62 @@
+import os
+import sys
+import re
+import datetime
+import argparse
+
+
+packagesByCategory={}
+
+
+def write_wiki_content(file_path):
+    wikiContent = ""
+    for category in packagesByCategory.keys():
+        wikiContent += "## " + category + "\n\n"
+        wikiContent += "| Package | Description |\n"
+        wikiContent += "|---|---|\n"
+        wikiContent += packagesByCategory[category] + "\n\n"
+    try:   
+        with open(file_path, 'w', encoding='utf-8') as f:
+           f.write(wikiContent)
+    except FileNotFoundError:
+        print(f"Error: File '{file_path}' not found.")
+
+  
+
+
+def process_packages_directory(packages_dir):
+    """Processes all.nuspec files in the 'packages' directory structure."""
path: scripts/utils/generate_package_wiki.py
user: login: Ana06
body: This description is not clear to me and has a typo. Please add more details.
created_at: 2025-03-05T10:50:31Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,62 @@
+import os
+import sys
+import re
+import datetime
+import argparse
+
+
+packagesByCategory={}
+
+
+def write_wiki_content(file_path):
path: scripts/utils/generate_package_wiki.py
user: login: Ana06
body: Please add a description to this function.
created_at: 2025-03-05T10:50:47Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,62 @@
+import os
+import sys
+import re
+import datetime
+import argparse
+
+
+packagesByCategory={}
+
+
+def write_wiki_content(file_path):
+    wikiContent = ""
+    for category in packagesByCategory.keys():
+        wikiContent += "## " + category + "\n\n"
+        wikiContent += "| Package | Description |\n"
+        wikiContent += "|---|---|\n"
+        wikiContent += packagesByCategory[category] + "\n\n"
path: scripts/utils/generate_package_wiki.py
user: login: Ana06
body: I find it confusing that the table format is defined in two different functions. This is an issue when adding new items to the table. :thinking: 
created_at: 2025-03-05T10:55:31Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,62 @@
+import os
+import sys
+import re
+import datetime
+import argparse
+
+
+packagesByCategory={}
+
+
+def write_wiki_content(file_path):
+    wikiContent = ""
+    for category in packagesByCategory.keys():
+        wikiContent += "## " + category + "\n\n"
+        wikiContent += "| Package | Description |\n"
+        wikiContent += "|---|---|\n"
+        wikiContent += packagesByCategory[category] + "\n\n"
path: scripts/utils/generate_package_wiki.py
user: login: sara-rn
body: yes,  that is because each value in the dictionary contains the package name and the description already formatted in markdown previously. It is not possible to add the markdown headers then as this can only be done when all the packages have been processed. It is done in the `write_wiki_content` also because dictionaries are sorted alphabetically by default.
created_at: 2025-03-05T11:30:49Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,62 @@
+import os
+import sys
+import re
+import datetime
+import argparse
+
+
+packagesByCategory={}
+
+
+def write_wiki_content(file_path):
+    wikiContent = ""
+    for category in packagesByCategory.keys():
+        wikiContent += "## " + category + "\n\n"
+        wikiContent += "| Package | Description |\n"
+        wikiContent += "|---|---|\n"
+        wikiContent += packagesByCategory[category] + "\n\n"
+    try:   
+        with open(file_path, 'w', encoding='utf-8') as f:
+           f.write(wikiContent)
+    except FileNotFoundError:
path: scripts/utils/generate_package_wiki.py
user: login: sara-rn
body: it shouldn't happen as this is executed from github actions, shall I just remove the try / except?
created_at: 2025-03-05T11:31:28Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,62 @@
+import os
+import sys
+import re
+import datetime
+import argparse
+
+
+packagesByCategory={}
+
+
+def write_wiki_content(file_path):
+    wikiContent = ""
+    for category in packagesByCategory.keys():
+        wikiContent += "## " + category + "\n\n"
+        wikiContent += "| Package | Description |\n"
+        wikiContent += "|---|---|\n"
+        wikiContent += packagesByCategory[category] + "\n\n"
path: scripts/utils/generate_package_wiki.py
user: login: Ana06
body: yes, but the current implementation makes it difficult to understand it. This can be enhanced for example in one of the following ways:
- `packagesByCategory` contains a dictionary or tuple so that you keep the logic from parsing the packages and the logic from creating markdown in two separate functions
- With the other code suggestions, the resulting code may be simpler enough that we can have all the logic in the same function.

In any case, documentation in the code (as function descriptions and comments) is needed.

created_at: 2025-03-06T10:31:15Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,62 @@
+import os
+import sys
+import re
+import datetime
+import argparse
+
+
+packagesByCategory={}
+
+
+def write_wiki_content(file_path):
+    wikiContent = ""
+    for category in packagesByCategory.keys():
+        wikiContent += "## " + category + "\n\n"
+        wikiContent += "| Package | Description |\n"
+        wikiContent += "|---|---|\n"
+        wikiContent += packagesByCategory[category] + "\n\n"
+    try:   
+        with open(file_path, 'w', encoding='utf-8') as f:
+           f.write(wikiContent)
+    except FileNotFoundError:
path: scripts/utils/generate_package_wiki.py
user: login: Ana06
body: We want the action to fail if something unexpected happens. So you should only catch errors that you can handle. In case you are not able to write the wiki page, the action should fail because you are capturing the error.  So yes, remove the try-catch.
created_at: 2025-03-06T10:36:38Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,62 @@
+import os
+import sys
+import re
+import datetime
+import argparse
+
+
+packagesByCategory={}
+
+
+def write_wiki_content(file_path):
+    wikiContent = ""
+    for category in packagesByCategory.keys():
+        wikiContent += "## " + category + "\n\n"
+        wikiContent += "| Package | Description |\n"
+        wikiContent += "|---|---|\n"
+        wikiContent += packagesByCategory[category] + "\n\n"
+    try:   
+        with open(file_path, 'w', encoding='utf-8') as f:
+           f.write(wikiContent)
+    except FileNotFoundError:
+        print(f"Error: File '{file_path}' not found.")
+
+  
+
+
+def process_packages_directory(packages_dir):
+    """Processes all.nuspec files in the 'packages' directory structure."""
+    if not os.path.isdir(packages_dir):
+        print(f"Error: Directory '{packages_dir}' not found.")
+        return
+
+    
+    for package_name in os.listdir(packages_dir):
+        package = os.path.splitext(package_name)[0]  
+        package_path = os.path.join(packages_dir, package_name)
+        nuspec_file = os.path.join(package_path, f"{package_name}.nuspec")
+        try:
+            with open(nuspec_file, 'r', encoding='utf-8') as f:
+                content = f.read()
+            category_match = re.search(r'<tags>([^<]*)</tags>', content)
+            if category_match:
+                category = category_match.group(1)
+                description_match = re.search (r'<description>([^<]*)</description>', content)
+                description = description_match.group(1)
+                if not category in packagesByCategory:
+            	    packagesByCategory[category]=""
+                packagesByCategory[category]+="|" + package + "|" + description + "|\n"
+                print (f"| {package} | {description} |")
path: scripts/utils/generate_package_wiki.py
user: login: sara-rn
body: none, I was just debugging it and forgot to remove it
created_at: 2025-03-06T11:18:30Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,62 @@
+import os
+import sys
+import re
+import datetime
+import argparse
+
+
+packagesByCategory={}
+
+
+def write_wiki_content(file_path):
+    wikiContent = ""
+    for category in packagesByCategory.keys():
+        wikiContent += "## " + category + "\n\n"
+        wikiContent += "| Package | Description |\n"
+        wikiContent += "|---|---|\n"
+        wikiContent += packagesByCategory[category] + "\n\n"
+    try:   
+        with open(file_path, 'w', encoding='utf-8') as f:
+           f.write(wikiContent)
+    except FileNotFoundError:
+        print(f"Error: File '{file_path}' not found.")
+
+  
+
+
+def process_packages_directory(packages_dir):
+    """Processes all.nuspec files in the 'packages' directory structure."""
+    if not os.path.isdir(packages_dir):
+        print(f"Error: Directory '{packages_dir}' not found.")
+        return
+
+    
+    for package_name in os.listdir(packages_dir):
+        package = os.path.splitext(package_name)[0]  
+        package_path = os.path.join(packages_dir, package_name)
+        nuspec_file = os.path.join(package_path, f"{package_name}.nuspec")
+        try:
+            with open(nuspec_file, 'r', encoding='utf-8') as f:
+                content = f.read()
+            category_match = re.search(r'<tags>([^<]*)</tags>', content)
+            if category_match:
+                category = category_match.group(1)
+                description_match = re.search (r'<description>([^<]*)</description>', content)
+                description = description_match.group(1)
path: scripts/utils/generate_package_wiki.py
user: login: sara-rn
body: some packages such as `common.vm` do not contain `tags` therefore a validation is required otherwise it fails
created_at: 2025-03-06T11:19:09Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,62 @@
+import os
+import sys
+import re
+import datetime
+import argparse
+
+
+packagesByCategory={}
+
+
+def write_wiki_content(file_path):
+    wikiContent = ""
+    for category in packagesByCategory.keys():
+        wikiContent += "## " + category + "\n\n"
+        wikiContent += "| Package | Description |\n"
+        wikiContent += "|---|---|\n"
+        wikiContent += packagesByCategory[category] + "\n\n"
path: scripts/utils/generate_package_wiki.py
user: login: sara-rn
body: this is the only thing I haven't implemented as I didn't see the solution you proposed
created_at: 2025-03-06T12:01:19Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,70 @@
+import argparse
+import os
+import pathlib
+import sys
+import xml.etree.ElementTree as ET
+from collections import defaultdict
+
+packagesByCategory = defaultdict(str)
+
+
+def sort_write_wiki_content(file_path):
+    """
+    iterates over each category from the dictionary packagesByCategory and
+    writes the Markdown formatted string that contains
+    - header with the category name
+    - table with two columns: Package and Description of each of the packages"""
path: scripts/utils/generate_package_wiki.py
user: login: Ana06
body: Use Google recommended style to document Python functions in this and the other functions: https://github.com/google/styleguide/blob/gh-pages/pyguide.md#383-functions-and-methods
created_at: 2025-03-07T14:00:50Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,62 @@
+import os
+import sys
+import re
+import datetime
+import argparse
+
+
+packagesByCategory={}
+
+
+def write_wiki_content(file_path):
+    wikiContent = ""
+    for category in packagesByCategory.keys():
+        wikiContent += "## " + category + "\n\n"
+        wikiContent += "| Package | Description |\n"
+        wikiContent += "|---|---|\n"
+        wikiContent += packagesByCategory[category] + "\n\n"
+    try:   
+        with open(file_path, 'w', encoding='utf-8') as f:
+           f.write(wikiContent)
+    except FileNotFoundError:
+        print(f"Error: File '{file_path}' not found.")
+
+  
+
+
+def process_packages_directory(packages_dir):
+    """Processes all.nuspec files in the 'packages' directory structure."""
+    if not os.path.isdir(packages_dir):
path: scripts/utils/generate_package_wiki.py
user: login: Ana06
body: This is still not addressed. `sys.exit()` defaults to return code `0`, which still does not make the action fail. I think you should raise an exception here instead of trying to re-implement the same logic with a `print` + `exit`. Something like:
```Python
raise FileNotFoundError(f"Packages directory not found: {packages_dir}")
```

Alternatively, what happens if you remove this `if`? If the script fails with an error that makes the issue clear, that would be good too.
created_at: 2025-03-07T14:05:56Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,70 @@
+import argparse
+import os
+import pathlib
+import sys
+import xml.etree.ElementTree as ET
+from collections import defaultdict
+
+packagesByCategory = defaultdict(str)
+
+
+def sort_write_wiki_content(file_path):
+    """
+    iterates over each category from the dictionary packagesByCategory and
+    writes the Markdown formatted string that contains
+    - header with the category name
+    - table with two columns: Package and Description of each of the packages"""
+    wikiContent = ""
+    for category in packagesByCategory.keys():
+        wikiContent += "## " + category + "\n\n"
+        wikiContent += "| Package | Description |\n"
+        wikiContent += "|---|---|\n"
+        wikiContent += packagesByCategory[category] + "\n\n"
+    with open(file_path, "w", encoding="utf-8") as f:
+        f.write(wikiContent)
+
+
+# Helper function to find elements
+def find_element(parent, tag_name):
+    return parent.find(f"{{*}}{tag_name}")
+
+
+def process_packages_directory(packages_dir):
+    """Processes all the nuspec files in the 'packages' directory structure.
+    Once a category is found it adds the following line to the dictionary:
+    packagesByCategory where the key is the category:"""
+    if not os.path.isdir(packages_dir):
+        print(f"Error: Directory '{packages_dir}' not found.")
+        sys.exit()
+
+    for nuspec_path in pathlib.Path(packages_dir).glob("**/*.nuspec"):
+
+        try:
+            nuspec_tree = ET.parse(nuspec_path)
+            nuspec_metadata = find_element(nuspec_tree, "metadata")
+            if nuspec_metadata is not None:
+                description = find_element(nuspec_metadata, "description").text
+                package = find_element(nuspec_metadata, "id").text
+                category_element = find_element(nuspec_metadata, "tags")
+                # several packages such as common.vm do not contain a category
path: scripts/utils/generate_package_wiki.py
user: login: Ana06
body: [nitpick] Add some more details
```suggestion
                # We are only interested in the packages with a category
                # Some packages that assist with the installation (such as common.vm) do not contain a category
```
created_at: 2025-03-07T14:07:46Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,70 @@
+import argparse
+import os
+import pathlib
+import sys
+import xml.etree.ElementTree as ET
+from collections import defaultdict
+
+packagesByCategory = defaultdict(str)
+
+
+def sort_write_wiki_content(file_path):
+    """
+    iterates over each category from the dictionary packagesByCategory and
+    writes the Markdown formatted string that contains
+    - header with the category name
+    - table with two columns: Package and Description of each of the packages"""
+    wikiContent = ""
+    for category in packagesByCategory.keys():
+        wikiContent += "## " + category + "\n\n"
+        wikiContent += "| Package | Description |\n"
+        wikiContent += "|---|---|\n"
+        wikiContent += packagesByCategory[category] + "\n\n"
+    with open(file_path, "w", encoding="utf-8") as f:
+        f.write(wikiContent)
+
+
+# Helper function to find elements
+def find_element(parent, tag_name):
+    return parent.find(f"{{*}}{tag_name}")
+
+
+def process_packages_directory(packages_dir):
+    """Processes all the nuspec files in the 'packages' directory structure.
+    Once a category is found it adds the following line to the dictionary:
+    packagesByCategory where the key is the category:"""
+    if not os.path.isdir(packages_dir):
+        print(f"Error: Directory '{packages_dir}' not found.")
+        sys.exit()
+
+    for nuspec_path in pathlib.Path(packages_dir).glob("**/*.nuspec"):
+
path: scripts/utils/generate_package_wiki.py
user: login: Ana06
body: [nitpick]
```suggestion
```
created_at: 2025-03-07T14:08:06Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,70 @@
+import argparse
+import os
+import pathlib
+import sys
+import xml.etree.ElementTree as ET
+from collections import defaultdict
+
+packagesByCategory = defaultdict(str)
+
+
+def sort_write_wiki_content(file_path):
+    """
+    iterates over each category from the dictionary packagesByCategory and
+    writes the Markdown formatted string that contains
+    - header with the category name
+    - table with two columns: Package and Description of each of the packages"""
+    wikiContent = ""
+    for category in packagesByCategory.keys():
+        wikiContent += "## " + category + "\n\n"
+        wikiContent += "| Package | Description |\n"
+        wikiContent += "|---|---|\n"
+        wikiContent += packagesByCategory[category] + "\n\n"
+    with open(file_path, "w", encoding="utf-8") as f:
+        f.write(wikiContent)
+
+
+# Helper function to find elements
+def find_element(parent, tag_name):
+    return parent.find(f"{{*}}{tag_name}")
+
+
+def process_packages_directory(packages_dir):
+    """Processes all the nuspec files in the 'packages' directory structure.
+    Once a category is found it adds the following line to the dictionary:
+    packagesByCategory where the key is the category:"""
path: scripts/utils/generate_package_wiki.py
user: login: Ana06
body: This description is confusing to me.
created_at: 2025-03-07T14:09:02Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,70 @@
+import argparse
+import os
+import pathlib
+import sys
+import xml.etree.ElementTree as ET
+from collections import defaultdict
+
+packagesByCategory = defaultdict(str)
+
+
+def sort_write_wiki_content(file_path):
+    """
+    iterates over each category from the dictionary packagesByCategory and
+    writes the Markdown formatted string that contains
+    - header with the category name
+    - table with two columns: Package and Description of each of the packages"""
+    wikiContent = ""
+    for category in packagesByCategory.keys():
+        wikiContent += "## " + category + "\n\n"
+        wikiContent += "| Package | Description |\n"
+        wikiContent += "|---|---|\n"
+        wikiContent += packagesByCategory[category] + "\n\n"
+    with open(file_path, "w", encoding="utf-8") as f:
+        f.write(wikiContent)
+
+
+# Helper function to find elements
+def find_element(parent, tag_name):
+    return parent.find(f"{{*}}{tag_name}")
path: scripts/utils/generate_package_wiki.py
user: login: Ana06
body: Extend this function to avoid having to repeat the `.text` when calling this function:
```suggestion
def find_element(parent, tag_name):
    tag = parent.find(f"{{*}}{tag_name}")
    if not tag:
        return None
    return tag.text
```
created_at: 2025-03-07T14:11:19Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,70 @@
+import argparse
+import os
+import pathlib
+import sys
+import xml.etree.ElementTree as ET
+from collections import defaultdict
+
+packagesByCategory = defaultdict(str)
+
+
+def sort_write_wiki_content(file_path):
+    """
+    iterates over each category from the dictionary packagesByCategory and
+    writes the Markdown formatted string that contains
+    - header with the category name
+    - table with two columns: Package and Description of each of the packages"""
+    wikiContent = ""
+    for category in packagesByCategory.keys():
+        wikiContent += "## " + category + "\n\n"
+        wikiContent += "| Package | Description |\n"
+        wikiContent += "|---|---|\n"
+        wikiContent += packagesByCategory[category] + "\n\n"
+    with open(file_path, "w", encoding="utf-8") as f:
+        f.write(wikiContent)
+
+
+# Helper function to find elements
+def find_element(parent, tag_name):
+    return parent.find(f"{{*}}{tag_name}")
+
+
+def process_packages_directory(packages_dir):
+    """Processes all the nuspec files in the 'packages' directory structure.
+    Once a category is found it adds the following line to the dictionary:
+    packagesByCategory where the key is the category:"""
+    if not os.path.isdir(packages_dir):
+        print(f"Error: Directory '{packages_dir}' not found.")
+        sys.exit()
+
+    for nuspec_path in pathlib.Path(packages_dir).glob("**/*.nuspec"):
+
+        try:
+            nuspec_tree = ET.parse(nuspec_path)
+            nuspec_metadata = find_element(nuspec_tree, "metadata")
+            if nuspec_metadata is not None:
+                description = find_element(nuspec_metadata, "description").text
+                package = find_element(nuspec_metadata, "id").text
path: scripts/utils/generate_package_wiki.py
user: login: Ana06
body: place this after accessing the category and checking it is not None as otherwise you do not use it.
created_at: 2025-03-07T14:11:52Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,70 @@
+import argparse
+import os
+import pathlib
+import sys
+import xml.etree.ElementTree as ET
+from collections import defaultdict
+
+packagesByCategory = defaultdict(str)
+
+
+def sort_write_wiki_content(file_path):
+    """
+    iterates over each category from the dictionary packagesByCategory and
+    writes the Markdown formatted string that contains
+    - header with the category name
+    - table with two columns: Package and Description of each of the packages"""
+    wikiContent = ""
+    for category in packagesByCategory.keys():
+        wikiContent += "## " + category + "\n\n"
+        wikiContent += "| Package | Description |\n"
+        wikiContent += "|---|---|\n"
+        wikiContent += packagesByCategory[category] + "\n\n"
+    with open(file_path, "w", encoding="utf-8") as f:
+        f.write(wikiContent)
+
+
+# Helper function to find elements
+def find_element(parent, tag_name):
+    return parent.find(f"{{*}}{tag_name}")
+
+
+def process_packages_directory(packages_dir):
+    """Processes all the nuspec files in the 'packages' directory structure.
+    Once a category is found it adds the following line to the dictionary:
+    packagesByCategory where the key is the category:"""
+    if not os.path.isdir(packages_dir):
+        print(f"Error: Directory '{packages_dir}' not found.")
+        sys.exit()
+
+    for nuspec_path in pathlib.Path(packages_dir).glob("**/*.nuspec"):
+
+        try:
+            nuspec_tree = ET.parse(nuspec_path)
+            nuspec_metadata = find_element(nuspec_tree, "metadata")
+            if nuspec_metadata is not None:
+                description = find_element(nuspec_metadata, "description").text
+                package = find_element(nuspec_metadata, "id").text
+                category_element = find_element(nuspec_metadata, "tags")
+                # several packages such as common.vm do not contain a category
+                if category_element is None:
+                    continue
+                else:
path: scripts/utils/generate_package_wiki.py
user: login: Ana06
body: [nitpick]
```suggestion
                if not category_element:
                    continue
```
created_at: 2025-03-07T14:12:17Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,62 @@
+import os
+import sys
+import re
+import datetime
+import argparse
+
+
+packagesByCategory={}
+
+
+def write_wiki_content(file_path):
+    wikiContent = ""
+    for category in packagesByCategory.keys():
+        wikiContent += "## " + category + "\n\n"
+        wikiContent += "| Package | Description |\n"
+        wikiContent += "|---|---|\n"
+        wikiContent += packagesByCategory[category] + "\n\n"
+    try:   
+        with open(file_path, 'w', encoding='utf-8') as f:
+           f.write(wikiContent)
+    except FileNotFoundError:
+        print(f"Error: File '{file_path}' not found.")
+
+  
+
+
+def process_packages_directory(packages_dir):
+    """Processes all.nuspec files in the 'packages' directory structure."""
+    if not os.path.isdir(packages_dir):
+        print(f"Error: Directory '{packages_dir}' not found.")
+        return
+
+    
+    for package_name in os.listdir(packages_dir):
+        package = os.path.splitext(package_name)[0]  
+        package_path = os.path.join(packages_dir, package_name)
+        nuspec_file = os.path.join(package_path, f"{package_name}.nuspec")
+        try:
+            with open(nuspec_file, 'r', encoding='utf-8') as f:
+                content = f.read()
+            category_match = re.search(r'<tags>([^<]*)</tags>', content)
+            if category_match:
+                category = category_match.group(1)
+                description_match = re.search (r'<description>([^<]*)</description>', content)
+                description = description_match.group(1)
+                if not category in packagesByCategory:
+            	    packagesByCategory[category]=""
+                packagesByCategory[category]+="|" + package + "|" + description + "|\n"
+                print (f"| {package} | {description} |")
+        except FileNotFoundError:
path: scripts/utils/generate_package_wiki.py
user: login: Ana06
body: In general: `sys.exit()` exists with return code `0` unless you provide another return code as argument, which means the script won't fail. Do not use `print` + `exit` and use Python exceptions instead. 

In this case: I do not understand how this error can happen and I think the `except` should be remove. Or am I missing something? When can this happen?
created_at: 2025-03-07T14:16:35Z
author_association: MEMBER
diff_hunk: @@ -24,7 +24,7 @@ jobs:
 
       - name: Generate Wiki Content
         id: generate-content
-        run: scripts/utils/generate_package_wiki.ps1
+        run: python scripts/utils/generate_package_wiki.py packages/ --wiki wiki/Packages.md
path: .github/workflows/generate_package_wiki.yml
user: login: Ana06
body: remove the arguments here and provide them as defaults in the script.
created_at: 2025-03-07T14:17:15Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,70 @@
+import argparse
+import os
+import pathlib
+import sys
+import xml.etree.ElementTree as ET
+from collections import defaultdict
+
+packagesByCategory = defaultdict(str)
+
+
+def sort_write_wiki_content(file_path):
+    """
+    iterates over each category from the dictionary packagesByCategory and
+    writes the Markdown formatted string that contains
+    - header with the category name
+    - table with two columns: Package and Description of each of the packages"""
+    wikiContent = ""
+    for category in packagesByCategory.keys():
+        wikiContent += "## " + category + "\n\n"
+        wikiContent += "| Package | Description |\n"
+        wikiContent += "|---|---|\n"
+        wikiContent += packagesByCategory[category] + "\n\n"
+    with open(file_path, "w", encoding="utf-8") as f:
+        f.write(wikiContent)
+
+
+# Helper function to find elements
+def find_element(parent, tag_name):
+    return parent.find(f"{{*}}{tag_name}")
+
+
+def process_packages_directory(packages_dir):
+    """Processes all the nuspec files in the 'packages' directory structure.
+    Once a category is found it adds the following line to the dictionary:
+    packagesByCategory where the key is the category:"""
+    if not os.path.isdir(packages_dir):
+        print(f"Error: Directory '{packages_dir}' not found.")
+        sys.exit()
+
+    for nuspec_path in pathlib.Path(packages_dir).glob("**/*.nuspec"):
+
+        try:
+            nuspec_tree = ET.parse(nuspec_path)
+            nuspec_metadata = find_element(nuspec_tree, "metadata")
+            if nuspec_metadata is not None:
+                description = find_element(nuspec_metadata, "description").text
+                package = find_element(nuspec_metadata, "id").text
+                category_element = find_element(nuspec_metadata, "tags")
+                # several packages such as common.vm do not contain a category
+                if category_element is None:
+                    continue
+                else:
+                    category = category_element.text
+                    """packagesByCategory is a dictionary of str where each package and description
+                    are appended formatted in Markdown"""
+                    packagesByCategory[category] += "|" + package + "|" + description + "|\n"
+
+        except FileNotFoundError:
+            print(f"Error: File '{nuspec_path}' not found.")
+            sys.exit()
+
+
+if __name__ == "__main__":
+
+    parser = argparse.ArgumentParser(description="Create wiki of packages.")
+    parser.add_argument("packages", type=str, help="Path to packages")
path: scripts/utils/generate_package_wiki.py
user: login: Ana06
body: make this argument optional to provide a default 
created_at: 2025-03-07T14:17:36Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,70 @@
+import argparse
+import os
+import pathlib
+import sys
+import xml.etree.ElementTree as ET
+from collections import defaultdict
+
+packagesByCategory = defaultdict(str)
+
+
+def sort_write_wiki_content(file_path):
+    """
+    iterates over each category from the dictionary packagesByCategory and
+    writes the Markdown formatted string that contains
+    - header with the category name
+    - table with two columns: Package and Description of each of the packages"""
+    wikiContent = ""
+    for category in packagesByCategory.keys():
+        wikiContent += "## " + category + "\n\n"
+        wikiContent += "| Package | Description |\n"
+        wikiContent += "|---|---|\n"
+        wikiContent += packagesByCategory[category] + "\n\n"
+    with open(file_path, "w", encoding="utf-8") as f:
+        f.write(wikiContent)
+
+
+# Helper function to find elements
+def find_element(parent, tag_name):
+    return parent.find(f"{{*}}{tag_name}")
+
+
+def process_packages_directory(packages_dir):
+    """Processes all the nuspec files in the 'packages' directory structure.
+    Once a category is found it adds the following line to the dictionary:
+    packagesByCategory where the key is the category:"""
+    if not os.path.isdir(packages_dir):
+        print(f"Error: Directory '{packages_dir}' not found.")
+        sys.exit()
+
+    for nuspec_path in pathlib.Path(packages_dir).glob("**/*.nuspec"):
+
+        try:
+            nuspec_tree = ET.parse(nuspec_path)
+            nuspec_metadata = find_element(nuspec_tree, "metadata")
+            if nuspec_metadata is not None:
+                description = find_element(nuspec_metadata, "description").text
+                package = find_element(nuspec_metadata, "id").text
+                category_element = find_element(nuspec_metadata, "tags")
+                # several packages such as common.vm do not contain a category
+                if category_element is None:
+                    continue
+                else:
+                    category = category_element.text
+                    """packagesByCategory is a dictionary of str where each package and description
+                    are appended formatted in Markdown"""
+                    packagesByCategory[category] += "|" + package + "|" + description + "|\n"
+
+        except FileNotFoundError:
+            print(f"Error: File '{nuspec_path}' not found.")
+            sys.exit()
+
+
+if __name__ == "__main__":
+
path: scripts/utils/generate_package_wiki.py
user: login: Ana06
body: [nitpick]
```suggestion
```
created_at: 2025-03-07T14:17:52Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,62 @@
+import os
+import sys
+import re
+import datetime
+import argparse
+
+
+packagesByCategory={}
+
+
+def write_wiki_content(file_path):
+    wikiContent = ""
+    for category in packagesByCategory.keys():
+        wikiContent += "## " + category + "\n\n"
+        wikiContent += "| Package | Description |\n"
+        wikiContent += "|---|---|\n"
+        wikiContent += packagesByCategory[category] + "\n\n"
+    try:   
+        with open(file_path, 'w', encoding='utf-8') as f:
+           f.write(wikiContent)
+    except FileNotFoundError:
+        print(f"Error: File '{file_path}' not found.")
+
+  
+
+
+def process_packages_directory(packages_dir):
+    """Processes all.nuspec files in the 'packages' directory structure."""
+    if not os.path.isdir(packages_dir):
+        print(f"Error: Directory '{packages_dir}' not found.")
+        return
+
+    
+    for package_name in os.listdir(packages_dir):
+        package = os.path.splitext(package_name)[0]  
+        package_path = os.path.join(packages_dir, package_name)
+        nuspec_file = os.path.join(package_path, f"{package_name}.nuspec")
+        try:
+            with open(nuspec_file, 'r', encoding='utf-8') as f:
+                content = f.read()
+            category_match = re.search(r'<tags>([^<]*)</tags>', content)
+            if category_match:
+                category = category_match.group(1)
+                description_match = re.search (r'<description>([^<]*)</description>', content)
+                description = description_match.group(1)
path: scripts/utils/generate_package_wiki.py
user: login: Ana06
body: add the validation into the function to avoid duplication. This is easy to implement, see https://github.com/mandiant/VM-Packages/pull/1320#discussion_r1985124571 for details.
created_at: 2025-03-07T14:20:04Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,70 @@
+import argparse
+import os
+import pathlib
+import sys
+import xml.etree.ElementTree as ET
+from collections import defaultdict
+
+packagesByCategory = defaultdict(str)
+
+
+def sort_write_wiki_content(file_path):
+    """
+    iterates over each category from the dictionary packagesByCategory and
+    writes the Markdown formatted string that contains
+    - header with the category name
+    - table with two columns: Package and Description of each of the packages"""
+    wikiContent = ""
+    for category in packagesByCategory.keys():
+        wikiContent += "## " + category + "\n\n"
+        wikiContent += "| Package | Description |\n"
+        wikiContent += "|---|---|\n"
+        wikiContent += packagesByCategory[category] + "\n\n"
+    with open(file_path, "w", encoding="utf-8") as f:
+        f.write(wikiContent)
+
+
+# Helper function to find elements
+def find_element(parent, tag_name):
+    return parent.find(f"{{*}}{tag_name}")
+
+
+def process_packages_directory(packages_dir):
+    """Processes all the nuspec files in the 'packages' directory structure.
+    Once a category is found it adds the following line to the dictionary:
+    packagesByCategory where the key is the category:"""
+    if not os.path.isdir(packages_dir):
+        print(f"Error: Directory '{packages_dir}' not found.")
+        sys.exit()
+
+    for nuspec_path in pathlib.Path(packages_dir).glob("**/*.nuspec"):
+
+        try:
+            nuspec_tree = ET.parse(nuspec_path)
+            nuspec_metadata = find_element(nuspec_tree, "metadata")
+            if nuspec_metadata is not None:
+                description = find_element(nuspec_metadata, "description").text
+                package = find_element(nuspec_metadata, "id").text
+                category_element = find_element(nuspec_metadata, "tags")
+                # several packages such as common.vm do not contain a category
+                if category_element is None:
+                    continue
+                else:
path: scripts/utils/generate_package_wiki.py
user: login: sara-rn
body: not sure if I understood this
created_at: 2025-03-07T17:00:33Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,70 @@
+import argparse
+import os
+import pathlib
+import sys
+import xml.etree.ElementTree as ET
+from collections import defaultdict
+
+packagesByCategory = defaultdict(str)
+
+
+def sort_write_wiki_content(file_path):
+    """
+    iterates over each category from the dictionary packagesByCategory and
+    writes the Markdown formatted string that contains
+    - header with the category name
+    - table with two columns: Package and Description of each of the packages"""
+    wikiContent = ""
+    for category in packagesByCategory.keys():
+        wikiContent += "## " + category + "\n\n"
+        wikiContent += "| Package | Description |\n"
+        wikiContent += "|---|---|\n"
+        wikiContent += packagesByCategory[category] + "\n\n"
+    with open(file_path, "w", encoding="utf-8") as f:
+        f.write(wikiContent)
+
+
+# Helper function to find elements
+def find_element(parent, tag_name):
+    return parent.find(f"{{*}}{tag_name}")
+
+
+def process_packages_directory(packages_dir):
+    """Processes all the nuspec files in the 'packages' directory structure.
+    Once a category is found it adds the following line to the dictionary:
+    packagesByCategory where the key is the category:"""
+    if not os.path.isdir(packages_dir):
+        print(f"Error: Directory '{packages_dir}' not found.")
+        sys.exit()
+
+    for nuspec_path in pathlib.Path(packages_dir).glob("**/*.nuspec"):
+
+        try:
+            nuspec_tree = ET.parse(nuspec_path)
+            nuspec_metadata = find_element(nuspec_tree, "metadata")
+            if nuspec_metadata is not None:
+                description = find_element(nuspec_metadata, "description").text
+                package = find_element(nuspec_metadata, "id").text
+                category_element = find_element(nuspec_metadata, "tags")
+                # several packages such as common.vm do not contain a category
+                if category_element is None:
+                    continue
+                else:
path: scripts/utils/generate_package_wiki.py
user: login: Ana06
body: the code can be simplified as suggested (removing the `is None` and using `not` instead and removing the `else`) as the code is equivalent (and slightly more robust). Is it clear now?
created_at: 2025-03-07T20:14:04Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,79 @@
+import argparse
+import os
+import pathlib
+import xml.etree.ElementTree as ET
+from collections import defaultdict
+
+packagesByCategory = defaultdict(str)
path: scripts/utils/generate_package_wiki.py
user: login: Ana06
body: As I already pointed out in [my previous comment](https://github.com/mandiant/VM-Packages/pull/1320#discussion_r1981151327), follow [variables naming convention in Google Python Style Guide](https://google.github.io/styleguide/pyguide.html#316-naming)
```suggestion
packages_by_category = defaultdict(str)
```

Ensure you replace it in the comments too.
created_at: 2025-03-07T20:16:09Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,79 @@
+import argparse
+import os
+import pathlib
+import xml.etree.ElementTree as ET
+from collections import defaultdict
+
+packagesByCategory = defaultdict(str)
+
+
+def sort_write_wiki_content(file_path):
+    """
+    Retrieves rows pertaining to the key category, as a result of
+    iterating over packagesByCategory.
+    Writes to the Markdown formatted string:
+    - header with the category name
+    - table with two columns: Package and Description of each of the packages
+    Args:
+       file_path: path of the wiki file
+    """
+    wikiContent = ""
+    for category in packagesByCategory.keys():
+        wikiContent += "## " + category + "\n\n"
+        wikiContent += "| Package | Description |\n"
+        wikiContent += "|---|---|\n"
+        wikiContent += packagesByCategory[category] + "\n\n"
+    with open(file_path, "w", encoding="utf-8") as f:
+        f.write(wikiContent)
+
+
+def find_element_text(parent, tag_name):
+    """Retrieves the text from a xml node, it validates if the tag exists
+    Args:
+       parent: parent tree to search
+       tag_name: tag to search the text
+    Returns:
+       the tag text value
+    """
path: scripts/utils/generate_package_wiki.py
user: login: Ana06
body: [nitpick] I think the format explained in https://github.com/google/styleguide/blob/gh-pages/pyguide.md is as follows (note the empty lines):
```
"""A one-line summary

Overall description of the module or program. Possibly multiline.

Section1:
"""
```

I asked Gemini to help me in this case (by providing the function code and asking it to write the docstring):
```suggestion
    """Retrieves the text content of a specified XML element.

    This function searches for a child element with the given tag name within the
    provided parent element. It handles XML namespaces by using a wildcard.

    Args:
        parent: The parent XML element.
        tag_name: The name of the child element to find.

    Returns:
        The text content of the found element, or None if the element is not found.
    """
```
created_at: 2025-03-07T20:27:47Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,88 @@
+import argparse
+import os
+import pathlib
+import xml.etree.ElementTree as ET
+from collections import defaultdict
+
+packages_by_category = defaultdict(str)
path: scripts/utils/generate_package_wiki.py
user: login: Ana06
body: [nitpick] I think it would be helpful to document the type of this variable:
```suggestion
# Dict[str (category), list (packages information)]
packages_by_category = defaultdict(str)
```
created_at: 2025-03-12T08:27:51Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,88 @@
+import argparse
+import os
+import pathlib
+import xml.etree.ElementTree as ET
+from collections import defaultdict
+
+packages_by_category = defaultdict(str)
+
+
+def sort_write_wiki_content(file_path):
+    """ Writes content of packages_by_category to the wiki file
+    
+    This function retrieves rows pertaining to the key category, as a result of
+    iterating over packages_by_category.
+    Writes to the Markdown formatted string:
+    - header with the category name
+    - table with two columns: Package and Description of each of the packages
+    
+    Args:
+       file_path: path of the wiki .md file
+    """
path: scripts/utils/generate_package_wiki.py
user: login: Ana06
body: [nitpick]
```suggestion
    """ Writes package information sorted by category to a Markdown wiki file.

    This function iterates through the `packages_by_category` dictionary, which
    contains package information organized by category. For each category, it
    generates a Markdown header and a table containing package names and
    descriptions. The resulting Markdown content is then written to the specified
    file.

    Args:
        file_path (str): The path to the output Markdown file.
    """
```
created_at: 2025-03-12T08:34:07Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,88 @@
+import argparse
+import os
+import pathlib
+import xml.etree.ElementTree as ET
+from collections import defaultdict
+
+packages_by_category = defaultdict(str)
+
+
+def sort_write_wiki_content(file_path):
+    """ Writes content of packages_by_category to the wiki file
+    
+    This function retrieves rows pertaining to the key category, as a result of
+    iterating over packages_by_category.
+    Writes to the Markdown formatted string:
+    - header with the category name
+    - table with two columns: Package and Description of each of the packages
+    
+    Args:
+       file_path: path of the wiki .md file
+    """
+    wikiContent = ""
+    for category in packages_by_category.keys():
+        wikiContent += "## " + category + "\n\n"
+        wikiContent += "| Package | Description |\n"
+        wikiContent += "|---|---|\n"
+        wikiContent += packages_by_category[category] + "\n\n"
+    with open(file_path, "w", encoding="utf-8") as f:
+        f.write(wikiContent)
+
+
+def find_element_text(parent, tag_name):
+    """Retrieves the text content of a specified XML element.
+
+    This function searches for a child element with the given tag name within the
+    provided parent element. It handles XML namespaces by using a wildcard.
+
+    Args:
+        parent: The parent XML element.
+        tag_name: The name of the child element to find.
+
+    Returns:
+        The text content of the found element, or None if the element is not found.
+    """
+    tag = parent.find(f"{{*}}{tag_name}")
+    if tag is None:
+        return None
+    return tag.text
+
+
+def process_packages_directory(packages_dir):
+    """ Obtains the package name, description and category from a directory
+    
+    This function parses all the nuspec files in the 'packages' directory structure.
path: scripts/utils/generate_package_wiki.py
user: login: Ana06
body: [nitpick]
```suggestion
    This function parses all the nuspec files in the specified packages directory.
```
created_at: 2025-03-12T08:35:17Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,88 @@
+import argparse
+import os
+import pathlib
+import xml.etree.ElementTree as ET
+from collections import defaultdict
+
+packages_by_category = defaultdict(str)
+
+
+def sort_write_wiki_content(file_path):
+    """ Writes content of packages_by_category to the wiki file
+    
+    This function retrieves rows pertaining to the key category, as a result of
+    iterating over packages_by_category.
+    Writes to the Markdown formatted string:
+    - header with the category name
+    - table with two columns: Package and Description of each of the packages
+    
+    Args:
+       file_path: path of the wiki .md file
+    """
+    wikiContent = ""
+    for category in packages_by_category.keys():
+        wikiContent += "## " + category + "\n\n"
+        wikiContent += "| Package | Description |\n"
+        wikiContent += "|---|---|\n"
+        wikiContent += packages_by_category[category] + "\n\n"
+    with open(file_path, "w", encoding="utf-8") as f:
+        f.write(wikiContent)
+
+
+def find_element_text(parent, tag_name):
+    """Retrieves the text content of a specified XML element.
+
+    This function searches for a child element with the given tag name within the
+    provided parent element. It handles XML namespaces by using a wildcard.
+
+    Args:
+        parent: The parent XML element.
+        tag_name: The name of the child element to find.
+
+    Returns:
+        The text content of the found element, or None if the element is not found.
+    """
+    tag = parent.find(f"{{*}}{tag_name}")
+    if tag is None:
+        return None
+    return tag.text
+
+
+def process_packages_directory(packages_dir):
+    """ Obtains the package name, description and category from a directory
+    
+    This function parses all the nuspec files in the 'packages' directory structure.
+    It saves into packages_by_category[category] a line containing the package
+    name and the description separated by |
path: scripts/utils/generate_package_wiki.py
user: login: Ana06
body: [nitpick]
```suggestion
    name and the description separated by "|".
```
created_at: 2025-03-12T08:36:14Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,88 @@
+import argparse
+import os
+import pathlib
+import xml.etree.ElementTree as ET
+from collections import defaultdict
+
+packages_by_category = defaultdict(str)
+
+
+def sort_write_wiki_content(file_path):
+    """ Writes content of packages_by_category to the wiki file
+    
+    This function retrieves rows pertaining to the key category, as a result of
+    iterating over packages_by_category.
+    Writes to the Markdown formatted string:
+    - header with the category name
+    - table with two columns: Package and Description of each of the packages
+    
+    Args:
+       file_path: path of the wiki .md file
+    """
+    wikiContent = ""
+    for category in packages_by_category.keys():
+        wikiContent += "## " + category + "\n\n"
+        wikiContent += "| Package | Description |\n"
+        wikiContent += "|---|---|\n"
+        wikiContent += packages_by_category[category] + "\n\n"
+    with open(file_path, "w", encoding="utf-8") as f:
+        f.write(wikiContent)
+
+
+def find_element_text(parent, tag_name):
+    """Retrieves the text content of a specified XML element.
+
+    This function searches for a child element with the given tag name within the
+    provided parent element. It handles XML namespaces by using a wildcard.
+
+    Args:
+        parent: The parent XML element.
+        tag_name: The name of the child element to find.
+
+    Returns:
+        The text content of the found element, or None if the element is not found.
+    """
+    tag = parent.find(f"{{*}}{tag_name}")
+    if tag is None:
+        return None
+    return tag.text
+
+
+def process_packages_directory(packages_dir):
+    """ Obtains the package name, description and category from a directory
+    
+    This function parses all the nuspec files in the 'packages' directory structure.
+    It saves into packages_by_category[category] a line containing the package
+    name and the description separated by |
+    
+    Args:
+        packages: directory where all the packages reside
path: scripts/utils/generate_package_wiki.py
user: login: Ana06
body: [nitpick]
```suggestion
        packages: directory where the packages reside.
```
created_at: 2025-03-12T08:36:47Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,88 @@
+import argparse
+import os
+import pathlib
+import xml.etree.ElementTree as ET
+from collections import defaultdict
+
+packages_by_category = defaultdict(str)
+
+
+def sort_write_wiki_content(file_path):
+    """ Writes content of packages_by_category to the wiki file
+    
+    This function retrieves rows pertaining to the key category, as a result of
+    iterating over packages_by_category.
+    Writes to the Markdown formatted string:
+    - header with the category name
+    - table with two columns: Package and Description of each of the packages
+    
+    Args:
+       file_path: path of the wiki .md file
+    """
+    wikiContent = ""
+    for category in packages_by_category.keys():
+        wikiContent += "## " + category + "\n\n"
+        wikiContent += "| Package | Description |\n"
+        wikiContent += "|---|---|\n"
+        wikiContent += packages_by_category[category] + "\n\n"
+    with open(file_path, "w", encoding="utf-8") as f:
+        f.write(wikiContent)
+
+
+def find_element_text(parent, tag_name):
+    """Retrieves the text content of a specified XML element.
+
+    This function searches for a child element with the given tag name within the
+    provided parent element. It handles XML namespaces by using a wildcard.
+
+    Args:
+        parent: The parent XML element.
+        tag_name: The name of the child element to find.
+
+    Returns:
+        The text content of the found element, or None if the element is not found.
+    """
+    tag = parent.find(f"{{*}}{tag_name}")
+    if tag is None:
+        return None
+    return tag.text
+
+
+def process_packages_directory(packages_dir):
+    """ Obtains the package name, description and category from a directory
+    
+    This function parses all the nuspec files in the 'packages' directory structure.
+    It saves into packages_by_category[category] a line containing the package
+    name and the description separated by |
+    
+    Args:
+        packages: directory where all the packages reside
+    """
+    if not os.path.isdir(packages_dir):
+        raise FileNotFoundError(f"Packages directory not found: {packages_dir}")
+
+    for nuspec_path in pathlib.Path(packages_dir).glob("**/*.nuspec"):
+        nuspec_tree = ET.parse(nuspec_path)
+        nuspec_metadata = nuspec_tree.find("{*}metadata")
+        if nuspec_metadata is not None:
+            category = find_element_text(nuspec_metadata, "tags")
+            # we are only interested in the packages with a category
+            # some packages that assist with the istallation (such as common.vm) do
+            # not contain a category
path: scripts/utils/generate_package_wiki.py
user: login: Ana06
body: [nitpick]
```suggestion
            # We are only interested in the packages with a category.
            # Some packages that assist with the istallation (such as common.vm) do
            # not contain a category
```
created_at: 2025-03-12T08:37:18Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,88 @@
+import argparse
+import os
+import pathlib
+import xml.etree.ElementTree as ET
+from collections import defaultdict
+
+packages_by_category = defaultdict(str)
+
+
+def sort_write_wiki_content(file_path):
+    """ Writes content of packages_by_category to the wiki file
+    
+    This function retrieves rows pertaining to the key category, as a result of
+    iterating over packages_by_category.
+    Writes to the Markdown formatted string:
+    - header with the category name
+    - table with two columns: Package and Description of each of the packages
+    
+    Args:
+       file_path: path of the wiki .md file
+    """
+    wikiContent = ""
+    for category in packages_by_category.keys():
+        wikiContent += "## " + category + "\n\n"
+        wikiContent += "| Package | Description |\n"
+        wikiContent += "|---|---|\n"
+        wikiContent += packages_by_category[category] + "\n\n"
+    with open(file_path, "w", encoding="utf-8") as f:
+        f.write(wikiContent)
+
+
+def find_element_text(parent, tag_name):
+    """Retrieves the text content of a specified XML element.
+
+    This function searches for a child element with the given tag name within the
+    provided parent element. It handles XML namespaces by using a wildcard.
+
+    Args:
+        parent: The parent XML element.
+        tag_name: The name of the child element to find.
+
+    Returns:
+        The text content of the found element, or None if the element is not found.
+    """
+    tag = parent.find(f"{{*}}{tag_name}")
+    if tag is None:
+        return None
+    return tag.text
+
+
+def process_packages_directory(packages_dir):
+    """ Obtains the package name, description and category from a directory
+    
+    This function parses all the nuspec files in the 'packages' directory structure.
+    It saves into packages_by_category[category] a line containing the package
+    name and the description separated by |
+    
+    Args:
+        packages: directory where all the packages reside
+    """
+    if not os.path.isdir(packages_dir):
+        raise FileNotFoundError(f"Packages directory not found: {packages_dir}")
+
+    for nuspec_path in pathlib.Path(packages_dir).glob("**/*.nuspec"):
+        nuspec_tree = ET.parse(nuspec_path)
+        nuspec_metadata = nuspec_tree.find("{*}metadata")
+        if nuspec_metadata is not None:
+            category = find_element_text(nuspec_metadata, "tags")
+            # we are only interested in the packages with a category
+            # some packages that assist with the istallation (such as common.vm) do
+            # not contain a category
+            if not category:
+                continue
+            description = find_element_text(nuspec_metadata, "description")
+            package = find_element_text(nuspec_metadata, "id")
+            """packages_by_category is a dictionary of str where each package and description
+            are appended formatted in Markdown"""
+            packages_by_category[category] += "|" + package + "|" + description + "|\n"
path: scripts/utils/generate_package_wiki.py
user: login: Ana06
body: Adding some spaces makes the markdown nicer to read:
```suggestion
            packages_by_category[category] += "| " + package + " | " + description + " |\n"
```
created_at: 2025-03-12T08:40:33Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,88 @@
+import argparse
+import os
+import pathlib
+import xml.etree.ElementTree as ET
+from collections import defaultdict
+
+packages_by_category = defaultdict(str)
+
+
+def sort_write_wiki_content(file_path):
+    """ Writes content of packages_by_category to the wiki file
+    
+    This function retrieves rows pertaining to the key category, as a result of
+    iterating over packages_by_category.
+    Writes to the Markdown formatted string:
+    - header with the category name
+    - table with two columns: Package and Description of each of the packages
+    
+    Args:
+       file_path: path of the wiki .md file
+    """
+    wikiContent = ""
+    for category in packages_by_category.keys():
+        wikiContent += "## " + category + "\n\n"
+        wikiContent += "| Package | Description |\n"
+        wikiContent += "|---|---|\n"
path: scripts/utils/generate_package_wiki.py
user: login: Ana06
body: [nitpick] It makes it the code and the markdown a bit nicer to read:
```suggestion
        wikiContent += "| ------- | ----------- |\n"
```
created_at: 2025-03-12T08:44:15Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,62 @@
+import os
+import sys
+import re
+import datetime
+import argparse
+
+
+packagesByCategory={}
+
+
+def write_wiki_content(file_path):
+    wikiContent = ""
+    for category in packagesByCategory.keys():
+        wikiContent += "## " + category + "\n\n"
+        wikiContent += "| Package | Description |\n"
+        wikiContent += "|---|---|\n"
+        wikiContent += packagesByCategory[category] + "\n\n"
path: scripts/utils/generate_package_wiki.py
user: login: Ana06
body: I feel with the added documentation, it is ok to keep the current code structure.
created_at: 2025-03-12T08:47:12Z
author_association: MEMBER

PR comments:
user: login: sara-rn
created_at: 2025-02-28T16:23:39Z
body: There is a formatting issue due to the description of the metapackage `vcbuildtools.vm` addressed in #1319
author_association: CONTRIBUTOR
user: login: sara-rn
created_at: 2025-03-04T22:38:38Z
body: added the python script in a separate commit just in case, I haven't deleted the Powershell script yet
example of the wiki: https://github.com/sara-rn/VM-Packages/wiki/Packages
author_association: CONTRIBUTOR
user: login: sara-rn
created_at: 2025-03-06T14:00:14Z
body: @Ana06 wiki in my local repo https://github.com/sara-rn/VM-Packages/wiki/Packages
author_association: CONTRIBUTOR