Repository: mozilla/bugbug

Issue:
ResponseError: too many 502 error responses: /report
https://community-tc.services.mozilla.com/tasks/BdiI4F8gRHiiK6sp4cAyEA/runs/1/logs/public/logs/live.log#L260:
```
urllib3.exceptions.ResponseError: too many 502 error responses
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 667, in send
    resp = conn.urlopen(
           ^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 948, in urlopen
    return self.urlopen(
           ^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 948, in urlopen
    return self.urlopen(
           ^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 948, in urlopen
    return self.urlopen(
           ^^^^^^^^^^^^^
  [Previous line repeated 6 more times]
  File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 938, in urlopen
    retries = retries.increment(method, url, response=response, _pool=self)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/urllib3/util/retry.py", line 515, in increment
    raise MaxRetryError(_pool, url, reason) from reason  # type: ignore[arg-type]
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='bugzilla.mozilla.org', port=443): Max retries exceeded with url: /report.cgi?f1=creation_ts&o1=greaterthan&v1=1191-03-05&x_axis_field=product&y_axis_field=component&action=wrap&ctype=csv&format=table (Caused by ResponseError('too many 502 error responses'))

```

Currently, we are seeing “502 Bad Gateway” when trying to load https://bugzilla.mozilla.org/report.cgi?f1=creation_ts&o1=greaterthan&v1=1191-03-05&x_axis_field=product&y_axis_field=component&action=wrap&ctype=csv&format=table


Discussion:

Resolving patch:
diff --git a/bugbug/bugzilla.py b/bugbug/bugzilla.py
--- a/bugbug/bugzilla.py
+++ b/bugbug/bugzilla.py
@@ -301,6 +301,33 @@ def handler(bug):
     return data["bug_count"]
 
 
+def fetch_components_list(product_types="accessible") -> list[tuple]:
+    """Fetch all components from all products.
+
+    Args:
+        product_types: The types of products to fetch components from. Defaults
+            to "accessible".
+
+    Returns:
+        A list of tuples where the first element is the product name and the
+        second element is the component name.
+    """
+    components: list[tuple] = []
+
+    def handler(product):
+        components.extend(
+            (product["name"], component["name"]) for component in product["components"]
+        )
+
+    BugzillaProduct(
+        product_types=product_types,
+        include_fields=["name", "components.name"],
+        product_handler=handler,
+    ).wait()
+
+    return components
+
+
 def get_product_component_count(months: int = 12) -> dict[str, int]:
     """Get the number of bugs per component.
 
diff --git a/scripts/bug_retriever.py b/scripts/bug_retriever.py
--- a/scripts/bug_retriever.py
+++ b/scripts/bug_retriever.py
@@ -45,12 +45,12 @@ def retrieve_bugs(self, limit: int | None = None) -> None:
 
         logger.info("Retrieved %d IDs.", len(changed_ids))
 
-        all_components = bugzilla.get_product_component_count(9999)
+        all_components = set(bugzilla.fetch_components_list())
 
         deleted_component_ids = set(
             bug["id"]
             for bug in bugzilla.get_bugs()
-            if "{}::{}".format(bug["product"], bug["component"]) not in all_components
+            if (bug["product"], bug["component"]) not in all_components
         )
         logger.info(
             "%d bugs belonging to deleted components", len(deleted_component_ids)


Test patch: