Repository: causify-ai/helpers
Pull request #598: HelpersTask597_Configure_Codecov_for_coverage
Author: login: Shaunak01
Created: 2025-04-21T22:50:48Z | Merged: 2025-05-02T14:47:21Z

Issue: https://github.com/causify-ai/helpers/issues/597


Diff:
diff --git a/.coveragerc b/.coveragerc
new file mode 100644
index 000000000..9d4fbe2a0
--- /dev/null
+++ b/.coveragerc
@@ -0,0 +1,5 @@
+[report]
+# Glob pattern(s) of files to omit from the report.
+omit =
+    */devops/compose/*
+    */helpers/test/outcomes/*/tmp.scratch/*
diff --git a/.github/gh_requirements.txt b/.github/gh_requirements.txt
index b6eb64ad6..6a8e6997b 100644
--- a/.github/gh_requirements.txt
+++ b/.github/gh_requirements.txt
@@ -1,4 +1,5 @@
+boto3 >= 1.20.17
+coverage
 invoke
-tqdm
 s3fs
-boto3 >= 1.20.17
+tqdm
diff --git a/.github/workflows/coverage_tests.yml b/.github/workflows/coverage_tests.yml
new file mode 100644
index 000000000..1f6e9f2ae
--- /dev/null
+++ b/.github/workflows/coverage_tests.yml
@@ -0,0 +1,173 @@
+name: Test coverage
+
+on:
+  workflow_dispatch: {}
+  # every day at 00:00 UTC.
+  schedule:
+    - cron: '0 0 * * *'
+
+env:
+  CSFY_CI: true
+
+permissions:
+  # Required to authenticate and retrieve temporary AWS credentials via OIDC.
+  id-token: write
+  # Required to fetch and check out code from the repository.
+  contents: read
+  # Required to authenticate and pull Docker images from GitHub Container Registry (GHCR).
+  packages: read
+
+
+jobs:
+  run_test_coverage:
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Configure AWS credentials
+        uses: aws-actions/configure-aws-credentials@v1
+        with:
+          role-to-assume: ${{ vars.GH_ACTION_AWS_ROLE_ARN }}
+          role-session-name: ${{ vars.GH_ACTION_AWS_SESSION_NAME }}
+          aws-region: ${{ vars.CSFY_AWS_DEFAULT_REGION }}
+
+      - name: Login to GHCR
+        run: docker login ghcr.io -u gpsaggese -p ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Cleanup
+        run: sudo chmod 777 -R .
+
+      - name: Checkout code
+        uses: actions/checkout@v3
+        with:
+          submodules: recursive
+          fetch-depth: 0
+          token: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Update PYTHONPATH
+        run: echo "PYTHONPATH=.:helpers" >> $GITHUB_ENV
+
+      - name: Install dependencies
+        run: |
+          python -m pip install --upgrade pip
+          pip install -r .github/gh_requirements.txt
+
+      - name: Pull image from GHCR
+        run: docker pull ghcr.io/${{ github.repository }}:dev
+
+      # Only on scheduled runs, capture ISO weekday (1=Mon … 7=Sun).
+      - name: Set DAY_OF_WEEK
+        if: github.event_name == 'schedule'
+        run: echo "DAY_OF_WEEK=$(date -u +'%u')" >> $GITHUB_ENV
+
+      # This step is used to trigger the fast test coverage generation using the invoke task.
+      - name: Run Fast test and generate report
+        id: run_fast
+        continue-on-error: true
+        run: |
+          echo "Simulating fast test failure"
+          exit 1
+        # env:
+        #   GH_ACTION_ACCESS_TOKEN: ${{ secrets.GH_ACTION_ACCESS_TOKEN }}
+        #   CSFY_AWS_ACCESS_KEY_ID: ${{ env.AWS_ACCESS_KEY_ID }}
+        #   CSFY_AWS_SECRET_ACCESS_KEY: ${{ env.AWS_SECRET_ACCESS_KEY }}
+        #   CSFY_AWS_SESSION_TOKEN: ${{ env.AWS_SESSION_TOKEN }}
+        #   CSFY_AWS_DEFAULT_REGION: ${{ env.AWS_DEFAULT_REGION }}
+        #   CSFY_ECR_BASE_PATH: ghcr.io/${{ github.repository_owner }}
+        #   CSFY_AWS_S3_BUCKET: ${{ vars.CSFY_AWS_S3_BUCKET }}
+        # run: invoke run_coverage --suite fast
+
+      - name: Upload Fast Test Coverage to Codecov
+        id: upload_fast
+        # Only upload if the previous fast test run step succeeded (i.r report generated).
+        # failed step don’t generate a coverage report, so there's nothing to upload.
+        if: steps.run_fast.outcome == 'success'
+        continue-on-error: true
+        uses: codecov/codecov-action@v5
+        with:
+          token: ${{ secrets.CODECOV_TOKEN }}
+          files: ./coverage.xml
+          # Specify the Codecov flag name associated with this test suite.
+          # Required to separate coverage reports by type (e.g., fast, slow, superslow) inside the Codecov UI.
+          flags: fast
+          name: fast-test-coverage
+
+      - name: Run Slow test and generate report
+        id: run_slow
+        continue-on-error: true
+        env:
+          GH_ACTION_ACCESS_TOKEN: ${{ secrets.GH_ACTION_ACCESS_TOKEN }}
+          CSFY_AWS_ACCESS_KEY_ID: ${{ env.AWS_ACCESS_KEY_ID }}
+          CSFY_AWS_SECRET_ACCESS_KEY: ${{ env.AWS_SECRET_ACCESS_KEY }}
+          CSFY_AWS_SESSION_TOKEN: ${{ env.AWS_SESSION_TOKEN }}
+          CSFY_AWS_DEFAULT_REGION: ${{ env.AWS_DEFAULT_REGION }}
+          CSFY_ECR_BASE_PATH: ghcr.io/${{ github.repository_owner }}
+          CSFY_AWS_S3_BUCKET: ${{ vars.CSFY_AWS_S3_BUCKET }}
+        run: invoke run_coverage --suite slow
+
+      - name: Upload Slow Test Coverage to Codecov
+        id: upload_slow
+        # Only upload if the previous slow test run step succeeded (i.r report generated).
+        if: steps.run_slow.outcome == 'success'
+        continue-on-error: true
+        uses: codecov/codecov-action@v5
+        with:
+          token: ${{ secrets.CODECOV_TOKEN }}
+          files: ./coverage.xml
+          flags: slow
+          name: slow-test-coverage
+
+      - name: Run Superslow test and generate report
+        # Run only on scheduled jobs or manual trigger
+        if: github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'
+        env:
+          GH_ACTION_ACCESS_TOKEN: ${{ secrets.GH_ACTION_ACCESS_TOKEN }}
+          CSFY_AWS_ACCESS_KEY_ID: ${{ env.AWS_ACCESS_KEY_ID }}
+          CSFY_AWS_SECRET_ACCESS_KEY: ${{ env.AWS_SECRET_ACCESS_KEY }}
+          CSFY_AWS_SESSION_TOKEN: ${{ env.AWS_SESSION_TOKEN }}
+          CSFY_AWS_DEFAULT_REGION: ${{ env.AWS_DEFAULT_REGION }}
+          CSFY_ECR_BASE_PATH: ghcr.io/${{ github.repository_owner }}
+          CSFY_AWS_S3_BUCKET: ${{ vars.CSFY_AWS_S3_BUCKET }}
+        run: |
+          # Only run superslow tests if it's Monday or a manual dispatch.
+          day_of_week=$(date +%u)
+          if [ "$day_of_week" = "1" ] || [ "${{ github.event_name }}" = "workflow_dispatch" ]; then
+            invoke run_coverage --suite superslow
+          else
+            echo "Skipping superslow tests — not Monday and not manually triggered"
+            exit 0
+          fi
+
+      - name: Upload Superslow Test Coverage to Codecov
+        #TODO(Shaunak): Consider removing it when we turn this workflow into a reusable one.
+        if: steps.run_superslow.outcome == 'success'
+        uses: codecov/codecov-action@v5
+        with:
+          token: ${{ secrets.CODECOV_TOKEN }}
+          files: ./coverage.xml
+          flags: superslow
+          name: superslow-test-coverage
+
+        # Fail the job in CI if any of the fast/ slow run/ upload steps above failed.
+      - name: Fail if fast/slow test or upload failed
+        run: |
+          failed=""
+          if [ "${{ steps.run_fast.outcome }}" != "success" ]; then
+            echo "Fast test run failed"
+            failed="true"
+          fi
+          if [ "${{ steps.upload_fast.outcome }}" != "success" ]; then
+            echo "Fast test coverage upload failed"
+            failed="true"
+          fi
+          if [ "${{ steps.run_slow.outcome }}" != "success" ]; then
+            echo "Slow test run failed"
+            failed="true"
+          fi
+          if [ "${{ steps.upload_slow.outcome }}" != "success" ]; then
+            echo "Slow test coverage upload failed"
+            failed="true"
+          fi
+          if [ "$failed" = "true" ]; then
+            echo "At least one fast/slow test or upload step failed."
+            exit 1
+          fi
diff --git a/dev_scripts_helpers/update_devops_packages/notebooks/Master_buildmeister_dashboard.ipynb b/dev_scripts_helpers/update_devops_packages/notebooks/Master_buildmeister_dashboard.ipynb
index 091e2595e..48f2b4c28 100644
--- a/dev_scripts_helpers/update_devops_packages/notebooks/Master_buildmeister_dashboard.ipynb
+++ b/dev_scripts_helpers/update_devops_packages/notebooks/Master_buildmeister_dashboard.ipynb
@@ -1,338 +1,377 @@
 {
- "cells": [
-  {
-   "cell_type": "markdown",
-   "id": "e3103ff3",
-   "metadata": {},
-   "source": [
-    " TODO(Grisha): does it belong to the `devops` dir?"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "9d992fed",
-   "metadata": {},
-   "source": [
-    "# Description"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "3e381a7d",
-   "metadata": {
-    "ExecuteTime": {
-     "end_time": "2024-02-02T08:02:05.889049Z",
-     "start_time": "2024-02-02T08:02:05.883420Z"
-    }
-   },
-   "source": [
-    "The notebook reports the latest build status for multiple repos."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "982f47f1",
-   "metadata": {},
-   "source": [
-    "# Imports"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "97bbec36",
-   "metadata": {
-    "ExecuteTime": {
-     "end_time": "2024-02-07T17:59:42.038091Z",
-     "start_time": "2024-02-07T17:59:42.002068Z"
-    }
-   },
-   "outputs": [],
-   "source": [
-    "%load_ext autoreload\n",
-    "%autoreload 2\n",
-    "%matplotlib inline"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "518df056",
-   "metadata": {
-    "ExecuteTime": {
-     "end_time": "2024-02-07T17:59:42.078514Z",
-     "start_time": "2024-02-07T17:59:42.041301Z"
-    }
-   },
-   "outputs": [],
-   "source": [
-    "import logging\n",
-    "from typing import Dict\n",
-    "\n",
-    "import pandas as pd\n",
-    "from IPython.display import Markdown, display\n",
-    "\n",
-    "import helpers.hdbg as hdbg\n",
-    "import helpers.henv as henv\n",
-    "import helpers.hpandas as hpandas\n",
-    "import helpers.hprint as hprint\n",
-    "import helpers.lib_tasks_gh as hlitagh"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "f0793aa5",
-   "metadata": {
-    "ExecuteTime": {
-     "end_time": "2024-02-07T17:59:42.268049Z",
-     "start_time": "2024-02-07T17:59:42.081426Z"
-    }
-   },
-   "outputs": [],
-   "source": [
-    "hdbg.init_logger(verbosity=logging.INFO)\n",
-    "_LOG = logging.getLogger(__name__)\n",
-    "_LOG.info(\"%s\", henv.get_system_signature()[0])\n",
-    "hprint.config_notebook()"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "93c2d39f",
-   "metadata": {
-    "ExecuteTime": {
-     "end_time": "2024-02-07T17:59:42.338614Z",
-     "start_time": "2024-02-07T17:59:42.271472Z"
-    }
-   },
-   "outputs": [],
-   "source": [
-    "# Set the display options to print the full table.\n",
-    "pd.set_option(\"display.max_colwidth\", None)\n",
-    "pd.set_option(\"display.max_columns\", None)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "14f379d5",
-   "metadata": {
-    "lines_to_next_cell": 2
-   },
-   "source": [
-    "# Utils"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "1f41a8dd",
-   "metadata": {
-    "ExecuteTime": {
-     "end_time": "2024-02-07T17:59:42.380319Z",
-     "start_time": "2024-02-07T17:59:42.343492Z"
-    }
-   },
-   "outputs": [],
-   "source": [
-    "def make_clickable(url: str) -> str:\n",
-    "    \"\"\"\n",
-    "    Wrapper to make the URL value clickable.\n",
-    "\n",
-    "    :param url: URL value to convert\n",
-    "    :return: clickable URL link\n",
-    "    \"\"\"\n",
-    "    return f'<a href=\"{url}\" target=\"_blank\">{url}</a>'\n",
-    "\n",
-    "\n",
-    "def color_format(val: str, status_color_mapping: Dict[str, str]) -> str:\n",
-    "    \"\"\"\n",
-    "    Return the color depends on status.\n",
-    "\n",
-    "    :param val: value of the status e.g. `failure`\n",
-    "    :param status_color_mapping: mapping statuses to the colors e.g.:\n",
-    "    ```\n",
-    "    {\n",
-    "       \"success\": \"green\",\n",
-    "       \"failure\": \"red\",\n",
-    "    }\n",
-    "    ```\n",
-    "    \"\"\"\n",
-    "    if val in status_color_mapping:\n",
-    "        color = status_color_mapping[val]\n",
-    "    else:\n",
-    "        color = \"grey\"\n",
-    "    return f\"background-color: {color}\""
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "189f2c75",
-   "metadata": {},
-   "source": [
-    "# GH workflows state"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "865bc9f2",
-   "metadata": {
-    "ExecuteTime": {
-     "end_time": "2024-02-07T17:59:57.513155Z",
-     "start_time": "2024-02-07T17:59:42.383039Z"
-    }
-   },
-   "outputs": [],
-   "source": [
-    "repo_list = [\n",
-    "    \"cryptokaizen/cmamp\",\n",
-    "    \"cryptokaizen/orange\",\n",
-    "    \"cryptokaizen/lemonade\",\n",
-    "    \"causify-ai/kaizenflow\",\n",
-    "]\n",
-    "workflow_df = hlitagh.gh_get_details_for_all_workflows(repo_list)\n",
-    "# Reorder columns.\n",
-    "columns_order = [\"repo_name\", \"workflow_name\", \"conclusion\", \"url\"]\n",
-    "workflow_df = workflow_df[columns_order]\n",
-    "# Make URL values clickable.\n",
-    "workflow_df[\"url\"] = workflow_df[\"url\"].apply(make_clickable)\n",
-    "_LOG.info(hpandas.df_to_str(workflow_df, log_level=logging.INFO))"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "f7e999ce",
-   "metadata": {
-    "ExecuteTime": {
-     "end_time": "2024-02-07T17:59:57.585606Z",
-     "start_time": "2024-02-07T17:59:57.515915Z"
-    }
-   },
-   "outputs": [],
-   "source": [
-    "status_color_mapping = {\n",
-    "    \"success\": \"green\",\n",
-    "    \"failure\": \"red\",\n",
-    "}\n",
-    "repos = workflow_df[\"repo_name\"].unique()\n",
-    "display(Markdown(\"## Overall Status\"))\n",
-    "current_timestamp = pd.Timestamp.now(tz=\"America/New_York\")\n",
-    "display(Markdown(f\"**Last run: {current_timestamp}**\"))\n",
-    "for repo in repos:\n",
-    "    # Calculate the overall status.\n",
-    "    repo_df = workflow_df[workflow_df[\"repo_name\"] == repo]\n",
-    "    overall_status = hlitagh.gh_get_overall_build_status_for_repo(repo_df)\n",
-    "    display(Markdown(f\"## {repo}: {overall_status}\"))\n",
-    "    repo_df = repo_df.drop(columns=[\"repo_name\"])\n",
-    "    display(\n",
-    "        repo_df.style.map(\n",
-    "            color_format,\n",
-    "            status_color_mapping=status_color_mapping,\n",
-    "            subset=[\"conclusion\"],\n",
-    "        )\n",
-    "    )"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "a00870a9",
-   "metadata": {},
-   "source": [
-    "# Allure reports"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "36e93fca",
-   "metadata": {},
-   "source": [
-    "- fast tests: http://172.30.2.44/allure_reports/cmamp/fast/latest/index.html\n",
-    "- slow tests: http://172.30.2.44/allure_reports/cmamp/slow/latest/index.html\n",
-    "- superslow tests: http://172.30.2.44/allure_reports/cmamp/superslow/latest/index.html"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "bb8ed505",
-   "metadata": {},
-   "source": [
-    "# Number of open pull requests"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "69dbda1d",
-   "metadata": {
-    "ExecuteTime": {
-     "end_time": "2024-02-07T17:59:59.309022Z",
-     "start_time": "2024-02-07T17:59:57.588291Z"
-    }
-   },
-   "outputs": [],
-   "source": [
-    "for repo in repo_list:\n",
-    "    number_prs = len(hlitagh.gh_get_open_prs(repo))\n",
-    "    _LOG.info(\"%s: %s\", repo, number_prs)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "ec63cb5e",
-   "metadata": {},
-   "source": [
-    "# Code coverage HTML-page"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "569f9404",
-   "metadata": {
-    "ExecuteTime": {
-     "end_time": "2024-01-24T14:40:31.379819Z",
-     "start_time": "2024-01-24T14:40:31.327151Z"
-    }
-   },
-   "source": [
-    "http://172.30.2.44/html_coverage/runner_master/"
-   ]
-  }
- ],
- "metadata": {
-  "kernelspec": {
-   "display_name": "Python 3 (ipykernel)",
-   "language": "python",
-   "name": "python3"
-  },
-  "language_info": {
-   "codemirror_mode": {
-    "name": "ipython",
-    "version": 3
-   },
-   "file_extension": ".py",
-   "mimetype": "text/x-python",
-   "name": "python",
-   "nbconvert_exporter": "python",
-   "pygments_lexer": "ipython3",
-   "version": "3.12.3"
-  },
-  "toc": {
-   "base_numbering": 1,
-   "nav_menu": {},
-   "number_sections": true,
-   "sideBar": true,
-   "skip_h1_title": false,
-   "title_cell": "Table of Contents",
-   "title_sidebar": "Contents",
-   "toc_cell": false,
-   "toc_position": {},
-   "toc_section_display": true,
-   "toc_window_display": false
-  }
- },
- "nbformat": 4,
- "nbformat_minor": 5
+    "cells": [
+        {
+            "cell_type": "markdown",
+            "metadata": {},
+            "source": [
+                "CONTENTS:\n",
+                "- [Description](#description)\n",
+                "- [Imports](#imports)\n",
+                "- [Utils](#utils)\n",
+                "- [GH workflows state](#gh-workflows-state)\n",
+                "- [Allure reports](#allure-reports)\n",
+                "- [Number of open pull requests](#number-of-open-pull-requests)\n",
+                "- [Code coverage HTML-page](#code-coverage-html-page)\n",
+                "- [Code Coverage Page - CodeCov](#code-coverage-page---codecov)"
+            ]
+        },
+        {
+            "cell_type": "markdown",
+            "id": "e3103ff3",
+            "metadata": {},
+            "source": [
+                " TODO(Grisha): does it belong to the `devops` dir?"
+            ]
+        },
+        {
+            "cell_type": "markdown",
+            "id": "9d992fed",
+            "metadata": {},
+            "source": [
+                "<a name='description'></a>\n",
+                "# Description"
+            ]
+        },
+        {
+            "cell_type": "markdown",
+            "id": "3e381a7d",
+            "metadata": {
+                "ExecuteTime": {
+                    "end_time": "2024-02-02T08:02:05.889049Z",
+                    "start_time": "2024-02-02T08:02:05.883420Z"
+                }
+            },
+            "source": [
+                "The notebook reports the latest build status for multiple repos."
+            ]
+        },
+        {
+            "cell_type": "markdown",
+            "id": "982f47f1",
+            "metadata": {},
+            "source": [
+                "<a name='imports'></a>\n",
+                "# Imports"
+            ]
+        },
+        {
+            "cell_type": "code",
+            "execution_count": null,
+            "id": "97bbec36",
+            "metadata": {
+                "ExecuteTime": {
+                    "end_time": "2024-02-07T17:59:42.038091Z",
+                    "start_time": "2024-02-07T17:59:42.002068Z"
+                }
+            },
+            "outputs": [],
+            "source": [
+                "%load_ext autoreload\n",
+                "%autoreload 2\n",
+                "%matplotlib inline"
+            ]
+        },
+        {
+            "cell_type": "code",
+            "execution_count": null,
+            "id": "518df056",
+            "metadata": {
+                "ExecuteTime": {
+                    "end_time": "2024-02-07T17:59:42.078514Z",
+                    "start_time": "2024-02-07T17:59:42.041301Z"
+                }
+            },
+            "outputs": [],
+            "source": [
+                "import logging\n",
+                "from typing import Dict\n",
+                "\n",
+                "import pandas as pd\n",
+                "from IPython.display import Markdown, display\n",
+                "\n",
+                "import helpers.hdbg as hdbg\n",
+                "import helpers.henv as henv\n",
+                "import helpers.hpandas as hpandas\n",
+                "import helpers.hprint as hprint\n",
+                "import helpers.lib_tasks_gh as hlitagh\n"
+            ]
+        },
+        {
+            "cell_type": "code",
+            "execution_count": null,
+            "id": "f0793aa5",
+            "metadata": {
+                "ExecuteTime": {
+                    "end_time": "2024-02-07T17:59:42.268049Z",
+                    "start_time": "2024-02-07T17:59:42.081426Z"
+                }
+            },
+            "outputs": [],
+            "source": [
+                "hdbg.init_logger(verbosity=logging.INFO)\n",
+                "_LOG = logging.getLogger(__name__)\n",
+                "_LOG.info(\"%s\", henv.get_system_signature()[0])\n",
+                "hprint.config_notebook()"
+            ]
+        },
+        {
+            "cell_type": "code",
+            "execution_count": null,
+            "id": "93c2d39f",
+            "metadata": {
+                "ExecuteTime": {
+                    "end_time": "2024-02-07T17:59:42.338614Z",
+                    "start_time": "2024-02-07T17:59:42.271472Z"
+                }
+            },
+            "outputs": [],
+            "source": [
+                "# Set the display options to print the full table.\n",
+                "pd.set_option(\"display.max_colwidth\", None)\n",
+                "pd.set_option(\"display.max_columns\", None)"
+            ]
+        },
+        {
+            "cell_type": "markdown",
+            "id": "14f379d5",
+            "metadata": {
+                "lines_to_next_cell": 2
+            },
+            "source": [
+                "<a name='utils'></a>\n",
+                "# Utils"
+            ]
+        },
+        {
+            "cell_type": "code",
+            "execution_count": null,
+            "id": "1f41a8dd",
+            "metadata": {
+                "ExecuteTime": {
+                    "end_time": "2024-02-07T17:59:42.380319Z",
+                    "start_time": "2024-02-07T17:59:42.343492Z"
+                }
+            },
+            "outputs": [],
+            "source": [
+                "def make_clickable(url: str) -> str:\n",
+                "    \"\"\"\n",
+                "    Wrapper to make the URL value clickable.\n",
+                "\n",
+                "    :param url: URL value to convert\n",
+                "    :return: clickable URL link\n",
+                "    \"\"\"\n",
+                "    return f'<a href=\"{url}\" target=\"_blank\">{url}</a>'\n",
+                "\n",
+                "\n",
+                "def color_format(val: str, status_color_mapping: Dict[str, str]) -> str:\n",
+                "    \"\"\"\n",
+                "    Return the color depends on status.\n",
+                "\n",
+                "    :param val: value of the status e.g. `failure`\n",
+                "    :param status_color_mapping: mapping statuses to the colors e.g.:\n",
+                "    ```\n",
+                "    {\n",
+                "       \"success\": \"green\",\n",
+                "       \"failure\": \"red\",\n",
+                "    }\n",
+                "    ```\n",
+                "    \"\"\"\n",
+                "    if val in status_color_mapping:\n",
+                "        color = status_color_mapping[val]\n",
+                "    else:\n",
+                "        color = \"grey\"\n",
+                "    return f\"background-color: {color}\""
+            ]
+        },
+        {
+            "cell_type": "markdown",
+            "id": "189f2c75",
+            "metadata": {},
+            "source": [
+                "<a name='gh-workflows-state'></a>\n",
+                "# GH workflows state"
+            ]
+        },
+        {
+            "cell_type": "code",
+            "execution_count": null,
+            "id": "865bc9f2",
+            "metadata": {
+                "ExecuteTime": {
+                    "end_time": "2024-02-07T17:59:57.513155Z",
+                    "start_time": "2024-02-07T17:59:42.383039Z"
+                }
+            },
+            "outputs": [],
+            "source": [
+                "repo_list = [\n",
+                "    \"cryptokaizen/cmamp\",\n",
+                "    \"cryptokaizen/orange\",\n",
+                "    \"cryptokaizen/lemonade\",\n",
+                "    \"causify-ai/kaizenflow\",\n",
+                "]\n",
+                "workflow_df = hlitagh.gh_get_details_for_all_workflows(repo_list)\n",
+                "# Reorder columns.\n",
+                "columns_order = [\"repo_name\", \"workflow_name\", \"conclusion\", \"url\"]\n",
+                "workflow_df = workflow_df[columns_order]\n",
+                "# Make URL values clickable.\n",
+                "workflow_df[\"url\"] = workflow_df[\"url\"].apply(make_clickable)\n",
+                "_LOG.info(hpandas.df_to_str(workflow_df, log_level=logging.INFO))"
+            ]
+        },
+        {
+            "cell_type": "code",
+            "execution_count": null,
+            "id": "f7e999ce",
+            "metadata": {
+                "ExecuteTime": {
+                    "end_time": "2024-02-07T17:59:57.585606Z",
+                    "start_time": "2024-02-07T17:59:57.515915Z"
+                }
+            },
+            "outputs": [],
+            "source": [
+                "status_color_mapping = {\n",
+                "    \"success\": \"green\",\n",
+                "    \"failure\": \"red\",\n",
+                "}\n",
+                "repos = workflow_df[\"repo_name\"].unique()\n",
+                "display(Markdown(\"## Overall Status\"))\n",
+                "current_timestamp = pd.Timestamp.now(tz=\"America/New_York\")\n",
+                "display(Markdown(f\"**Last run: {current_timestamp}**\"))\n",
+                "for repo in repos:\n",
+                "    # Calculate the overall status.\n",
+                "    repo_df = workflow_df[workflow_df[\"repo_name\"] == repo]\n",
+                "    overall_status = hlitagh.gh_get_overall_build_status_for_repo(repo_df)\n",
+                "    display(Markdown(f\"## {repo}: {overall_status}\"))\n",
+                "    repo_df = repo_df.drop(columns=[\"repo_name\"])\n",
+                "    display(\n",
+                "        repo_df.style.map(\n",
+                "            color_format,\n",
+                "            status_color_mapping=status_color_mapping,\n",
+                "            subset=[\"conclusion\"],\n",
+                "        )\n",
+                "    )"
+            ]
+        },
+        {
+            "cell_type": "markdown",
+            "id": "a00870a9",
+            "metadata": {},
+            "source": [
+                "<a name='allure-reports'></a>\n",
+                "# Allure reports"
+            ]
+        },
+        {
+            "cell_type": "markdown",
+            "id": "36e93fca",
+            "metadata": {},
+            "source": [
+                "- fast tests: http://172.30.2.44/allure_reports/cmamp/fast/latest/index.html\n",
+                "- slow tests: http://172.30.2.44/allure_reports/cmamp/slow/latest/index.html\n",
+                "- superslow tests: http://172.30.2.44/allure_reports/cmamp/superslow/latest/index.html"
+            ]
+        },
+        {
+            "cell_type": "markdown",
+            "id": "bb8ed505",
+            "metadata": {},
+            "source": [
+                "<a name='number-of-open-pull-requests'></a>\n",
+                "# Number of open pull requests"
+            ]
+        },
+        {
+            "cell_type": "code",
+            "execution_count": null,
+            "id": "69dbda1d",
+            "metadata": {
+                "ExecuteTime": {
+                    "end_time": "2024-02-07T17:59:59.309022Z",
+                    "start_time": "2024-02-07T17:59:57.588291Z"
+                }
+            },
+            "outputs": [],
+            "source": [
+                "for repo in repo_list:\n",
+                "    number_prs = len(hlitagh.gh_get_open_prs(repo))\n",
+                "    _LOG.info(\"%s: %s\", repo, number_prs)"
+            ]
+        },
+        {
+            "cell_type": "markdown",
+            "id": "ec63cb5e",
+            "metadata": {},
+            "source": [
+                "<a name='code-coverage-html-page'></a>\n",
+                "# Code coverage HTML-page"
+            ]
+        },
+        {
+            "cell_type": "markdown",
+            "id": "569f9404",
+            "metadata": {
+                "ExecuteTime": {
+                    "end_time": "2024-01-24T14:40:31.379819Z",
+                    "start_time": "2024-01-24T14:40:31.327151Z"
+                }
+            },
+            "source": [
+                "http://172.30.2.44/html_coverage/runner_master/"
+            ]
+        },
+        {
+            "cell_type": "markdown",
+            "id": "027d1b3d",
+            "metadata": {},
+            "source": [
+                "<a name='code-coverage-page---codecov'></a>\n",
+                "# Code Coverage Page - CodeCov"
+            ]
+        },
+        {
+            "cell_type": "markdown",
+            "id": "6c9219e1",
+            "metadata": {},
+            "source": [
+                "- Helpers: https://app.codecov.io/gh/causify-ai/helpers"
+            ]
+        }
+    ],
+    "metadata": {
+        "kernelspec": {
+            "display_name": "Python 3 (ipykernel)",
+            "language": "python",
+            "name": "python3"
+        },
+        "language_info": {
+            "codemirror_mode": {
+                "name": "ipython",
+                "version": 3
+            },
+            "file_extension": ".py",
+            "mimetype": "text/x-python",
+            "name": "python",
+            "nbconvert_exporter": "python",
+            "pygments_lexer": "ipython3",
+            "version": "3.12.3"
+        },
+        "toc": {
+            "base_numbering": 1,
+            "nav_menu": {},
+            "number_sections": true,
+            "sideBar": true,
+            "skip_h1_title": false,
+            "title_cell": "Table of Contents",
+            "title_sidebar": "Contents",
+            "toc_cell": false,
+            "toc_position": {},
+            "toc_section_display": true,
+            "toc_window_display": false
+        }
+    },
+    "nbformat": 4,
+    "nbformat_minor": 5
 }
diff --git a/dev_scripts_helpers/update_devops_packages/notebooks/Master_buildmeister_dashboard.py b/dev_scripts_helpers/update_devops_packages/notebooks/Master_buildmeister_dashboard.py
index c7940be98..d0095209c 100644
--- a/dev_scripts_helpers/update_devops_packages/notebooks/Master_buildmeister_dashboard.py
+++ b/dev_scripts_helpers/update_devops_packages/notebooks/Master_buildmeister_dashboard.py
@@ -3,33 +3,40 @@
 #   jupytext:
 #     text_representation:
 #       extension: .py
-#       format_name: percent
-#       format_version: '1.3'
-#       jupytext_version: 1.15.2
+#       format_name: light
+#       format_version: '1.5'
+#       jupytext_version: 1.16.7
 #   kernelspec:
 #     display_name: Python 3 (ipykernel)
 #     language: python
 #     name: python3
 # ---
 
-# %% [markdown]
+# CONTENTS:
+# - [Description](#description)
+# - [Imports](#imports)
+# - [Utils](#utils)
+# - [GH workflows state](#gh-workflows-state)
+# - [Allure reports](#allure-reports)
+# - [Number of open pull requests](#number-of-open-pull-requests)
+# - [Code coverage HTML-page](#code-coverage-html-page)
+# - [Code Coverage Page - CodeCov](#code-coverage-page---codecov)
+
 #  TODO(Grisha): does it belong to the `devops` dir?
 
-# %% [markdown]
+# <a name='description'></a>
 # # Description
 
-# %% [markdown]
 # The notebook reports the latest build status for multiple repos.
 
-# %% [markdown]
+# <a name='imports'></a>
 # # Imports
 
-# %%
 # %load_ext autoreload
 # %autoreload 2
 # %matplotlib inline
 
-# %%
+# +
 import logging
 from typing import Dict
 
@@ -42,22 +49,22 @@
 import helpers.hprint as hprint
 import helpers.lib_tasks_gh as hlitagh
 
-# %%
+# -
+
 hdbg.init_logger(verbosity=logging.INFO)
 _LOG = logging.getLogger(__name__)
 _LOG.info("%s", henv.get_system_signature()[0])
 hprint.config_notebook()
 
-# %%
 # Set the display options to print the full table.
 pd.set_option("display.max_colwidth", None)
 pd.set_option("display.max_columns", None)
 
-# %% [markdown]
+# <a name='utils'></a>
 # # Utils
 
 
-# %%
+# +
 def make_clickable(url: str) -> str:
     """
     Wrapper to make the URL value clickable.
@@ -88,10 +95,11 @@ def color_format(val: str, status_color_mapping: Dict[str, str]) -> str:
     return f"background-color: {color}"
 
 
-# %% [markdown]
+# -
+
+# <a name='gh-workflows-state'></a>
 # # GH workflows state
 
-# %%
 repo_list = [
     "cryptokaizen/cmamp",
     "cryptokaizen/orange",
@@ -106,7 +114,6 @@ def color_format(val: str, status_color_mapping: Dict[str, str]) -> str:
 workflow_df["url"] = workflow_df["url"].apply(make_clickable)
 _LOG.info(hpandas.df_to_str(workflow_df, log_level=logging.INFO))
 
-# %%
 status_color_mapping = {
     "success": "green",
     "failure": "red",
@@ -129,24 +136,26 @@ def color_format(val: str, status_color_mapping: Dict[str, str]) -> str:
         )
     )
 
-# %% [markdown]
+# <a name='allure-reports'></a>
 # # Allure reports
 
-# %% [markdown]
 # - fast tests: http://172.30.2.44/allure_reports/cmamp/fast/latest/index.html
 # - slow tests: http://172.30.2.44/allure_reports/cmamp/slow/latest/index.html
 # - superslow tests: http://172.30.2.44/allure_reports/cmamp/superslow/latest/index.html
 
-# %% [markdown]
+# <a name='number-of-open-pull-requests'></a>
 # # Number of open pull requests
 
-# %%
 for repo in repo_list:
     number_prs = len(hlitagh.gh_get_open_prs(repo))
     _LOG.info("%s: %s", repo, number_prs)
 
-# %% [markdown]
+# <a name='code-coverage-html-page'></a>
 # # Code coverage HTML-page
 
-# %% [markdown]
 # http://172.30.2.44/html_coverage/runner_master/
+
+# <a name='code-coverage-page---codecov'></a>
+# # Code Coverage Page - CodeCov
+
+# - Helpers: https://app.codecov.io/gh/causify-ai/helpers
diff --git a/helpers/lib_tasks_pytest.py b/helpers/lib_tasks_pytest.py
index 3ad73449a..6c9047be9 100644
--- a/helpers/lib_tasks_pytest.py
+++ b/helpers/lib_tasks_pytest.py
@@ -790,7 +790,7 @@ def _publish_html_coverage_report_on_s3(aws_profile: str) -> None:
 def run_coverage_report(  # type: ignore
     ctx,
     target_dir,
-    generate_html_report=True,
+    generate_html_report=False,
     publish_html_on_s3=True,
     aws_profile="ck",
 ):
@@ -874,6 +874,111 @@ def run_coverage_report(  # type: ignore
         _publish_html_coverage_report_on_s3(aws_profile)
 
 
+def _get_inclusion_settings(target_dir: str) -> Tuple[str, Optional[str]]:
+    """
+    Determine include/omit glob patterns for the coverage report for both text
+    and HTML coverage reports.
+
+    :param target_dir: directory for coverage stats; use "." to indicate all directories
+    :return: glob pattern to include and a comma-separated glob pattern to omit
+
+    Examples:
+        1. Cover everything (no submodules to omit):
+        ```
+        _get_inclusion_settings(".")
+        ```
+        ("*", "")
+
+        2. Only cover code under a specific directory:
+        ```
+        _get_inclusion_settings("helpers")
+        ```
+        ("*/helpers/*", None)
+
+    Usage in `_run_coverage`:
+
+         # Entire repo coverage (e.g. your 'helpers' project root):
+         ```
+        include, omit = _get_inclusion_settings(".")
+        ```
+        #   coverage report --include=* --sort=Cover
+        #   coverage html   --include=* [--omit=submodule1/*,submodule2/*]
+
+        # Single-directory coverage:
+        ```
+        include, omit = _get_inclusion_settings("helpers")
+        ```
+        #   coverage report --include=*/helpers/* --sort=Cover
+        #   coverage html   --include=*/helpers/* [--omit=...]
+    """
+    if target_dir == ".":
+        include_in_report = "*"
+        exclude_from_report = ""
+        if hserver.skip_submodules_test():
+            submodule_paths: List[str] = hgit.get_submodule_paths()
+            exclude_from_report = ",".join(
+                f"{path}/*" for path in submodule_paths
+            )
+    else:
+        include_in_report = f"*/{target_dir}/*"
+        exclude_from_report = None
+    return include_in_report, exclude_from_report
+
+
+@task
+def run_coverage(
+    ctx, suite: str, target_dir: str = ".", generate_html_report: bool = False
+):
+    """
+    Unified task to run coverage for any test suite.
+
+    :param ctx: invoke context
+    :param suite: suite to run ("fast", "slow", "superslow")
+    :param target_dir: directory to measure coverage
+    """
+    hdbg.dassert_in(suite, ("fast", "slow", "superslow"))
+    # Build the command line.
+    test_cmd_parts = [
+        # Invoke the "<suite>_tests" task.
+        "invoke",
+        f"run_{suite}_tests",
+        # Enable coverage computation.
+        "--coverage",
+        # Specify which directory to test.
+        "-p",
+        target_dir,
+    ]
+    test_cmd = hlitauti.to_multi_line_cmd(test_cmd_parts)
+    # Run the tests under coverage.
+    hlitauti.run(ctx, test_cmd, use_system=False)
+    hdbg.dassert_file_exists(".coverage")
+    # Compute which files/dirs to include and omit in the report.
+    include_in_report, exclude_from_report = _get_inclusion_settings(target_dir)
+    report_cmd: List[str] = [
+        # Reset any previous coverage data to avoid contamination.
+        "coverage erase"
+    ]
+    # Generate a text report, including only our target paths.
+    report_stats_cmd: str = (
+        f"coverage report --include={include_in_report} --sort=Cover"
+    )
+    if exclude_from_report:
+        report_stats_cmd += f" --omit={exclude_from_report}"
+    report_cmd.append(report_stats_cmd)
+    # Produce HTML output for interactive browsing.
+    if generate_html_report:
+        report_html_cmd: str = f"coverage html --include={include_in_report}"
+        if exclude_from_report:
+            report_html_cmd += f" --omit={exclude_from_report}"
+        report_cmd.append(report_html_cmd)
+    # Export XML coverage report to integrate with Codecov.
+    report_cmd.append("coverage xml -o coverage.xml")
+    full_report_cmd: str = " && ".join(report_cmd)
+    docker_cmd_ = f"invoke docker_cmd --use-bash --cmd '{full_report_cmd}'"
+    # Execute the full coverage/Docker pipeline.
+    hlitauti.run(ctx, docker_cmd_)
+
+
 # #############################################################################
 # Traceback.
 # #############################################################################
@@ -998,10 +1103,6 @@ def pytest_repro(  # type: ignore
             pytest helpers/test/test_cache.py::TestCachingOnS3_2
             ```
         - "files": files with the failed tests, e.g.,
-            ```
-            pytest helpers/test/test_cache.py
-            pytest helpers/test/test_lib_tasks.py
-            ```
     :param file_name: the name of the file containing the pytest output file to parse
     :param show_stacktrace: whether to show the stacktrace of the failed tests
       - only if it is available in the pytest output file
@@ -1151,10 +1252,6 @@ def pytest_rename_test(ctx, old_test_class_name, new_test_class_name):  # type:
     Rename the test and move its golden outcome.
 
     E.g., to rename a test class and all the test methods:
-    ```
-    > i pytest_rename_test TestCacheUpdateFunction1 \
-            TestCacheUpdateFunction_new
-    ```
 
     :param old_test_class_name: old class name
     :param new_test_class_name: new class name
@@ -1385,9 +1482,6 @@ def pytest_buildmeister(  # type: ignore
     Run the regression tests.
 
     - Run updating all the tests
-    ```
-    > pytest_buildmeister --pytest-opts "--update_outcomes"
-    ```
 
     :param docker_clean: remove all dead Docker instances
     :param opts: options to pass to the invoke (e.g., `--version 1.2.0` to test
diff --git a/tasks.py b/tasks.py
index 86b2e0643..72f35fb6c 100644
--- a/tasks.py
+++ b/tasks.py
@@ -92,6 +92,7 @@
     pytest_repro,
     run_blank_tests,
     run_coverage_report,
+    run_coverage,
     run_fast_slow_superslow_tests,
     run_fast_slow_tests,
     run_fast_tests,


Reviews:
user: login: dremdem
body: Good job. 
Some comments.
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-23T16:38:30Z
user: login: gpsaggese
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-23T18:34:25Z
user: login: gpsaggese
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-23T18:35:26Z
user: login: Shaunak01
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-24T13:09:18Z
user: login: Shaunak01
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-24T14:33:10Z
user: login: Shaunak01
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-24T14:43:47Z
user: login: dremdem
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-24T14:50:34Z
user: login: dremdem
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-24T14:54:58Z
user: login: dremdem
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-24T15:10:09Z
user: login: Shaunak01
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-24T15:12:15Z
user: login: dremdem
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-24T15:14:58Z
user: login: Shaunak01
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-24T15:33:05Z
user: login: Shaunak01
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-24T15:38:27Z
user: login: dremdem
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-24T16:36:36Z
user: login: Shaunak01
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-24T19:29:59Z
user: login: dremdem
body: Good job. 
Some questions and comments. 
I'd propose to make a doc with a pipeline explanation. 
For readers and users with zero `coverage` and `Codecov` experience will be hard to figure out.
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-25T16:02:07Z
user: login: Shaunak01
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-28T01:51:32Z
user: login: Shaunak01
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-28T02:01:55Z
user: login: Shaunak01
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-28T02:30:53Z
user: login: Shaunak01
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-28T02:35:02Z
user: login: Shaunak01
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-28T02:40:30Z
user: login: Shaunak01
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-28T02:53:13Z
user: login: Shaunak01
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-28T03:07:02Z
user: login: dremdem
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-28T16:22:39Z
user: login: dremdem
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-28T16:24:59Z
user: login: dremdem
body: Good job. 
Some comments.
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-29T15:05:50Z
user: login: Shaunak01
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-29T16:31:20Z
user: login: Shaunak01
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-29T17:57:22Z
user: login: Shaunak01
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-29T17:59:32Z
user: login: Shaunak01
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-29T18:00:59Z
user: login: dremdem
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-29T18:09:23Z
user: login: Shaunak01
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-29T18:51:41Z
user: login: dremdem
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-29T19:00:43Z
user: login: dremdem
body: Good progress.
Some nits and questions.
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-30T12:00:57Z
user: login: Shaunak01
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-30T18:32:16Z
user: login: Shaunak01
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-30T18:36:30Z
user: login: Shaunak01
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-30T18:41:19Z
user: login: Shaunak01
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-30T18:53:52Z
user: login: dremdem
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-01T12:46:09Z
user: login: dremdem
body: LG. Just some nits and comments.
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-01T13:29:00Z
user: login: Shaunak01
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-01T14:33:35Z
user: login: Shaunak01
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-01T15:17:57Z
user: login: Shaunak01
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-01T15:20:31Z
user: login: gpsaggese
body: LGTM

Can we merge
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-01T15:24:03Z
user: login: dremdem
body: LGTM.
Let’s address a few nits, and feel free to ask Samarth for a review.
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-05-02T08:38:43Z

Review comments:
diff_hunk: @@ -874,6 +874,121 @@ def run_coverage_report(  # type: ignore
         _publish_html_coverage_report_on_s3(aws_profile)
 
 
+def _get_inclusion_settings(target_dir: str) -> Tuple[str, Optional[str]]:
+    """
+    Determine include/omit glob patterns for the coverage report.
+
+    :param target_dir: directory for coverage stats; use "." to indicate all directories
+    :return: tuple (include_in_report, exclude_from_report) where
+        - include_in_report is the glob pattern to include
+        - exclude_from_report is a comma‑separated glob pattern to omit, or ``None``
+    """
+    if target_dir == ".":
+        include_in_report: str = "*"
+        exclude_from_report: Optional[str] = None
+        if hserver.skip_submodules_test():
+            submodule_paths: List[str] = hgit.get_submodule_paths()
+            exclude_from_report = ",".join(
+                f"{path}/*" for path in submodule_paths
+            )
+    else:
+        include_in_report = f"*/{target_dir}/*"
+        exclude_from_report = None
+    return include_in_report, exclude_from_report
+
+
+def _run_coverage(
+    ctx,
+    suite: str,
+    target_dir: str,
+    generate_html_report: bool,
+) -> str:
+    """
+    Internal helper to run coverage for a given suite (fast/slow) and emit XML.
+
+    :param suite: one of "fast" or "slow"
+    :param target_dir: coverage target directory
+    :param generate_html_report: whether to produce HTML output
+    :return: filename of the suite‑specific coverage data file
+    """
+
+    if suite not in ("fast", "slow"):
+        raise ValueError(
+            f"Unsupported suite='{suite}'. Expected 'fast' or 'slow'."
+        )
+    test_cmd = f"invoke run_{suite}_tests --coverage -p {target_dir} -- --junitxml=reports/junit_{suite}.xml -o junit_family=legacy"
+    coverage_file = f".coverage_{suite}_tests"
+    hlitauti.run(ctx, test_cmd, use_system=False)
+    hsystem.system(f"mv .coverage {coverage_file}")
+    hdbg.dassert_file_exists(coverage_file)
+    include_in_report, exclude_from_report = _get_inclusion_settings(target_dir)
+    report_cmd: List[str] = [
+        "coverage erase",
+        f"coverage combine --keep {coverage_file}",
+    ]
+    report_stats_cmd: str = (
+        f"coverage report --include={include_in_report} --sort=Cover"
+    )
+    if exclude_from_report:
+        report_stats_cmd += f" --omit={exclude_from_report}"
+    report_cmd.append(report_stats_cmd)
+    if generate_html_report:
+        report_html_cmd: str = f"coverage html --include={include_in_report}"
+        if exclude_from_report:
+            report_html_cmd += f" --omit={exclude_from_report}"
+        report_cmd.append(report_html_cmd)
+    report_cmd.append("coverage xml -o coverage.xml")
+    full_report_cmd: str = " && ".join(report_cmd)
+    docker_cmd_: str = f"invoke docker_cmd --use-bash --cmd '{full_report_cmd}'"
+    hlitauti.run(ctx, docker_cmd_)
+    return coverage_file
+
+
+@task
+def run_fast_coverage(
+    ctx, target_dir: str, generate_html_report: bool = True
+) -> str:
+    """
+    Task wrapper to run *fast* test suite with coverage and emit reports.
+
+    :param ctx: Invoke context
+    :param target_dir: coverage target directory
+    :param generate_html_report: whether to produce HTML output
+    :return: filename of the fast‑suite coverage data file
+    """
+    return _run_coverage(ctx, "fast", target_dir, generate_html_report=False)
path: helpers/lib_tasks_pytest.py
user: login: dremdem
body: Can we assign "fast" to variable and then pass it to the function? 

Same everywhere. 
created_at: 2025-04-23T16:13:42Z
author_association: CONTRIBUTOR
diff_hunk: @@ -874,6 +874,121 @@ def run_coverage_report(  # type: ignore
         _publish_html_coverage_report_on_s3(aws_profile)
 
 
+def _get_inclusion_settings(target_dir: str) -> Tuple[str, Optional[str]]:
+    """
+    Determine include/omit glob patterns for the coverage report.
+
+    :param target_dir: directory for coverage stats; use "." to indicate all directories
+    :return: tuple (include_in_report, exclude_from_report) where
+        - include_in_report is the glob pattern to include
+        - exclude_from_report is a comma‑separated glob pattern to omit, or ``None``
+    """
+    if target_dir == ".":
+        include_in_report: str = "*"
+        exclude_from_report: Optional[str] = None
+        if hserver.skip_submodules_test():
+            submodule_paths: List[str] = hgit.get_submodule_paths()
+            exclude_from_report = ",".join(
+                f"{path}/*" for path in submodule_paths
+            )
+    else:
+        include_in_report = f"*/{target_dir}/*"
+        exclude_from_report = None
+    return include_in_report, exclude_from_report
+
+
+def _run_coverage(
+    ctx,
+    suite: str,
+    target_dir: str,
+    generate_html_report: bool,
+) -> str:
+    """
+    Internal helper to run coverage for a given suite (fast/slow) and emit XML.
+
+    :param suite: one of "fast" or "slow"
+    :param target_dir: coverage target directory
+    :param generate_html_report: whether to produce HTML output
+    :return: filename of the suite‑specific coverage data file
+    """
+
+    if suite not in ("fast", "slow"):
+        raise ValueError(
+            f"Unsupported suite='{suite}'. Expected 'fast' or 'slow'."
+        )
path: helpers/lib_tasks_pytest.py
user: login: dremdem
body: I'd change it to `hdbg.dassert_in()`.
created_at: 2025-04-23T16:14:40Z
author_association: CONTRIBUTOR
diff_hunk: @@ -874,6 +874,121 @@ def run_coverage_report(  # type: ignore
         _publish_html_coverage_report_on_s3(aws_profile)
 
 
+def _get_inclusion_settings(target_dir: str) -> Tuple[str, Optional[str]]:
+    """
+    Determine include/omit glob patterns for the coverage report.
+
+    :param target_dir: directory for coverage stats; use "." to indicate all directories
+    :return: tuple (include_in_report, exclude_from_report) where
+        - include_in_report is the glob pattern to include
+        - exclude_from_report is a comma‑separated glob pattern to omit, or ``None``
+    """
+    if target_dir == ".":
+        include_in_report: str = "*"
+        exclude_from_report: Optional[str] = None
+        if hserver.skip_submodules_test():
+            submodule_paths: List[str] = hgit.get_submodule_paths()
+            exclude_from_report = ",".join(
+                f"{path}/*" for path in submodule_paths
+            )
+    else:
+        include_in_report = f"*/{target_dir}/*"
+        exclude_from_report = None
+    return include_in_report, exclude_from_report
+
+
+def _run_coverage(
+    ctx,
+    suite: str,
+    target_dir: str,
+    generate_html_report: bool,
+) -> str:
+    """
+    Internal helper to run coverage for a given suite (fast/slow) and emit XML.
path: helpers/lib_tasks_pytest.py
user: login: dremdem
body: I'd simplify it like `run coverage for a ....`
created_at: 2025-04-23T16:15:28Z
author_association: CONTRIBUTOR
diff_hunk: @@ -874,6 +874,121 @@ def run_coverage_report(  # type: ignore
         _publish_html_coverage_report_on_s3(aws_profile)
 
 
+def _get_inclusion_settings(target_dir: str) -> Tuple[str, Optional[str]]:
+    """
+    Determine include/omit glob patterns for the coverage report.
+
+    :param target_dir: directory for coverage stats; use "." to indicate all directories
+    :return: tuple (include_in_report, exclude_from_report) where
+        - include_in_report is the glob pattern to include
+        - exclude_from_report is a comma‑separated glob pattern to omit, or ``None``
+    """
+    if target_dir == ".":
+        include_in_report: str = "*"
+        exclude_from_report: Optional[str] = None
+        if hserver.skip_submodules_test():
+            submodule_paths: List[str] = hgit.get_submodule_paths()
+            exclude_from_report = ",".join(
+                f"{path}/*" for path in submodule_paths
+            )
+    else:
+        include_in_report = f"*/{target_dir}/*"
+        exclude_from_report = None
+    return include_in_report, exclude_from_report
+
+
+def _run_coverage(
+    ctx,
+    suite: str,
+    target_dir: str,
+    generate_html_report: bool,
+) -> str:
+    """
+    Internal helper to run coverage for a given suite (fast/slow) and emit XML.
+
+    :param suite: one of "fast" or "slow"
+    :param target_dir: coverage target directory
+    :param generate_html_report: whether to produce HTML output
+    :return: filename of the suite‑specific coverage data file
+    """
+
+    if suite not in ("fast", "slow"):
path: helpers/lib_tasks_pytest.py
user: login: dremdem
body: How about `superslow`?
created_at: 2025-04-23T16:15:57Z
author_association: CONTRIBUTOR
diff_hunk: @@ -874,6 +874,121 @@ def run_coverage_report(  # type: ignore
         _publish_html_coverage_report_on_s3(aws_profile)
 
 
+def _get_inclusion_settings(target_dir: str) -> Tuple[str, Optional[str]]:
+    """
+    Determine include/omit glob patterns for the coverage report.
+
+    :param target_dir: directory for coverage stats; use "." to indicate all directories
+    :return: tuple (include_in_report, exclude_from_report) where
+        - include_in_report is the glob pattern to include
+        - exclude_from_report is a comma‑separated glob pattern to omit, or ``None``
+    """
+    if target_dir == ".":
+        include_in_report: str = "*"
+        exclude_from_report: Optional[str] = None
path: helpers/lib_tasks_pytest.py
user: login: dremdem
body: Let's avoid type hints for variables that are not part of a function or method interface.
created_at: 2025-04-23T16:17:38Z
author_association: CONTRIBUTOR
diff_hunk: @@ -874,6 +874,121 @@ def run_coverage_report(  # type: ignore
         _publish_html_coverage_report_on_s3(aws_profile)
 
 
+def _get_inclusion_settings(target_dir: str) -> Tuple[str, Optional[str]]:
+    """
+    Determine include/omit glob patterns for the coverage report.
+
+    :param target_dir: directory for coverage stats; use "." to indicate all directories
+    :return: tuple (include_in_report, exclude_from_report) where
+        - include_in_report is the glob pattern to include
+        - exclude_from_report is a comma‑separated glob pattern to omit, or ``None``
+    """
+    if target_dir == ".":
+        include_in_report: str = "*"
+        exclude_from_report: Optional[str] = None
path: helpers/lib_tasks_pytest.py
user: login: dremdem
body: Same everywhere.
created_at: 2025-04-23T16:17:48Z
author_association: CONTRIBUTOR
diff_hunk: @@ -874,6 +874,121 @@ def run_coverage_report(  # type: ignore
         _publish_html_coverage_report_on_s3(aws_profile)
 
 
+def _get_inclusion_settings(target_dir: str) -> Tuple[str, Optional[str]]:
+    """
+    Determine include/omit glob patterns for the coverage report.
+
+    :param target_dir: directory for coverage stats; use "." to indicate all directories
+    :return: tuple (include_in_report, exclude_from_report) where
+        - include_in_report is the glob pattern to include
+        - exclude_from_report is a comma‑separated glob pattern to omit, or ``None``
+    """
+    if target_dir == ".":
+        include_in_report: str = "*"
+        exclude_from_report: Optional[str] = None
+        if hserver.skip_submodules_test():
+            submodule_paths: List[str] = hgit.get_submodule_paths()
+            exclude_from_report = ",".join(
+                f"{path}/*" for path in submodule_paths
+            )
+    else:
+        include_in_report = f"*/{target_dir}/*"
+        exclude_from_report = None
+    return include_in_report, exclude_from_report
+
+
+def _run_coverage(
+    ctx,
+    suite: str,
+    target_dir: str,
+    generate_html_report: bool,
+) -> str:
+    """
+    Internal helper to run coverage for a given suite (fast/slow) and emit XML.
+
+    :param suite: one of "fast" or "slow"
+    :param target_dir: coverage target directory
+    :param generate_html_report: whether to produce HTML output
+    :return: filename of the suite‑specific coverage data file
+    """
+
+    if suite not in ("fast", "slow"):
+        raise ValueError(
+            f"Unsupported suite='{suite}'. Expected 'fast' or 'slow'."
+        )
+    test_cmd = f"invoke run_{suite}_tests --coverage -p {target_dir} -- --junitxml=reports/junit_{suite}.xml -o junit_family=legacy"
+    coverage_file = f".coverage_{suite}_tests"
+    hlitauti.run(ctx, test_cmd, use_system=False)
+    hsystem.system(f"mv .coverage {coverage_file}")
+    hdbg.dassert_file_exists(coverage_file)
+    include_in_report, exclude_from_report = _get_inclusion_settings(target_dir)
+    report_cmd: List[str] = [
+        "coverage erase",
+        f"coverage combine --keep {coverage_file}",
+    ]
+    report_stats_cmd: str = (
+        f"coverage report --include={include_in_report} --sort=Cover"
+    )
+    if exclude_from_report:
+        report_stats_cmd += f" --omit={exclude_from_report}"
+    report_cmd.append(report_stats_cmd)
+    if generate_html_report:
+        report_html_cmd: str = f"coverage html --include={include_in_report}"
+        if exclude_from_report:
+            report_html_cmd += f" --omit={exclude_from_report}"
+        report_cmd.append(report_html_cmd)
path: helpers/lib_tasks_pytest.py
user: login: dremdem
body: Just for my understanding: why we need HTML-report? 
created_at: 2025-04-23T16:20:20Z
author_association: CONTRIBUTOR
diff_hunk: @@ -874,6 +874,121 @@ def run_coverage_report(  # type: ignore
         _publish_html_coverage_report_on_s3(aws_profile)
 
 
+def _get_inclusion_settings(target_dir: str) -> Tuple[str, Optional[str]]:
+    """
+    Determine include/omit glob patterns for the coverage report.
+
+    :param target_dir: directory for coverage stats; use "." to indicate all directories
+    :return: tuple (include_in_report, exclude_from_report) where
+        - include_in_report is the glob pattern to include
+        - exclude_from_report is a comma‑separated glob pattern to omit, or ``None``
+    """
+    if target_dir == ".":
+        include_in_report: str = "*"
+        exclude_from_report: Optional[str] = None
+        if hserver.skip_submodules_test():
+            submodule_paths: List[str] = hgit.get_submodule_paths()
+            exclude_from_report = ",".join(
+                f"{path}/*" for path in submodule_paths
+            )
+    else:
+        include_in_report = f"*/{target_dir}/*"
+        exclude_from_report = None
+    return include_in_report, exclude_from_report
+
+
+def _run_coverage(
+    ctx,
+    suite: str,
+    target_dir: str,
+    generate_html_report: bool,
+) -> str:
+    """
+    Internal helper to run coverage for a given suite (fast/slow) and emit XML.
+
+    :param suite: one of "fast" or "slow"
+    :param target_dir: coverage target directory
+    :param generate_html_report: whether to produce HTML output
+    :return: filename of the suite‑specific coverage data file
+    """
+
+    if suite not in ("fast", "slow"):
+        raise ValueError(
+            f"Unsupported suite='{suite}'. Expected 'fast' or 'slow'."
+        )
+    test_cmd = f"invoke run_{suite}_tests --coverage -p {target_dir} -- --junitxml=reports/junit_{suite}.xml -o junit_family=legacy"
path: helpers/lib_tasks_pytest.py
user: login: dremdem
body: Let's add a comment for each command explaining what it does and why we're doing it.
created_at: 2025-04-23T16:27:40Z
author_association: CONTRIBUTOR
diff_hunk: @@ -874,6 +874,121 @@ def run_coverage_report(  # type: ignore
         _publish_html_coverage_report_on_s3(aws_profile)
 
 
+def _get_inclusion_settings(target_dir: str) -> Tuple[str, Optional[str]]:
+    """
+    Determine include/omit glob patterns for the coverage report.
+
+    :param target_dir: directory for coverage stats; use "." to indicate all directories
+    :return: tuple (include_in_report, exclude_from_report) where
+        - include_in_report is the glob pattern to include
+        - exclude_from_report is a comma‑separated glob pattern to omit, or ``None``
+    """
+    if target_dir == ".":
+        include_in_report: str = "*"
+        exclude_from_report: Optional[str] = None
+        if hserver.skip_submodules_test():
+            submodule_paths: List[str] = hgit.get_submodule_paths()
+            exclude_from_report = ",".join(
+                f"{path}/*" for path in submodule_paths
+            )
+    else:
+        include_in_report = f"*/{target_dir}/*"
+        exclude_from_report = None
+    return include_in_report, exclude_from_report
+
+
+def _run_coverage(
+    ctx,
+    suite: str,
+    target_dir: str,
+    generate_html_report: bool,
+) -> str:
+    """
+    Internal helper to run coverage for a given suite (fast/slow) and emit XML.
+
+    :param suite: one of "fast" or "slow"
+    :param target_dir: coverage target directory
+    :param generate_html_report: whether to produce HTML output
+    :return: filename of the suite‑specific coverage data file
path: helpers/lib_tasks_pytest.py
user: login: dremdem
body: Can use provide use-cases for this return value? 
If we don't use it, let's remove it everywhere. 
created_at: 2025-04-23T16:29:57Z
author_association: CONTRIBUTOR
diff_hunk: @@ -874,6 +874,121 @@ def run_coverage_report(  # type: ignore
         _publish_html_coverage_report_on_s3(aws_profile)
 
 
+def _get_inclusion_settings(target_dir: str) -> Tuple[str, Optional[str]]:
+    """
+    Determine include/omit glob patterns for the coverage report.
path: helpers/lib_tasks_pytest.py
user: login: dremdem
body: I'd add some examples, since it's not so clear why we need this function. 
created_at: 2025-04-23T16:31:56Z
author_association: CONTRIBUTOR
diff_hunk: @@ -299,6 +299,23 @@
    "source": [
     "http://172.30.2.44/html_coverage/runner_master/"
    ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "027d1b3d",
+   "metadata": {},
+   "source": [
+    "# Code Coverage Page - CodeCov"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "6c9219e1",
+   "metadata": {},
+   "source": [
+    "- Helpers: https://app.codecov.io/gh/causify-ai/helpers\n",
+    "- Other Repos: https://app.codecov.io/gh/causify-ai/<repo_name>"
path: dev_scripts_helpers/update_devops_packages/notebooks/Master_buildmeister_dashboard.ipynb
user: login: dremdem
body: Can you remove it for now, since it's not implemented in the other repositories.
created_at: 2025-04-23T16:36:10Z
author_association: CONTRIBUTOR
diff_hunk: @@ -299,6 +299,23 @@
    "source": [
     "http://172.30.2.44/html_coverage/runner_master/"
    ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "027d1b3d",
+   "metadata": {},
+   "source": [
+    "# Code Coverage Page - CodeCov"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "6c9219e1",
+   "metadata": {},
+   "source": [
+    "- Helpers: https://app.codecov.io/gh/causify-ai/helpers\n",
path: dev_scripts_helpers/update_devops_packages/notebooks/Master_buildmeister_dashboard.ipynb
user: login: dremdem
body: I can't find the slow mark in the report. Would you mind running it as well, please?
created_at: 2025-04-23T16:37:36Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,160 @@
+name: Test coverage
path: .github/workflows/coverage_tests.yml
user: login: gpsaggese
body: Ok but can we reuse our factored out approach, reusing some of the templates?

@dremdem 
created_at: 2025-04-23T18:26:41Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,29 @@
+flag_management:
+  default_rules:
+    carryforward: false
path: codecov.yml
user: login: gpsaggese
body: Can we explain each value?
created_at: 2025-04-23T18:27:14Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,29 @@
+flag_management:
+  default_rules:
+    carryforward: false
+  individual_flags:
+    - name: fast
+      carryforward: true
+    - name: slow
+      carryforward: true
+    - name: combined_fast_slow
+      carryforward: true
+    - name: superslow
+
+# no per-line comments.
path: codecov.yml
user: login: gpsaggese
body: -> No
created_at: 2025-04-23T18:27:19Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,29 @@
+flag_management:
+  default_rules:
+    carryforward: false
+  individual_flags:
+    - name: fast
+      carryforward: true
+    - name: slow
+      carryforward: true
+    - name: combined_fast_slow
+      carryforward: true
+    - name: superslow
+
+# no per-line comments.
+comment: false
+
+coverage:
+  status:
+    project:
+      # post the project‑level GitHub Check.
path: codecov.yml
user: login: gpsaggese
body: -> Post
created_at: 2025-04-23T18:27:26Z
author_association: CONTRIBUTOR
diff_hunk: @@ -874,6 +874,121 @@ def run_coverage_report(  # type: ignore
         _publish_html_coverage_report_on_s3(aws_profile)
 
 
+def _get_inclusion_settings(target_dir: str) -> Tuple[str, Optional[str]]:
+    """
+    Determine include/omit glob patterns for the coverage report.
+
+    :param target_dir: directory for coverage stats; use "." to indicate all directories
+    :return: tuple (include_in_report, exclude_from_report) where
+        - include_in_report is the glob pattern to include
+        - exclude_from_report is a comma‑separated glob pattern to omit, or ``None``
+    """
+    if target_dir == ".":
+        include_in_report: str = "*"
path: helpers/lib_tasks_pytest.py
user: login: gpsaggese
body: This is not needed
created_at: 2025-04-23T18:28:21Z
author_association: CONTRIBUTOR
diff_hunk: @@ -874,6 +874,121 @@ def run_coverage_report(  # type: ignore
         _publish_html_coverage_report_on_s3(aws_profile)
 
 
+def _get_inclusion_settings(target_dir: str) -> Tuple[str, Optional[str]]:
+    """
+    Determine include/omit glob patterns for the coverage report.
+
+    :param target_dir: directory for coverage stats; use "." to indicate all directories
+    :return: tuple (include_in_report, exclude_from_report) where
+        - include_in_report is the glob pattern to include
+        - exclude_from_report is a comma‑separated glob pattern to omit, or ``None``
+    """
+    if target_dir == ".":
+        include_in_report: str = "*"
+        exclude_from_report: Optional[str] = None
path: helpers/lib_tasks_pytest.py
user: login: gpsaggese
body: I'd say "let's avoid type hints when it can automatically inferred".

E.g., `exclude_from_report: Optional[str] = None` is needed to make the linter happy

Some times I prefer to use a token as empty string (e.g., `exclude_from_report = ""`) so you avoid typing
created_at: 2025-04-23T18:28:55Z
author_association: CONTRIBUTOR
diff_hunk: @@ -874,6 +874,121 @@ def run_coverage_report(  # type: ignore
         _publish_html_coverage_report_on_s3(aws_profile)
 
 
+def _get_inclusion_settings(target_dir: str) -> Tuple[str, Optional[str]]:
+    """
+    Determine include/omit glob patterns for the coverage report.
path: helpers/lib_tasks_pytest.py
user: login: gpsaggese
body: Yes let's explain how it's used (and not only what it does)
created_at: 2025-04-23T18:30:01Z
author_association: CONTRIBUTOR
diff_hunk: @@ -874,6 +874,121 @@ def run_coverage_report(  # type: ignore
         _publish_html_coverage_report_on_s3(aws_profile)
 
 
+def _get_inclusion_settings(target_dir: str) -> Tuple[str, Optional[str]]:
+    """
+    Determine include/omit glob patterns for the coverage report.
+
+    :param target_dir: directory for coverage stats; use "." to indicate all directories
+    :return: tuple (include_in_report, exclude_from_report) where
+        - include_in_report is the glob pattern to include
+        - exclude_from_report is a comma‑separated glob pattern to omit, or ``None``
+    """
+    if target_dir == ".":
+        include_in_report: str = "*"
+        exclude_from_report: Optional[str] = None
+        if hserver.skip_submodules_test():
+            submodule_paths: List[str] = hgit.get_submodule_paths()
+            exclude_from_report = ",".join(
+                f"{path}/*" for path in submodule_paths
+            )
+    else:
+        include_in_report = f"*/{target_dir}/*"
+        exclude_from_report = None
+    return include_in_report, exclude_from_report
+
+
+def _run_coverage(
+    ctx,
+    suite: str,
+    target_dir: str,
+    generate_html_report: bool,
+) -> str:
+    """
+    Internal helper to run coverage for a given suite (fast/slow) and emit XML.
+
+    :param suite: one of "fast" or "slow"
+    :param target_dir: coverage target directory
+    :param generate_html_report: whether to produce HTML output
+    :return: filename of the suite‑specific coverage data file
+    """
+
+    if suite not in ("fast", "slow"):
+        raise ValueError(
+            f"Unsupported suite='{suite}'. Expected 'fast' or 'slow'."
+        )
+    test_cmd = f"invoke run_{suite}_tests --coverage -p {target_dir} -- --junitxml=reports/junit_{suite}.xml -o junit_family=legacy"
path: helpers/lib_tasks_pytest.py
user: login: gpsaggese
body: Also you can do

```
test_cmd = [
    "invoke run_...",
    # ...
    "--coverage" 
```

Let's use the long cmd line options since they are often self-commenting
created_at: 2025-04-23T18:31:11Z
author_association: CONTRIBUTOR
diff_hunk: @@ -874,6 +874,121 @@ def run_coverage_report(  # type: ignore
         _publish_html_coverage_report_on_s3(aws_profile)
 
 
+def _get_inclusion_settings(target_dir: str) -> Tuple[str, Optional[str]]:
+    """
+    Determine include/omit glob patterns for the coverage report.
+
+    :param target_dir: directory for coverage stats; use "." to indicate all directories
+    :return: tuple (include_in_report, exclude_from_report) where
+        - include_in_report is the glob pattern to include
+        - exclude_from_report is a comma‑separated glob pattern to omit, or ``None``
+    """
+    if target_dir == ".":
+        include_in_report: str = "*"
+        exclude_from_report: Optional[str] = None
+        if hserver.skip_submodules_test():
+            submodule_paths: List[str] = hgit.get_submodule_paths()
+            exclude_from_report = ",".join(
+                f"{path}/*" for path in submodule_paths
+            )
+    else:
+        include_in_report = f"*/{target_dir}/*"
+        exclude_from_report = None
+    return include_in_report, exclude_from_report
+
+
+def _run_coverage(
+    ctx,
+    suite: str,
+    target_dir: str,
+    generate_html_report: bool,
+) -> str:
+    """
+    Internal helper to run coverage for a given suite (fast/slow) and emit XML.
+
+    :param suite: one of "fast" or "slow"
+    :param target_dir: coverage target directory
+    :param generate_html_report: whether to produce HTML output
+    :return: filename of the suite‑specific coverage data file
+    """
+
+    if suite not in ("fast", "slow"):
+        raise ValueError(
+            f"Unsupported suite='{suite}'. Expected 'fast' or 'slow'."
+        )
+    test_cmd = f"invoke run_{suite}_tests --coverage -p {target_dir} -- --junitxml=reports/junit_{suite}.xml -o junit_family=legacy"
+    coverage_file = f".coverage_{suite}_tests"
+    hlitauti.run(ctx, test_cmd, use_system=False)
+    hsystem.system(f"mv .coverage {coverage_file}")
+    hdbg.dassert_file_exists(coverage_file)
+    include_in_report, exclude_from_report = _get_inclusion_settings(target_dir)
path: helpers/lib_tasks_pytest.py
user: login: gpsaggese
body: Let's add some comments to split the code in chunks.

```
# Build the command line.
...
# Run the command.
...
# Compute statistics.
...
```
created_at: 2025-04-23T18:31:55Z
author_association: CONTRIBUTOR
diff_hunk: @@ -874,6 +874,121 @@ def run_coverage_report(  # type: ignore
         _publish_html_coverage_report_on_s3(aws_profile)
 
 
+def _get_inclusion_settings(target_dir: str) -> Tuple[str, Optional[str]]:
+    """
+    Determine include/omit glob patterns for the coverage report.
+
+    :param target_dir: directory for coverage stats; use "." to indicate all directories
+    :return: tuple (include_in_report, exclude_from_report) where
+        - include_in_report is the glob pattern to include
+        - exclude_from_report is a comma‑separated glob pattern to omit, or ``None``
+    """
+    if target_dir == ".":
+        include_in_report: str = "*"
+        exclude_from_report: Optional[str] = None
+        if hserver.skip_submodules_test():
+            submodule_paths: List[str] = hgit.get_submodule_paths()
+            exclude_from_report = ",".join(
+                f"{path}/*" for path in submodule_paths
+            )
+    else:
+        include_in_report = f"*/{target_dir}/*"
+        exclude_from_report = None
+    return include_in_report, exclude_from_report
+
+
+def _run_coverage(
+    ctx,
+    suite: str,
+    target_dir: str,
+    generate_html_report: bool,
+) -> str:
+    """
+    Internal helper to run coverage for a given suite (fast/slow) and emit XML.
+
+    :param suite: one of "fast" or "slow"
+    :param target_dir: coverage target directory
+    :param generate_html_report: whether to produce HTML output
+    :return: filename of the suite‑specific coverage data file
+    """
+
+    if suite not in ("fast", "slow"):
+        raise ValueError(
+            f"Unsupported suite='{suite}'. Expected 'fast' or 'slow'."
+        )
+    test_cmd = f"invoke run_{suite}_tests --coverage -p {target_dir} -- --junitxml=reports/junit_{suite}.xml -o junit_family=legacy"
+    coverage_file = f".coverage_{suite}_tests"
+    hlitauti.run(ctx, test_cmd, use_system=False)
+    hsystem.system(f"mv .coverage {coverage_file}")
+    hdbg.dassert_file_exists(coverage_file)
+    include_in_report, exclude_from_report = _get_inclusion_settings(target_dir)
+    report_cmd: List[str] = [
+        "coverage erase",
+        f"coverage combine --keep {coverage_file}",
+    ]
+    report_stats_cmd: str = (
+        f"coverage report --include={include_in_report} --sort=Cover"
+    )
+    if exclude_from_report:
+        report_stats_cmd += f" --omit={exclude_from_report}"
+    report_cmd.append(report_stats_cmd)
+    if generate_html_report:
+        report_html_cmd: str = f"coverage html --include={include_in_report}"
+        if exclude_from_report:
+            report_html_cmd += f" --omit={exclude_from_report}"
+        report_cmd.append(report_html_cmd)
+    report_cmd.append("coverage xml -o coverage.xml")
+    full_report_cmd: str = " && ".join(report_cmd)
+    docker_cmd_: str = f"invoke docker_cmd --use-bash --cmd '{full_report_cmd}'"
path: helpers/lib_tasks_pytest.py
user: login: gpsaggese
body: No need for these type hints. The linter will infer it automatically. It tells you when it can't
created_at: 2025-04-23T18:32:36Z
author_association: CONTRIBUTOR
diff_hunk: @@ -874,6 +874,121 @@ def run_coverage_report(  # type: ignore
         _publish_html_coverage_report_on_s3(aws_profile)
 
 
+def _get_inclusion_settings(target_dir: str) -> Tuple[str, Optional[str]]:
+    """
+    Determine include/omit glob patterns for the coverage report.
+
+    :param target_dir: directory for coverage stats; use "." to indicate all directories
+    :return: tuple (include_in_report, exclude_from_report) where
+        - include_in_report is the glob pattern to include
+        - exclude_from_report is a comma‑separated glob pattern to omit, or ``None``
+    """
+    if target_dir == ".":
+        include_in_report: str = "*"
+        exclude_from_report: Optional[str] = None
+        if hserver.skip_submodules_test():
+            submodule_paths: List[str] = hgit.get_submodule_paths()
+            exclude_from_report = ",".join(
+                f"{path}/*" for path in submodule_paths
+            )
+    else:
+        include_in_report = f"*/{target_dir}/*"
+        exclude_from_report = None
+    return include_in_report, exclude_from_report
+
+
+def _run_coverage(
+    ctx,
+    suite: str,
+    target_dir: str,
+    generate_html_report: bool,
+) -> str:
+    """
+    Internal helper to run coverage for a given suite (fast/slow) and emit XML.
+
+    :param suite: one of "fast" or "slow"
+    :param target_dir: coverage target directory
+    :param generate_html_report: whether to produce HTML output
+    :return: filename of the suite‑specific coverage data file
+    """
+
+    if suite not in ("fast", "slow"):
+        raise ValueError(
+            f"Unsupported suite='{suite}'. Expected 'fast' or 'slow'."
+        )
+    test_cmd = f"invoke run_{suite}_tests --coverage -p {target_dir} -- --junitxml=reports/junit_{suite}.xml -o junit_family=legacy"
+    coverage_file = f".coverage_{suite}_tests"
+    hlitauti.run(ctx, test_cmd, use_system=False)
+    hsystem.system(f"mv .coverage {coverage_file}")
+    hdbg.dassert_file_exists(coverage_file)
+    include_in_report, exclude_from_report = _get_inclusion_settings(target_dir)
+    report_cmd: List[str] = [
+        "coverage erase",
+        f"coverage combine --keep {coverage_file}",
+    ]
+    report_stats_cmd: str = (
+        f"coverage report --include={include_in_report} --sort=Cover"
+    )
+    if exclude_from_report:
+        report_stats_cmd += f" --omit={exclude_from_report}"
+    report_cmd.append(report_stats_cmd)
+    if generate_html_report:
+        report_html_cmd: str = f"coverage html --include={include_in_report}"
+        if exclude_from_report:
+            report_html_cmd += f" --omit={exclude_from_report}"
+        report_cmd.append(report_html_cmd)
+    report_cmd.append("coverage xml -o coverage.xml")
+    full_report_cmd: str = " && ".join(report_cmd)
+    docker_cmd_: str = f"invoke docker_cmd --use-bash --cmd '{full_report_cmd}'"
+    hlitauti.run(ctx, docker_cmd_)
+    return coverage_file
+
+
+@task
+def run_fast_coverage(
+    ctx, target_dir: str, generate_html_report: bool = True
+) -> str:
+    """
+    Task wrapper to run *fast* test suite with coverage and emit reports.
+
+    :param ctx: Invoke context
+    :param target_dir: coverage target directory
+    :param generate_html_report: whether to produce HTML output
+    :return: filename of the fast‑suite coverage data file
+    """
+    return _run_coverage(ctx, "fast", target_dir, generate_html_report=False)
path: helpers/lib_tasks_pytest.py
user: login: gpsaggese
body: It's a thin layer (the function body is one line and there is 20 lines around it)

But do you need to bind the task? If that's the reason, I'm ok with it
created_at: 2025-04-23T18:33:33Z
author_association: CONTRIBUTOR
diff_hunk: @@ -41,23 +36,21 @@
 import helpers.hpandas as hpandas
 import helpers.hprint as hprint
 import helpers.lib_tasks_gh as hlitagh
+# -
 
-# %%
 hdbg.init_logger(verbosity=logging.INFO)
 _LOG = logging.getLogger(__name__)
 _LOG.info("%s", henv.get_system_signature()[0])
 hprint.config_notebook()
 
-# %%
 # Set the display options to print the full table.
 pd.set_option("display.max_colwidth", None)
 pd.set_option("display.max_columns", None)
 
-# %% [markdown]
 # # Utils
 
 
-# %%
+# +
path: dev_scripts_helpers/update_devops_packages/notebooks/Master_buildmeister_dashboard.py
user: login: gpsaggese
body: Can we do a quick follow up moving the code from the notebook to a lib?
I can be called Master_buildmeister_utils.py
created_at: 2025-04-23T18:35:21Z
author_association: CONTRIBUTOR
diff_hunk: @@ -874,6 +874,121 @@ def run_coverage_report(  # type: ignore
         _publish_html_coverage_report_on_s3(aws_profile)
 
 
+def _get_inclusion_settings(target_dir: str) -> Tuple[str, Optional[str]]:
+    """
+    Determine include/omit glob patterns for the coverage report.
+
+    :param target_dir: directory for coverage stats; use "." to indicate all directories
+    :return: tuple (include_in_report, exclude_from_report) where
+        - include_in_report is the glob pattern to include
+        - exclude_from_report is a comma‑separated glob pattern to omit, or ``None``
+    """
+    if target_dir == ".":
+        include_in_report: str = "*"
+        exclude_from_report: Optional[str] = None
+        if hserver.skip_submodules_test():
+            submodule_paths: List[str] = hgit.get_submodule_paths()
+            exclude_from_report = ",".join(
+                f"{path}/*" for path in submodule_paths
+            )
+    else:
+        include_in_report = f"*/{target_dir}/*"
+        exclude_from_report = None
+    return include_in_report, exclude_from_report
+
+
+def _run_coverage(
+    ctx,
+    suite: str,
+    target_dir: str,
+    generate_html_report: bool,
+) -> str:
+    """
+    Internal helper to run coverage for a given suite (fast/slow) and emit XML.
+
+    :param suite: one of "fast" or "slow"
+    :param target_dir: coverage target directory
+    :param generate_html_report: whether to produce HTML output
+    :return: filename of the suite‑specific coverage data file
+    """
+
+    if suite not in ("fast", "slow"):
+        raise ValueError(
+            f"Unsupported suite='{suite}'. Expected 'fast' or 'slow'."
+        )
+    test_cmd = f"invoke run_{suite}_tests --coverage -p {target_dir} -- --junitxml=reports/junit_{suite}.xml -o junit_family=legacy"
+    coverage_file = f".coverage_{suite}_tests"
+    hlitauti.run(ctx, test_cmd, use_system=False)
+    hsystem.system(f"mv .coverage {coverage_file}")
+    hdbg.dassert_file_exists(coverage_file)
+    include_in_report, exclude_from_report = _get_inclusion_settings(target_dir)
+    report_cmd: List[str] = [
+        "coverage erase",
+        f"coverage combine --keep {coverage_file}",
+    ]
+    report_stats_cmd: str = (
+        f"coverage report --include={include_in_report} --sort=Cover"
+    )
+    if exclude_from_report:
+        report_stats_cmd += f" --omit={exclude_from_report}"
+    report_cmd.append(report_stats_cmd)
+    if generate_html_report:
+        report_html_cmd: str = f"coverage html --include={include_in_report}"
+        if exclude_from_report:
+            report_html_cmd += f" --omit={exclude_from_report}"
+        report_cmd.append(report_html_cmd)
path: helpers/lib_tasks_pytest.py
user: login: Shaunak01
body: We dont need it for codecov. But in case codecov is down or we want to check our reports locally we can generate html coverage and check in browser
created_at: 2025-04-24T13:09:18Z
author_association: CONTRIBUTOR
diff_hunk: @@ -299,6 +299,23 @@
    "source": [
     "http://172.30.2.44/html_coverage/runner_master/"
    ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "027d1b3d",
+   "metadata": {},
+   "source": [
+    "# Code Coverage Page - CodeCov"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "6c9219e1",
+   "metadata": {},
+   "source": [
+    "- Helpers: https://app.codecov.io/gh/causify-ai/helpers\n",
path: dev_scripts_helpers/update_devops_packages/notebooks/Master_buildmeister_dashboard.ipynb
user: login: Shaunak01
body: https://app.codecov.io/gh/causify-ai/helpers/tree/HelpersTask597_Configure_Codecov_for_coverage?flags%5B0%5D=slow

FYI: We can see the master coverage once PR is merged and the scheduled task runs or we invoke manually via GitActions
created_at: 2025-04-24T14:33:10Z
author_association: CONTRIBUTOR
diff_hunk: @@ -41,23 +36,21 @@
 import helpers.hpandas as hpandas
 import helpers.hprint as hprint
 import helpers.lib_tasks_gh as hlitagh
+# -
 
-# %%
 hdbg.init_logger(verbosity=logging.INFO)
 _LOG = logging.getLogger(__name__)
 _LOG.info("%s", henv.get_system_signature()[0])
 hprint.config_notebook()
 
-# %%
 # Set the display options to print the full table.
 pd.set_option("display.max_colwidth", None)
 pd.set_option("display.max_columns", None)
 
-# %% [markdown]
 # # Utils
 
 
-# %%
+# +
path: dev_scripts_helpers/update_devops_packages/notebooks/Master_buildmeister_dashboard.py
user: login: Shaunak01
body: Filed: https://github.com/causify-ai/helpers/issues/612
created_at: 2025-04-24T14:43:47Z
author_association: CONTRIBUTOR
diff_hunk: @@ -299,6 +299,23 @@
    "source": [
     "http://172.30.2.44/html_coverage/runner_master/"
    ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "027d1b3d",
+   "metadata": {},
+   "source": [
+    "# Code Coverage Page - CodeCov"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "6c9219e1",
+   "metadata": {},
+   "source": [
+    "- Helpers: https://app.codecov.io/gh/causify-ai/helpers\n",
path: dev_scripts_helpers/update_devops_packages/notebooks/Master_buildmeister_dashboard.ipynb
user: login: dremdem
body: > we invoke manually via GitActions

Can you pls run it in order to see complete picture in the `Codecov UI`.
created_at: 2025-04-24T14:50:34Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,160 @@
+name: Test coverage
path: .github/workflows/coverage_tests.yml
user: login: dremdem
body: Yes, once we finish setting up `Codecov` in the `helpers` repo, we can turn it into a reusable workflow and call it from the other repositories.
created_at: 2025-04-24T14:54:58Z
author_association: CONTRIBUTOR
diff_hunk: @@ -874,6 +874,121 @@ def run_coverage_report(  # type: ignore
         _publish_html_coverage_report_on_s3(aws_profile)
 
 
+def _get_inclusion_settings(target_dir: str) -> Tuple[str, Optional[str]]:
+    """
+    Determine include/omit glob patterns for the coverage report.
+
+    :param target_dir: directory for coverage stats; use "." to indicate all directories
+    :return: tuple (include_in_report, exclude_from_report) where
+        - include_in_report is the glob pattern to include
+        - exclude_from_report is a comma‑separated glob pattern to omit, or ``None``
+    """
+    if target_dir == ".":
+        include_in_report: str = "*"
+        exclude_from_report: Optional[str] = None
+        if hserver.skip_submodules_test():
+            submodule_paths: List[str] = hgit.get_submodule_paths()
+            exclude_from_report = ",".join(
+                f"{path}/*" for path in submodule_paths
+            )
+    else:
+        include_in_report = f"*/{target_dir}/*"
+        exclude_from_report = None
+    return include_in_report, exclude_from_report
+
+
+def _run_coverage(
+    ctx,
+    suite: str,
+    target_dir: str,
+    generate_html_report: bool,
+) -> str:
+    """
+    Internal helper to run coverage for a given suite (fast/slow) and emit XML.
+
+    :param suite: one of "fast" or "slow"
+    :param target_dir: coverage target directory
+    :param generate_html_report: whether to produce HTML output
+    :return: filename of the suite‑specific coverage data file
+    """
+
+    if suite not in ("fast", "slow"):
+        raise ValueError(
+            f"Unsupported suite='{suite}'. Expected 'fast' or 'slow'."
+        )
+    test_cmd = f"invoke run_{suite}_tests --coverage -p {target_dir} -- --junitxml=reports/junit_{suite}.xml -o junit_family=legacy"
+    coverage_file = f".coverage_{suite}_tests"
+    hlitauti.run(ctx, test_cmd, use_system=False)
+    hsystem.system(f"mv .coverage {coverage_file}")
+    hdbg.dassert_file_exists(coverage_file)
+    include_in_report, exclude_from_report = _get_inclusion_settings(target_dir)
+    report_cmd: List[str] = [
+        "coverage erase",
+        f"coverage combine --keep {coverage_file}",
+    ]
+    report_stats_cmd: str = (
+        f"coverage report --include={include_in_report} --sort=Cover"
+    )
+    if exclude_from_report:
+        report_stats_cmd += f" --omit={exclude_from_report}"
+    report_cmd.append(report_stats_cmd)
+    if generate_html_report:
+        report_html_cmd: str = f"coverage html --include={include_in_report}"
+        if exclude_from_report:
+            report_html_cmd += f" --omit={exclude_from_report}"
+        report_cmd.append(report_html_cmd)
path: helpers/lib_tasks_pytest.py
user: login: dremdem
body: I'd say let's make it optional and not generate by default in CI. 
created_at: 2025-04-24T15:10:09Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,160 @@
+name: Test coverage
path: .github/workflows/coverage_tests.yml
user: login: Shaunak01
body: @dremdem can you guide
created_at: 2025-04-24T15:12:15Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,160 @@
+name: Test coverage
path: .github/workflows/coverage_tests.yml
user: login: dremdem
body: > @dremdem can you guide

Yes, sure - but let's first make this action workflow functional and complete, then we can make it reusable in a follow-up.
created_at: 2025-04-24T15:14:58Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,160 @@
+name: Test coverage
path: .github/workflows/coverage_tests.yml
user: login: Shaunak01
body: Oops i didnt see ur comment. Git glitch sorry
created_at: 2025-04-24T15:33:05Z
author_association: CONTRIBUTOR
diff_hunk: @@ -299,6 +299,23 @@
    "source": [
     "http://172.30.2.44/html_coverage/runner_master/"
    ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "027d1b3d",
+   "metadata": {},
+   "source": [
+    "# Code Coverage Page - CodeCov"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "6c9219e1",
+   "metadata": {},
+   "source": [
+    "- Helpers: https://app.codecov.io/gh/causify-ai/helpers\n",
path: dev_scripts_helpers/update_devops_packages/notebooks/Master_buildmeister_dashboard.ipynb
user: login: Shaunak01
body: I am not sure I understand. We can see both the fast and slow test on UI rn

![image](https://github.com/user-attachments/assets/f608a3ef-2061-4704-9aaf-0f66a3db0c6e)

![image](https://github.com/user-attachments/assets/574381e0-46b0-4a02-834e-ab4789b36296)

We can only invoke manually after the PR is merged not rn.


created_at: 2025-04-24T15:38:27Z
author_association: CONTRIBUTOR
diff_hunk: @@ -299,6 +299,23 @@
    "source": [
     "http://172.30.2.44/html_coverage/runner_master/"
    ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "027d1b3d",
+   "metadata": {},
+   "source": [
+    "# Code Coverage Page - CodeCov"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "6c9219e1",
+   "metadata": {},
+   "source": [
+    "- Helpers: https://app.codecov.io/gh/causify-ai/helpers\n",
path: dev_scripts_helpers/update_devops_packages/notebooks/Master_buildmeister_dashboard.ipynb
user: login: dremdem
body: Oops, sorry, I meant superslow ofc. 
created_at: 2025-04-24T16:36:36Z
author_association: CONTRIBUTOR
diff_hunk: @@ -299,6 +299,23 @@
    "source": [
     "http://172.30.2.44/html_coverage/runner_master/"
    ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "027d1b3d",
+   "metadata": {},
+   "source": [
+    "# Code Coverage Page - CodeCov"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "6c9219e1",
+   "metadata": {},
+   "source": [
+    "- Helpers: https://app.codecov.io/gh/causify-ai/helpers\n",
path: dev_scripts_helpers/update_devops_packages/notebooks/Master_buildmeister_dashboard.ipynb
user: login: Shaunak01
body: ![image](https://github.com/user-attachments/assets/e57f2e99-229a-41fe-a16b-238f267bc704)


created_at: 2025-04-24T19:29:59Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,134 @@
+name: Test coverage
+
+on:
+  workflow_dispatch: {}
+  # every day at 00:00 UTC.
+  schedule:
+    - cron: '0 0 * * *'
+
+env:
+  CSFY_CI: true
+
+permissions:
+  id-token: write
+  contents: read
+  packages: read
+
+jobs:
+  run_test_coverage:
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Configure AWS credentials
+        uses: aws-actions/configure-aws-credentials@v1
+        with:
+          role-to-assume: ${{ vars.GH_ACTION_AWS_ROLE_ARN }}
+          role-session-name: ${{ vars.GH_ACTION_AWS_SESSION_NAME }}
+          aws-region: ${{ vars.CSFY_AWS_DEFAULT_REGION }}
+
+      - name: Login to GHCR
+        run: docker login ghcr.io -u gpsaggese -p ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Cleanup
+        run: sudo chmod 777 -R .
+
+      - name: Checkout code
+        uses: actions/checkout@v3
+        with:
+          submodules: true
path: .github/workflows/coverage_tests.yml
user: login: dremdem
body: Let's change it to `submodules: recursive` to prepare for using it as a reusable workflow.
created_at: 2025-04-25T14:52:49Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,134 @@
+name: Test coverage
+
+on:
+  workflow_dispatch: {}
+  # every day at 00:00 UTC.
+  schedule:
+    - cron: '0 0 * * *'
+
+env:
+  CSFY_CI: true
+
+permissions:
+  id-token: write
+  contents: read
+  packages: read
+
+jobs:
+  run_test_coverage:
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Configure AWS credentials
+        uses: aws-actions/configure-aws-credentials@v1
+        with:
+          role-to-assume: ${{ vars.GH_ACTION_AWS_ROLE_ARN }}
+          role-session-name: ${{ vars.GH_ACTION_AWS_SESSION_NAME }}
+          aws-region: ${{ vars.CSFY_AWS_DEFAULT_REGION }}
+
+      - name: Login to GHCR
+        run: docker login ghcr.io -u gpsaggese -p ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Cleanup
+        run: sudo chmod 777 -R .
+
+      - name: Checkout code
+        uses: actions/checkout@v3
+        with:
+          submodules: true
+          fetch-depth: 0
+          token: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Update PYTHONPATH
+        run: echo "PYTHONPATH=.:helpers" >> $GITHUB_ENV
+
+      - name: Install dependencies
+        run: |
+          python -m pip install --upgrade pip
+          pip install -r .github/gh_requirements.txt
+
+      - name: Pull image from GHCR
+        run: docker pull ghcr.io/${{ github.repository }}:dev
+
+      # Only on scheduled runs, capture ISO weekday (1=Mon … 7=Sun).
+      - name: Set DAY_OF_WEEK
+        if: github.event_name == 'schedule'
+        run: echo "DAY_OF_WEEK=$(date -u +'%u')" >> $GITHUB_ENV
+
+      # This step is used to trigger the fast test coverage generation using the invoke task.
+      - name: Run Fast test and generate report
+        env:
+          GH_ACTION_ACCESS_TOKEN: ${{ secrets.GH_ACTION_ACCESS_TOKEN }}
+          CSFY_AWS_ACCESS_KEY_ID: ${{ env.AWS_ACCESS_KEY_ID }}
+          CSFY_AWS_SECRET_ACCESS_KEY: ${{ env.AWS_SECRET_ACCESS_KEY }}
+          CSFY_AWS_SESSION_TOKEN: ${{ env.AWS_SESSION_TOKEN }}
+          CSFY_AWS_DEFAULT_REGION: ${{ env.AWS_DEFAULT_REGION }}
+          CSFY_ECR_BASE_PATH: ghcr.io/${{ github.repository_owner }}
+          CSFY_AWS_S3_BUCKET: ${{ vars.CSFY_AWS_S3_BUCKET }}
+        run: invoke run_fast_coverage .
+
+      - name: Upload Fast Test Coverage to Codecov
+        # `if: always()` ensures this step runs regardless of prior failures.
+        # It is used here to always attempt uploading the coverage report,
+        # even if earlier steps have failed.
+        if: always()
+        uses: codecov/codecov-action@v5
+        with:
+          token: ${{ secrets.CODECOV_TOKEN }}
+          files: ./coverage.xml
+          flags: fast
+          name: fast-test-coverage
+          # Combined coverage for tests will be checked rather than individual.
+          fail_ci: false
+          # `fail_ci_if_error: true` instructs the action to mark the CI as failed
+          # if there is any error during the upload to Codecov.
+          fail_ci_if_error: true
+
+      - name: Run Slow test and generate report
+        env:
+          GH_ACTION_ACCESS_TOKEN: ${{ secrets.GH_ACTION_ACCESS_TOKEN }}
+          CSFY_AWS_ACCESS_KEY_ID: ${{ env.AWS_ACCESS_KEY_ID }}
+          CSFY_AWS_SECRET_ACCESS_KEY: ${{ env.AWS_SECRET_ACCESS_KEY }}
+          CSFY_AWS_SESSION_TOKEN: ${{ env.AWS_SESSION_TOKEN }}
+          CSFY_AWS_DEFAULT_REGION: ${{ env.AWS_DEFAULT_REGION }}
+          CSFY_ECR_BASE_PATH: ghcr.io/${{ github.repository_owner }}
+          CSFY_AWS_S3_BUCKET: ${{ vars.CSFY_AWS_S3_BUCKET }}
+        run: invoke run_slow_coverage .
+
+      - name: Upload Slow Test Coverage to Codecov
+        # "if: always()" ensures this step is executed regardless of previous outcomes.
+        if: always()
+        uses: codecov/codecov-action@v5
+        with:
+          token: ${{ secrets.CODECOV_TOKEN }}
+          files: ./coverage.xml
+          flags: slow
+          name: slow-test-coverage
+          fail_ci: false
+          # "fail_ci_if_error: true" causes the job to fail if the upload to Codecov fails,
+          # helping to catch issues with the coverage reporting.
+          fail_ci_if_error: true
+
+      - name: Run Superslow test and generate report
+        # Scheduled for Monday (i.e Day 1) or when manually triggered.
+        if: (github.event_name == 'schedule' && env.DAY_OF_WEEK == '1') || github.event_name == 'workflow_dispatch'
path: .github/workflows/coverage_tests.yml
user: login: dremdem
body: It's looking a bit hackish. Let's add a TODO to consider removing it when we turn this workflow into a reusable one.
created_at: 2025-04-25T15:06:58Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,134 @@
+name: Test coverage
+
+on:
+  workflow_dispatch: {}
+  # every day at 00:00 UTC.
+  schedule:
+    - cron: '0 0 * * *'
+
+env:
+  CSFY_CI: true
+
+permissions:
+  id-token: write
+  contents: read
+  packages: read
+
+jobs:
+  run_test_coverage:
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Configure AWS credentials
+        uses: aws-actions/configure-aws-credentials@v1
+        with:
+          role-to-assume: ${{ vars.GH_ACTION_AWS_ROLE_ARN }}
+          role-session-name: ${{ vars.GH_ACTION_AWS_SESSION_NAME }}
+          aws-region: ${{ vars.CSFY_AWS_DEFAULT_REGION }}
+
+      - name: Login to GHCR
+        run: docker login ghcr.io -u gpsaggese -p ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Cleanup
+        run: sudo chmod 777 -R .
+
+      - name: Checkout code
+        uses: actions/checkout@v3
+        with:
+          submodules: true
+          fetch-depth: 0
+          token: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Update PYTHONPATH
+        run: echo "PYTHONPATH=.:helpers" >> $GITHUB_ENV
+
+      - name: Install dependencies
+        run: |
+          python -m pip install --upgrade pip
+          pip install -r .github/gh_requirements.txt
+
+      - name: Pull image from GHCR
+        run: docker pull ghcr.io/${{ github.repository }}:dev
+
+      # Only on scheduled runs, capture ISO weekday (1=Mon … 7=Sun).
+      - name: Set DAY_OF_WEEK
+        if: github.event_name == 'schedule'
+        run: echo "DAY_OF_WEEK=$(date -u +'%u')" >> $GITHUB_ENV
+
+      # This step is used to trigger the fast test coverage generation using the invoke task.
+      - name: Run Fast test and generate report
+        env:
+          GH_ACTION_ACCESS_TOKEN: ${{ secrets.GH_ACTION_ACCESS_TOKEN }}
+          CSFY_AWS_ACCESS_KEY_ID: ${{ env.AWS_ACCESS_KEY_ID }}
+          CSFY_AWS_SECRET_ACCESS_KEY: ${{ env.AWS_SECRET_ACCESS_KEY }}
+          CSFY_AWS_SESSION_TOKEN: ${{ env.AWS_SESSION_TOKEN }}
+          CSFY_AWS_DEFAULT_REGION: ${{ env.AWS_DEFAULT_REGION }}
+          CSFY_ECR_BASE_PATH: ghcr.io/${{ github.repository_owner }}
+          CSFY_AWS_S3_BUCKET: ${{ vars.CSFY_AWS_S3_BUCKET }}
+        run: invoke run_fast_coverage .
+
+      - name: Upload Fast Test Coverage to Codecov
+        # `if: always()` ensures this step runs regardless of prior failures.
+        # It is used here to always attempt uploading the coverage report,
+        # even if earlier steps have failed.
+        if: always()
+        uses: codecov/codecov-action@v5
+        with:
+          token: ${{ secrets.CODECOV_TOKEN }}
+          files: ./coverage.xml
+          flags: fast
+          name: fast-test-coverage
+          # Combined coverage for tests will be checked rather than individual.
+          fail_ci: false
+          # `fail_ci_if_error: true` instructs the action to mark the CI as failed
+          # if there is any error during the upload to Codecov.
+          fail_ci_if_error: true
+
+      - name: Run Slow test and generate report
+        env:
+          GH_ACTION_ACCESS_TOKEN: ${{ secrets.GH_ACTION_ACCESS_TOKEN }}
+          CSFY_AWS_ACCESS_KEY_ID: ${{ env.AWS_ACCESS_KEY_ID }}
+          CSFY_AWS_SECRET_ACCESS_KEY: ${{ env.AWS_SECRET_ACCESS_KEY }}
+          CSFY_AWS_SESSION_TOKEN: ${{ env.AWS_SESSION_TOKEN }}
+          CSFY_AWS_DEFAULT_REGION: ${{ env.AWS_DEFAULT_REGION }}
+          CSFY_ECR_BASE_PATH: ghcr.io/${{ github.repository_owner }}
+          CSFY_AWS_S3_BUCKET: ${{ vars.CSFY_AWS_S3_BUCKET }}
+        run: invoke run_slow_coverage .
+
+      - name: Upload Slow Test Coverage to Codecov
+        # "if: always()" ensures this step is executed regardless of previous outcomes.
+        if: always()
+        uses: codecov/codecov-action@v5
+        with:
+          token: ${{ secrets.CODECOV_TOKEN }}
+          files: ./coverage.xml
+          flags: slow
+          name: slow-test-coverage
+          fail_ci: false
+          # "fail_ci_if_error: true" causes the job to fail if the upload to Codecov fails,
+          # helping to catch issues with the coverage reporting.
+          fail_ci_if_error: true
+
+      - name: Run Superslow test and generate report
+        # Scheduled for Monday (i.e Day 1) or when manually triggered.
+        if: (github.event_name == 'schedule' && env.DAY_OF_WEEK == '1') || github.event_name == 'workflow_dispatch'
+        env:
+          GH_ACTION_ACCESS_TOKEN: ${{ secrets.GH_ACTION_ACCESS_TOKEN }}
+          CSFY_AWS_ACCESS_KEY_ID: ${{ env.AWS_ACCESS_KEY_ID }}
+          CSFY_AWS_SECRET_ACCESS_KEY: ${{ env.AWS_SECRET_ACCESS_KEY }}
+          CSFY_AWS_SESSION_TOKEN: ${{ env.AWS_SESSION_TOKEN }}
+          CSFY_AWS_DEFAULT_REGION: ${{ env.AWS_DEFAULT_REGION }}
+          CSFY_ECR_BASE_PATH: ghcr.io/${{ github.repository_owner }}
+          CSFY_AWS_S3_BUCKET: ${{ vars.CSFY_AWS_S3_BUCKET }}
+        run: invoke run_superslow_coverage .
+
+      - name: Upload Superslow Test Coverage to Codecov
+        if: (github.event_name == 'schedule' && env.DAY_OF_WEEK == '1') || github.event_name == 'workflow_dispatch'
path: .github/workflows/coverage_tests.yml
user: login: dremdem
body: I'd not duplicate the `if` here, just use 
https://stackoverflow.com/questions/60453924/running-a-github-actions-step-only-if-previous-step-has-run

Something like: 
```
  - name: Build app
    id: build
    run: |
      <build command>

  - name: Run tests

  - name: Create artifact of test coverage
    if: steps.build.outcome == 'success'
```
created_at: 2025-04-25T15:10:13Z
author_association: CONTRIBUTOR
diff_hunk: @@ -874,6 +875,177 @@ def run_coverage_report(  # type: ignore
         _publish_html_coverage_report_on_s3(aws_profile)
 
 
+def _get_inclusion_settings(target_dir: str) -> Tuple[str, Optional[str]]:
+    """
+    Determine include/omit glob patterns for the coverage report for both text
+    and HTML coverage reports.
+
+    :param target_dir: directory for coverage stats; use "." to indicate all directories
+    :return: tuple (include_in_report, exclude_from_report) where
+        - include_in_report is the glob pattern to include
+        - exclude_from_report is a comma‑separated glob pattern to omit, or ``None``
+
+    Examples:
+        1. Cover everything (no submodules to omit):
+        ```
+        _get_inclusion_settings(".")
+        ```
+        ("*", "")
+
+        2. Only cover code under a specific directory:
+        ```
+        _get_inclusion_settings("helpers")
+        ```
+        ("*/helpers/*", None)
+
+    Usage in `_run_coverage`:
+
+         # Entire repo coverage (e.g. your 'helpers' project root):
+         ```
+        include, omit = _get_inclusion_settings(".")
+        ```
+        #   coverage report --include=* --sort=Cover
+        #   coverage html   --include=* [--omit=ext/foo/*,ext/bar/*]
+
+        # Single-directory coverage:
+        ```
+        include, omit = _get_inclusion_settings("helpers")
+        ```
+        #   coverage report --include=*/helpers/* --sort=Cover
+        #   coverage html   --include=*/helpers/* [--omit=...]
+    """
+    if target_dir == ".":
+        include_in_report = "*"
+        exclude_from_report = ""
+        if hserver.skip_submodules_test():
+            submodule_paths: List[str] = hgit.get_submodule_paths()
+            exclude_from_report = ",".join(
+                f"{path}/*" for path in submodule_paths
+            )
+    else:
+        include_in_report = f"*/{target_dir}/*"
+        exclude_from_report = None
+    return include_in_report, exclude_from_report
+
+
+def _run_coverage(
+    ctx,
+    suite: str,
+    target_dir: str,
+    generate_html_report: bool = False,
+) -> str:
+    """
+    Run coverage for a given suite (fast/slow/superslow).
+
+    :param suite: one of "fast" or "slow"
path: helpers/lib_tasks_pytest.py
user: login: dremdem
body: Let's add the `superslow` also.
created_at: 2025-04-25T15:22:38Z
author_association: CONTRIBUTOR
diff_hunk: @@ -874,6 +875,177 @@ def run_coverage_report(  # type: ignore
         _publish_html_coverage_report_on_s3(aws_profile)
 
 
+def _get_inclusion_settings(target_dir: str) -> Tuple[str, Optional[str]]:
+    """
+    Determine include/omit glob patterns for the coverage report for both text
+    and HTML coverage reports.
+
+    :param target_dir: directory for coverage stats; use "." to indicate all directories
+    :return: tuple (include_in_report, exclude_from_report) where
+        - include_in_report is the glob pattern to include
+        - exclude_from_report is a comma‑separated glob pattern to omit, or ``None``
+
+    Examples:
+        1. Cover everything (no submodules to omit):
+        ```
+        _get_inclusion_settings(".")
+        ```
+        ("*", "")
+
+        2. Only cover code under a specific directory:
+        ```
+        _get_inclusion_settings("helpers")
+        ```
+        ("*/helpers/*", None)
+
+    Usage in `_run_coverage`:
+
+         # Entire repo coverage (e.g. your 'helpers' project root):
+         ```
+        include, omit = _get_inclusion_settings(".")
+        ```
+        #   coverage report --include=* --sort=Cover
+        #   coverage html   --include=* [--omit=ext/foo/*,ext/bar/*]
+
+        # Single-directory coverage:
+        ```
+        include, omit = _get_inclusion_settings("helpers")
+        ```
+        #   coverage report --include=*/helpers/* --sort=Cover
+        #   coverage html   --include=*/helpers/* [--omit=...]
+    """
+    if target_dir == ".":
+        include_in_report = "*"
+        exclude_from_report = ""
+        if hserver.skip_submodules_test():
+            submodule_paths: List[str] = hgit.get_submodule_paths()
+            exclude_from_report = ",".join(
+                f"{path}/*" for path in submodule_paths
+            )
+    else:
+        include_in_report = f"*/{target_dir}/*"
+        exclude_from_report = None
+    return include_in_report, exclude_from_report
+
+
+def _run_coverage(
+    ctx,
+    suite: str,
+    target_dir: str,
+    generate_html_report: bool = False,
+) -> str:
+    """
+    Run coverage for a given suite (fast/slow/superslow).
+
+    :param suite: one of "fast" or "slow"
+    :param target_dir: coverage target directory
+    :param generate_html_report: whether to produce HTML output
+    :return: xml file named 'coverage.xml'
+    """
+
path: helpers/lib_tasks_pytest.py
user: login: dremdem
body: Linter?
created_at: 2025-04-25T15:22:58Z
author_association: CONTRIBUTOR
diff_hunk: @@ -874,6 +875,177 @@ def run_coverage_report(  # type: ignore
         _publish_html_coverage_report_on_s3(aws_profile)
 
 
+def _get_inclusion_settings(target_dir: str) -> Tuple[str, Optional[str]]:
+    """
+    Determine include/omit glob patterns for the coverage report for both text
+    and HTML coverage reports.
+
+    :param target_dir: directory for coverage stats; use "." to indicate all directories
+    :return: tuple (include_in_report, exclude_from_report) where
+        - include_in_report is the glob pattern to include
+        - exclude_from_report is a comma‑separated glob pattern to omit, or ``None``
+
+    Examples:
+        1. Cover everything (no submodules to omit):
+        ```
+        _get_inclusion_settings(".")
+        ```
+        ("*", "")
+
+        2. Only cover code under a specific directory:
+        ```
+        _get_inclusion_settings("helpers")
+        ```
+        ("*/helpers/*", None)
+
+    Usage in `_run_coverage`:
+
+         # Entire repo coverage (e.g. your 'helpers' project root):
+         ```
+        include, omit = _get_inclusion_settings(".")
+        ```
+        #   coverage report --include=* --sort=Cover
+        #   coverage html   --include=* [--omit=ext/foo/*,ext/bar/*]
+
+        # Single-directory coverage:
+        ```
+        include, omit = _get_inclusion_settings("helpers")
+        ```
+        #   coverage report --include=*/helpers/* --sort=Cover
+        #   coverage html   --include=*/helpers/* [--omit=...]
+    """
+    if target_dir == ".":
+        include_in_report = "*"
+        exclude_from_report = ""
+        if hserver.skip_submodules_test():
+            submodule_paths: List[str] = hgit.get_submodule_paths()
+            exclude_from_report = ",".join(
+                f"{path}/*" for path in submodule_paths
+            )
+    else:
+        include_in_report = f"*/{target_dir}/*"
+        exclude_from_report = None
+    return include_in_report, exclude_from_report
+
+
+def _run_coverage(
+    ctx,
+    suite: str,
+    target_dir: str,
+    generate_html_report: bool = False,
+) -> str:
+    """
+    Run coverage for a given suite (fast/slow/superslow).
+
+    :param suite: one of "fast" or "slow"
+    :param target_dir: coverage target directory
+    :param generate_html_report: whether to produce HTML output
+    :return: xml file named 'coverage.xml'
+    """
+
+    hdbg.dassert_in(
+        suite,
+        ("fast", "slow", "superslow"),
+        "Expected one of 'fast', 'slow', or 'superslow'.",
+    )
+    # Build the command line.
+    test_cmd_parts = [
+        # Invoke the "<suite>_tests" task.
+        "invoke",
+        f"run_{suite}_tests",
+        # Enable coverage collection.
+        "--coverage",
+        # Specify which directory to test.
+        "-p",
+        target_dir,
path: helpers/lib_tasks_pytest.py
user: login: dremdem
body: Perhaps better reflow to one line -> `"-p", target_dir,`
created_at: 2025-04-25T15:24:11Z
author_association: CONTRIBUTOR
diff_hunk: @@ -874,6 +875,177 @@ def run_coverage_report(  # type: ignore
         _publish_html_coverage_report_on_s3(aws_profile)
 
 
+def _get_inclusion_settings(target_dir: str) -> Tuple[str, Optional[str]]:
+    """
+    Determine include/omit glob patterns for the coverage report for both text
+    and HTML coverage reports.
+
+    :param target_dir: directory for coverage stats; use "." to indicate all directories
+    :return: tuple (include_in_report, exclude_from_report) where
+        - include_in_report is the glob pattern to include
+        - exclude_from_report is a comma‑separated glob pattern to omit, or ``None``
+
+    Examples:
+        1. Cover everything (no submodules to omit):
+        ```
+        _get_inclusion_settings(".")
+        ```
+        ("*", "")
+
+        2. Only cover code under a specific directory:
+        ```
+        _get_inclusion_settings("helpers")
+        ```
+        ("*/helpers/*", None)
+
+    Usage in `_run_coverage`:
+
+         # Entire repo coverage (e.g. your 'helpers' project root):
+         ```
+        include, omit = _get_inclusion_settings(".")
+        ```
+        #   coverage report --include=* --sort=Cover
+        #   coverage html   --include=* [--omit=ext/foo/*,ext/bar/*]
+
+        # Single-directory coverage:
+        ```
+        include, omit = _get_inclusion_settings("helpers")
+        ```
+        #   coverage report --include=*/helpers/* --sort=Cover
+        #   coverage html   --include=*/helpers/* [--omit=...]
+    """
+    if target_dir == ".":
+        include_in_report = "*"
+        exclude_from_report = ""
+        if hserver.skip_submodules_test():
+            submodule_paths: List[str] = hgit.get_submodule_paths()
+            exclude_from_report = ",".join(
+                f"{path}/*" for path in submodule_paths
+            )
+    else:
+        include_in_report = f"*/{target_dir}/*"
+        exclude_from_report = None
+    return include_in_report, exclude_from_report
+
+
+def _run_coverage(
+    ctx,
+    suite: str,
+    target_dir: str,
+    generate_html_report: bool = False,
+) -> str:
+    """
+    Run coverage for a given suite (fast/slow/superslow).
+
+    :param suite: one of "fast" or "slow"
+    :param target_dir: coverage target directory
+    :param generate_html_report: whether to produce HTML output
+    :return: xml file named 'coverage.xml'
+    """
+
+    hdbg.dassert_in(
+        suite,
+        ("fast", "slow", "superslow"),
+        "Expected one of 'fast', 'slow', or 'superslow'.",
+    )
+    # Build the command line.
+    test_cmd_parts = [
+        # Invoke the "<suite>_tests" task.
+        "invoke",
+        f"run_{suite}_tests",
+        # Enable coverage collection.
+        "--coverage",
+        # Specify which directory to test.
+        "-p",
+        target_dir,
+        # Pass through remaining args to pytest.
+        "--",
+        # Emit JUnit XML report for CI systems.
path: helpers/lib_tasks_pytest.py
user: login: dremdem
body: Just wondering: what is it? 
created_at: 2025-04-25T15:25:35Z
author_association: CONTRIBUTOR
diff_hunk: @@ -874,6 +875,177 @@ def run_coverage_report(  # type: ignore
         _publish_html_coverage_report_on_s3(aws_profile)
 
 
+def _get_inclusion_settings(target_dir: str) -> Tuple[str, Optional[str]]:
+    """
+    Determine include/omit glob patterns for the coverage report for both text
+    and HTML coverage reports.
+
+    :param target_dir: directory for coverage stats; use "." to indicate all directories
+    :return: tuple (include_in_report, exclude_from_report) where
+        - include_in_report is the glob pattern to include
+        - exclude_from_report is a comma‑separated glob pattern to omit, or ``None``
+
+    Examples:
+        1. Cover everything (no submodules to omit):
+        ```
+        _get_inclusion_settings(".")
+        ```
+        ("*", "")
+
+        2. Only cover code under a specific directory:
+        ```
+        _get_inclusion_settings("helpers")
+        ```
+        ("*/helpers/*", None)
+
+    Usage in `_run_coverage`:
+
+         # Entire repo coverage (e.g. your 'helpers' project root):
+         ```
+        include, omit = _get_inclusion_settings(".")
+        ```
+        #   coverage report --include=* --sort=Cover
+        #   coverage html   --include=* [--omit=ext/foo/*,ext/bar/*]
+
+        # Single-directory coverage:
+        ```
+        include, omit = _get_inclusion_settings("helpers")
+        ```
+        #   coverage report --include=*/helpers/* --sort=Cover
+        #   coverage html   --include=*/helpers/* [--omit=...]
+    """
+    if target_dir == ".":
+        include_in_report = "*"
+        exclude_from_report = ""
+        if hserver.skip_submodules_test():
+            submodule_paths: List[str] = hgit.get_submodule_paths()
+            exclude_from_report = ",".join(
+                f"{path}/*" for path in submodule_paths
+            )
+    else:
+        include_in_report = f"*/{target_dir}/*"
+        exclude_from_report = None
+    return include_in_report, exclude_from_report
+
+
+def _run_coverage(
+    ctx,
+    suite: str,
+    target_dir: str,
+    generate_html_report: bool = False,
+) -> str:
+    """
+    Run coverage for a given suite (fast/slow/superslow).
+
+    :param suite: one of "fast" or "slow"
+    :param target_dir: coverage target directory
+    :param generate_html_report: whether to produce HTML output
+    :return: xml file named 'coverage.xml'
+    """
+
+    hdbg.dassert_in(
+        suite,
+        ("fast", "slow", "superslow"),
+        "Expected one of 'fast', 'slow', or 'superslow'.",
+    )
+    # Build the command line.
+    test_cmd_parts = [
+        # Invoke the "<suite>_tests" task.
+        "invoke",
+        f"run_{suite}_tests",
+        # Enable coverage collection.
+        "--coverage",
+        # Specify which directory to test.
+        "-p",
+        target_dir,
+        # Pass through remaining args to pytest.
+        "--",
+        # Emit JUnit XML report for CI systems.
+        f"--junitxml=reports/junit_{suite}.xml",
+        # Force legacy JUnit output format.
+        "-o",
+        "junit_family=legacy",
path: helpers/lib_tasks_pytest.py
user: login: dremdem
body: Out of curiosity: why do we use "legacy" here
created_at: 2025-04-25T15:26:56Z
author_association: CONTRIBUTOR
diff_hunk: @@ -874,6 +875,177 @@ def run_coverage_report(  # type: ignore
         _publish_html_coverage_report_on_s3(aws_profile)
 
 
+def _get_inclusion_settings(target_dir: str) -> Tuple[str, Optional[str]]:
+    """
+    Determine include/omit glob patterns for the coverage report for both text
+    and HTML coverage reports.
+
+    :param target_dir: directory for coverage stats; use "." to indicate all directories
+    :return: tuple (include_in_report, exclude_from_report) where
+        - include_in_report is the glob pattern to include
+        - exclude_from_report is a comma‑separated glob pattern to omit, or ``None``
+
+    Examples:
+        1. Cover everything (no submodules to omit):
+        ```
+        _get_inclusion_settings(".")
+        ```
+        ("*", "")
+
+        2. Only cover code under a specific directory:
+        ```
+        _get_inclusion_settings("helpers")
+        ```
+        ("*/helpers/*", None)
+
+    Usage in `_run_coverage`:
+
+         # Entire repo coverage (e.g. your 'helpers' project root):
+         ```
+        include, omit = _get_inclusion_settings(".")
+        ```
+        #   coverage report --include=* --sort=Cover
+        #   coverage html   --include=* [--omit=ext/foo/*,ext/bar/*]
+
+        # Single-directory coverage:
+        ```
+        include, omit = _get_inclusion_settings("helpers")
+        ```
+        #   coverage report --include=*/helpers/* --sort=Cover
+        #   coverage html   --include=*/helpers/* [--omit=...]
+    """
+    if target_dir == ".":
+        include_in_report = "*"
+        exclude_from_report = ""
+        if hserver.skip_submodules_test():
+            submodule_paths: List[str] = hgit.get_submodule_paths()
+            exclude_from_report = ",".join(
+                f"{path}/*" for path in submodule_paths
+            )
+    else:
+        include_in_report = f"*/{target_dir}/*"
+        exclude_from_report = None
+    return include_in_report, exclude_from_report
+
+
+def _run_coverage(
+    ctx,
+    suite: str,
+    target_dir: str,
+    generate_html_report: bool = False,
+) -> str:
+    """
+    Run coverage for a given suite (fast/slow/superslow).
+
+    :param suite: one of "fast" or "slow"
+    :param target_dir: coverage target directory
+    :param generate_html_report: whether to produce HTML output
+    :return: xml file named 'coverage.xml'
+    """
+
+    hdbg.dassert_in(
+        suite,
+        ("fast", "slow", "superslow"),
+        "Expected one of 'fast', 'slow', or 'superslow'.",
+    )
+    # Build the command line.
+    test_cmd_parts = [
+        # Invoke the "<suite>_tests" task.
+        "invoke",
+        f"run_{suite}_tests",
+        # Enable coverage collection.
+        "--coverage",
+        # Specify which directory to test.
+        "-p",
+        target_dir,
+        # Pass through remaining args to pytest.
+        "--",
+        # Emit JUnit XML report for CI systems.
+        f"--junitxml=reports/junit_{suite}.xml",
+        # Force legacy JUnit output format.
+        "-o",
+        "junit_family=legacy",
+    ]
+    # join and quote them into a single shell command
+    test_cmd = " ".join(shlex.quote(p) for p in test_cmd_parts)
path: helpers/lib_tasks_pytest.py
user: login: dremdem
body: It might better to use `helpers.lib_tasks_utils.to_multi_line_cmd()` here.
Your call.
created_at: 2025-04-25T15:32:01Z
author_association: CONTRIBUTOR
diff_hunk: @@ -874,6 +875,177 @@ def run_coverage_report(  # type: ignore
         _publish_html_coverage_report_on_s3(aws_profile)
 
 
+def _get_inclusion_settings(target_dir: str) -> Tuple[str, Optional[str]]:
+    """
+    Determine include/omit glob patterns for the coverage report for both text
+    and HTML coverage reports.
+
+    :param target_dir: directory for coverage stats; use "." to indicate all directories
+    :return: tuple (include_in_report, exclude_from_report) where
+        - include_in_report is the glob pattern to include
+        - exclude_from_report is a comma‑separated glob pattern to omit, or ``None``
+
+    Examples:
+        1. Cover everything (no submodules to omit):
+        ```
+        _get_inclusion_settings(".")
+        ```
+        ("*", "")
+
+        2. Only cover code under a specific directory:
+        ```
+        _get_inclusion_settings("helpers")
+        ```
+        ("*/helpers/*", None)
+
+    Usage in `_run_coverage`:
+
+         # Entire repo coverage (e.g. your 'helpers' project root):
+         ```
+        include, omit = _get_inclusion_settings(".")
+        ```
+        #   coverage report --include=* --sort=Cover
+        #   coverage html   --include=* [--omit=ext/foo/*,ext/bar/*]
+
+        # Single-directory coverage:
+        ```
+        include, omit = _get_inclusion_settings("helpers")
+        ```
+        #   coverage report --include=*/helpers/* --sort=Cover
+        #   coverage html   --include=*/helpers/* [--omit=...]
+    """
+    if target_dir == ".":
+        include_in_report = "*"
+        exclude_from_report = ""
+        if hserver.skip_submodules_test():
+            submodule_paths: List[str] = hgit.get_submodule_paths()
+            exclude_from_report = ",".join(
+                f"{path}/*" for path in submodule_paths
+            )
+    else:
+        include_in_report = f"*/{target_dir}/*"
+        exclude_from_report = None
+    return include_in_report, exclude_from_report
+
+
+def _run_coverage(
+    ctx,
+    suite: str,
+    target_dir: str,
+    generate_html_report: bool = False,
+) -> str:
+    """
+    Run coverage for a given suite (fast/slow/superslow).
+
+    :param suite: one of "fast" or "slow"
+    :param target_dir: coverage target directory
+    :param generate_html_report: whether to produce HTML output
+    :return: xml file named 'coverage.xml'
+    """
+
+    hdbg.dassert_in(
+        suite,
+        ("fast", "slow", "superslow"),
+        "Expected one of 'fast', 'slow', or 'superslow'.",
+    )
+    # Build the command line.
+    test_cmd_parts = [
+        # Invoke the "<suite>_tests" task.
+        "invoke",
+        f"run_{suite}_tests",
+        # Enable coverage collection.
+        "--coverage",
+        # Specify which directory to test.
+        "-p",
+        target_dir,
+        # Pass through remaining args to pytest.
+        "--",
+        # Emit JUnit XML report for CI systems.
+        f"--junitxml=reports/junit_{suite}.xml",
+        # Force legacy JUnit output format.
+        "-o",
+        "junit_family=legacy",
+    ]
+    # join and quote them into a single shell command
+    test_cmd = " ".join(shlex.quote(p) for p in test_cmd_parts)
+    # Name output coverage data file for this suite.
+    coverage_file = f".coverage_{suite}_tests"
path: helpers/lib_tasks_pytest.py
user: login: dremdem
body: Let's move it close where it's using.
created_at: 2025-04-25T15:46:45Z
author_association: CONTRIBUTOR
diff_hunk: @@ -874,6 +875,177 @@ def run_coverage_report(  # type: ignore
         _publish_html_coverage_report_on_s3(aws_profile)
 
 
+def _get_inclusion_settings(target_dir: str) -> Tuple[str, Optional[str]]:
+    """
+    Determine include/omit glob patterns for the coverage report for both text
+    and HTML coverage reports.
+
+    :param target_dir: directory for coverage stats; use "." to indicate all directories
+    :return: tuple (include_in_report, exclude_from_report) where
+        - include_in_report is the glob pattern to include
+        - exclude_from_report is a comma‑separated glob pattern to omit, or ``None``
+
+    Examples:
+        1. Cover everything (no submodules to omit):
+        ```
+        _get_inclusion_settings(".")
+        ```
+        ("*", "")
+
+        2. Only cover code under a specific directory:
+        ```
+        _get_inclusion_settings("helpers")
+        ```
+        ("*/helpers/*", None)
+
+    Usage in `_run_coverage`:
+
+         # Entire repo coverage (e.g. your 'helpers' project root):
+         ```
+        include, omit = _get_inclusion_settings(".")
+        ```
+        #   coverage report --include=* --sort=Cover
+        #   coverage html   --include=* [--omit=ext/foo/*,ext/bar/*]
+
+        # Single-directory coverage:
+        ```
+        include, omit = _get_inclusion_settings("helpers")
+        ```
+        #   coverage report --include=*/helpers/* --sort=Cover
+        #   coverage html   --include=*/helpers/* [--omit=...]
+    """
+    if target_dir == ".":
+        include_in_report = "*"
+        exclude_from_report = ""
+        if hserver.skip_submodules_test():
+            submodule_paths: List[str] = hgit.get_submodule_paths()
+            exclude_from_report = ",".join(
+                f"{path}/*" for path in submodule_paths
+            )
+    else:
+        include_in_report = f"*/{target_dir}/*"
+        exclude_from_report = None
+    return include_in_report, exclude_from_report
+
+
+def _run_coverage(
+    ctx,
+    suite: str,
+    target_dir: str,
+    generate_html_report: bool = False,
+) -> str:
+    """
+    Run coverage for a given suite (fast/slow/superslow).
+
+    :param suite: one of "fast" or "slow"
+    :param target_dir: coverage target directory
+    :param generate_html_report: whether to produce HTML output
+    :return: xml file named 'coverage.xml'
+    """
+
+    hdbg.dassert_in(
+        suite,
+        ("fast", "slow", "superslow"),
+        "Expected one of 'fast', 'slow', or 'superslow'.",
+    )
+    # Build the command line.
+    test_cmd_parts = [
+        # Invoke the "<suite>_tests" task.
+        "invoke",
+        f"run_{suite}_tests",
+        # Enable coverage collection.
+        "--coverage",
+        # Specify which directory to test.
+        "-p",
+        target_dir,
+        # Pass through remaining args to pytest.
+        "--",
+        # Emit JUnit XML report for CI systems.
+        f"--junitxml=reports/junit_{suite}.xml",
+        # Force legacy JUnit output format.
+        "-o",
+        "junit_family=legacy",
+    ]
+    # join and quote them into a single shell command
+    test_cmd = " ".join(shlex.quote(p) for p in test_cmd_parts)
+    # Name output coverage data file for this suite.
+    coverage_file = f".coverage_{suite}_tests"
+    # Run the tests under coverage.
+    hlitauti.run(ctx, test_cmd, use_system=False)
+    # Move the default ".coverage" file to our suite-specific filename.
+    hsystem.system(f"mv .coverage {coverage_file}")
+    hdbg.dassert_file_exists(coverage_file)
path: helpers/lib_tasks_pytest.py
user: login: dremdem
body: I'd check the existence of .coverage before renaming.
created_at: 2025-04-25T15:47:50Z
author_association: CONTRIBUTOR
diff_hunk: @@ -874,6 +875,177 @@ def run_coverage_report(  # type: ignore
         _publish_html_coverage_report_on_s3(aws_profile)
 
 
+def _get_inclusion_settings(target_dir: str) -> Tuple[str, Optional[str]]:
+    """
+    Determine include/omit glob patterns for the coverage report for both text
+    and HTML coverage reports.
+
+    :param target_dir: directory for coverage stats; use "." to indicate all directories
+    :return: tuple (include_in_report, exclude_from_report) where
+        - include_in_report is the glob pattern to include
+        - exclude_from_report is a comma‑separated glob pattern to omit, or ``None``
+
+    Examples:
+        1. Cover everything (no submodules to omit):
+        ```
+        _get_inclusion_settings(".")
+        ```
+        ("*", "")
+
+        2. Only cover code under a specific directory:
+        ```
+        _get_inclusion_settings("helpers")
+        ```
+        ("*/helpers/*", None)
+
+    Usage in `_run_coverage`:
+
+         # Entire repo coverage (e.g. your 'helpers' project root):
+         ```
+        include, omit = _get_inclusion_settings(".")
+        ```
+        #   coverage report --include=* --sort=Cover
+        #   coverage html   --include=* [--omit=ext/foo/*,ext/bar/*]
+
+        # Single-directory coverage:
+        ```
+        include, omit = _get_inclusion_settings("helpers")
+        ```
+        #   coverage report --include=*/helpers/* --sort=Cover
+        #   coverage html   --include=*/helpers/* [--omit=...]
+    """
+    if target_dir == ".":
+        include_in_report = "*"
+        exclude_from_report = ""
+        if hserver.skip_submodules_test():
+            submodule_paths: List[str] = hgit.get_submodule_paths()
+            exclude_from_report = ",".join(
+                f"{path}/*" for path in submodule_paths
+            )
+    else:
+        include_in_report = f"*/{target_dir}/*"
+        exclude_from_report = None
+    return include_in_report, exclude_from_report
+
+
+def _run_coverage(
+    ctx,
+    suite: str,
+    target_dir: str,
+    generate_html_report: bool = False,
+) -> str:
+    """
+    Run coverage for a given suite (fast/slow/superslow).
+
+    :param suite: one of "fast" or "slow"
+    :param target_dir: coverage target directory
+    :param generate_html_report: whether to produce HTML output
+    :return: xml file named 'coverage.xml'
+    """
+
+    hdbg.dassert_in(
+        suite,
+        ("fast", "slow", "superslow"),
+        "Expected one of 'fast', 'slow', or 'superslow'.",
+    )
+    # Build the command line.
+    test_cmd_parts = [
+        # Invoke the "<suite>_tests" task.
+        "invoke",
+        f"run_{suite}_tests",
+        # Enable coverage collection.
+        "--coverage",
+        # Specify which directory to test.
+        "-p",
+        target_dir,
+        # Pass through remaining args to pytest.
+        "--",
+        # Emit JUnit XML report for CI systems.
+        f"--junitxml=reports/junit_{suite}.xml",
+        # Force legacy JUnit output format.
+        "-o",
+        "junit_family=legacy",
+    ]
+    # join and quote them into a single shell command
+    test_cmd = " ".join(shlex.quote(p) for p in test_cmd_parts)
+    # Name output coverage data file for this suite.
+    coverage_file = f".coverage_{suite}_tests"
+    # Run the tests under coverage.
+    hlitauti.run(ctx, test_cmd, use_system=False)
+    # Move the default ".coverage" file to our suite-specific filename.
+    hsystem.system(f"mv .coverage {coverage_file}")
+    hdbg.dassert_file_exists(coverage_file)
+    # Compute which files/dirs to include and omit in the report.
path: helpers/lib_tasks_pytest.py
user: login: dremdem
body: I'm lost. What report do you meant here? 
created_at: 2025-04-25T15:49:15Z
author_association: CONTRIBUTOR
diff_hunk: @@ -874,6 +875,177 @@ def run_coverage_report(  # type: ignore
         _publish_html_coverage_report_on_s3(aws_profile)
 
 
+def _get_inclusion_settings(target_dir: str) -> Tuple[str, Optional[str]]:
+    """
+    Determine include/omit glob patterns for the coverage report for both text
+    and HTML coverage reports.
+
+    :param target_dir: directory for coverage stats; use "." to indicate all directories
+    :return: tuple (include_in_report, exclude_from_report) where
+        - include_in_report is the glob pattern to include
+        - exclude_from_report is a comma‑separated glob pattern to omit, or ``None``
+
+    Examples:
+        1. Cover everything (no submodules to omit):
+        ```
+        _get_inclusion_settings(".")
+        ```
+        ("*", "")
+
+        2. Only cover code under a specific directory:
+        ```
+        _get_inclusion_settings("helpers")
+        ```
+        ("*/helpers/*", None)
+
+    Usage in `_run_coverage`:
+
+         # Entire repo coverage (e.g. your 'helpers' project root):
+         ```
+        include, omit = _get_inclusion_settings(".")
+        ```
+        #   coverage report --include=* --sort=Cover
+        #   coverage html   --include=* [--omit=ext/foo/*,ext/bar/*]
+
+        # Single-directory coverage:
+        ```
+        include, omit = _get_inclusion_settings("helpers")
+        ```
+        #   coverage report --include=*/helpers/* --sort=Cover
+        #   coverage html   --include=*/helpers/* [--omit=...]
+    """
+    if target_dir == ".":
+        include_in_report = "*"
+        exclude_from_report = ""
+        if hserver.skip_submodules_test():
+            submodule_paths: List[str] = hgit.get_submodule_paths()
+            exclude_from_report = ",".join(
+                f"{path}/*" for path in submodule_paths
+            )
+    else:
+        include_in_report = f"*/{target_dir}/*"
+        exclude_from_report = None
+    return include_in_report, exclude_from_report
+
+
+def _run_coverage(
+    ctx,
+    suite: str,
+    target_dir: str,
+    generate_html_report: bool = False,
+) -> str:
+    """
+    Run coverage for a given suite (fast/slow/superslow).
+
+    :param suite: one of "fast" or "slow"
+    :param target_dir: coverage target directory
+    :param generate_html_report: whether to produce HTML output
+    :return: xml file named 'coverage.xml'
+    """
+
+    hdbg.dassert_in(
+        suite,
+        ("fast", "slow", "superslow"),
+        "Expected one of 'fast', 'slow', or 'superslow'.",
+    )
+    # Build the command line.
+    test_cmd_parts = [
+        # Invoke the "<suite>_tests" task.
+        "invoke",
+        f"run_{suite}_tests",
+        # Enable coverage collection.
+        "--coverage",
+        # Specify which directory to test.
+        "-p",
+        target_dir,
+        # Pass through remaining args to pytest.
+        "--",
+        # Emit JUnit XML report for CI systems.
+        f"--junitxml=reports/junit_{suite}.xml",
+        # Force legacy JUnit output format.
+        "-o",
+        "junit_family=legacy",
+    ]
+    # join and quote them into a single shell command
+    test_cmd = " ".join(shlex.quote(p) for p in test_cmd_parts)
+    # Name output coverage data file for this suite.
+    coverage_file = f".coverage_{suite}_tests"
+    # Run the tests under coverage.
+    hlitauti.run(ctx, test_cmd, use_system=False)
+    # Move the default ".coverage" file to our suite-specific filename.
+    hsystem.system(f"mv .coverage {coverage_file}")
+    hdbg.dassert_file_exists(coverage_file)
+    # Compute which files/dirs to include and omit in the report.
path: helpers/lib_tasks_pytest.py
user: login: dremdem
body: Probably it worth to split the pipeline to blocks with comments. 

created_at: 2025-04-25T15:50:17Z
author_association: CONTRIBUTOR
diff_hunk: @@ -874,6 +875,177 @@ def run_coverage_report(  # type: ignore
         _publish_html_coverage_report_on_s3(aws_profile)
 
 
+def _get_inclusion_settings(target_dir: str) -> Tuple[str, Optional[str]]:
+    """
+    Determine include/omit glob patterns for the coverage report for both text
+    and HTML coverage reports.
+
+    :param target_dir: directory for coverage stats; use "." to indicate all directories
+    :return: tuple (include_in_report, exclude_from_report) where
+        - include_in_report is the glob pattern to include
+        - exclude_from_report is a comma‑separated glob pattern to omit, or ``None``
+
+    Examples:
+        1. Cover everything (no submodules to omit):
+        ```
+        _get_inclusion_settings(".")
+        ```
+        ("*", "")
+
+        2. Only cover code under a specific directory:
+        ```
+        _get_inclusion_settings("helpers")
+        ```
+        ("*/helpers/*", None)
+
+    Usage in `_run_coverage`:
+
+         # Entire repo coverage (e.g. your 'helpers' project root):
+         ```
+        include, omit = _get_inclusion_settings(".")
+        ```
+        #   coverage report --include=* --sort=Cover
+        #   coverage html   --include=* [--omit=ext/foo/*,ext/bar/*]
+
+        # Single-directory coverage:
+        ```
+        include, omit = _get_inclusion_settings("helpers")
+        ```
+        #   coverage report --include=*/helpers/* --sort=Cover
+        #   coverage html   --include=*/helpers/* [--omit=...]
+    """
+    if target_dir == ".":
+        include_in_report = "*"
+        exclude_from_report = ""
+        if hserver.skip_submodules_test():
+            submodule_paths: List[str] = hgit.get_submodule_paths()
+            exclude_from_report = ",".join(
+                f"{path}/*" for path in submodule_paths
+            )
+    else:
+        include_in_report = f"*/{target_dir}/*"
+        exclude_from_report = None
+    return include_in_report, exclude_from_report
+
+
+def _run_coverage(
+    ctx,
+    suite: str,
+    target_dir: str,
+    generate_html_report: bool = False,
+) -> str:
+    """
+    Run coverage for a given suite (fast/slow/superslow).
+
+    :param suite: one of "fast" or "slow"
+    :param target_dir: coverage target directory
+    :param generate_html_report: whether to produce HTML output
+    :return: xml file named 'coverage.xml'
+    """
+
+    hdbg.dassert_in(
+        suite,
+        ("fast", "slow", "superslow"),
+        "Expected one of 'fast', 'slow', or 'superslow'.",
+    )
+    # Build the command line.
+    test_cmd_parts = [
+        # Invoke the "<suite>_tests" task.
+        "invoke",
+        f"run_{suite}_tests",
+        # Enable coverage collection.
+        "--coverage",
+        # Specify which directory to test.
+        "-p",
+        target_dir,
+        # Pass through remaining args to pytest.
+        "--",
+        # Emit JUnit XML report for CI systems.
+        f"--junitxml=reports/junit_{suite}.xml",
+        # Force legacy JUnit output format.
+        "-o",
+        "junit_family=legacy",
+    ]
+    # join and quote them into a single shell command
+    test_cmd = " ".join(shlex.quote(p) for p in test_cmd_parts)
+    # Name output coverage data file for this suite.
+    coverage_file = f".coverage_{suite}_tests"
+    # Run the tests under coverage.
+    hlitauti.run(ctx, test_cmd, use_system=False)
+    # Move the default ".coverage" file to our suite-specific filename.
+    hsystem.system(f"mv .coverage {coverage_file}")
+    hdbg.dassert_file_exists(coverage_file)
+    # Compute which files/dirs to include and omit in the report.
+    include_in_report, exclude_from_report = _get_inclusion_settings(target_dir)
+    report_cmd: List[str] = [
+        # Clear out any prior coverage data.
+        "coverage erase",
+        # Combine this suite's data into the coverage database.
path: helpers/lib_tasks_pytest.py
user: login: dremdem
body: Again, I'm puzzled. Why we need it? 
created_at: 2025-04-25T15:51:11Z
author_association: CONTRIBUTOR
diff_hunk: @@ -874,6 +875,177 @@ def run_coverage_report(  # type: ignore
         _publish_html_coverage_report_on_s3(aws_profile)
 
 
+def _get_inclusion_settings(target_dir: str) -> Tuple[str, Optional[str]]:
+    """
+    Determine include/omit glob patterns for the coverage report for both text
+    and HTML coverage reports.
+
+    :param target_dir: directory for coverage stats; use "." to indicate all directories
+    :return: tuple (include_in_report, exclude_from_report) where
+        - include_in_report is the glob pattern to include
+        - exclude_from_report is a comma‑separated glob pattern to omit, or ``None``
+
+    Examples:
+        1. Cover everything (no submodules to omit):
+        ```
+        _get_inclusion_settings(".")
+        ```
+        ("*", "")
+
+        2. Only cover code under a specific directory:
+        ```
+        _get_inclusion_settings("helpers")
+        ```
+        ("*/helpers/*", None)
+
+    Usage in `_run_coverage`:
+
+         # Entire repo coverage (e.g. your 'helpers' project root):
+         ```
+        include, omit = _get_inclusion_settings(".")
+        ```
+        #   coverage report --include=* --sort=Cover
+        #   coverage html   --include=* [--omit=ext/foo/*,ext/bar/*]
+
+        # Single-directory coverage:
+        ```
+        include, omit = _get_inclusion_settings("helpers")
+        ```
+        #   coverage report --include=*/helpers/* --sort=Cover
+        #   coverage html   --include=*/helpers/* [--omit=...]
+    """
+    if target_dir == ".":
+        include_in_report = "*"
+        exclude_from_report = ""
+        if hserver.skip_submodules_test():
+            submodule_paths: List[str] = hgit.get_submodule_paths()
+            exclude_from_report = ",".join(
+                f"{path}/*" for path in submodule_paths
+            )
+    else:
+        include_in_report = f"*/{target_dir}/*"
+        exclude_from_report = None
+    return include_in_report, exclude_from_report
+
+
+def _run_coverage(
+    ctx,
+    suite: str,
+    target_dir: str,
+    generate_html_report: bool = False,
+) -> str:
+    """
+    Run coverage for a given suite (fast/slow/superslow).
+
+    :param suite: one of "fast" or "slow"
+    :param target_dir: coverage target directory
+    :param generate_html_report: whether to produce HTML output
+    :return: xml file named 'coverage.xml'
+    """
+
+    hdbg.dassert_in(
+        suite,
+        ("fast", "slow", "superslow"),
+        "Expected one of 'fast', 'slow', or 'superslow'.",
+    )
+    # Build the command line.
+    test_cmd_parts = [
+        # Invoke the "<suite>_tests" task.
+        "invoke",
+        f"run_{suite}_tests",
+        # Enable coverage collection.
+        "--coverage",
+        # Specify which directory to test.
+        "-p",
+        target_dir,
+        # Pass through remaining args to pytest.
+        "--",
+        # Emit JUnit XML report for CI systems.
+        f"--junitxml=reports/junit_{suite}.xml",
+        # Force legacy JUnit output format.
+        "-o",
+        "junit_family=legacy",
+    ]
+    # join and quote them into a single shell command
+    test_cmd = " ".join(shlex.quote(p) for p in test_cmd_parts)
+    # Name output coverage data file for this suite.
+    coverage_file = f".coverage_{suite}_tests"
+    # Run the tests under coverage.
+    hlitauti.run(ctx, test_cmd, use_system=False)
+    # Move the default ".coverage" file to our suite-specific filename.
+    hsystem.system(f"mv .coverage {coverage_file}")
+    hdbg.dassert_file_exists(coverage_file)
+    # Compute which files/dirs to include and omit in the report.
+    include_in_report, exclude_from_report = _get_inclusion_settings(target_dir)
+    report_cmd: List[str] = [
+        # Clear out any prior coverage data.
+        "coverage erase",
+        # Combine this suite's data into the coverage database.
+        f"coverage combine --keep {coverage_file}",
+    ]
+    # Generate a text report, including only our target paths.
+    report_stats_cmd: str = (
+        f"coverage report --include={include_in_report} --sort=Cover"
+    )
+    if exclude_from_report:
+        report_stats_cmd += f" --omit={exclude_from_report}"
+    report_cmd.append(report_stats_cmd)
+    # Produce HTML output for interactive browsing.
+    if generate_html_report:
+        report_html_cmd: str = f"coverage html --include={include_in_report}"
+        if exclude_from_report:
+            report_html_cmd += f" --omit={exclude_from_report}"
+        report_cmd.append(report_html_cmd)
+    # Export XML coverage report.
+    report_cmd.append("coverage xml -o coverage.xml")
+    full_report_cmd: str = " && ".join(report_cmd)
+    docker_cmd_ = f"invoke docker_cmd --use-bash --cmd '{full_report_cmd}'"
path: helpers/lib_tasks_pytest.py
user: login: dremdem
body: Dumb question: why we need to run it inside the Docker container? 
created_at: 2025-04-25T15:53:02Z
author_association: CONTRIBUTOR
diff_hunk: @@ -874,6 +875,177 @@ def run_coverage_report(  # type: ignore
         _publish_html_coverage_report_on_s3(aws_profile)
 
 
+def _get_inclusion_settings(target_dir: str) -> Tuple[str, Optional[str]]:
+    """
+    Determine include/omit glob patterns for the coverage report for both text
+    and HTML coverage reports.
+
+    :param target_dir: directory for coverage stats; use "." to indicate all directories
+    :return: tuple (include_in_report, exclude_from_report) where
+        - include_in_report is the glob pattern to include
+        - exclude_from_report is a comma‑separated glob pattern to omit, or ``None``
+
+    Examples:
+        1. Cover everything (no submodules to omit):
+        ```
+        _get_inclusion_settings(".")
+        ```
+        ("*", "")
+
+        2. Only cover code under a specific directory:
+        ```
+        _get_inclusion_settings("helpers")
+        ```
+        ("*/helpers/*", None)
+
+    Usage in `_run_coverage`:
+
+         # Entire repo coverage (e.g. your 'helpers' project root):
+         ```
+        include, omit = _get_inclusion_settings(".")
+        ```
+        #   coverage report --include=* --sort=Cover
+        #   coverage html   --include=* [--omit=ext/foo/*,ext/bar/*]
+
+        # Single-directory coverage:
+        ```
+        include, omit = _get_inclusion_settings("helpers")
+        ```
+        #   coverage report --include=*/helpers/* --sort=Cover
+        #   coverage html   --include=*/helpers/* [--omit=...]
+    """
+    if target_dir == ".":
+        include_in_report = "*"
+        exclude_from_report = ""
+        if hserver.skip_submodules_test():
+            submodule_paths: List[str] = hgit.get_submodule_paths()
+            exclude_from_report = ",".join(
+                f"{path}/*" for path in submodule_paths
+            )
+    else:
+        include_in_report = f"*/{target_dir}/*"
+        exclude_from_report = None
+    return include_in_report, exclude_from_report
+
+
+def _run_coverage(
path: helpers/lib_tasks_pytest.py
user: login: dremdem
body: Can we promote it to an invoke target and get rid of: 
- run_fast_coverage
- run_slow_coverage
- run_superslow_coverage
WDYT?
created_at: 2025-04-25T15:58:11Z
author_association: CONTRIBUTOR
diff_hunk: @@ -874,6 +875,177 @@ def run_coverage_report(  # type: ignore
         _publish_html_coverage_report_on_s3(aws_profile)
 
 
+def _get_inclusion_settings(target_dir: str) -> Tuple[str, Optional[str]]:
+    """
+    Determine include/omit glob patterns for the coverage report for both text
+    and HTML coverage reports.
+
+    :param target_dir: directory for coverage stats; use "." to indicate all directories
+    :return: tuple (include_in_report, exclude_from_report) where
+        - include_in_report is the glob pattern to include
+        - exclude_from_report is a comma‑separated glob pattern to omit, or ``None``
+
+    Examples:
+        1. Cover everything (no submodules to omit):
+        ```
+        _get_inclusion_settings(".")
+        ```
+        ("*", "")
+
+        2. Only cover code under a specific directory:
+        ```
+        _get_inclusion_settings("helpers")
+        ```
+        ("*/helpers/*", None)
+
+    Usage in `_run_coverage`:
+
+         # Entire repo coverage (e.g. your 'helpers' project root):
+         ```
+        include, omit = _get_inclusion_settings(".")
+        ```
+        #   coverage report --include=* --sort=Cover
+        #   coverage html   --include=* [--omit=ext/foo/*,ext/bar/*]
+
+        # Single-directory coverage:
+        ```
+        include, omit = _get_inclusion_settings("helpers")
+        ```
+        #   coverage report --include=*/helpers/* --sort=Cover
+        #   coverage html   --include=*/helpers/* [--omit=...]
+    """
+    if target_dir == ".":
+        include_in_report = "*"
+        exclude_from_report = ""
+        if hserver.skip_submodules_test():
+            submodule_paths: List[str] = hgit.get_submodule_paths()
+            exclude_from_report = ",".join(
+                f"{path}/*" for path in submodule_paths
+            )
+    else:
+        include_in_report = f"*/{target_dir}/*"
+        exclude_from_report = None
+    return include_in_report, exclude_from_report
+
+
+def _run_coverage(
+    ctx,
+    suite: str,
+    target_dir: str,
+    generate_html_report: bool = False,
+) -> str:
+    """
+    Run coverage for a given suite (fast/slow/superslow).
+
+    :param suite: one of "fast" or "slow"
+    :param target_dir: coverage target directory
+    :param generate_html_report: whether to produce HTML output
+    :return: xml file named 'coverage.xml'
+    """
+
+    hdbg.dassert_in(
+        suite,
+        ("fast", "slow", "superslow"),
+        "Expected one of 'fast', 'slow', or 'superslow'.",
+    )
+    # Build the command line.
+    test_cmd_parts = [
+        # Invoke the "<suite>_tests" task.
+        "invoke",
+        f"run_{suite}_tests",
+        # Enable coverage collection.
+        "--coverage",
+        # Specify which directory to test.
+        "-p",
+        target_dir,
path: helpers/lib_tasks_pytest.py
user: login: Shaunak01
body: linter turns it back to 2 lines but agreed
created_at: 2025-04-28T01:51:32Z
author_association: CONTRIBUTOR
diff_hunk: @@ -874,6 +875,177 @@ def run_coverage_report(  # type: ignore
         _publish_html_coverage_report_on_s3(aws_profile)
 
 
+def _get_inclusion_settings(target_dir: str) -> Tuple[str, Optional[str]]:
+    """
+    Determine include/omit glob patterns for the coverage report for both text
+    and HTML coverage reports.
+
+    :param target_dir: directory for coverage stats; use "." to indicate all directories
+    :return: tuple (include_in_report, exclude_from_report) where
+        - include_in_report is the glob pattern to include
+        - exclude_from_report is a comma‑separated glob pattern to omit, or ``None``
+
+    Examples:
+        1. Cover everything (no submodules to omit):
+        ```
+        _get_inclusion_settings(".")
+        ```
+        ("*", "")
+
+        2. Only cover code under a specific directory:
+        ```
+        _get_inclusion_settings("helpers")
+        ```
+        ("*/helpers/*", None)
+
+    Usage in `_run_coverage`:
+
+         # Entire repo coverage (e.g. your 'helpers' project root):
+         ```
+        include, omit = _get_inclusion_settings(".")
+        ```
+        #   coverage report --include=* --sort=Cover
+        #   coverage html   --include=* [--omit=ext/foo/*,ext/bar/*]
+
+        # Single-directory coverage:
+        ```
+        include, omit = _get_inclusion_settings("helpers")
+        ```
+        #   coverage report --include=*/helpers/* --sort=Cover
+        #   coverage html   --include=*/helpers/* [--omit=...]
+    """
+    if target_dir == ".":
+        include_in_report = "*"
+        exclude_from_report = ""
+        if hserver.skip_submodules_test():
+            submodule_paths: List[str] = hgit.get_submodule_paths()
+            exclude_from_report = ",".join(
+                f"{path}/*" for path in submodule_paths
+            )
+    else:
+        include_in_report = f"*/{target_dir}/*"
+        exclude_from_report = None
+    return include_in_report, exclude_from_report
+
+
+def _run_coverage(
+    ctx,
+    suite: str,
+    target_dir: str,
+    generate_html_report: bool = False,
+) -> str:
+    """
+    Run coverage for a given suite (fast/slow/superslow).
+
+    :param suite: one of "fast" or "slow"
+    :param target_dir: coverage target directory
+    :param generate_html_report: whether to produce HTML output
+    :return: xml file named 'coverage.xml'
+    """
+
+    hdbg.dassert_in(
+        suite,
+        ("fast", "slow", "superslow"),
+        "Expected one of 'fast', 'slow', or 'superslow'.",
+    )
+    # Build the command line.
+    test_cmd_parts = [
+        # Invoke the "<suite>_tests" task.
+        "invoke",
+        f"run_{suite}_tests",
+        # Enable coverage collection.
+        "--coverage",
+        # Specify which directory to test.
+        "-p",
+        target_dir,
+        # Pass through remaining args to pytest.
+        "--",
+        # Emit JUnit XML report for CI systems.
path: helpers/lib_tasks_pytest.py
user: login: Shaunak01
body: an additional func of codecov i was testing. Not needed rn
created_at: 2025-04-28T02:01:55Z
author_association: CONTRIBUTOR
diff_hunk: @@ -874,6 +875,177 @@ def run_coverage_report(  # type: ignore
         _publish_html_coverage_report_on_s3(aws_profile)
 
 
+def _get_inclusion_settings(target_dir: str) -> Tuple[str, Optional[str]]:
+    """
+    Determine include/omit glob patterns for the coverage report for both text
+    and HTML coverage reports.
+
+    :param target_dir: directory for coverage stats; use "." to indicate all directories
+    :return: tuple (include_in_report, exclude_from_report) where
+        - include_in_report is the glob pattern to include
+        - exclude_from_report is a comma‑separated glob pattern to omit, or ``None``
+
+    Examples:
+        1. Cover everything (no submodules to omit):
+        ```
+        _get_inclusion_settings(".")
+        ```
+        ("*", "")
+
+        2. Only cover code under a specific directory:
+        ```
+        _get_inclusion_settings("helpers")
+        ```
+        ("*/helpers/*", None)
+
+    Usage in `_run_coverage`:
+
+         # Entire repo coverage (e.g. your 'helpers' project root):
+         ```
+        include, omit = _get_inclusion_settings(".")
+        ```
+        #   coverage report --include=* --sort=Cover
+        #   coverage html   --include=* [--omit=ext/foo/*,ext/bar/*]
+
+        # Single-directory coverage:
+        ```
+        include, omit = _get_inclusion_settings("helpers")
+        ```
+        #   coverage report --include=*/helpers/* --sort=Cover
+        #   coverage html   --include=*/helpers/* [--omit=...]
+    """
+    if target_dir == ".":
+        include_in_report = "*"
+        exclude_from_report = ""
+        if hserver.skip_submodules_test():
+            submodule_paths: List[str] = hgit.get_submodule_paths()
+            exclude_from_report = ",".join(
+                f"{path}/*" for path in submodule_paths
+            )
+    else:
+        include_in_report = f"*/{target_dir}/*"
+        exclude_from_report = None
+    return include_in_report, exclude_from_report
+
+
+def _run_coverage(
+    ctx,
+    suite: str,
+    target_dir: str,
+    generate_html_report: bool = False,
+) -> str:
+    """
+    Run coverage for a given suite (fast/slow/superslow).
+
+    :param suite: one of "fast" or "slow"
+    :param target_dir: coverage target directory
+    :param generate_html_report: whether to produce HTML output
+    :return: xml file named 'coverage.xml'
+    """
+
+    hdbg.dassert_in(
+        suite,
+        ("fast", "slow", "superslow"),
+        "Expected one of 'fast', 'slow', or 'superslow'.",
+    )
+    # Build the command line.
+    test_cmd_parts = [
+        # Invoke the "<suite>_tests" task.
+        "invoke",
+        f"run_{suite}_tests",
+        # Enable coverage collection.
+        "--coverage",
+        # Specify which directory to test.
+        "-p",
+        target_dir,
+        # Pass through remaining args to pytest.
+        "--",
+        # Emit JUnit XML report for CI systems.
+        f"--junitxml=reports/junit_{suite}.xml",
+        # Force legacy JUnit output format.
+        "-o",
+        "junit_family=legacy",
+    ]
+    # join and quote them into a single shell command
+    test_cmd = " ".join(shlex.quote(p) for p in test_cmd_parts)
+    # Name output coverage data file for this suite.
+    coverage_file = f".coverage_{suite}_tests"
+    # Run the tests under coverage.
+    hlitauti.run(ctx, test_cmd, use_system=False)
+    # Move the default ".coverage" file to our suite-specific filename.
+    hsystem.system(f"mv .coverage {coverage_file}")
+    hdbg.dassert_file_exists(coverage_file)
+    # Compute which files/dirs to include and omit in the report.
path: helpers/lib_tasks_pytest.py
user: login: Shaunak01
body: > I'm lost. What report do you meant here?

we are preparing for different reports:

    - Text report
    - HTML report 
    - XML report
    
- `include_in_report` is a glob to say which files you want to measure (e.g., only our repo, not dependencies).
- `exclude_from_report` is used to skip submodules or other unwanted folders.

We are preparing which parts of the codebase coverage will show up for

- First → we prepare include_in_report, exclude_from_report.
- Then → we use them in the coverage report (text report).
- Then → we use them again in coverage html (HTML report, only if generate_html_report=True).
- Finally → we generate an XML file with coverage xml -o coverage.xml.





created_at: 2025-04-28T02:30:53Z
author_association: CONTRIBUTOR
diff_hunk: @@ -874,6 +875,177 @@ def run_coverage_report(  # type: ignore
         _publish_html_coverage_report_on_s3(aws_profile)
 
 
+def _get_inclusion_settings(target_dir: str) -> Tuple[str, Optional[str]]:
+    """
+    Determine include/omit glob patterns for the coverage report for both text
+    and HTML coverage reports.
+
+    :param target_dir: directory for coverage stats; use "." to indicate all directories
+    :return: tuple (include_in_report, exclude_from_report) where
+        - include_in_report is the glob pattern to include
+        - exclude_from_report is a comma‑separated glob pattern to omit, or ``None``
+
+    Examples:
+        1. Cover everything (no submodules to omit):
+        ```
+        _get_inclusion_settings(".")
+        ```
+        ("*", "")
+
+        2. Only cover code under a specific directory:
+        ```
+        _get_inclusion_settings("helpers")
+        ```
+        ("*/helpers/*", None)
+
+    Usage in `_run_coverage`:
+
+         # Entire repo coverage (e.g. your 'helpers' project root):
+         ```
+        include, omit = _get_inclusion_settings(".")
+        ```
+        #   coverage report --include=* --sort=Cover
+        #   coverage html   --include=* [--omit=ext/foo/*,ext/bar/*]
+
+        # Single-directory coverage:
+        ```
+        include, omit = _get_inclusion_settings("helpers")
+        ```
+        #   coverage report --include=*/helpers/* --sort=Cover
+        #   coverage html   --include=*/helpers/* [--omit=...]
+    """
+    if target_dir == ".":
+        include_in_report = "*"
+        exclude_from_report = ""
+        if hserver.skip_submodules_test():
+            submodule_paths: List[str] = hgit.get_submodule_paths()
+            exclude_from_report = ",".join(
+                f"{path}/*" for path in submodule_paths
+            )
+    else:
+        include_in_report = f"*/{target_dir}/*"
+        exclude_from_report = None
+    return include_in_report, exclude_from_report
+
+
+def _run_coverage(
+    ctx,
+    suite: str,
+    target_dir: str,
+    generate_html_report: bool = False,
+) -> str:
+    """
+    Run coverage for a given suite (fast/slow/superslow).
+
+    :param suite: one of "fast" or "slow"
+    :param target_dir: coverage target directory
+    :param generate_html_report: whether to produce HTML output
+    :return: xml file named 'coverage.xml'
+    """
+
+    hdbg.dassert_in(
+        suite,
+        ("fast", "slow", "superslow"),
+        "Expected one of 'fast', 'slow', or 'superslow'.",
+    )
+    # Build the command line.
+    test_cmd_parts = [
+        # Invoke the "<suite>_tests" task.
+        "invoke",
+        f"run_{suite}_tests",
+        # Enable coverage collection.
+        "--coverage",
+        # Specify which directory to test.
+        "-p",
+        target_dir,
+        # Pass through remaining args to pytest.
+        "--",
+        # Emit JUnit XML report for CI systems.
+        f"--junitxml=reports/junit_{suite}.xml",
+        # Force legacy JUnit output format.
+        "-o",
+        "junit_family=legacy",
+    ]
+    # join and quote them into a single shell command
+    test_cmd = " ".join(shlex.quote(p) for p in test_cmd_parts)
+    # Name output coverage data file for this suite.
+    coverage_file = f".coverage_{suite}_tests"
+    # Run the tests under coverage.
+    hlitauti.run(ctx, test_cmd, use_system=False)
+    # Move the default ".coverage" file to our suite-specific filename.
+    hsystem.system(f"mv .coverage {coverage_file}")
+    hdbg.dassert_file_exists(coverage_file)
+    # Compute which files/dirs to include and omit in the report.
path: helpers/lib_tasks_pytest.py
user: login: Shaunak01
body: > Probably it worth to split the pipeline to blocks with comments.

yes sir!
created_at: 2025-04-28T02:35:02Z
author_association: CONTRIBUTOR
diff_hunk: @@ -874,6 +875,177 @@ def run_coverage_report(  # type: ignore
         _publish_html_coverage_report_on_s3(aws_profile)
 
 
+def _get_inclusion_settings(target_dir: str) -> Tuple[str, Optional[str]]:
+    """
+    Determine include/omit glob patterns for the coverage report for both text
+    and HTML coverage reports.
+
+    :param target_dir: directory for coverage stats; use "." to indicate all directories
+    :return: tuple (include_in_report, exclude_from_report) where
+        - include_in_report is the glob pattern to include
+        - exclude_from_report is a comma‑separated glob pattern to omit, or ``None``
+
+    Examples:
+        1. Cover everything (no submodules to omit):
+        ```
+        _get_inclusion_settings(".")
+        ```
+        ("*", "")
+
+        2. Only cover code under a specific directory:
+        ```
+        _get_inclusion_settings("helpers")
+        ```
+        ("*/helpers/*", None)
+
+    Usage in `_run_coverage`:
+
+         # Entire repo coverage (e.g. your 'helpers' project root):
+         ```
+        include, omit = _get_inclusion_settings(".")
+        ```
+        #   coverage report --include=* --sort=Cover
+        #   coverage html   --include=* [--omit=ext/foo/*,ext/bar/*]
+
+        # Single-directory coverage:
+        ```
+        include, omit = _get_inclusion_settings("helpers")
+        ```
+        #   coverage report --include=*/helpers/* --sort=Cover
+        #   coverage html   --include=*/helpers/* [--omit=...]
+    """
+    if target_dir == ".":
+        include_in_report = "*"
+        exclude_from_report = ""
+        if hserver.skip_submodules_test():
+            submodule_paths: List[str] = hgit.get_submodule_paths()
+            exclude_from_report = ",".join(
+                f"{path}/*" for path in submodule_paths
+            )
+    else:
+        include_in_report = f"*/{target_dir}/*"
+        exclude_from_report = None
+    return include_in_report, exclude_from_report
+
+
+def _run_coverage(
+    ctx,
+    suite: str,
+    target_dir: str,
+    generate_html_report: bool = False,
+) -> str:
+    """
+    Run coverage for a given suite (fast/slow/superslow).
+
+    :param suite: one of "fast" or "slow"
+    :param target_dir: coverage target directory
+    :param generate_html_report: whether to produce HTML output
+    :return: xml file named 'coverage.xml'
+    """
+
+    hdbg.dassert_in(
+        suite,
+        ("fast", "slow", "superslow"),
+        "Expected one of 'fast', 'slow', or 'superslow'.",
+    )
+    # Build the command line.
+    test_cmd_parts = [
+        # Invoke the "<suite>_tests" task.
+        "invoke",
+        f"run_{suite}_tests",
+        # Enable coverage collection.
+        "--coverage",
+        # Specify which directory to test.
+        "-p",
+        target_dir,
+        # Pass through remaining args to pytest.
+        "--",
+        # Emit JUnit XML report for CI systems.
+        f"--junitxml=reports/junit_{suite}.xml",
+        # Force legacy JUnit output format.
+        "-o",
+        "junit_family=legacy",
+    ]
+    # join and quote them into a single shell command
+    test_cmd = " ".join(shlex.quote(p) for p in test_cmd_parts)
+    # Name output coverage data file for this suite.
+    coverage_file = f".coverage_{suite}_tests"
+    # Run the tests under coverage.
+    hlitauti.run(ctx, test_cmd, use_system=False)
+    # Move the default ".coverage" file to our suite-specific filename.
+    hsystem.system(f"mv .coverage {coverage_file}")
+    hdbg.dassert_file_exists(coverage_file)
+    # Compute which files/dirs to include and omit in the report.
+    include_in_report, exclude_from_report = _get_inclusion_settings(target_dir)
+    report_cmd: List[str] = [
+        # Clear out any prior coverage data.
+        "coverage erase",
+        # Combine this suite's data into the coverage database.
path: helpers/lib_tasks_pytest.py
user: login: Shaunak01
body: So originally we were running multiple tests thus workflow like this. I think its an overkill now. I will simplify the func and do a commit.
created_at: 2025-04-28T02:40:30Z
author_association: CONTRIBUTOR
diff_hunk: @@ -874,6 +875,177 @@ def run_coverage_report(  # type: ignore
         _publish_html_coverage_report_on_s3(aws_profile)
 
 
+def _get_inclusion_settings(target_dir: str) -> Tuple[str, Optional[str]]:
+    """
+    Determine include/omit glob patterns for the coverage report for both text
+    and HTML coverage reports.
+
+    :param target_dir: directory for coverage stats; use "." to indicate all directories
+    :return: tuple (include_in_report, exclude_from_report) where
+        - include_in_report is the glob pattern to include
+        - exclude_from_report is a comma‑separated glob pattern to omit, or ``None``
+
+    Examples:
+        1. Cover everything (no submodules to omit):
+        ```
+        _get_inclusion_settings(".")
+        ```
+        ("*", "")
+
+        2. Only cover code under a specific directory:
+        ```
+        _get_inclusion_settings("helpers")
+        ```
+        ("*/helpers/*", None)
+
+    Usage in `_run_coverage`:
+
+         # Entire repo coverage (e.g. your 'helpers' project root):
+         ```
+        include, omit = _get_inclusion_settings(".")
+        ```
+        #   coverage report --include=* --sort=Cover
+        #   coverage html   --include=* [--omit=ext/foo/*,ext/bar/*]
+
+        # Single-directory coverage:
+        ```
+        include, omit = _get_inclusion_settings("helpers")
+        ```
+        #   coverage report --include=*/helpers/* --sort=Cover
+        #   coverage html   --include=*/helpers/* [--omit=...]
+    """
+    if target_dir == ".":
+        include_in_report = "*"
+        exclude_from_report = ""
+        if hserver.skip_submodules_test():
+            submodule_paths: List[str] = hgit.get_submodule_paths()
+            exclude_from_report = ",".join(
+                f"{path}/*" for path in submodule_paths
+            )
+    else:
+        include_in_report = f"*/{target_dir}/*"
+        exclude_from_report = None
+    return include_in_report, exclude_from_report
+
+
+def _run_coverage(
+    ctx,
+    suite: str,
+    target_dir: str,
+    generate_html_report: bool = False,
+) -> str:
+    """
+    Run coverage for a given suite (fast/slow/superslow).
+
+    :param suite: one of "fast" or "slow"
+    :param target_dir: coverage target directory
+    :param generate_html_report: whether to produce HTML output
+    :return: xml file named 'coverage.xml'
+    """
+
+    hdbg.dassert_in(
+        suite,
+        ("fast", "slow", "superslow"),
+        "Expected one of 'fast', 'slow', or 'superslow'.",
+    )
+    # Build the command line.
+    test_cmd_parts = [
+        # Invoke the "<suite>_tests" task.
+        "invoke",
+        f"run_{suite}_tests",
+        # Enable coverage collection.
+        "--coverage",
+        # Specify which directory to test.
+        "-p",
+        target_dir,
+        # Pass through remaining args to pytest.
+        "--",
+        # Emit JUnit XML report for CI systems.
+        f"--junitxml=reports/junit_{suite}.xml",
+        # Force legacy JUnit output format.
+        "-o",
+        "junit_family=legacy",
+    ]
+    # join and quote them into a single shell command
+    test_cmd = " ".join(shlex.quote(p) for p in test_cmd_parts)
+    # Name output coverage data file for this suite.
+    coverage_file = f".coverage_{suite}_tests"
+    # Run the tests under coverage.
+    hlitauti.run(ctx, test_cmd, use_system=False)
+    # Move the default ".coverage" file to our suite-specific filename.
+    hsystem.system(f"mv .coverage {coverage_file}")
+    hdbg.dassert_file_exists(coverage_file)
+    # Compute which files/dirs to include and omit in the report.
+    include_in_report, exclude_from_report = _get_inclusion_settings(target_dir)
+    report_cmd: List[str] = [
+        # Clear out any prior coverage data.
+        "coverage erase",
+        # Combine this suite's data into the coverage database.
+        f"coverage combine --keep {coverage_file}",
+    ]
+    # Generate a text report, including only our target paths.
+    report_stats_cmd: str = (
+        f"coverage report --include={include_in_report} --sort=Cover"
+    )
+    if exclude_from_report:
+        report_stats_cmd += f" --omit={exclude_from_report}"
+    report_cmd.append(report_stats_cmd)
+    # Produce HTML output for interactive browsing.
+    if generate_html_report:
+        report_html_cmd: str = f"coverage html --include={include_in_report}"
+        if exclude_from_report:
+            report_html_cmd += f" --omit={exclude_from_report}"
+        report_cmd.append(report_html_cmd)
+    # Export XML coverage report.
+    report_cmd.append("coverage xml -o coverage.xml")
+    full_report_cmd: str = " && ".join(report_cmd)
+    docker_cmd_ = f"invoke docker_cmd --use-bash --cmd '{full_report_cmd}'"
path: helpers/lib_tasks_pytest.py
user: login: Shaunak01
body: well I just took the logic from how we used to run coverage before from the other funcs. But reasons could be

- If we run coverage report outside, it might say “missing source file” or “cannot find file” errors.
- Since we are running tests inside Docker, we also want to run reporting inside the exact same environment so that coverage correctly maps file paths, modules, etc.

created_at: 2025-04-28T02:53:13Z
author_association: CONTRIBUTOR
diff_hunk: @@ -874,6 +875,177 @@ def run_coverage_report(  # type: ignore
         _publish_html_coverage_report_on_s3(aws_profile)
 
 
+def _get_inclusion_settings(target_dir: str) -> Tuple[str, Optional[str]]:
+    """
+    Determine include/omit glob patterns for the coverage report for both text
+    and HTML coverage reports.
+
+    :param target_dir: directory for coverage stats; use "." to indicate all directories
+    :return: tuple (include_in_report, exclude_from_report) where
+        - include_in_report is the glob pattern to include
+        - exclude_from_report is a comma‑separated glob pattern to omit, or ``None``
+
+    Examples:
+        1. Cover everything (no submodules to omit):
+        ```
+        _get_inclusion_settings(".")
+        ```
+        ("*", "")
+
+        2. Only cover code under a specific directory:
+        ```
+        _get_inclusion_settings("helpers")
+        ```
+        ("*/helpers/*", None)
+
+    Usage in `_run_coverage`:
+
+         # Entire repo coverage (e.g. your 'helpers' project root):
+         ```
+        include, omit = _get_inclusion_settings(".")
+        ```
+        #   coverage report --include=* --sort=Cover
+        #   coverage html   --include=* [--omit=ext/foo/*,ext/bar/*]
+
+        # Single-directory coverage:
+        ```
+        include, omit = _get_inclusion_settings("helpers")
+        ```
+        #   coverage report --include=*/helpers/* --sort=Cover
+        #   coverage html   --include=*/helpers/* [--omit=...]
+    """
+    if target_dir == ".":
+        include_in_report = "*"
+        exclude_from_report = ""
+        if hserver.skip_submodules_test():
+            submodule_paths: List[str] = hgit.get_submodule_paths()
+            exclude_from_report = ",".join(
+                f"{path}/*" for path in submodule_paths
+            )
+    else:
+        include_in_report = f"*/{target_dir}/*"
+        exclude_from_report = None
+    return include_in_report, exclude_from_report
+
+
+def _run_coverage(
path: helpers/lib_tasks_pytest.py
user: login: Shaunak01
body: ooh nice one!
created_at: 2025-04-28T03:07:01Z
author_association: CONTRIBUTOR
diff_hunk: @@ -874,6 +875,177 @@ def run_coverage_report(  # type: ignore
         _publish_html_coverage_report_on_s3(aws_profile)
 
 
+def _get_inclusion_settings(target_dir: str) -> Tuple[str, Optional[str]]:
+    """
+    Determine include/omit glob patterns for the coverage report for both text
+    and HTML coverage reports.
+
+    :param target_dir: directory for coverage stats; use "." to indicate all directories
+    :return: tuple (include_in_report, exclude_from_report) where
+        - include_in_report is the glob pattern to include
+        - exclude_from_report is a comma‑separated glob pattern to omit, or ``None``
+
+    Examples:
+        1. Cover everything (no submodules to omit):
+        ```
+        _get_inclusion_settings(".")
+        ```
+        ("*", "")
+
+        2. Only cover code under a specific directory:
+        ```
+        _get_inclusion_settings("helpers")
+        ```
+        ("*/helpers/*", None)
+
+    Usage in `_run_coverage`:
+
+         # Entire repo coverage (e.g. your 'helpers' project root):
+         ```
+        include, omit = _get_inclusion_settings(".")
+        ```
+        #   coverage report --include=* --sort=Cover
+        #   coverage html   --include=* [--omit=ext/foo/*,ext/bar/*]
+
+        # Single-directory coverage:
+        ```
+        include, omit = _get_inclusion_settings("helpers")
+        ```
+        #   coverage report --include=*/helpers/* --sort=Cover
+        #   coverage html   --include=*/helpers/* [--omit=...]
+    """
+    if target_dir == ".":
+        include_in_report = "*"
+        exclude_from_report = ""
+        if hserver.skip_submodules_test():
+            submodule_paths: List[str] = hgit.get_submodule_paths()
+            exclude_from_report = ",".join(
+                f"{path}/*" for path in submodule_paths
+            )
+    else:
+        include_in_report = f"*/{target_dir}/*"
+        exclude_from_report = None
+    return include_in_report, exclude_from_report
+
+
+def _run_coverage(
+    ctx,
+    suite: str,
+    target_dir: str,
+    generate_html_report: bool = False,
+) -> str:
+    """
+    Run coverage for a given suite (fast/slow/superslow).
+
+    :param suite: one of "fast" or "slow"
+    :param target_dir: coverage target directory
+    :param generate_html_report: whether to produce HTML output
+    :return: xml file named 'coverage.xml'
+    """
+
+    hdbg.dassert_in(
+        suite,
+        ("fast", "slow", "superslow"),
+        "Expected one of 'fast', 'slow', or 'superslow'.",
+    )
+    # Build the command line.
+    test_cmd_parts = [
+        # Invoke the "<suite>_tests" task.
+        "invoke",
+        f"run_{suite}_tests",
+        # Enable coverage collection.
+        "--coverage",
+        # Specify which directory to test.
+        "-p",
+        target_dir,
+        # Pass through remaining args to pytest.
+        "--",
+        # Emit JUnit XML report for CI systems.
+        f"--junitxml=reports/junit_{suite}.xml",
+        # Force legacy JUnit output format.
+        "-o",
+        "junit_family=legacy",
+    ]
+    # join and quote them into a single shell command
+    test_cmd = " ".join(shlex.quote(p) for p in test_cmd_parts)
+    # Name output coverage data file for this suite.
+    coverage_file = f".coverage_{suite}_tests"
+    # Run the tests under coverage.
+    hlitauti.run(ctx, test_cmd, use_system=False)
+    # Move the default ".coverage" file to our suite-specific filename.
+    hsystem.system(f"mv .coverage {coverage_file}")
+    hdbg.dassert_file_exists(coverage_file)
+    # Compute which files/dirs to include and omit in the report.
path: helpers/lib_tasks_pytest.py
user: login: dremdem
body: > We are preparing which parts of the codebase coverage will show up for
> 
> * First → we prepare include_in_report, exclude_from_report.
> * Then → we use them in the coverage report (text report).
> * Then → we use them again in coverage html (HTML report, only if generate_html_report=True).
> * Finally → we generate an XML file with coverage xml -o coverage.xml.

It's a good draft to start a doc 👍
created_at: 2025-04-28T16:22:39Z
author_association: CONTRIBUTOR
diff_hunk: @@ -874,6 +875,177 @@ def run_coverage_report(  # type: ignore
         _publish_html_coverage_report_on_s3(aws_profile)
 
 
+def _get_inclusion_settings(target_dir: str) -> Tuple[str, Optional[str]]:
+    """
+    Determine include/omit glob patterns for the coverage report for both text
+    and HTML coverage reports.
+
+    :param target_dir: directory for coverage stats; use "." to indicate all directories
+    :return: tuple (include_in_report, exclude_from_report) where
+        - include_in_report is the glob pattern to include
+        - exclude_from_report is a comma‑separated glob pattern to omit, or ``None``
+
+    Examples:
+        1. Cover everything (no submodules to omit):
+        ```
+        _get_inclusion_settings(".")
+        ```
+        ("*", "")
+
+        2. Only cover code under a specific directory:
+        ```
+        _get_inclusion_settings("helpers")
+        ```
+        ("*/helpers/*", None)
+
+    Usage in `_run_coverage`:
+
+         # Entire repo coverage (e.g. your 'helpers' project root):
+         ```
+        include, omit = _get_inclusion_settings(".")
+        ```
+        #   coverage report --include=* --sort=Cover
+        #   coverage html   --include=* [--omit=ext/foo/*,ext/bar/*]
+
+        # Single-directory coverage:
+        ```
+        include, omit = _get_inclusion_settings("helpers")
+        ```
+        #   coverage report --include=*/helpers/* --sort=Cover
+        #   coverage html   --include=*/helpers/* [--omit=...]
+    """
+    if target_dir == ".":
+        include_in_report = "*"
+        exclude_from_report = ""
+        if hserver.skip_submodules_test():
+            submodule_paths: List[str] = hgit.get_submodule_paths()
+            exclude_from_report = ",".join(
+                f"{path}/*" for path in submodule_paths
+            )
+    else:
+        include_in_report = f"*/{target_dir}/*"
+        exclude_from_report = None
+    return include_in_report, exclude_from_report
+
+
+def _run_coverage(
+    ctx,
+    suite: str,
+    target_dir: str,
+    generate_html_report: bool = False,
+) -> str:
+    """
+    Run coverage for a given suite (fast/slow/superslow).
+
+    :param suite: one of "fast" or "slow"
+    :param target_dir: coverage target directory
+    :param generate_html_report: whether to produce HTML output
+    :return: xml file named 'coverage.xml'
+    """
+
+    hdbg.dassert_in(
+        suite,
+        ("fast", "slow", "superslow"),
+        "Expected one of 'fast', 'slow', or 'superslow'.",
+    )
+    # Build the command line.
+    test_cmd_parts = [
+        # Invoke the "<suite>_tests" task.
+        "invoke",
+        f"run_{suite}_tests",
+        # Enable coverage collection.
+        "--coverage",
+        # Specify which directory to test.
+        "-p",
+        target_dir,
+        # Pass through remaining args to pytest.
+        "--",
+        # Emit JUnit XML report for CI systems.
+        f"--junitxml=reports/junit_{suite}.xml",
+        # Force legacy JUnit output format.
+        "-o",
+        "junit_family=legacy",
+    ]
+    # join and quote them into a single shell command
+    test_cmd = " ".join(shlex.quote(p) for p in test_cmd_parts)
+    # Name output coverage data file for this suite.
+    coverage_file = f".coverage_{suite}_tests"
+    # Run the tests under coverage.
+    hlitauti.run(ctx, test_cmd, use_system=False)
+    # Move the default ".coverage" file to our suite-specific filename.
+    hsystem.system(f"mv .coverage {coverage_file}")
+    hdbg.dassert_file_exists(coverage_file)
+    # Compute which files/dirs to include and omit in the report.
+    include_in_report, exclude_from_report = _get_inclusion_settings(target_dir)
+    report_cmd: List[str] = [
+        # Clear out any prior coverage data.
+        "coverage erase",
+        # Combine this suite's data into the coverage database.
+        f"coverage combine --keep {coverage_file}",
+    ]
+    # Generate a text report, including only our target paths.
+    report_stats_cmd: str = (
+        f"coverage report --include={include_in_report} --sort=Cover"
+    )
+    if exclude_from_report:
+        report_stats_cmd += f" --omit={exclude_from_report}"
+    report_cmd.append(report_stats_cmd)
+    # Produce HTML output for interactive browsing.
+    if generate_html_report:
+        report_html_cmd: str = f"coverage html --include={include_in_report}"
+        if exclude_from_report:
+            report_html_cmd += f" --omit={exclude_from_report}"
+        report_cmd.append(report_html_cmd)
+    # Export XML coverage report.
+    report_cmd.append("coverage xml -o coverage.xml")
+    full_report_cmd: str = " && ".join(report_cmd)
+    docker_cmd_ = f"invoke docker_cmd --use-bash --cmd '{full_report_cmd}'"
path: helpers/lib_tasks_pytest.py
user: login: dremdem
body: > * Since we are running tests inside Docker, we also want to run reporting inside the exact same environment so that coverage correctly maps file paths, modules, etc.

Fair enough. 
created_at: 2025-04-28T16:24:59Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,136 @@
+name: Test coverage
+
+on:
+  workflow_dispatch: {}
+  # every day at 00:00 UTC.
+  schedule:
+    - cron: '0 0 * * *'
+
+env:
+  CSFY_CI: true
+
+permissions:
+  id-token: write
+  contents: read
+  packages: read
path: .github/workflows/coverage_tests.yml
user: login: dremdem
body: I'd provide comments above each line in order to explain why we need such permission. 
Feel free to take a look at another workflows. 
created_at: 2025-04-29T14:25:49Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,136 @@
+name: Test coverage
+
+on:
+  workflow_dispatch: {}
+  # every day at 00:00 UTC.
+  schedule:
+    - cron: '0 0 * * *'
+
+env:
+  CSFY_CI: true
+
+permissions:
+  id-token: write
+  contents: read
+  packages: read
+
+jobs:
+  run_test_coverage:
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Configure AWS credentials
+        uses: aws-actions/configure-aws-credentials@v1
+        with:
+          role-to-assume: ${{ vars.GH_ACTION_AWS_ROLE_ARN }}
+          role-session-name: ${{ vars.GH_ACTION_AWS_SESSION_NAME }}
+          aws-region: ${{ vars.CSFY_AWS_DEFAULT_REGION }}
+
+      - name: Login to GHCR
+        run: docker login ghcr.io -u gpsaggese -p ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Cleanup
+        run: sudo chmod 777 -R .
+
+      - name: Checkout code
+        uses: actions/checkout@v3
+        with:
+          submodules: recursive
+          fetch-depth: 0
+          token: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Update PYTHONPATH
+        run: echo "PYTHONPATH=.:helpers" >> $GITHUB_ENV
+
+      - name: Install dependencies
+        run: |
+          python -m pip install --upgrade pip
+          pip install -r .github/gh_requirements.txt
+
+      - name: Pull image from GHCR
+        run: docker pull ghcr.io/${{ github.repository }}:dev
+
+      # Only on scheduled runs, capture ISO weekday (1=Mon … 7=Sun).
+      - name: Set DAY_OF_WEEK
+        if: github.event_name == 'schedule'
+        run: echo "DAY_OF_WEEK=$(date -u +'%u')" >> $GITHUB_ENV
+
+      # This step is used to trigger the fast test coverage generation using the invoke task.
+      - name: Run Fast test and generate report
+        env:
+          GH_ACTION_ACCESS_TOKEN: ${{ secrets.GH_ACTION_ACCESS_TOKEN }}
+          CSFY_AWS_ACCESS_KEY_ID: ${{ env.AWS_ACCESS_KEY_ID }}
+          CSFY_AWS_SECRET_ACCESS_KEY: ${{ env.AWS_SECRET_ACCESS_KEY }}
+          CSFY_AWS_SESSION_TOKEN: ${{ env.AWS_SESSION_TOKEN }}
+          CSFY_AWS_DEFAULT_REGION: ${{ env.AWS_DEFAULT_REGION }}
+          CSFY_ECR_BASE_PATH: ghcr.io/${{ github.repository_owner }}
+          CSFY_AWS_S3_BUCKET: ${{ vars.CSFY_AWS_S3_BUCKET }}
+        run: invoke run_coverage --suite fast -- .
+
+      - name: Upload Fast Test Coverage to Codecov
+        # `if: always()` ensures this step runs regardless of prior failures.
+        # It is used here to always attempt uploading the coverage report,
+        # even if earlier steps have failed.
+        if: always()
+        uses: codecov/codecov-action@v5
+        with:
+          token: ${{ secrets.CODECOV_TOKEN }}
+          files: ./coverage.xml
+          flags: fast
+          name: fast-test-coverage
+          # Combined coverage for tests will be checked rather than individual.
+          fail_ci: false
+          # `fail_ci_if_error: true` instructs the action to mark the CI as failed
+          # if there is any error during the upload to Codecov.
+          fail_ci_if_error: true
+
+      - name: Run Slow test and generate report
+        env:
path: .github/workflows/coverage_tests.yml
user: login: dremdem
body: Out of curiosity: if the `fast` tests fail, will the `slow` tests still run?

Can you check pls?
created_at: 2025-04-29T14:31:32Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,136 @@
+name: Test coverage
+
+on:
+  workflow_dispatch: {}
+  # every day at 00:00 UTC.
+  schedule:
+    - cron: '0 0 * * *'
+
+env:
+  CSFY_CI: true
+
+permissions:
+  id-token: write
+  contents: read
+  packages: read
+
+jobs:
+  run_test_coverage:
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Configure AWS credentials
+        uses: aws-actions/configure-aws-credentials@v1
+        with:
+          role-to-assume: ${{ vars.GH_ACTION_AWS_ROLE_ARN }}
+          role-session-name: ${{ vars.GH_ACTION_AWS_SESSION_NAME }}
+          aws-region: ${{ vars.CSFY_AWS_DEFAULT_REGION }}
+
+      - name: Login to GHCR
+        run: docker login ghcr.io -u gpsaggese -p ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Cleanup
+        run: sudo chmod 777 -R .
+
+      - name: Checkout code
+        uses: actions/checkout@v3
+        with:
+          submodules: recursive
+          fetch-depth: 0
+          token: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Update PYTHONPATH
+        run: echo "PYTHONPATH=.:helpers" >> $GITHUB_ENV
+
+      - name: Install dependencies
+        run: |
+          python -m pip install --upgrade pip
+          pip install -r .github/gh_requirements.txt
+
+      - name: Pull image from GHCR
+        run: docker pull ghcr.io/${{ github.repository }}:dev
+
+      # Only on scheduled runs, capture ISO weekday (1=Mon … 7=Sun).
+      - name: Set DAY_OF_WEEK
+        if: github.event_name == 'schedule'
+        run: echo "DAY_OF_WEEK=$(date -u +'%u')" >> $GITHUB_ENV
+
+      # This step is used to trigger the fast test coverage generation using the invoke task.
+      - name: Run Fast test and generate report
+        env:
+          GH_ACTION_ACCESS_TOKEN: ${{ secrets.GH_ACTION_ACCESS_TOKEN }}
+          CSFY_AWS_ACCESS_KEY_ID: ${{ env.AWS_ACCESS_KEY_ID }}
+          CSFY_AWS_SECRET_ACCESS_KEY: ${{ env.AWS_SECRET_ACCESS_KEY }}
+          CSFY_AWS_SESSION_TOKEN: ${{ env.AWS_SESSION_TOKEN }}
+          CSFY_AWS_DEFAULT_REGION: ${{ env.AWS_DEFAULT_REGION }}
+          CSFY_ECR_BASE_PATH: ghcr.io/${{ github.repository_owner }}
+          CSFY_AWS_S3_BUCKET: ${{ vars.CSFY_AWS_S3_BUCKET }}
+        run: invoke run_coverage --suite fast -- .
+
+      - name: Upload Fast Test Coverage to Codecov
+        # `if: always()` ensures this step runs regardless of prior failures.
+        # It is used here to always attempt uploading the coverage report,
+        # even if earlier steps have failed.
+        if: always()
+        uses: codecov/codecov-action@v5
+        with:
+          token: ${{ secrets.CODECOV_TOKEN }}
+          files: ./coverage.xml
+          flags: fast
+          name: fast-test-coverage
+          # Combined coverage for tests will be checked rather than individual.
+          fail_ci: false
+          # `fail_ci_if_error: true` instructs the action to mark the CI as failed
+          # if there is any error during the upload to Codecov.
+          fail_ci_if_error: true
+
+      - name: Run Slow test and generate report
+        env:
path: .github/workflows/coverage_tests.yml
user: login: dremdem
body: For `superslow` also.
created_at: 2025-04-29T14:31:56Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,136 @@
+name: Test coverage
+
+on:
+  workflow_dispatch: {}
+  # every day at 00:00 UTC.
+  schedule:
+    - cron: '0 0 * * *'
+
+env:
+  CSFY_CI: true
+
+permissions:
+  id-token: write
+  contents: read
+  packages: read
+
+jobs:
+  run_test_coverage:
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Configure AWS credentials
+        uses: aws-actions/configure-aws-credentials@v1
+        with:
+          role-to-assume: ${{ vars.GH_ACTION_AWS_ROLE_ARN }}
+          role-session-name: ${{ vars.GH_ACTION_AWS_SESSION_NAME }}
+          aws-region: ${{ vars.CSFY_AWS_DEFAULT_REGION }}
+
+      - name: Login to GHCR
+        run: docker login ghcr.io -u gpsaggese -p ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Cleanup
+        run: sudo chmod 777 -R .
+
+      - name: Checkout code
+        uses: actions/checkout@v3
+        with:
+          submodules: recursive
+          fetch-depth: 0
+          token: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Update PYTHONPATH
+        run: echo "PYTHONPATH=.:helpers" >> $GITHUB_ENV
+
+      - name: Install dependencies
+        run: |
+          python -m pip install --upgrade pip
+          pip install -r .github/gh_requirements.txt
+
+      - name: Pull image from GHCR
+        run: docker pull ghcr.io/${{ github.repository }}:dev
+
+      # Only on scheduled runs, capture ISO weekday (1=Mon … 7=Sun).
+      - name: Set DAY_OF_WEEK
+        if: github.event_name == 'schedule'
+        run: echo "DAY_OF_WEEK=$(date -u +'%u')" >> $GITHUB_ENV
+
+      # This step is used to trigger the fast test coverage generation using the invoke task.
+      - name: Run Fast test and generate report
+        env:
+          GH_ACTION_ACCESS_TOKEN: ${{ secrets.GH_ACTION_ACCESS_TOKEN }}
+          CSFY_AWS_ACCESS_KEY_ID: ${{ env.AWS_ACCESS_KEY_ID }}
+          CSFY_AWS_SECRET_ACCESS_KEY: ${{ env.AWS_SECRET_ACCESS_KEY }}
+          CSFY_AWS_SESSION_TOKEN: ${{ env.AWS_SESSION_TOKEN }}
+          CSFY_AWS_DEFAULT_REGION: ${{ env.AWS_DEFAULT_REGION }}
+          CSFY_ECR_BASE_PATH: ghcr.io/${{ github.repository_owner }}
+          CSFY_AWS_S3_BUCKET: ${{ vars.CSFY_AWS_S3_BUCKET }}
+        run: invoke run_coverage --suite fast -- .
+
+      - name: Upload Fast Test Coverage to Codecov
+        # `if: always()` ensures this step runs regardless of prior failures.
+        # It is used here to always attempt uploading the coverage report,
+        # even if earlier steps have failed.
+        if: always()
+        uses: codecov/codecov-action@v5
+        with:
+          token: ${{ secrets.CODECOV_TOKEN }}
+          files: ./coverage.xml
+          flags: fast
+          name: fast-test-coverage
+          # Combined coverage for tests will be checked rather than individual.
+          fail_ci: false
+          # `fail_ci_if_error: true` instructs the action to mark the CI as failed
+          # if there is any error during the upload to Codecov.
+          fail_ci_if_error: true
+
+      - name: Run Slow test and generate report
+        env:
+          GH_ACTION_ACCESS_TOKEN: ${{ secrets.GH_ACTION_ACCESS_TOKEN }}
+          CSFY_AWS_ACCESS_KEY_ID: ${{ env.AWS_ACCESS_KEY_ID }}
+          CSFY_AWS_SECRET_ACCESS_KEY: ${{ env.AWS_SECRET_ACCESS_KEY }}
+          CSFY_AWS_SESSION_TOKEN: ${{ env.AWS_SESSION_TOKEN }}
+          CSFY_AWS_DEFAULT_REGION: ${{ env.AWS_DEFAULT_REGION }}
+          CSFY_ECR_BASE_PATH: ghcr.io/${{ github.repository_owner }}
+          CSFY_AWS_S3_BUCKET: ${{ vars.CSFY_AWS_S3_BUCKET }}
+        run: invoke run_coverage --suite slow -- .
+
+      - name: Upload Slow Test Coverage to Codecov
+        # "if: always()" ensures this step is executed regardless of previous outcomes.
+        if: always()
+        uses: codecov/codecov-action@v5
+        with:
+          token: ${{ secrets.CODECOV_TOKEN }}
+          files: ./coverage.xml
+          flags: slow
+          name: slow-test-coverage
+          fail_ci: false
path: .github/workflows/coverage_tests.yml
user: login: dremdem
body: Can you explain a bit: what is the flag? 
created_at: 2025-04-29T14:34:32Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,33 @@
+flag_management:
+  default_rules:
+    # Do not reuse last coverage if missing.
+    carryforward: false
+  individual_flags:
+    - name: fast
+      # Reuse last coverage if missing for true.
path: codecov.yml
user: login: dremdem
body: Let's remove it since the code is self-explanatory. 
created_at: 2025-04-29T14:43:41Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,33 @@
+flag_management:
+  default_rules:
+    # Do not reuse last coverage if missing.
+    carryforward: false
+  individual_flags:
+    - name: fast
+      # Reuse last coverage if missing for true.
+      carryforward: true
+    - name: slow
+      carryforward: true
+    - name: superslow
+      carryforward: true
+
+
+# No per-line comments.
+comment: false
+
+coverage:
+  status:
+    project:
+      # Post the project‑level GitHub Check.
+      enabled: true
+      target_branch: master
+      # Measure % drop relative to baseline.
+      comparator: relative
+      # Fail if drop ≥1%.
+      threshold: 1
+      flags:
+        - fast
+        - slow
+        - superslow
+    patch:
path: codecov.yml
user: login: dremdem
body: Can we add some comments above. 
created_at: 2025-04-29T14:44:31Z
author_association: CONTRIBUTOR
diff_hunk: @@ -874,6 +874,143 @@ def run_coverage_report(  # type: ignore
         _publish_html_coverage_report_on_s3(aws_profile)
 
 
+def _get_inclusion_settings(target_dir: str) -> Tuple[str, Optional[str]]:
+    """
+    Determine include/omit glob patterns for the coverage report for both text
+    and HTML coverage reports.
+
+    :param target_dir: directory for coverage stats; use "." to indicate all directories
+    :return: tuple (include_in_report, exclude_from_report) where
+        - include_in_report is the glob pattern to include
+        - exclude_from_report is a comma‑separated glob pattern to omit, or ``None``
+
+    Examples:
+        1. Cover everything (no submodules to omit):
+        ```
+        _get_inclusion_settings(".")
+        ```
+        ("*", "")
+
+        2. Only cover code under a specific directory:
+        ```
+        _get_inclusion_settings("helpers")
+        ```
+        ("*/helpers/*", None)
+
+    Usage in `_run_coverage`:
+
+         # Entire repo coverage (e.g. your 'helpers' project root):
+         ```
+        include, omit = _get_inclusion_settings(".")
+        ```
+        #   coverage report --include=* --sort=Cover
+        #   coverage html   --include=* [--omit=ext/foo/*,ext/bar/*]
path: helpers/lib_tasks_pytest.py
user: login: dremdem
body: I'd use more meaningful submodule names to omit. Even: `sumbodule1/*`, `sumbodule2/*` will be better. 
WDYT? 
created_at: 2025-04-29T14:48:42Z
author_association: CONTRIBUTOR
diff_hunk: @@ -874,6 +874,143 @@ def run_coverage_report(  # type: ignore
         _publish_html_coverage_report_on_s3(aws_profile)
 
 
+def _get_inclusion_settings(target_dir: str) -> Tuple[str, Optional[str]]:
+    """
+    Determine include/omit glob patterns for the coverage report for both text
+    and HTML coverage reports.
+
+    :param target_dir: directory for coverage stats; use "." to indicate all directories
+    :return: tuple (include_in_report, exclude_from_report) where
+        - include_in_report is the glob pattern to include
+        - exclude_from_report is a comma‑separated glob pattern to omit, or ``None``
+
+    Examples:
+        1. Cover everything (no submodules to omit):
+        ```
+        _get_inclusion_settings(".")
+        ```
+        ("*", "")
+
+        2. Only cover code under a specific directory:
+        ```
+        _get_inclusion_settings("helpers")
+        ```
+        ("*/helpers/*", None)
+
+    Usage in `_run_coverage`:
+
+         # Entire repo coverage (e.g. your 'helpers' project root):
+         ```
+        include, omit = _get_inclusion_settings(".")
+        ```
+        #   coverage report --include=* --sort=Cover
+        #   coverage html   --include=* [--omit=ext/foo/*,ext/bar/*]
+
+        # Single-directory coverage:
+        ```
+        include, omit = _get_inclusion_settings("helpers")
+        ```
+        #   coverage report --include=*/helpers/* --sort=Cover
+        #   coverage html   --include=*/helpers/* [--omit=...]
+    """
+    if target_dir == ".":
+        include_in_report = "*"
+        exclude_from_report = ""
+        if hserver.skip_submodules_test():
+            submodule_paths: List[str] = hgit.get_submodule_paths()
+            exclude_from_report = ",".join(
+                f"{path}/*" for path in submodule_paths
+            )
+    else:
+        include_in_report = f"*/{target_dir}/*"
+        exclude_from_report = None
+    return include_in_report, exclude_from_report
+
+
+def _run_coverage(
+    ctx,
+    suite: str,
+    target_dir: str,
+    generate_html_report: bool = False,
+) -> str:
+    """
+    Run coverage for a given suite (fast/slow/superslow).
+
+    :param suite: one of "fast", "slow" or "superslow"
+    :param target_dir: coverage target directory
+    :param generate_html_report: whether to produce HTML output
+    :return: xml file named 'coverage.xml'
+    """
+    hdbg.dassert_in(
+        suite,
+        ("fast", "slow", "superslow"),
+        "Expected one of 'fast', 'slow', or 'superslow'.",
+    )
+    # Build the command line.
+    test_cmd_parts = [
+        # Invoke the "<suite>_tests" task.
+        "invoke",
+        f"run_{suite}_tests",
+        # Enable coverage collection.
path: helpers/lib_tasks_pytest.py
user: login: dremdem
body: Let's explain a little bit more that "coverage collection" is.
created_at: 2025-04-29T14:50:20Z
author_association: CONTRIBUTOR
diff_hunk: @@ -874,6 +874,143 @@ def run_coverage_report(  # type: ignore
         _publish_html_coverage_report_on_s3(aws_profile)
 
 
+def _get_inclusion_settings(target_dir: str) -> Tuple[str, Optional[str]]:
+    """
+    Determine include/omit glob patterns for the coverage report for both text
+    and HTML coverage reports.
+
+    :param target_dir: directory for coverage stats; use "." to indicate all directories
+    :return: tuple (include_in_report, exclude_from_report) where
+        - include_in_report is the glob pattern to include
+        - exclude_from_report is a comma‑separated glob pattern to omit, or ``None``
+
+    Examples:
+        1. Cover everything (no submodules to omit):
+        ```
+        _get_inclusion_settings(".")
+        ```
+        ("*", "")
+
+        2. Only cover code under a specific directory:
+        ```
+        _get_inclusion_settings("helpers")
+        ```
+        ("*/helpers/*", None)
+
+    Usage in `_run_coverage`:
+
+         # Entire repo coverage (e.g. your 'helpers' project root):
+         ```
+        include, omit = _get_inclusion_settings(".")
+        ```
+        #   coverage report --include=* --sort=Cover
+        #   coverage html   --include=* [--omit=ext/foo/*,ext/bar/*]
+
+        # Single-directory coverage:
+        ```
+        include, omit = _get_inclusion_settings("helpers")
+        ```
+        #   coverage report --include=*/helpers/* --sort=Cover
+        #   coverage html   --include=*/helpers/* [--omit=...]
+    """
+    if target_dir == ".":
+        include_in_report = "*"
+        exclude_from_report = ""
+        if hserver.skip_submodules_test():
+            submodule_paths: List[str] = hgit.get_submodule_paths()
+            exclude_from_report = ",".join(
+                f"{path}/*" for path in submodule_paths
+            )
+    else:
+        include_in_report = f"*/{target_dir}/*"
+        exclude_from_report = None
+    return include_in_report, exclude_from_report
+
+
+def _run_coverage(
+    ctx,
+    suite: str,
+    target_dir: str,
+    generate_html_report: bool = False,
+) -> str:
+    """
+    Run coverage for a given suite (fast/slow/superslow).
+
+    :param suite: one of "fast", "slow" or "superslow"
+    :param target_dir: coverage target directory
+    :param generate_html_report: whether to produce HTML output
+    :return: xml file named 'coverage.xml'
+    """
+    hdbg.dassert_in(
+        suite,
+        ("fast", "slow", "superslow"),
+        "Expected one of 'fast', 'slow', or 'superslow'.",
+    )
+    # Build the command line.
+    test_cmd_parts = [
+        # Invoke the "<suite>_tests" task.
+        "invoke",
+        f"run_{suite}_tests",
+        # Enable coverage collection.
+        "--coverage",
+        # Specify which directory to test.
+        "-p", target_dir,
+    ]
+    # join and quote them into a single shell command
+    test_cmd = hlitauti.to_multi_line_cmd(test_cmd_parts)
+    # Run the tests under coverage.
+    hlitauti.run(ctx, test_cmd, use_system=False)
+    # Check that '.coverage' was created.
path: helpers/lib_tasks_pytest.py
user: login: dremdem
body: No need to explain `dassert`.
created_at: 2025-04-29T14:51:12Z
author_association: CONTRIBUTOR
diff_hunk: @@ -874,6 +874,143 @@ def run_coverage_report(  # type: ignore
         _publish_html_coverage_report_on_s3(aws_profile)
 
 
+def _get_inclusion_settings(target_dir: str) -> Tuple[str, Optional[str]]:
+    """
+    Determine include/omit glob patterns for the coverage report for both text
+    and HTML coverage reports.
+
+    :param target_dir: directory for coverage stats; use "." to indicate all directories
+    :return: tuple (include_in_report, exclude_from_report) where
+        - include_in_report is the glob pattern to include
+        - exclude_from_report is a comma‑separated glob pattern to omit, or ``None``
+
+    Examples:
+        1. Cover everything (no submodules to omit):
+        ```
+        _get_inclusion_settings(".")
+        ```
+        ("*", "")
+
+        2. Only cover code under a specific directory:
+        ```
+        _get_inclusion_settings("helpers")
+        ```
+        ("*/helpers/*", None)
+
+    Usage in `_run_coverage`:
+
+         # Entire repo coverage (e.g. your 'helpers' project root):
+         ```
+        include, omit = _get_inclusion_settings(".")
+        ```
+        #   coverage report --include=* --sort=Cover
+        #   coverage html   --include=* [--omit=ext/foo/*,ext/bar/*]
+
+        # Single-directory coverage:
+        ```
+        include, omit = _get_inclusion_settings("helpers")
+        ```
+        #   coverage report --include=*/helpers/* --sort=Cover
+        #   coverage html   --include=*/helpers/* [--omit=...]
+    """
+    if target_dir == ".":
+        include_in_report = "*"
+        exclude_from_report = ""
+        if hserver.skip_submodules_test():
+            submodule_paths: List[str] = hgit.get_submodule_paths()
+            exclude_from_report = ",".join(
+                f"{path}/*" for path in submodule_paths
+            )
+    else:
+        include_in_report = f"*/{target_dir}/*"
+        exclude_from_report = None
+    return include_in_report, exclude_from_report
+
+
+def _run_coverage(
+    ctx,
+    suite: str,
+    target_dir: str,
+    generate_html_report: bool = False,
+) -> str:
+    """
+    Run coverage for a given suite (fast/slow/superslow).
+
+    :param suite: one of "fast", "slow" or "superslow"
+    :param target_dir: coverage target directory
+    :param generate_html_report: whether to produce HTML output
+    :return: xml file named 'coverage.xml'
+    """
+    hdbg.dassert_in(
+        suite,
+        ("fast", "slow", "superslow"),
+        "Expected one of 'fast', 'slow', or 'superslow'.",
+    )
+    # Build the command line.
+    test_cmd_parts = [
+        # Invoke the "<suite>_tests" task.
+        "invoke",
+        f"run_{suite}_tests",
+        # Enable coverage collection.
+        "--coverage",
+        # Specify which directory to test.
+        "-p", target_dir,
+    ]
+    # join and quote them into a single shell command
+    test_cmd = hlitauti.to_multi_line_cmd(test_cmd_parts)
+    # Run the tests under coverage.
+    hlitauti.run(ctx, test_cmd, use_system=False)
+    # Check that '.coverage' was created.
+    hdbg.dassert_file_exists(".coverage")
+    # Compute which files/dirs to include and omit in the report.
+    include_in_report, exclude_from_report = _get_inclusion_settings(target_dir)
+    report_cmd: List[str] = [
+        # Clear out any prior coverage data.
path: helpers/lib_tasks_pytest.py
user: login: dremdem
body: Are we considering a previous run?
If so, let's mention it.
created_at: 2025-04-29T14:52:57Z
author_association: CONTRIBUTOR
diff_hunk: @@ -874,6 +874,143 @@ def run_coverage_report(  # type: ignore
         _publish_html_coverage_report_on_s3(aws_profile)
 
 
+def _get_inclusion_settings(target_dir: str) -> Tuple[str, Optional[str]]:
+    """
+    Determine include/omit glob patterns for the coverage report for both text
+    and HTML coverage reports.
+
+    :param target_dir: directory for coverage stats; use "." to indicate all directories
+    :return: tuple (include_in_report, exclude_from_report) where
+        - include_in_report is the glob pattern to include
+        - exclude_from_report is a comma‑separated glob pattern to omit, or ``None``
+
+    Examples:
+        1. Cover everything (no submodules to omit):
+        ```
+        _get_inclusion_settings(".")
+        ```
+        ("*", "")
+
+        2. Only cover code under a specific directory:
+        ```
+        _get_inclusion_settings("helpers")
+        ```
+        ("*/helpers/*", None)
+
+    Usage in `_run_coverage`:
+
+         # Entire repo coverage (e.g. your 'helpers' project root):
+         ```
+        include, omit = _get_inclusion_settings(".")
+        ```
+        #   coverage report --include=* --sort=Cover
+        #   coverage html   --include=* [--omit=ext/foo/*,ext/bar/*]
+
+        # Single-directory coverage:
+        ```
+        include, omit = _get_inclusion_settings("helpers")
+        ```
+        #   coverage report --include=*/helpers/* --sort=Cover
+        #   coverage html   --include=*/helpers/* [--omit=...]
+    """
+    if target_dir == ".":
+        include_in_report = "*"
+        exclude_from_report = ""
+        if hserver.skip_submodules_test():
+            submodule_paths: List[str] = hgit.get_submodule_paths()
+            exclude_from_report = ",".join(
+                f"{path}/*" for path in submodule_paths
+            )
+    else:
+        include_in_report = f"*/{target_dir}/*"
+        exclude_from_report = None
+    return include_in_report, exclude_from_report
+
+
+def _run_coverage(
+    ctx,
+    suite: str,
+    target_dir: str,
+    generate_html_report: bool = False,
+) -> str:
+    """
+    Run coverage for a given suite (fast/slow/superslow).
+
+    :param suite: one of "fast", "slow" or "superslow"
+    :param target_dir: coverage target directory
+    :param generate_html_report: whether to produce HTML output
+    :return: xml file named 'coverage.xml'
+    """
+    hdbg.dassert_in(
+        suite,
+        ("fast", "slow", "superslow"),
+        "Expected one of 'fast', 'slow', or 'superslow'.",
+    )
+    # Build the command line.
+    test_cmd_parts = [
+        # Invoke the "<suite>_tests" task.
+        "invoke",
+        f"run_{suite}_tests",
+        # Enable coverage collection.
+        "--coverage",
+        # Specify which directory to test.
+        "-p", target_dir,
+    ]
+    # join and quote them into a single shell command
+    test_cmd = hlitauti.to_multi_line_cmd(test_cmd_parts)
+    # Run the tests under coverage.
+    hlitauti.run(ctx, test_cmd, use_system=False)
+    # Check that '.coverage' was created.
+    hdbg.dassert_file_exists(".coverage")
+    # Compute which files/dirs to include and omit in the report.
+    include_in_report, exclude_from_report = _get_inclusion_settings(target_dir)
+    report_cmd: List[str] = [
+        # Clear out any prior coverage data.
+        "coverage erase"
+    ]
+    # Generate a text report, including only our target paths.
+    report_stats_cmd: str = (
+        f"coverage report --include={include_in_report} --sort=Cover"
+    )
+    if exclude_from_report:
+        report_stats_cmd += f" --omit={exclude_from_report}"
+    report_cmd.append(report_stats_cmd)
+    # Produce HTML output for interactive browsing.
+    if generate_html_report:
+        report_html_cmd: str = f"coverage html --include={include_in_report}"
+        if exclude_from_report:
+            report_html_cmd += f" --omit={exclude_from_report}"
+        report_cmd.append(report_html_cmd)
+    # Export XML coverage report.
path: helpers/lib_tasks_pytest.py
user: login: dremdem
body: I'd add the reason why we are doing it, i.g: `... for Codecov integration`
created_at: 2025-04-29T14:57:26Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,136 @@
+name: Test coverage
+
+on:
+  workflow_dispatch: {}
+  # every day at 00:00 UTC.
+  schedule:
+    - cron: '0 0 * * *'
+
+env:
+  CSFY_CI: true
+
+permissions:
+  id-token: write
+  contents: read
+  packages: read
+
+jobs:
+  run_test_coverage:
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Configure AWS credentials
+        uses: aws-actions/configure-aws-credentials@v1
+        with:
+          role-to-assume: ${{ vars.GH_ACTION_AWS_ROLE_ARN }}
+          role-session-name: ${{ vars.GH_ACTION_AWS_SESSION_NAME }}
+          aws-region: ${{ vars.CSFY_AWS_DEFAULT_REGION }}
+
+      - name: Login to GHCR
+        run: docker login ghcr.io -u gpsaggese -p ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Cleanup
+        run: sudo chmod 777 -R .
+
+      - name: Checkout code
+        uses: actions/checkout@v3
+        with:
+          submodules: recursive
+          fetch-depth: 0
+          token: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Update PYTHONPATH
+        run: echo "PYTHONPATH=.:helpers" >> $GITHUB_ENV
+
+      - name: Install dependencies
+        run: |
+          python -m pip install --upgrade pip
+          pip install -r .github/gh_requirements.txt
+
+      - name: Pull image from GHCR
+        run: docker pull ghcr.io/${{ github.repository }}:dev
+
+      # Only on scheduled runs, capture ISO weekday (1=Mon … 7=Sun).
+      - name: Set DAY_OF_WEEK
+        if: github.event_name == 'schedule'
+        run: echo "DAY_OF_WEEK=$(date -u +'%u')" >> $GITHUB_ENV
+
+      # This step is used to trigger the fast test coverage generation using the invoke task.
+      - name: Run Fast test and generate report
+        env:
+          GH_ACTION_ACCESS_TOKEN: ${{ secrets.GH_ACTION_ACCESS_TOKEN }}
+          CSFY_AWS_ACCESS_KEY_ID: ${{ env.AWS_ACCESS_KEY_ID }}
+          CSFY_AWS_SECRET_ACCESS_KEY: ${{ env.AWS_SECRET_ACCESS_KEY }}
+          CSFY_AWS_SESSION_TOKEN: ${{ env.AWS_SESSION_TOKEN }}
+          CSFY_AWS_DEFAULT_REGION: ${{ env.AWS_DEFAULT_REGION }}
+          CSFY_ECR_BASE_PATH: ghcr.io/${{ github.repository_owner }}
+          CSFY_AWS_S3_BUCKET: ${{ vars.CSFY_AWS_S3_BUCKET }}
+        run: invoke run_coverage --suite fast -- .
path: .github/workflows/coverage_tests.yml
user: login: dremdem
body: Can we run it without `.`?
created_at: 2025-04-29T15:02:32Z
author_association: CONTRIBUTOR
diff_hunk: @@ -874,6 +874,143 @@ def run_coverage_report(  # type: ignore
         _publish_html_coverage_report_on_s3(aws_profile)
 
 
+def _get_inclusion_settings(target_dir: str) -> Tuple[str, Optional[str]]:
+    """
+    Determine include/omit glob patterns for the coverage report for both text
+    and HTML coverage reports.
+
+    :param target_dir: directory for coverage stats; use "." to indicate all directories
+    :return: tuple (include_in_report, exclude_from_report) where
+        - include_in_report is the glob pattern to include
+        - exclude_from_report is a comma‑separated glob pattern to omit, or ``None``
+
+    Examples:
+        1. Cover everything (no submodules to omit):
+        ```
+        _get_inclusion_settings(".")
+        ```
+        ("*", "")
+
+        2. Only cover code under a specific directory:
+        ```
+        _get_inclusion_settings("helpers")
+        ```
+        ("*/helpers/*", None)
+
+    Usage in `_run_coverage`:
+
+         # Entire repo coverage (e.g. your 'helpers' project root):
+         ```
+        include, omit = _get_inclusion_settings(".")
+        ```
+        #   coverage report --include=* --sort=Cover
+        #   coverage html   --include=* [--omit=ext/foo/*,ext/bar/*]
+
+        # Single-directory coverage:
+        ```
+        include, omit = _get_inclusion_settings("helpers")
+        ```
+        #   coverage report --include=*/helpers/* --sort=Cover
+        #   coverage html   --include=*/helpers/* [--omit=...]
+    """
+    if target_dir == ".":
+        include_in_report = "*"
+        exclude_from_report = ""
+        if hserver.skip_submodules_test():
+            submodule_paths: List[str] = hgit.get_submodule_paths()
+            exclude_from_report = ",".join(
+                f"{path}/*" for path in submodule_paths
+            )
+    else:
+        include_in_report = f"*/{target_dir}/*"
+        exclude_from_report = None
+    return include_in_report, exclude_from_report
+
+
+def _run_coverage(
+    ctx,
+    suite: str,
+    target_dir: str,
+    generate_html_report: bool = False,
+) -> str:
+    """
+    Run coverage for a given suite (fast/slow/superslow).
+
+    :param suite: one of "fast", "slow" or "superslow"
+    :param target_dir: coverage target directory
+    :param generate_html_report: whether to produce HTML output
+    :return: xml file named 'coverage.xml'
+    """
+    hdbg.dassert_in(
+        suite,
+        ("fast", "slow", "superslow"),
+        "Expected one of 'fast', 'slow', or 'superslow'.",
+    )
+    # Build the command line.
+    test_cmd_parts = [
+        # Invoke the "<suite>_tests" task.
+        "invoke",
+        f"run_{suite}_tests",
+        # Enable coverage collection.
+        "--coverage",
+        # Specify which directory to test.
+        "-p", target_dir,
+    ]
+    # join and quote them into a single shell command
+    test_cmd = hlitauti.to_multi_line_cmd(test_cmd_parts)
+    # Run the tests under coverage.
+    hlitauti.run(ctx, test_cmd, use_system=False)
+    # Check that '.coverage' was created.
+    hdbg.dassert_file_exists(".coverage")
+    # Compute which files/dirs to include and omit in the report.
+    include_in_report, exclude_from_report = _get_inclusion_settings(target_dir)
+    report_cmd: List[str] = [
+        # Clear out any prior coverage data.
+        "coverage erase"
+    ]
+    # Generate a text report, including only our target paths.
+    report_stats_cmd: str = (
+        f"coverage report --include={include_in_report} --sort=Cover"
+    )
+    if exclude_from_report:
+        report_stats_cmd += f" --omit={exclude_from_report}"
+    report_cmd.append(report_stats_cmd)
+    # Produce HTML output for interactive browsing.
+    if generate_html_report:
+        report_html_cmd: str = f"coverage html --include={include_in_report}"
+        if exclude_from_report:
+            report_html_cmd += f" --omit={exclude_from_report}"
+        report_cmd.append(report_html_cmd)
+    # Export XML coverage report.
+    report_cmd.append("coverage xml -o coverage.xml")
+    full_report_cmd: str = " && ".join(report_cmd)
+    docker_cmd_ = f"invoke docker_cmd --use-bash --cmd '{full_report_cmd}'"
+    # Execute the full coverage/Docker pipeline.
+    hlitauti.run(ctx, docker_cmd_)
+
+
+@task
+def run_coverage(
+    ctx,
+    suite: str,
+    target_dir: str = ".",
+):
+    """
+    Unified task to run coverage for any test suite.
+
+    :param ctx: Invoke context
+    :param suite: Test suite to run ("fast", "slow", "superslow")
+    :param target_dir: Target directory to measure coverage for (default
+        ".")
+    """
+    hdbg.dassert_in(
+        suite,
+        ("fast", "slow", "superslow"),
+        "Expected one of 'fast', 'slow', or 'superslow'.",
path: helpers/lib_tasks_pytest.py
user: login: dremdem
body: I think no need to repeat default message, so it's safe to delete a message.
created_at: 2025-04-29T15:04:06Z
author_association: CONTRIBUTOR
diff_hunk: @@ -874,6 +874,143 @@ def run_coverage_report(  # type: ignore
         _publish_html_coverage_report_on_s3(aws_profile)
 
 
+def _get_inclusion_settings(target_dir: str) -> Tuple[str, Optional[str]]:
+    """
+    Determine include/omit glob patterns for the coverage report for both text
+    and HTML coverage reports.
+
+    :param target_dir: directory for coverage stats; use "." to indicate all directories
+    :return: tuple (include_in_report, exclude_from_report) where
+        - include_in_report is the glob pattern to include
+        - exclude_from_report is a comma‑separated glob pattern to omit, or ``None``
+
+    Examples:
+        1. Cover everything (no submodules to omit):
+        ```
+        _get_inclusion_settings(".")
+        ```
+        ("*", "")
+
+        2. Only cover code under a specific directory:
+        ```
+        _get_inclusion_settings("helpers")
+        ```
+        ("*/helpers/*", None)
+
+    Usage in `_run_coverage`:
+
+         # Entire repo coverage (e.g. your 'helpers' project root):
+         ```
+        include, omit = _get_inclusion_settings(".")
+        ```
+        #   coverage report --include=* --sort=Cover
+        #   coverage html   --include=* [--omit=ext/foo/*,ext/bar/*]
+
+        # Single-directory coverage:
+        ```
+        include, omit = _get_inclusion_settings("helpers")
+        ```
+        #   coverage report --include=*/helpers/* --sort=Cover
+        #   coverage html   --include=*/helpers/* [--omit=...]
+    """
+    if target_dir == ".":
+        include_in_report = "*"
+        exclude_from_report = ""
+        if hserver.skip_submodules_test():
+            submodule_paths: List[str] = hgit.get_submodule_paths()
+            exclude_from_report = ",".join(
+                f"{path}/*" for path in submodule_paths
+            )
+    else:
+        include_in_report = f"*/{target_dir}/*"
+        exclude_from_report = None
+    return include_in_report, exclude_from_report
+
+
+def _run_coverage(
+    ctx,
+    suite: str,
+    target_dir: str,
+    generate_html_report: bool = False,
+) -> str:
+    """
+    Run coverage for a given suite (fast/slow/superslow).
+
+    :param suite: one of "fast", "slow" or "superslow"
+    :param target_dir: coverage target directory
+    :param generate_html_report: whether to produce HTML output
+    :return: xml file named 'coverage.xml'
+    """
+    hdbg.dassert_in(
+        suite,
+        ("fast", "slow", "superslow"),
+        "Expected one of 'fast', 'slow', or 'superslow'.",
+    )
+    # Build the command line.
+    test_cmd_parts = [
+        # Invoke the "<suite>_tests" task.
+        "invoke",
+        f"run_{suite}_tests",
+        # Enable coverage collection.
+        "--coverage",
+        # Specify which directory to test.
+        "-p", target_dir,
+    ]
+    # join and quote them into a single shell command
+    test_cmd = hlitauti.to_multi_line_cmd(test_cmd_parts)
+    # Run the tests under coverage.
+    hlitauti.run(ctx, test_cmd, use_system=False)
+    # Check that '.coverage' was created.
+    hdbg.dassert_file_exists(".coverage")
+    # Compute which files/dirs to include and omit in the report.
+    include_in_report, exclude_from_report = _get_inclusion_settings(target_dir)
+    report_cmd: List[str] = [
+        # Clear out any prior coverage data.
+        "coverage erase"
+    ]
+    # Generate a text report, including only our target paths.
+    report_stats_cmd: str = (
+        f"coverage report --include={include_in_report} --sort=Cover"
+    )
+    if exclude_from_report:
+        report_stats_cmd += f" --omit={exclude_from_report}"
+    report_cmd.append(report_stats_cmd)
+    # Produce HTML output for interactive browsing.
+    if generate_html_report:
+        report_html_cmd: str = f"coverage html --include={include_in_report}"
+        if exclude_from_report:
+            report_html_cmd += f" --omit={exclude_from_report}"
+        report_cmd.append(report_html_cmd)
+    # Export XML coverage report.
+    report_cmd.append("coverage xml -o coverage.xml")
+    full_report_cmd: str = " && ".join(report_cmd)
+    docker_cmd_ = f"invoke docker_cmd --use-bash --cmd '{full_report_cmd}'"
+    # Execute the full coverage/Docker pipeline.
+    hlitauti.run(ctx, docker_cmd_)
+
+
+@task
+def run_coverage(
+    ctx,
+    suite: str,
+    target_dir: str = ".",
+):
+    """
+    Unified task to run coverage for any test suite.
+
+    :param ctx: Invoke context
+    :param suite: Test suite to run ("fast", "slow", "superslow")
+    :param target_dir: Target directory to measure coverage for (default
+        ".")
path: helpers/lib_tasks_pytest.py
user: login: dremdem
body: Let's not repeat info about default values.
created_at: 2025-04-29T15:04:56Z
author_association: CONTRIBUTOR
diff_hunk: @@ -874,6 +874,143 @@ def run_coverage_report(  # type: ignore
         _publish_html_coverage_report_on_s3(aws_profile)
 
 
+def _get_inclusion_settings(target_dir: str) -> Tuple[str, Optional[str]]:
+    """
+    Determine include/omit glob patterns for the coverage report for both text
+    and HTML coverage reports.
+
+    :param target_dir: directory for coverage stats; use "." to indicate all directories
+    :return: tuple (include_in_report, exclude_from_report) where
+        - include_in_report is the glob pattern to include
+        - exclude_from_report is a comma‑separated glob pattern to omit, or ``None``
+
+    Examples:
+        1. Cover everything (no submodules to omit):
+        ```
+        _get_inclusion_settings(".")
+        ```
+        ("*", "")
+
+        2. Only cover code under a specific directory:
+        ```
+        _get_inclusion_settings("helpers")
+        ```
+        ("*/helpers/*", None)
+
+    Usage in `_run_coverage`:
+
+         # Entire repo coverage (e.g. your 'helpers' project root):
+         ```
+        include, omit = _get_inclusion_settings(".")
+        ```
+        #   coverage report --include=* --sort=Cover
+        #   coverage html   --include=* [--omit=ext/foo/*,ext/bar/*]
+
+        # Single-directory coverage:
+        ```
+        include, omit = _get_inclusion_settings("helpers")
+        ```
+        #   coverage report --include=*/helpers/* --sort=Cover
+        #   coverage html   --include=*/helpers/* [--omit=...]
+    """
+    if target_dir == ".":
+        include_in_report = "*"
+        exclude_from_report = ""
+        if hserver.skip_submodules_test():
+            submodule_paths: List[str] = hgit.get_submodule_paths()
+            exclude_from_report = ",".join(
+                f"{path}/*" for path in submodule_paths
+            )
+    else:
+        include_in_report = f"*/{target_dir}/*"
+        exclude_from_report = None
+    return include_in_report, exclude_from_report
+
+
+def _run_coverage(
+    ctx,
+    suite: str,
+    target_dir: str,
+    generate_html_report: bool = False,
+) -> str:
+    """
+    Run coverage for a given suite (fast/slow/superslow).
+
+    :param suite: one of "fast", "slow" or "superslow"
+    :param target_dir: coverage target directory
+    :param generate_html_report: whether to produce HTML output
+    :return: xml file named 'coverage.xml'
+    """
+    hdbg.dassert_in(
+        suite,
+        ("fast", "slow", "superslow"),
+        "Expected one of 'fast', 'slow', or 'superslow'.",
+    )
+    # Build the command line.
+    test_cmd_parts = [
+        # Invoke the "<suite>_tests" task.
+        "invoke",
+        f"run_{suite}_tests",
+        # Enable coverage collection.
+        "--coverage",
+        # Specify which directory to test.
+        "-p", target_dir,
+    ]
+    # join and quote them into a single shell command
+    test_cmd = hlitauti.to_multi_line_cmd(test_cmd_parts)
+    # Run the tests under coverage.
+    hlitauti.run(ctx, test_cmd, use_system=False)
+    # Check that '.coverage' was created.
+    hdbg.dassert_file_exists(".coverage")
+    # Compute which files/dirs to include and omit in the report.
+    include_in_report, exclude_from_report = _get_inclusion_settings(target_dir)
+    report_cmd: List[str] = [
+        # Clear out any prior coverage data.
+        "coverage erase"
+    ]
+    # Generate a text report, including only our target paths.
+    report_stats_cmd: str = (
+        f"coverage report --include={include_in_report} --sort=Cover"
+    )
+    if exclude_from_report:
+        report_stats_cmd += f" --omit={exclude_from_report}"
+    report_cmd.append(report_stats_cmd)
+    # Produce HTML output for interactive browsing.
+    if generate_html_report:
+        report_html_cmd: str = f"coverage html --include={include_in_report}"
+        if exclude_from_report:
+            report_html_cmd += f" --omit={exclude_from_report}"
+        report_cmd.append(report_html_cmd)
+    # Export XML coverage report.
+    report_cmd.append("coverage xml -o coverage.xml")
+    full_report_cmd: str = " && ".join(report_cmd)
+    docker_cmd_ = f"invoke docker_cmd --use-bash --cmd '{full_report_cmd}'"
+    # Execute the full coverage/Docker pipeline.
+    hlitauti.run(ctx, docker_cmd_)
+
+
+@task
+def run_coverage(
+    ctx,
+    suite: str,
+    target_dir: str = ".",
+):
+    """
+    Unified task to run coverage for any test suite.
+
+    :param ctx: Invoke context
+    :param suite: Test suite to run ("fast", "slow", "superslow")
+    :param target_dir: Target directory to measure coverage for (default
+        ".")
path: helpers/lib_tasks_pytest.py
user: login: dremdem
body: Same everywhere.
created_at: 2025-04-29T15:05:07Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,136 @@
+name: Test coverage
+
+on:
+  workflow_dispatch: {}
+  # every day at 00:00 UTC.
+  schedule:
+    - cron: '0 0 * * *'
+
+env:
+  CSFY_CI: true
+
+permissions:
+  id-token: write
+  contents: read
+  packages: read
+
+jobs:
+  run_test_coverage:
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Configure AWS credentials
+        uses: aws-actions/configure-aws-credentials@v1
+        with:
+          role-to-assume: ${{ vars.GH_ACTION_AWS_ROLE_ARN }}
+          role-session-name: ${{ vars.GH_ACTION_AWS_SESSION_NAME }}
+          aws-region: ${{ vars.CSFY_AWS_DEFAULT_REGION }}
+
+      - name: Login to GHCR
+        run: docker login ghcr.io -u gpsaggese -p ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Cleanup
+        run: sudo chmod 777 -R .
+
+      - name: Checkout code
+        uses: actions/checkout@v3
+        with:
+          submodules: recursive
+          fetch-depth: 0
+          token: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Update PYTHONPATH
+        run: echo "PYTHONPATH=.:helpers" >> $GITHUB_ENV
+
+      - name: Install dependencies
+        run: |
+          python -m pip install --upgrade pip
+          pip install -r .github/gh_requirements.txt
+
+      - name: Pull image from GHCR
+        run: docker pull ghcr.io/${{ github.repository }}:dev
+
+      # Only on scheduled runs, capture ISO weekday (1=Mon … 7=Sun).
+      - name: Set DAY_OF_WEEK
+        if: github.event_name == 'schedule'
+        run: echo "DAY_OF_WEEK=$(date -u +'%u')" >> $GITHUB_ENV
+
+      # This step is used to trigger the fast test coverage generation using the invoke task.
+      - name: Run Fast test and generate report
+        env:
+          GH_ACTION_ACCESS_TOKEN: ${{ secrets.GH_ACTION_ACCESS_TOKEN }}
+          CSFY_AWS_ACCESS_KEY_ID: ${{ env.AWS_ACCESS_KEY_ID }}
+          CSFY_AWS_SECRET_ACCESS_KEY: ${{ env.AWS_SECRET_ACCESS_KEY }}
+          CSFY_AWS_SESSION_TOKEN: ${{ env.AWS_SESSION_TOKEN }}
+          CSFY_AWS_DEFAULT_REGION: ${{ env.AWS_DEFAULT_REGION }}
+          CSFY_ECR_BASE_PATH: ghcr.io/${{ github.repository_owner }}
+          CSFY_AWS_S3_BUCKET: ${{ vars.CSFY_AWS_S3_BUCKET }}
+        run: invoke run_coverage --suite fast -- .
+
+      - name: Upload Fast Test Coverage to Codecov
+        # `if: always()` ensures this step runs regardless of prior failures.
+        # It is used here to always attempt uploading the coverage report,
+        # even if earlier steps have failed.
+        if: always()
+        uses: codecov/codecov-action@v5
+        with:
+          token: ${{ secrets.CODECOV_TOKEN }}
+          files: ./coverage.xml
+          flags: fast
+          name: fast-test-coverage
+          # Combined coverage for tests will be checked rather than individual.
+          fail_ci: false
+          # `fail_ci_if_error: true` instructs the action to mark the CI as failed
+          # if there is any error during the upload to Codecov.
+          fail_ci_if_error: true
+
+      - name: Run Slow test and generate report
+        env:
path: .github/workflows/coverage_tests.yml
user: login: Shaunak01
body: I am adding a commit where the logic is better:

All test suites always run even if earlier ones fail. Each coverage upload only runs if its corresponding test passes, so a failure in one test does not block others from running or uploading their coverage.
created_at: 2025-04-29T16:31:20Z
author_association: CONTRIBUTOR
diff_hunk: @@ -874,6 +874,143 @@ def run_coverage_report(  # type: ignore
         _publish_html_coverage_report_on_s3(aws_profile)
 
 
+def _get_inclusion_settings(target_dir: str) -> Tuple[str, Optional[str]]:
+    """
+    Determine include/omit glob patterns for the coverage report for both text
+    and HTML coverage reports.
+
+    :param target_dir: directory for coverage stats; use "." to indicate all directories
+    :return: tuple (include_in_report, exclude_from_report) where
+        - include_in_report is the glob pattern to include
+        - exclude_from_report is a comma‑separated glob pattern to omit, or ``None``
+
+    Examples:
+        1. Cover everything (no submodules to omit):
+        ```
+        _get_inclusion_settings(".")
+        ```
+        ("*", "")
+
+        2. Only cover code under a specific directory:
+        ```
+        _get_inclusion_settings("helpers")
+        ```
+        ("*/helpers/*", None)
+
+    Usage in `_run_coverage`:
+
+         # Entire repo coverage (e.g. your 'helpers' project root):
+         ```
+        include, omit = _get_inclusion_settings(".")
+        ```
+        #   coverage report --include=* --sort=Cover
+        #   coverage html   --include=* [--omit=ext/foo/*,ext/bar/*]
+
+        # Single-directory coverage:
+        ```
+        include, omit = _get_inclusion_settings("helpers")
+        ```
+        #   coverage report --include=*/helpers/* --sort=Cover
+        #   coverage html   --include=*/helpers/* [--omit=...]
+    """
+    if target_dir == ".":
+        include_in_report = "*"
+        exclude_from_report = ""
+        if hserver.skip_submodules_test():
+            submodule_paths: List[str] = hgit.get_submodule_paths()
+            exclude_from_report = ",".join(
+                f"{path}/*" for path in submodule_paths
+            )
+    else:
+        include_in_report = f"*/{target_dir}/*"
+        exclude_from_report = None
+    return include_in_report, exclude_from_report
+
+
+def _run_coverage(
+    ctx,
+    suite: str,
+    target_dir: str,
+    generate_html_report: bool = False,
+) -> str:
+    """
+    Run coverage for a given suite (fast/slow/superslow).
+
+    :param suite: one of "fast", "slow" or "superslow"
+    :param target_dir: coverage target directory
+    :param generate_html_report: whether to produce HTML output
+    :return: xml file named 'coverage.xml'
+    """
+    hdbg.dassert_in(
+        suite,
+        ("fast", "slow", "superslow"),
+        "Expected one of 'fast', 'slow', or 'superslow'.",
+    )
+    # Build the command line.
+    test_cmd_parts = [
+        # Invoke the "<suite>_tests" task.
+        "invoke",
+        f"run_{suite}_tests",
+        # Enable coverage collection.
path: helpers/lib_tasks_pytest.py
user: login: Shaunak01
body: umm # Enable coverage collection to track code execution.?
created_at: 2025-04-29T17:57:22Z
author_association: CONTRIBUTOR
diff_hunk: @@ -874,6 +874,143 @@ def run_coverage_report(  # type: ignore
         _publish_html_coverage_report_on_s3(aws_profile)
 
 
+def _get_inclusion_settings(target_dir: str) -> Tuple[str, Optional[str]]:
+    """
+    Determine include/omit glob patterns for the coverage report for both text
+    and HTML coverage reports.
+
+    :param target_dir: directory for coverage stats; use "." to indicate all directories
+    :return: tuple (include_in_report, exclude_from_report) where
+        - include_in_report is the glob pattern to include
+        - exclude_from_report is a comma‑separated glob pattern to omit, or ``None``
+
+    Examples:
+        1. Cover everything (no submodules to omit):
+        ```
+        _get_inclusion_settings(".")
+        ```
+        ("*", "")
+
+        2. Only cover code under a specific directory:
+        ```
+        _get_inclusion_settings("helpers")
+        ```
+        ("*/helpers/*", None)
+
+    Usage in `_run_coverage`:
+
+         # Entire repo coverage (e.g. your 'helpers' project root):
+         ```
+        include, omit = _get_inclusion_settings(".")
+        ```
+        #   coverage report --include=* --sort=Cover
+        #   coverage html   --include=* [--omit=ext/foo/*,ext/bar/*]
+
+        # Single-directory coverage:
+        ```
+        include, omit = _get_inclusion_settings("helpers")
+        ```
+        #   coverage report --include=*/helpers/* --sort=Cover
+        #   coverage html   --include=*/helpers/* [--omit=...]
+    """
+    if target_dir == ".":
+        include_in_report = "*"
+        exclude_from_report = ""
+        if hserver.skip_submodules_test():
+            submodule_paths: List[str] = hgit.get_submodule_paths()
+            exclude_from_report = ",".join(
+                f"{path}/*" for path in submodule_paths
+            )
+    else:
+        include_in_report = f"*/{target_dir}/*"
+        exclude_from_report = None
+    return include_in_report, exclude_from_report
+
+
+def _run_coverage(
+    ctx,
+    suite: str,
+    target_dir: str,
+    generate_html_report: bool = False,
+) -> str:
+    """
+    Run coverage for a given suite (fast/slow/superslow).
+
+    :param suite: one of "fast", "slow" or "superslow"
+    :param target_dir: coverage target directory
+    :param generate_html_report: whether to produce HTML output
+    :return: xml file named 'coverage.xml'
+    """
+    hdbg.dassert_in(
+        suite,
+        ("fast", "slow", "superslow"),
+        "Expected one of 'fast', 'slow', or 'superslow'.",
+    )
+    # Build the command line.
+    test_cmd_parts = [
+        # Invoke the "<suite>_tests" task.
+        "invoke",
+        f"run_{suite}_tests",
+        # Enable coverage collection.
+        "--coverage",
+        # Specify which directory to test.
+        "-p", target_dir,
+    ]
+    # join and quote them into a single shell command
+    test_cmd = hlitauti.to_multi_line_cmd(test_cmd_parts)
+    # Run the tests under coverage.
+    hlitauti.run(ctx, test_cmd, use_system=False)
+    # Check that '.coverage' was created.
+    hdbg.dassert_file_exists(".coverage")
+    # Compute which files/dirs to include and omit in the report.
+    include_in_report, exclude_from_report = _get_inclusion_settings(target_dir)
+    report_cmd: List[str] = [
+        # Clear out any prior coverage data.
path: helpers/lib_tasks_pytest.py
user: login: Shaunak01
body: Nopes we are not. Previously we were combining. Lets remove if we want to stick to this workflow
created_at: 2025-04-29T17:59:32Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,136 @@
+name: Test coverage
+
+on:
+  workflow_dispatch: {}
+  # every day at 00:00 UTC.
+  schedule:
+    - cron: '0 0 * * *'
+
+env:
+  CSFY_CI: true
+
+permissions:
+  id-token: write
+  contents: read
+  packages: read
+
+jobs:
+  run_test_coverage:
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Configure AWS credentials
+        uses: aws-actions/configure-aws-credentials@v1
+        with:
+          role-to-assume: ${{ vars.GH_ACTION_AWS_ROLE_ARN }}
+          role-session-name: ${{ vars.GH_ACTION_AWS_SESSION_NAME }}
+          aws-region: ${{ vars.CSFY_AWS_DEFAULT_REGION }}
+
+      - name: Login to GHCR
+        run: docker login ghcr.io -u gpsaggese -p ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Cleanup
+        run: sudo chmod 777 -R .
+
+      - name: Checkout code
+        uses: actions/checkout@v3
+        with:
+          submodules: recursive
+          fetch-depth: 0
+          token: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Update PYTHONPATH
+        run: echo "PYTHONPATH=.:helpers" >> $GITHUB_ENV
+
+      - name: Install dependencies
+        run: |
+          python -m pip install --upgrade pip
+          pip install -r .github/gh_requirements.txt
+
+      - name: Pull image from GHCR
+        run: docker pull ghcr.io/${{ github.repository }}:dev
+
+      # Only on scheduled runs, capture ISO weekday (1=Mon … 7=Sun).
+      - name: Set DAY_OF_WEEK
+        if: github.event_name == 'schedule'
+        run: echo "DAY_OF_WEEK=$(date -u +'%u')" >> $GITHUB_ENV
+
+      # This step is used to trigger the fast test coverage generation using the invoke task.
+      - name: Run Fast test and generate report
+        env:
+          GH_ACTION_ACCESS_TOKEN: ${{ secrets.GH_ACTION_ACCESS_TOKEN }}
+          CSFY_AWS_ACCESS_KEY_ID: ${{ env.AWS_ACCESS_KEY_ID }}
+          CSFY_AWS_SECRET_ACCESS_KEY: ${{ env.AWS_SECRET_ACCESS_KEY }}
+          CSFY_AWS_SESSION_TOKEN: ${{ env.AWS_SESSION_TOKEN }}
+          CSFY_AWS_DEFAULT_REGION: ${{ env.AWS_DEFAULT_REGION }}
+          CSFY_ECR_BASE_PATH: ghcr.io/${{ github.repository_owner }}
+          CSFY_AWS_S3_BUCKET: ${{ vars.CSFY_AWS_S3_BUCKET }}
+        run: invoke run_coverage --suite fast -- .
path: .github/workflows/coverage_tests.yml
user: login: Shaunak01
body: Yess
created_at: 2025-04-29T18:00:58Z
author_association: CONTRIBUTOR
diff_hunk: @@ -874,6 +874,143 @@ def run_coverage_report(  # type: ignore
         _publish_html_coverage_report_on_s3(aws_profile)
 
 
+def _get_inclusion_settings(target_dir: str) -> Tuple[str, Optional[str]]:
+    """
+    Determine include/omit glob patterns for the coverage report for both text
+    and HTML coverage reports.
+
+    :param target_dir: directory for coverage stats; use "." to indicate all directories
+    :return: tuple (include_in_report, exclude_from_report) where
+        - include_in_report is the glob pattern to include
+        - exclude_from_report is a comma‑separated glob pattern to omit, or ``None``
+
+    Examples:
+        1. Cover everything (no submodules to omit):
+        ```
+        _get_inclusion_settings(".")
+        ```
+        ("*", "")
+
+        2. Only cover code under a specific directory:
+        ```
+        _get_inclusion_settings("helpers")
+        ```
+        ("*/helpers/*", None)
+
+    Usage in `_run_coverage`:
+
+         # Entire repo coverage (e.g. your 'helpers' project root):
+         ```
+        include, omit = _get_inclusion_settings(".")
+        ```
+        #   coverage report --include=* --sort=Cover
+        #   coverage html   --include=* [--omit=ext/foo/*,ext/bar/*]
+
+        # Single-directory coverage:
+        ```
+        include, omit = _get_inclusion_settings("helpers")
+        ```
+        #   coverage report --include=*/helpers/* --sort=Cover
+        #   coverage html   --include=*/helpers/* [--omit=...]
+    """
+    if target_dir == ".":
+        include_in_report = "*"
+        exclude_from_report = ""
+        if hserver.skip_submodules_test():
+            submodule_paths: List[str] = hgit.get_submodule_paths()
+            exclude_from_report = ",".join(
+                f"{path}/*" for path in submodule_paths
+            )
+    else:
+        include_in_report = f"*/{target_dir}/*"
+        exclude_from_report = None
+    return include_in_report, exclude_from_report
+
+
+def _run_coverage(
+    ctx,
+    suite: str,
+    target_dir: str,
+    generate_html_report: bool = False,
+) -> str:
+    """
+    Run coverage for a given suite (fast/slow/superslow).
+
+    :param suite: one of "fast", "slow" or "superslow"
+    :param target_dir: coverage target directory
+    :param generate_html_report: whether to produce HTML output
+    :return: xml file named 'coverage.xml'
+    """
+    hdbg.dassert_in(
+        suite,
+        ("fast", "slow", "superslow"),
+        "Expected one of 'fast', 'slow', or 'superslow'.",
+    )
+    # Build the command line.
+    test_cmd_parts = [
+        # Invoke the "<suite>_tests" task.
+        "invoke",
+        f"run_{suite}_tests",
+        # Enable coverage collection.
path: helpers/lib_tasks_pytest.py
user: login: dremdem
body: I think in the function docstring: 
https://github.com/causify-ai/helpers/blob/7659aea362fd2c63c2843f942e7f7b09074c6bf1/helpers/lib_tasks_pytest.py#L471
It's more clear. 
So, let's it be: `enable coverage computation`
created_at: 2025-04-29T18:09:23Z
author_association: CONTRIBUTOR
diff_hunk: @@ -874,6 +874,143 @@ def run_coverage_report(  # type: ignore
         _publish_html_coverage_report_on_s3(aws_profile)
 
 
+def _get_inclusion_settings(target_dir: str) -> Tuple[str, Optional[str]]:
+    """
+    Determine include/omit glob patterns for the coverage report for both text
+    and HTML coverage reports.
+
+    :param target_dir: directory for coverage stats; use "." to indicate all directories
+    :return: tuple (include_in_report, exclude_from_report) where
+        - include_in_report is the glob pattern to include
+        - exclude_from_report is a comma‑separated glob pattern to omit, or ``None``
+
+    Examples:
+        1. Cover everything (no submodules to omit):
+        ```
+        _get_inclusion_settings(".")
+        ```
+        ("*", "")
+
+        2. Only cover code under a specific directory:
+        ```
+        _get_inclusion_settings("helpers")
+        ```
+        ("*/helpers/*", None)
+
+    Usage in `_run_coverage`:
+
+         # Entire repo coverage (e.g. your 'helpers' project root):
+         ```
+        include, omit = _get_inclusion_settings(".")
+        ```
+        #   coverage report --include=* --sort=Cover
+        #   coverage html   --include=* [--omit=ext/foo/*,ext/bar/*]
+
+        # Single-directory coverage:
+        ```
+        include, omit = _get_inclusion_settings("helpers")
+        ```
+        #   coverage report --include=*/helpers/* --sort=Cover
+        #   coverage html   --include=*/helpers/* [--omit=...]
+    """
+    if target_dir == ".":
+        include_in_report = "*"
+        exclude_from_report = ""
+        if hserver.skip_submodules_test():
+            submodule_paths: List[str] = hgit.get_submodule_paths()
+            exclude_from_report = ",".join(
+                f"{path}/*" for path in submodule_paths
+            )
+    else:
+        include_in_report = f"*/{target_dir}/*"
+        exclude_from_report = None
+    return include_in_report, exclude_from_report
+
+
+def _run_coverage(
+    ctx,
+    suite: str,
+    target_dir: str,
+    generate_html_report: bool = False,
+) -> str:
+    """
+    Run coverage for a given suite (fast/slow/superslow).
+
+    :param suite: one of "fast", "slow" or "superslow"
+    :param target_dir: coverage target directory
+    :param generate_html_report: whether to produce HTML output
+    :return: xml file named 'coverage.xml'
+    """
+    hdbg.dassert_in(
+        suite,
+        ("fast", "slow", "superslow"),
+        "Expected one of 'fast', 'slow', or 'superslow'.",
+    )
+    # Build the command line.
+    test_cmd_parts = [
+        # Invoke the "<suite>_tests" task.
+        "invoke",
+        f"run_{suite}_tests",
+        # Enable coverage collection.
+        "--coverage",
+        # Specify which directory to test.
+        "-p", target_dir,
+    ]
+    # join and quote them into a single shell command
+    test_cmd = hlitauti.to_multi_line_cmd(test_cmd_parts)
+    # Run the tests under coverage.
+    hlitauti.run(ctx, test_cmd, use_system=False)
+    # Check that '.coverage' was created.
+    hdbg.dassert_file_exists(".coverage")
+    # Compute which files/dirs to include and omit in the report.
+    include_in_report, exclude_from_report = _get_inclusion_settings(target_dir)
+    report_cmd: List[str] = [
+        # Clear out any prior coverage data.
path: helpers/lib_tasks_pytest.py
user: login: Shaunak01
body: Actually, now I think It is better to keep coverage erase as a precaution to avoid accidental contamination if the coverage workflow changes in the future, such as running multiple suites together or merging reports. (cause its a possibility). No harm to keep it for sure
created_at: 2025-04-29T18:51:40Z
author_association: CONTRIBUTOR
diff_hunk: @@ -874,6 +874,143 @@ def run_coverage_report(  # type: ignore
         _publish_html_coverage_report_on_s3(aws_profile)
 
 
+def _get_inclusion_settings(target_dir: str) -> Tuple[str, Optional[str]]:
+    """
+    Determine include/omit glob patterns for the coverage report for both text
+    and HTML coverage reports.
+
+    :param target_dir: directory for coverage stats; use "." to indicate all directories
+    :return: tuple (include_in_report, exclude_from_report) where
+        - include_in_report is the glob pattern to include
+        - exclude_from_report is a comma‑separated glob pattern to omit, or ``None``
+
+    Examples:
+        1. Cover everything (no submodules to omit):
+        ```
+        _get_inclusion_settings(".")
+        ```
+        ("*", "")
+
+        2. Only cover code under a specific directory:
+        ```
+        _get_inclusion_settings("helpers")
+        ```
+        ("*/helpers/*", None)
+
+    Usage in `_run_coverage`:
+
+         # Entire repo coverage (e.g. your 'helpers' project root):
+         ```
+        include, omit = _get_inclusion_settings(".")
+        ```
+        #   coverage report --include=* --sort=Cover
+        #   coverage html   --include=* [--omit=ext/foo/*,ext/bar/*]
+
+        # Single-directory coverage:
+        ```
+        include, omit = _get_inclusion_settings("helpers")
+        ```
+        #   coverage report --include=*/helpers/* --sort=Cover
+        #   coverage html   --include=*/helpers/* [--omit=...]
+    """
+    if target_dir == ".":
+        include_in_report = "*"
+        exclude_from_report = ""
+        if hserver.skip_submodules_test():
+            submodule_paths: List[str] = hgit.get_submodule_paths()
+            exclude_from_report = ",".join(
+                f"{path}/*" for path in submodule_paths
+            )
+    else:
+        include_in_report = f"*/{target_dir}/*"
+        exclude_from_report = None
+    return include_in_report, exclude_from_report
+
+
+def _run_coverage(
+    ctx,
+    suite: str,
+    target_dir: str,
+    generate_html_report: bool = False,
+) -> str:
+    """
+    Run coverage for a given suite (fast/slow/superslow).
+
+    :param suite: one of "fast", "slow" or "superslow"
+    :param target_dir: coverage target directory
+    :param generate_html_report: whether to produce HTML output
+    :return: xml file named 'coverage.xml'
+    """
+    hdbg.dassert_in(
+        suite,
+        ("fast", "slow", "superslow"),
+        "Expected one of 'fast', 'slow', or 'superslow'.",
+    )
+    # Build the command line.
+    test_cmd_parts = [
+        # Invoke the "<suite>_tests" task.
+        "invoke",
+        f"run_{suite}_tests",
+        # Enable coverage collection.
+        "--coverage",
+        # Specify which directory to test.
+        "-p", target_dir,
+    ]
+    # join and quote them into a single shell command
+    test_cmd = hlitauti.to_multi_line_cmd(test_cmd_parts)
+    # Run the tests under coverage.
+    hlitauti.run(ctx, test_cmd, use_system=False)
+    # Check that '.coverage' was created.
+    hdbg.dassert_file_exists(".coverage")
+    # Compute which files/dirs to include and omit in the report.
+    include_in_report, exclude_from_report = _get_inclusion_settings(target_dir)
+    report_cmd: List[str] = [
+        # Clear out any prior coverage data.
path: helpers/lib_tasks_pytest.py
user: login: dremdem
body: SG.
created_at: 2025-04-29T19:00:43Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,147 @@
+name: Test coverage
+
+on:
+  workflow_dispatch: {}
+  # every day at 00:00 UTC.
+  schedule:
+    - cron: '0 0 * * *'
+
+env:
+  CSFY_CI: true
+
+permissions:
+  # Required to authenticate and retrieve temporary AWS credentials via OIDC.
+  id-token: write
+  # Required to fetch and check out code from the repository.
+  contents: read
+  # Required to authenticate and pull Docker images from GitHub Container Registry (GHCR).
+  packages: read
+
+
+jobs:
+  run_test_coverage:
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Configure AWS credentials
+        uses: aws-actions/configure-aws-credentials@v1
+        with:
+          role-to-assume: ${{ vars.GH_ACTION_AWS_ROLE_ARN }}
+          role-session-name: ${{ vars.GH_ACTION_AWS_SESSION_NAME }}
+          aws-region: ${{ vars.CSFY_AWS_DEFAULT_REGION }}
+
+      - name: Login to GHCR
+        run: docker login ghcr.io -u gpsaggese -p ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Cleanup
+        run: sudo chmod 777 -R .
+
+      - name: Checkout code
+        uses: actions/checkout@v3
+        with:
+          submodules: recursive
+          fetch-depth: 0
+          token: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Update PYTHONPATH
+        run: echo "PYTHONPATH=.:helpers" >> $GITHUB_ENV
+
+      - name: Install dependencies
+        run: |
+          python -m pip install --upgrade pip
+          pip install -r .github/gh_requirements.txt
+
+      - name: Pull image from GHCR
+        run: docker pull ghcr.io/${{ github.repository }}:dev
+
+      # Only on scheduled runs, capture ISO weekday (1=Mon … 7=Sun).
+      - name: Set DAY_OF_WEEK
+        if: github.event_name == 'schedule'
+        run: echo "DAY_OF_WEEK=$(date -u +'%u')" >> $GITHUB_ENV
+
+      # This step is used to trigger the fast test coverage generation using the invoke task.
+      - name: Run Fast test and generate report
+        id: run_fast
+        # `if: always()` ensures this step runs regardless of prior failures.
+        if: always()
path: .github/workflows/coverage_tests.yml
user: login: dremdem
body: I believe all of the previous steps are necessary for further processing. 
So if any of them fail then we need to fail whole workflow. 
WDYT? 
created_at: 2025-04-30T10:51:51Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,147 @@
+name: Test coverage
+
+on:
+  workflow_dispatch: {}
+  # every day at 00:00 UTC.
+  schedule:
+    - cron: '0 0 * * *'
+
+env:
+  CSFY_CI: true
+
+permissions:
+  # Required to authenticate and retrieve temporary AWS credentials via OIDC.
+  id-token: write
+  # Required to fetch and check out code from the repository.
+  contents: read
+  # Required to authenticate and pull Docker images from GitHub Container Registry (GHCR).
+  packages: read
+
+
+jobs:
+  run_test_coverage:
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Configure AWS credentials
+        uses: aws-actions/configure-aws-credentials@v1
+        with:
+          role-to-assume: ${{ vars.GH_ACTION_AWS_ROLE_ARN }}
+          role-session-name: ${{ vars.GH_ACTION_AWS_SESSION_NAME }}
+          aws-region: ${{ vars.CSFY_AWS_DEFAULT_REGION }}
+
+      - name: Login to GHCR
+        run: docker login ghcr.io -u gpsaggese -p ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Cleanup
+        run: sudo chmod 777 -R .
+
+      - name: Checkout code
+        uses: actions/checkout@v3
+        with:
+          submodules: recursive
+          fetch-depth: 0
+          token: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Update PYTHONPATH
+        run: echo "PYTHONPATH=.:helpers" >> $GITHUB_ENV
+
+      - name: Install dependencies
+        run: |
+          python -m pip install --upgrade pip
+          pip install -r .github/gh_requirements.txt
+
+      - name: Pull image from GHCR
+        run: docker pull ghcr.io/${{ github.repository }}:dev
+
+      # Only on scheduled runs, capture ISO weekday (1=Mon … 7=Sun).
+      - name: Set DAY_OF_WEEK
+        if: github.event_name == 'schedule'
+        run: echo "DAY_OF_WEEK=$(date -u +'%u')" >> $GITHUB_ENV
+
+      # This step is used to trigger the fast test coverage generation using the invoke task.
+      - name: Run Fast test and generate report
+        id: run_fast
+        # `if: always()` ensures this step runs regardless of prior failures.
+        if: always()
+        continue-on-error: true
path: .github/workflows/coverage_tests.yml
user: login: dremdem
body: I read the docs: 
https://docs.github.com/en/actions/writing-workflows/workflow-syntax-for-github-actions#jobsjob_idstepscontinue-on-error

I'm wondering, if: 
```
Prevents a job from failing when a step fails. Set to true to allow a job to pass when this step fails.
```
If some steps will fail then the job/workflow will fail or pass? 
Can you check pls.
created_at: 2025-04-30T11:01:38Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,147 @@
+name: Test coverage
+
+on:
+  workflow_dispatch: {}
+  # every day at 00:00 UTC.
+  schedule:
+    - cron: '0 0 * * *'
+
+env:
+  CSFY_CI: true
+
+permissions:
+  # Required to authenticate and retrieve temporary AWS credentials via OIDC.
+  id-token: write
+  # Required to fetch and check out code from the repository.
+  contents: read
+  # Required to authenticate and pull Docker images from GitHub Container Registry (GHCR).
+  packages: read
+
+
+jobs:
+  run_test_coverage:
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Configure AWS credentials
+        uses: aws-actions/configure-aws-credentials@v1
+        with:
+          role-to-assume: ${{ vars.GH_ACTION_AWS_ROLE_ARN }}
+          role-session-name: ${{ vars.GH_ACTION_AWS_SESSION_NAME }}
+          aws-region: ${{ vars.CSFY_AWS_DEFAULT_REGION }}
+
+      - name: Login to GHCR
+        run: docker login ghcr.io -u gpsaggese -p ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Cleanup
+        run: sudo chmod 777 -R .
+
+      - name: Checkout code
+        uses: actions/checkout@v3
+        with:
+          submodules: recursive
+          fetch-depth: 0
+          token: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Update PYTHONPATH
+        run: echo "PYTHONPATH=.:helpers" >> $GITHUB_ENV
+
+      - name: Install dependencies
+        run: |
+          python -m pip install --upgrade pip
+          pip install -r .github/gh_requirements.txt
+
+      - name: Pull image from GHCR
+        run: docker pull ghcr.io/${{ github.repository }}:dev
+
+      # Only on scheduled runs, capture ISO weekday (1=Mon … 7=Sun).
+      - name: Set DAY_OF_WEEK
+        if: github.event_name == 'schedule'
+        run: echo "DAY_OF_WEEK=$(date -u +'%u')" >> $GITHUB_ENV
+
+      # This step is used to trigger the fast test coverage generation using the invoke task.
+      - name: Run Fast test and generate report
+        id: run_fast
+        # `if: always()` ensures this step runs regardless of prior failures.
+        if: always()
+        continue-on-error: true
path: .github/workflows/coverage_tests.yml
user: login: dremdem
body: I was just thinking that if some tests fail, the job will still show as green, which is not the expected behavior.
created_at: 2025-04-30T11:25:21Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,147 @@
+name: Test coverage
+
+on:
+  workflow_dispatch: {}
+  # every day at 00:00 UTC.
+  schedule:
+    - cron: '0 0 * * *'
+
+env:
+  CSFY_CI: true
+
+permissions:
+  # Required to authenticate and retrieve temporary AWS credentials via OIDC.
+  id-token: write
+  # Required to fetch and check out code from the repository.
+  contents: read
+  # Required to authenticate and pull Docker images from GitHub Container Registry (GHCR).
+  packages: read
+
+
+jobs:
+  run_test_coverage:
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Configure AWS credentials
+        uses: aws-actions/configure-aws-credentials@v1
+        with:
+          role-to-assume: ${{ vars.GH_ACTION_AWS_ROLE_ARN }}
+          role-session-name: ${{ vars.GH_ACTION_AWS_SESSION_NAME }}
+          aws-region: ${{ vars.CSFY_AWS_DEFAULT_REGION }}
+
+      - name: Login to GHCR
+        run: docker login ghcr.io -u gpsaggese -p ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Cleanup
+        run: sudo chmod 777 -R .
+
+      - name: Checkout code
+        uses: actions/checkout@v3
+        with:
+          submodules: recursive
+          fetch-depth: 0
+          token: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Update PYTHONPATH
+        run: echo "PYTHONPATH=.:helpers" >> $GITHUB_ENV
+
+      - name: Install dependencies
+        run: |
+          python -m pip install --upgrade pip
+          pip install -r .github/gh_requirements.txt
+
+      - name: Pull image from GHCR
+        run: docker pull ghcr.io/${{ github.repository }}:dev
+
+      # Only on scheduled runs, capture ISO weekday (1=Mon … 7=Sun).
+      - name: Set DAY_OF_WEEK
+        if: github.event_name == 'schedule'
+        run: echo "DAY_OF_WEEK=$(date -u +'%u')" >> $GITHUB_ENV
+
+      # This step is used to trigger the fast test coverage generation using the invoke task.
+      - name: Run Fast test and generate report
+        id: run_fast
+        # `if: always()` ensures this step runs regardless of prior failures.
+        if: always()
+        continue-on-error: true
+        env:
+          GH_ACTION_ACCESS_TOKEN: ${{ secrets.GH_ACTION_ACCESS_TOKEN }}
+          CSFY_AWS_ACCESS_KEY_ID: ${{ env.AWS_ACCESS_KEY_ID }}
+          CSFY_AWS_SECRET_ACCESS_KEY: ${{ env.AWS_SECRET_ACCESS_KEY }}
+          CSFY_AWS_SESSION_TOKEN: ${{ env.AWS_SESSION_TOKEN }}
+          CSFY_AWS_DEFAULT_REGION: ${{ env.AWS_DEFAULT_REGION }}
+          CSFY_ECR_BASE_PATH: ghcr.io/${{ github.repository_owner }}
+          CSFY_AWS_S3_BUCKET: ${{ vars.CSFY_AWS_S3_BUCKET }}
+        run: invoke run_coverage --suite fast
+
+      - name: Upload Fast Test Coverage to Codecov
+        # Upload report if the fast test run successfully.
+        if: steps.run_fast.outcome == 'success'
path: .github/workflows/coverage_tests.yml
user: login: dremdem
body: Let's mention this in the docs and add in this comment why we are doing it.
As I got: we are not considering failed tests as a source data for the `Codecov`.
created_at: 2025-04-30T11:34:58Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,33 @@
+flag_management:
+  default_rules:
+    # Do not reuse last coverage if missing.
+    carryforward: false
+  individual_flags:
+    - name: fast
+      # Reuse last coverage if missing for true.
+      carryforward: true
+    - name: slow
+      carryforward: true
+    - name: superslow
+      carryforward: true
+
+
+# No per-line comments.
+comment: false
+
+coverage:
+  status:
+    project:
+      # Post the project‑level GitHub Check.
+      enabled: true
+      target_branch: master
+      # Measure % drop relative to baseline.
+      comparator: relative
+      # Fail if drop ≥1%.
+      threshold: 1
+      flags:
+        - fast
+        - slow
+        - superslow
+    patch:
path: codecov.yml
user: login: dremdem
body: ```
# Post patch-level coverage checks for pull requests.
```
This comment is very `Codecov` specific. Can you explain a bit more?
As I got from the ChatGPT explanation 
```
enabled: true: By setting enabled to true, this configuration activates patch-level coverage checks. When a pull request is created, Codecov will calculate the coverage for the modified lines and post the results as a comment or status check in the pull request. This helps developers quickly identify whether their changes are adequately tested.
```
this is not for `Codecov` UI, but for GitHub interface itself. 
Not sure we need it in the current iteration.
created_at: 2025-04-30T11:52:32Z
author_association: CONTRIBUTOR
diff_hunk: @@ -874,6 +874,135 @@ def run_coverage_report(  # type: ignore
         _publish_html_coverage_report_on_s3(aws_profile)
 
 
+def _get_inclusion_settings(target_dir: str) -> Tuple[str, Optional[str]]:
+    """
+    Determine include/omit glob patterns for the coverage report for both text
+    and HTML coverage reports.
+
+    :param target_dir: directory for coverage stats; use "." to indicate all directories
+    :return: glob pattern to include and a comma-separated glob pattern to omit
+
+    Examples:
+        1. Cover everything (no submodules to omit):
+        ```
+        _get_inclusion_settings(".")
+        ```
+        ("*", "")
+
+        2. Only cover code under a specific directory:
+        ```
+        _get_inclusion_settings("helpers")
+        ```
+        ("*/helpers/*", None)
+
+    Usage in `_run_coverage`:
+
+         # Entire repo coverage (e.g. your 'helpers' project root):
+         ```
+        include, omit = _get_inclusion_settings(".")
+        ```
+        #   coverage report --include=* --sort=Cover
+        #   coverage html   --include=* [--omit=submodule1/*,submodule2/*]
+
+        # Single-directory coverage:
+        ```
+        include, omit = _get_inclusion_settings("helpers")
+        ```
+        #   coverage report --include=*/helpers/* --sort=Cover
+        #   coverage html   --include=*/helpers/* [--omit=...]
+    """
+    if target_dir == ".":
+        include_in_report = "*"
+        exclude_from_report = ""
+        if hserver.skip_submodules_test():
+            submodule_paths: List[str] = hgit.get_submodule_paths()
+            exclude_from_report = ",".join(
+                f"{path}/*" for path in submodule_paths
+            )
+    else:
+        include_in_report = f"*/{target_dir}/*"
+        exclude_from_report = None
+    return include_in_report, exclude_from_report
+
+
+def _run_coverage(
+    ctx,
+    suite: str,
+    target_dir: str,
+    generate_html_report: bool = False,
+) -> str:
+    """
+    Run coverage for a given suite (fast/slow/superslow).
+
+    :param suite: one of "fast", "slow" or "superslow"
+    :param target_dir: coverage target directory
+    :param generate_html_report: whether to produce HTML output
+    :return: xml file named 'coverage.xml'
path: helpers/lib_tasks_pytest.py
user: login: dremdem
body: Not sure about it.
created_at: 2025-04-30T11:58:41Z
author_association: CONTRIBUTOR
diff_hunk: @@ -874,6 +874,135 @@ def run_coverage_report(  # type: ignore
         _publish_html_coverage_report_on_s3(aws_profile)
 
 
+def _get_inclusion_settings(target_dir: str) -> Tuple[str, Optional[str]]:
+    """
+    Determine include/omit glob patterns for the coverage report for both text
+    and HTML coverage reports.
+
+    :param target_dir: directory for coverage stats; use "." to indicate all directories
+    :return: glob pattern to include and a comma-separated glob pattern to omit
+
+    Examples:
+        1. Cover everything (no submodules to omit):
+        ```
+        _get_inclusion_settings(".")
+        ```
+        ("*", "")
+
+        2. Only cover code under a specific directory:
+        ```
+        _get_inclusion_settings("helpers")
+        ```
+        ("*/helpers/*", None)
+
+    Usage in `_run_coverage`:
+
+         # Entire repo coverage (e.g. your 'helpers' project root):
+         ```
+        include, omit = _get_inclusion_settings(".")
+        ```
+        #   coverage report --include=* --sort=Cover
+        #   coverage html   --include=* [--omit=submodule1/*,submodule2/*]
+
+        # Single-directory coverage:
+        ```
+        include, omit = _get_inclusion_settings("helpers")
+        ```
+        #   coverage report --include=*/helpers/* --sort=Cover
+        #   coverage html   --include=*/helpers/* [--omit=...]
+    """
+    if target_dir == ".":
+        include_in_report = "*"
+        exclude_from_report = ""
+        if hserver.skip_submodules_test():
+            submodule_paths: List[str] = hgit.get_submodule_paths()
+            exclude_from_report = ",".join(
+                f"{path}/*" for path in submodule_paths
+            )
+    else:
+        include_in_report = f"*/{target_dir}/*"
+        exclude_from_report = None
+    return include_in_report, exclude_from_report
+
+
+def _run_coverage(
+    ctx,
+    suite: str,
+    target_dir: str,
+    generate_html_report: bool = False,
+) -> str:
path: helpers/lib_tasks_pytest.py
user: login: dremdem
body: Can you check it please?
created_at: 2025-04-30T11:59:09Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,147 @@
+name: Test coverage
+
+on:
+  workflow_dispatch: {}
+  # every day at 00:00 UTC.
+  schedule:
+    - cron: '0 0 * * *'
+
+env:
+  CSFY_CI: true
+
+permissions:
+  # Required to authenticate and retrieve temporary AWS credentials via OIDC.
+  id-token: write
+  # Required to fetch and check out code from the repository.
+  contents: read
+  # Required to authenticate and pull Docker images from GitHub Container Registry (GHCR).
+  packages: read
+
+
+jobs:
+  run_test_coverage:
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Configure AWS credentials
+        uses: aws-actions/configure-aws-credentials@v1
+        with:
+          role-to-assume: ${{ vars.GH_ACTION_AWS_ROLE_ARN }}
+          role-session-name: ${{ vars.GH_ACTION_AWS_SESSION_NAME }}
+          aws-region: ${{ vars.CSFY_AWS_DEFAULT_REGION }}
+
+      - name: Login to GHCR
+        run: docker login ghcr.io -u gpsaggese -p ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Cleanup
+        run: sudo chmod 777 -R .
+
+      - name: Checkout code
+        uses: actions/checkout@v3
+        with:
+          submodules: recursive
+          fetch-depth: 0
+          token: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Update PYTHONPATH
+        run: echo "PYTHONPATH=.:helpers" >> $GITHUB_ENV
+
+      - name: Install dependencies
+        run: |
+          python -m pip install --upgrade pip
+          pip install -r .github/gh_requirements.txt
+
+      - name: Pull image from GHCR
+        run: docker pull ghcr.io/${{ github.repository }}:dev
+
+      # Only on scheduled runs, capture ISO weekday (1=Mon … 7=Sun).
+      - name: Set DAY_OF_WEEK
+        if: github.event_name == 'schedule'
+        run: echo "DAY_OF_WEEK=$(date -u +'%u')" >> $GITHUB_ENV
+
+      # This step is used to trigger the fast test coverage generation using the invoke task.
+      - name: Run Fast test and generate report
+        id: run_fast
+        # `if: always()` ensures this step runs regardless of prior failures.
+        if: always()
path: .github/workflows/coverage_tests.yml
user: login: Shaunak01
body: Added new architecture
created_at: 2025-04-30T18:32:16Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,147 @@
+name: Test coverage
+
+on:
+  workflow_dispatch: {}
+  # every day at 00:00 UTC.
+  schedule:
+    - cron: '0 0 * * *'
+
+env:
+  CSFY_CI: true
+
+permissions:
+  # Required to authenticate and retrieve temporary AWS credentials via OIDC.
+  id-token: write
+  # Required to fetch and check out code from the repository.
+  contents: read
+  # Required to authenticate and pull Docker images from GitHub Container Registry (GHCR).
+  packages: read
+
+
+jobs:
+  run_test_coverage:
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Configure AWS credentials
+        uses: aws-actions/configure-aws-credentials@v1
+        with:
+          role-to-assume: ${{ vars.GH_ACTION_AWS_ROLE_ARN }}
+          role-session-name: ${{ vars.GH_ACTION_AWS_SESSION_NAME }}
+          aws-region: ${{ vars.CSFY_AWS_DEFAULT_REGION }}
+
+      - name: Login to GHCR
+        run: docker login ghcr.io -u gpsaggese -p ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Cleanup
+        run: sudo chmod 777 -R .
+
+      - name: Checkout code
+        uses: actions/checkout@v3
+        with:
+          submodules: recursive
+          fetch-depth: 0
+          token: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Update PYTHONPATH
+        run: echo "PYTHONPATH=.:helpers" >> $GITHUB_ENV
+
+      - name: Install dependencies
+        run: |
+          python -m pip install --upgrade pip
+          pip install -r .github/gh_requirements.txt
+
+      - name: Pull image from GHCR
+        run: docker pull ghcr.io/${{ github.repository }}:dev
+
+      # Only on scheduled runs, capture ISO weekday (1=Mon … 7=Sun).
+      - name: Set DAY_OF_WEEK
+        if: github.event_name == 'schedule'
+        run: echo "DAY_OF_WEEK=$(date -u +'%u')" >> $GITHUB_ENV
+
+      # This step is used to trigger the fast test coverage generation using the invoke task.
+      - name: Run Fast test and generate report
+        id: run_fast
+        # `if: always()` ensures this step runs regardless of prior failures.
+        if: always()
+        continue-on-error: true
path: .github/workflows/coverage_tests.yml
user: login: Shaunak01
body: Now in new architecture,

Fast and slow test steps use `continue-on-error: true` so that all suites run regardless of individual failures (cause we want to run slow test even if fast fails). However, we added a final failure check step at the end that explicitly inspects the outcomes of `run_fast`, `upload_fast`, `run_slow`, and `upload_slow`. If any of these fail, the job fails with exit 1 and aborts, ensuring the workflow doesn't falsely succeed.

`superslow tests` are scheduled to run on Mondays or via manual trigger. These steps do not use `continue-on-error`, so if they fail, the job fails immediately.

Setup steps like dependency installation or AWS configuration are also not error-tolerant — if they fail, the job aborts before any tests run.


This should work and will add a note in the doc
created_at: 2025-04-30T18:36:30Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,33 @@
+flag_management:
+  default_rules:
+    # Do not reuse last coverage if missing.
+    carryforward: false
+  individual_flags:
+    - name: fast
+      # Reuse last coverage if missing for true.
+      carryforward: true
+    - name: slow
+      carryforward: true
+    - name: superslow
+      carryforward: true
+
+
+# No per-line comments.
+comment: false
+
+coverage:
+  status:
+    project:
+      # Post the project‑level GitHub Check.
+      enabled: true
+      target_branch: master
+      # Measure % drop relative to baseline.
+      comparator: relative
+      # Fail if drop ≥1%.
+      threshold: 1
+      flags:
+        - fast
+        - slow
+        - superslow
+    patch:
path: codecov.yml
user: login: Shaunak01
body: We are not going to keep coverage tests for all PRs, but GP does want it in cases he wants to see for a particular PR so lets keep it and I will update the comment
created_at: 2025-04-30T18:41:18Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,147 @@
+name: Test coverage
+
+on:
+  workflow_dispatch: {}
+  # every day at 00:00 UTC.
+  schedule:
+    - cron: '0 0 * * *'
+
+env:
+  CSFY_CI: true
+
+permissions:
+  # Required to authenticate and retrieve temporary AWS credentials via OIDC.
+  id-token: write
+  # Required to fetch and check out code from the repository.
+  contents: read
+  # Required to authenticate and pull Docker images from GitHub Container Registry (GHCR).
+  packages: read
+
+
+jobs:
+  run_test_coverage:
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Configure AWS credentials
+        uses: aws-actions/configure-aws-credentials@v1
+        with:
+          role-to-assume: ${{ vars.GH_ACTION_AWS_ROLE_ARN }}
+          role-session-name: ${{ vars.GH_ACTION_AWS_SESSION_NAME }}
+          aws-region: ${{ vars.CSFY_AWS_DEFAULT_REGION }}
+
+      - name: Login to GHCR
+        run: docker login ghcr.io -u gpsaggese -p ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Cleanup
+        run: sudo chmod 777 -R .
+
+      - name: Checkout code
+        uses: actions/checkout@v3
+        with:
+          submodules: recursive
+          fetch-depth: 0
+          token: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Update PYTHONPATH
+        run: echo "PYTHONPATH=.:helpers" >> $GITHUB_ENV
+
+      - name: Install dependencies
+        run: |
+          python -m pip install --upgrade pip
+          pip install -r .github/gh_requirements.txt
+
+      - name: Pull image from GHCR
+        run: docker pull ghcr.io/${{ github.repository }}:dev
+
+      # Only on scheduled runs, capture ISO weekday (1=Mon … 7=Sun).
+      - name: Set DAY_OF_WEEK
+        if: github.event_name == 'schedule'
+        run: echo "DAY_OF_WEEK=$(date -u +'%u')" >> $GITHUB_ENV
+
+      # This step is used to trigger the fast test coverage generation using the invoke task.
+      - name: Run Fast test and generate report
+        id: run_fast
+        # `if: always()` ensures this step runs regardless of prior failures.
+        if: always()
+        continue-on-error: true
+        env:
+          GH_ACTION_ACCESS_TOKEN: ${{ secrets.GH_ACTION_ACCESS_TOKEN }}
+          CSFY_AWS_ACCESS_KEY_ID: ${{ env.AWS_ACCESS_KEY_ID }}
+          CSFY_AWS_SECRET_ACCESS_KEY: ${{ env.AWS_SECRET_ACCESS_KEY }}
+          CSFY_AWS_SESSION_TOKEN: ${{ env.AWS_SESSION_TOKEN }}
+          CSFY_AWS_DEFAULT_REGION: ${{ env.AWS_DEFAULT_REGION }}
+          CSFY_ECR_BASE_PATH: ghcr.io/${{ github.repository_owner }}
+          CSFY_AWS_S3_BUCKET: ${{ vars.CSFY_AWS_S3_BUCKET }}
+        run: invoke run_coverage --suite fast
+
+      - name: Upload Fast Test Coverage to Codecov
+        # Upload report if the fast test run successfully.
+        if: steps.run_fast.outcome == 'success'
path: .github/workflows/coverage_tests.yml
user: login: Shaunak01
body: Isn't it self explanatory?

Upload report if the fast test run successfully.
The step runs only if the previous step i.e the report was generated. We are uploading if the report is generated or the test is skipped anyways and the job fails later
created_at: 2025-04-30T18:53:52Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,147 @@
+name: Test coverage
+
+on:
+  workflow_dispatch: {}
+  # every day at 00:00 UTC.
+  schedule:
+    - cron: '0 0 * * *'
+
+env:
+  CSFY_CI: true
+
+permissions:
+  # Required to authenticate and retrieve temporary AWS credentials via OIDC.
+  id-token: write
+  # Required to fetch and check out code from the repository.
+  contents: read
+  # Required to authenticate and pull Docker images from GitHub Container Registry (GHCR).
+  packages: read
+
+
+jobs:
+  run_test_coverage:
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Configure AWS credentials
+        uses: aws-actions/configure-aws-credentials@v1
+        with:
+          role-to-assume: ${{ vars.GH_ACTION_AWS_ROLE_ARN }}
+          role-session-name: ${{ vars.GH_ACTION_AWS_SESSION_NAME }}
+          aws-region: ${{ vars.CSFY_AWS_DEFAULT_REGION }}
+
+      - name: Login to GHCR
+        run: docker login ghcr.io -u gpsaggese -p ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Cleanup
+        run: sudo chmod 777 -R .
+
+      - name: Checkout code
+        uses: actions/checkout@v3
+        with:
+          submodules: recursive
+          fetch-depth: 0
+          token: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Update PYTHONPATH
+        run: echo "PYTHONPATH=.:helpers" >> $GITHUB_ENV
+
+      - name: Install dependencies
+        run: |
+          python -m pip install --upgrade pip
+          pip install -r .github/gh_requirements.txt
+
+      - name: Pull image from GHCR
+        run: docker pull ghcr.io/${{ github.repository }}:dev
+
+      # Only on scheduled runs, capture ISO weekday (1=Mon … 7=Sun).
+      - name: Set DAY_OF_WEEK
+        if: github.event_name == 'schedule'
+        run: echo "DAY_OF_WEEK=$(date -u +'%u')" >> $GITHUB_ENV
+
+      # This step is used to trigger the fast test coverage generation using the invoke task.
+      - name: Run Fast test and generate report
+        id: run_fast
+        # `if: always()` ensures this step runs regardless of prior failures.
+        if: always()
+        continue-on-error: true
+        env:
+          GH_ACTION_ACCESS_TOKEN: ${{ secrets.GH_ACTION_ACCESS_TOKEN }}
+          CSFY_AWS_ACCESS_KEY_ID: ${{ env.AWS_ACCESS_KEY_ID }}
+          CSFY_AWS_SECRET_ACCESS_KEY: ${{ env.AWS_SECRET_ACCESS_KEY }}
+          CSFY_AWS_SESSION_TOKEN: ${{ env.AWS_SESSION_TOKEN }}
+          CSFY_AWS_DEFAULT_REGION: ${{ env.AWS_DEFAULT_REGION }}
+          CSFY_ECR_BASE_PATH: ghcr.io/${{ github.repository_owner }}
+          CSFY_AWS_S3_BUCKET: ${{ vars.CSFY_AWS_S3_BUCKET }}
+        run: invoke run_coverage --suite fast
+
+      - name: Upload Fast Test Coverage to Codecov
+        # Upload report if the fast test run successfully.
+        if: steps.run_fast.outcome == 'success'
path: .github/workflows/coverage_tests.yml
user: login: dremdem
body: > Isn't it self explanatory?

It’s not very clear. For example, if some of the fast tests fail, then the “Run Fast Test and Generate Report” step will also fail.
As a result, the “Upload Fast Test Coverage to Codecov” step won’t run.

Summary: If some tests fail, we won’t be able to use this run in `Codecov`.
created_at: 2025-05-01T12:46:09Z
author_association: CONTRIBUTOR
diff_hunk: @@ -874,6 +874,127 @@ def run_coverage_report(  # type: ignore
         _publish_html_coverage_report_on_s3(aws_profile)
 
 
+def _get_inclusion_settings(target_dir: str) -> Tuple[str, Optional[str]]:
+    """
+    Determine include/omit glob patterns for the coverage report for both text
+    and HTML coverage reports.
+
+    :param target_dir: directory for coverage stats; use "." to indicate all directories
+    :return: glob pattern to include and a comma-separated glob pattern to omit
+
+    Examples:
+        1. Cover everything (no submodules to omit):
+        ```
+        _get_inclusion_settings(".")
+        ```
+        ("*", "")
+
+        2. Only cover code under a specific directory:
+        ```
+        _get_inclusion_settings("helpers")
+        ```
+        ("*/helpers/*", None)
+
+    Usage in `_run_coverage`:
+
+         # Entire repo coverage (e.g. your 'helpers' project root):
+         ```
+        include, omit = _get_inclusion_settings(".")
+        ```
+        #   coverage report --include=* --sort=Cover
+        #   coverage html   --include=* [--omit=submodule1/*,submodule2/*]
+
+        # Single-directory coverage:
+        ```
+        include, omit = _get_inclusion_settings("helpers")
+        ```
+        #   coverage report --include=*/helpers/* --sort=Cover
+        #   coverage html   --include=*/helpers/* [--omit=...]
+    """
+    if target_dir == ".":
+        include_in_report = "*"
+        exclude_from_report = ""
+        if hserver.skip_submodules_test():
+            submodule_paths: List[str] = hgit.get_submodule_paths()
+            exclude_from_report = ",".join(
+                f"{path}/*" for path in submodule_paths
+            )
+    else:
+        include_in_report = f"*/{target_dir}/*"
+        exclude_from_report = None
+    return include_in_report, exclude_from_report
+
+
+def _run_coverage(
+    ctx,
+    suite: str,
+    target_dir: str,
+    generate_html_report: bool = False,
+) -> None:
+    """
+    Run coverage for a given suite (fast/slow/superslow).
+
+    :param suite: one of "fast", "slow" or "superslow"
+    :param target_dir: coverage target directory
+    :param generate_html_report: whether to produce HTML output
+    """
+    hdbg.dassert_in(suite, ("fast", "slow", "superslow"))
+    # Build the command line.
+    test_cmd_parts = [
+        # Invoke the "<suite>_tests" task.
+        "invoke",
+        f"run_{suite}_tests",
+        # Enable coverage computation.
+        "--coverage",
+        # Specify which directory to test.
+        "-p", target_dir,
+    ]
+    test_cmd = hlitauti.to_multi_line_cmd(test_cmd_parts)
+    # Run the tests under coverage.
+    hlitauti.run(ctx, test_cmd, use_system=False)
+    hdbg.dassert_file_exists(".coverage")
+    # Compute which files/dirs to include and omit in the report.
+    include_in_report, exclude_from_report = _get_inclusion_settings(target_dir)
+    report_cmd: List[str] = [
+        # Reset any previous coverage data to avoid contamination.
+        "coverage erase"
+    ]
+    # Generate a text report, including only our target paths.
+    report_stats_cmd: str = (
+        f"coverage report --include={include_in_report} --sort=Cover"
+    )
+    if exclude_from_report:
+        report_stats_cmd += f" --omit={exclude_from_report}"
+    report_cmd.append(report_stats_cmd)
+    # Produce HTML output for interactive browsing.
+    if generate_html_report:
+        report_html_cmd: str = f"coverage html --include={include_in_report}"
+        if exclude_from_report:
+            report_html_cmd += f" --omit={exclude_from_report}"
+        report_cmd.append(report_html_cmd)
+    # Export XML coverage report to integrate with Codecov.
+    report_cmd.append("coverage xml -o coverage.xml")
+    full_report_cmd: str = " && ".join(report_cmd)
+    docker_cmd_ = f"invoke docker_cmd --use-bash --cmd '{full_report_cmd}'"
+    # Execute the full coverage/Docker pipeline.
+    hlitauti.run(ctx, docker_cmd_)
+
+
+@task
+def run_coverage(
+    ctx, suite: str, target_dir: str = ".", generate_html_report: bool = False
+):
+    """
+    Unified task to run coverage for any test suite.
+
+    :param ctx: invoke context
+    :param suite: suite to run ("fast", "slow", "superslow")
+    :param target_dir: directory to measure coverage
+    """
+    hdbg.dassert_in(suite, ("fast", "slow", "superslow"))
+    _run_coverage(ctx, suite, target_dir, generate_html_report)
path: helpers/lib_tasks_pytest.py
user: login: dremdem
body: Are we still need this helper giving that it's calling only once?
Probably better to move the body of `_run_coverage()` here.
created_at: 2025-05-01T12:55:17Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,146 @@
+name: Test coverage
+
+on:
+  workflow_dispatch: {}
+  # every day at 00:00 UTC.
+  schedule:
+    - cron: '0 0 * * *'
+
+env:
+  CSFY_CI: true
+
+permissions:
+  # Required to authenticate and retrieve temporary AWS credentials via OIDC.
+  id-token: write
+  # Required to fetch and check out code from the repository.
+  contents: read
+  # Required to authenticate and pull Docker images from GitHub Container Registry (GHCR).
+  packages: read
+
+
+jobs:
+  run_test_coverage:
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Configure AWS credentials
+        uses: aws-actions/configure-aws-credentials@v1
+        with:
+          role-to-assume: ${{ vars.GH_ACTION_AWS_ROLE_ARN }}
+          role-session-name: ${{ vars.GH_ACTION_AWS_SESSION_NAME }}
+          aws-region: ${{ vars.CSFY_AWS_DEFAULT_REGION }}
+
+      - name: Login to GHCR
+        run: docker login ghcr.io -u gpsaggese -p ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Cleanup
+        run: sudo chmod 777 -R .
+
+      - name: Checkout code
+        uses: actions/checkout@v3
+        with:
+          submodules: recursive
+          fetch-depth: 0
+          token: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Update PYTHONPATH
+        run: echo "PYTHONPATH=.:helpers" >> $GITHUB_ENV
+
+      - name: Install dependencies
+        run: |
+          python -m pip install --upgrade pip
+          pip install -r .github/gh_requirements.txt
+
+      - name: Pull image from GHCR
+        run: docker pull ghcr.io/${{ github.repository }}:dev
+
+      # Only on scheduled runs, capture ISO weekday (1=Mon … 7=Sun).
+      - name: Set DAY_OF_WEEK
+        if: github.event_name == 'schedule'
+        run: echo "DAY_OF_WEEK=$(date -u +'%u')" >> $GITHUB_ENV
+
+      # This step is used to trigger the fast test coverage generation using the invoke task.
+      - name: Run Fast test and generate report
+        id: run_fast
+        continue-on-error: true
+        env:
+          GH_ACTION_ACCESS_TOKEN: ${{ secrets.GH_ACTION_ACCESS_TOKEN }}
+          CSFY_AWS_ACCESS_KEY_ID: ${{ env.AWS_ACCESS_KEY_ID }}
+          CSFY_AWS_SECRET_ACCESS_KEY: ${{ env.AWS_SECRET_ACCESS_KEY }}
+          CSFY_AWS_SESSION_TOKEN: ${{ env.AWS_SESSION_TOKEN }}
+          CSFY_AWS_DEFAULT_REGION: ${{ env.AWS_DEFAULT_REGION }}
+          CSFY_ECR_BASE_PATH: ghcr.io/${{ github.repository_owner }}
+          CSFY_AWS_S3_BUCKET: ${{ vars.CSFY_AWS_S3_BUCKET }}
+        run: invoke run_coverage --suite fast
+
+      - name: Upload Fast Test Coverage to Codecov
+        id: upload_fast
+        # Upload report if the fast test run successfully and report is generated.
+        if: steps.run_fast.outcome == 'success'
+        continue-on-error: true
+        uses: codecov/codecov-action@v5
+        with:
+          token: ${{ secrets.CODECOV_TOKEN }}
+          files: ./coverage.xml
+          # Specify the Codecov flag name associated with this test suite.
+          # Required to separate coverage reports by type (e.g., fast, slow, superslow) inside the Codecov UI.
+          flags: fast
+          name: fast-test-coverage
+
+      - name: Run Slow test and generate report
+        id: run_slow
+        continue-on-error: true
+        env:
+          GH_ACTION_ACCESS_TOKEN: ${{ secrets.GH_ACTION_ACCESS_TOKEN }}
+          CSFY_AWS_ACCESS_KEY_ID: ${{ env.AWS_ACCESS_KEY_ID }}
+          CSFY_AWS_SECRET_ACCESS_KEY: ${{ env.AWS_SECRET_ACCESS_KEY }}
+          CSFY_AWS_SESSION_TOKEN: ${{ env.AWS_SESSION_TOKEN }}
+          CSFY_AWS_DEFAULT_REGION: ${{ env.AWS_DEFAULT_REGION }}
+          CSFY_ECR_BASE_PATH: ghcr.io/${{ github.repository_owner }}
+          CSFY_AWS_S3_BUCKET: ${{ vars.CSFY_AWS_S3_BUCKET }}
+        run: invoke run_coverage --suite slow
+
+      - name: Upload Slow Test Coverage to Codecov
+        id: upload_slow
+        if: steps.run_slow.outcome == 'success'
+        continue-on-error: true
+        uses: codecov/codecov-action@v5
+        with:
+          token: ${{ secrets.CODECOV_TOKEN }}
+          files: ./coverage.xml
+          flags: slow
+          name: slow-test-coverage
+
+      - name: Run Superslow test and generate report
+        # Scheduled for Monday (i.e Day 1) or when manually triggered.
+        if: (github.event_name == 'schedule' && env.DAY_OF_WEEK == '1') || github.event_name == 'workflow_dispatch'
path: .github/workflows/coverage_tests.yml
user: login: dremdem
body: I'd get rid of extra step "Set DAY_OF_WEEK" and put all the conditions inside the step. 
It could something like: 
```
run: |
         day_of_week=$(date +%u)
         if [ "$day_of_week" == "1" ] || [ "${{ github.event_name }}" == "workflow_dispatch" ]; then
            invoke run_coverage --suite superslow
         else
            exit 1
         fi
```
created_at: 2025-05-01T13:24:53Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,146 @@
+name: Test coverage
+
+on:
+  workflow_dispatch: {}
+  # every day at 00:00 UTC.
+  schedule:
+    - cron: '0 0 * * *'
+
+env:
+  CSFY_CI: true
+
+permissions:
+  # Required to authenticate and retrieve temporary AWS credentials via OIDC.
+  id-token: write
+  # Required to fetch and check out code from the repository.
+  contents: read
+  # Required to authenticate and pull Docker images from GitHub Container Registry (GHCR).
+  packages: read
+
+
+jobs:
+  run_test_coverage:
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Configure AWS credentials
+        uses: aws-actions/configure-aws-credentials@v1
+        with:
+          role-to-assume: ${{ vars.GH_ACTION_AWS_ROLE_ARN }}
+          role-session-name: ${{ vars.GH_ACTION_AWS_SESSION_NAME }}
+          aws-region: ${{ vars.CSFY_AWS_DEFAULT_REGION }}
+
+      - name: Login to GHCR
+        run: docker login ghcr.io -u gpsaggese -p ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Cleanup
+        run: sudo chmod 777 -R .
+
+      - name: Checkout code
+        uses: actions/checkout@v3
+        with:
+          submodules: recursive
+          fetch-depth: 0
+          token: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Update PYTHONPATH
+        run: echo "PYTHONPATH=.:helpers" >> $GITHUB_ENV
+
+      - name: Install dependencies
+        run: |
+          python -m pip install --upgrade pip
+          pip install -r .github/gh_requirements.txt
+
+      - name: Pull image from GHCR
+        run: docker pull ghcr.io/${{ github.repository }}:dev
+
+      # Only on scheduled runs, capture ISO weekday (1=Mon … 7=Sun).
+      - name: Set DAY_OF_WEEK
+        if: github.event_name == 'schedule'
+        run: echo "DAY_OF_WEEK=$(date -u +'%u')" >> $GITHUB_ENV
+
+      # This step is used to trigger the fast test coverage generation using the invoke task.
+      - name: Run Fast test and generate report
+        id: run_fast
+        continue-on-error: true
+        env:
+          GH_ACTION_ACCESS_TOKEN: ${{ secrets.GH_ACTION_ACCESS_TOKEN }}
+          CSFY_AWS_ACCESS_KEY_ID: ${{ env.AWS_ACCESS_KEY_ID }}
+          CSFY_AWS_SECRET_ACCESS_KEY: ${{ env.AWS_SECRET_ACCESS_KEY }}
+          CSFY_AWS_SESSION_TOKEN: ${{ env.AWS_SESSION_TOKEN }}
+          CSFY_AWS_DEFAULT_REGION: ${{ env.AWS_DEFAULT_REGION }}
+          CSFY_ECR_BASE_PATH: ghcr.io/${{ github.repository_owner }}
+          CSFY_AWS_S3_BUCKET: ${{ vars.CSFY_AWS_S3_BUCKET }}
+        run: invoke run_coverage --suite fast
+
+      - name: Upload Fast Test Coverage to Codecov
+        id: upload_fast
+        # Upload report if the fast test run successfully and report is generated.
+        if: steps.run_fast.outcome == 'success'
+        continue-on-error: true
+        uses: codecov/codecov-action@v5
+        with:
+          token: ${{ secrets.CODECOV_TOKEN }}
+          files: ./coverage.xml
+          # Specify the Codecov flag name associated with this test suite.
+          # Required to separate coverage reports by type (e.g., fast, slow, superslow) inside the Codecov UI.
+          flags: fast
+          name: fast-test-coverage
+
+      - name: Run Slow test and generate report
+        id: run_slow
+        continue-on-error: true
+        env:
+          GH_ACTION_ACCESS_TOKEN: ${{ secrets.GH_ACTION_ACCESS_TOKEN }}
+          CSFY_AWS_ACCESS_KEY_ID: ${{ env.AWS_ACCESS_KEY_ID }}
+          CSFY_AWS_SECRET_ACCESS_KEY: ${{ env.AWS_SECRET_ACCESS_KEY }}
+          CSFY_AWS_SESSION_TOKEN: ${{ env.AWS_SESSION_TOKEN }}
+          CSFY_AWS_DEFAULT_REGION: ${{ env.AWS_DEFAULT_REGION }}
+          CSFY_ECR_BASE_PATH: ghcr.io/${{ github.repository_owner }}
+          CSFY_AWS_S3_BUCKET: ${{ vars.CSFY_AWS_S3_BUCKET }}
+        run: invoke run_coverage --suite slow
+
+      - name: Upload Slow Test Coverage to Codecov
+        id: upload_slow
+        if: steps.run_slow.outcome == 'success'
+        continue-on-error: true
+        uses: codecov/codecov-action@v5
+        with:
+          token: ${{ secrets.CODECOV_TOKEN }}
+          files: ./coverage.xml
+          flags: slow
+          name: slow-test-coverage
+
+      - name: Run Superslow test and generate report
+        # Scheduled for Monday (i.e Day 1) or when manually triggered.
+        if: (github.event_name == 'schedule' && env.DAY_OF_WEEK == '1') || github.event_name == 'workflow_dispatch'
+        env:
+          GH_ACTION_ACCESS_TOKEN: ${{ secrets.GH_ACTION_ACCESS_TOKEN }}
+          CSFY_AWS_ACCESS_KEY_ID: ${{ env.AWS_ACCESS_KEY_ID }}
+          CSFY_AWS_SECRET_ACCESS_KEY: ${{ env.AWS_SECRET_ACCESS_KEY }}
+          CSFY_AWS_SESSION_TOKEN: ${{ env.AWS_SESSION_TOKEN }}
+          CSFY_AWS_DEFAULT_REGION: ${{ env.AWS_DEFAULT_REGION }}
+          CSFY_ECR_BASE_PATH: ghcr.io/${{ github.repository_owner }}
+          CSFY_AWS_S3_BUCKET: ${{ vars.CSFY_AWS_S3_BUCKET }}
+        run: invoke run_coverage --suite superslow
+
+      - name: Upload Superslow Test Coverage to Codecov
+        #TODO(Shaunak): Consider removing it when we turn this workflow into a reusable one.
+        if: steps.run_superslow.outcome == 'success'
+        uses: codecov/codecov-action@v5
+        with:
+          token: ${{ secrets.CODECOV_TOKEN }}
+          files: ./coverage.xml
+          flags: superslow
+          name: superslow-test-coverage
+
+        # Fail the job in CI if any of the fast/ slow run/ upload steps above failed.
+      - name: Fail if fast/slow test or upload failed
path: .github/workflows/coverage_tests.yml
user: login: dremdem
body: Did you check it? Is it works? 
created_at: 2025-05-01T13:27:50Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,147 @@
+name: Test coverage
+
+on:
+  workflow_dispatch: {}
+  # every day at 00:00 UTC.
+  schedule:
+    - cron: '0 0 * * *'
+
+env:
+  CSFY_CI: true
+
+permissions:
+  # Required to authenticate and retrieve temporary AWS credentials via OIDC.
+  id-token: write
+  # Required to fetch and check out code from the repository.
+  contents: read
+  # Required to authenticate and pull Docker images from GitHub Container Registry (GHCR).
+  packages: read
+
+
+jobs:
+  run_test_coverage:
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Configure AWS credentials
+        uses: aws-actions/configure-aws-credentials@v1
+        with:
+          role-to-assume: ${{ vars.GH_ACTION_AWS_ROLE_ARN }}
+          role-session-name: ${{ vars.GH_ACTION_AWS_SESSION_NAME }}
+          aws-region: ${{ vars.CSFY_AWS_DEFAULT_REGION }}
+
+      - name: Login to GHCR
+        run: docker login ghcr.io -u gpsaggese -p ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Cleanup
+        run: sudo chmod 777 -R .
+
+      - name: Checkout code
+        uses: actions/checkout@v3
+        with:
+          submodules: recursive
+          fetch-depth: 0
+          token: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Update PYTHONPATH
+        run: echo "PYTHONPATH=.:helpers" >> $GITHUB_ENV
+
+      - name: Install dependencies
+        run: |
+          python -m pip install --upgrade pip
+          pip install -r .github/gh_requirements.txt
+
+      - name: Pull image from GHCR
+        run: docker pull ghcr.io/${{ github.repository }}:dev
+
+      # Only on scheduled runs, capture ISO weekday (1=Mon … 7=Sun).
+      - name: Set DAY_OF_WEEK
+        if: github.event_name == 'schedule'
+        run: echo "DAY_OF_WEEK=$(date -u +'%u')" >> $GITHUB_ENV
+
+      # This step is used to trigger the fast test coverage generation using the invoke task.
+      - name: Run Fast test and generate report
+        id: run_fast
+        # `if: always()` ensures this step runs regardless of prior failures.
+        if: always()
+        continue-on-error: true
+        env:
+          GH_ACTION_ACCESS_TOKEN: ${{ secrets.GH_ACTION_ACCESS_TOKEN }}
+          CSFY_AWS_ACCESS_KEY_ID: ${{ env.AWS_ACCESS_KEY_ID }}
+          CSFY_AWS_SECRET_ACCESS_KEY: ${{ env.AWS_SECRET_ACCESS_KEY }}
+          CSFY_AWS_SESSION_TOKEN: ${{ env.AWS_SESSION_TOKEN }}
+          CSFY_AWS_DEFAULT_REGION: ${{ env.AWS_DEFAULT_REGION }}
+          CSFY_ECR_BASE_PATH: ghcr.io/${{ github.repository_owner }}
+          CSFY_AWS_S3_BUCKET: ${{ vars.CSFY_AWS_S3_BUCKET }}
+        run: invoke run_coverage --suite fast
+
+      - name: Upload Fast Test Coverage to Codecov
+        # Upload report if the fast test run successfully.
+        if: steps.run_fast.outcome == 'success'
path: .github/workflows/coverage_tests.yml
user: login: Shaunak01
body: > It’s not very clear. For example, if some of the fast tests fail, then the “Run Fast Test and Generate Report” step will also fail.

For whatever reason “Run Fast Test and Generate Report” fails, we dont want to run the upload step.
- Upload step (Run if report generated or skip)

Thats all it meas,
I will clarify the comment to
```
# Only upload if the previous fast test run step succeeded (i.r report generated).
# failed step don’t generate a coverage report, so there's nothing to upload.
```

created_at: 2025-05-01T14:33:35Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,146 @@
+name: Test coverage
+
+on:
+  workflow_dispatch: {}
+  # every day at 00:00 UTC.
+  schedule:
+    - cron: '0 0 * * *'
+
+env:
+  CSFY_CI: true
+
+permissions:
+  # Required to authenticate and retrieve temporary AWS credentials via OIDC.
+  id-token: write
+  # Required to fetch and check out code from the repository.
+  contents: read
+  # Required to authenticate and pull Docker images from GitHub Container Registry (GHCR).
+  packages: read
+
+
+jobs:
+  run_test_coverage:
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Configure AWS credentials
+        uses: aws-actions/configure-aws-credentials@v1
+        with:
+          role-to-assume: ${{ vars.GH_ACTION_AWS_ROLE_ARN }}
+          role-session-name: ${{ vars.GH_ACTION_AWS_SESSION_NAME }}
+          aws-region: ${{ vars.CSFY_AWS_DEFAULT_REGION }}
+
+      - name: Login to GHCR
+        run: docker login ghcr.io -u gpsaggese -p ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Cleanup
+        run: sudo chmod 777 -R .
+
+      - name: Checkout code
+        uses: actions/checkout@v3
+        with:
+          submodules: recursive
+          fetch-depth: 0
+          token: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Update PYTHONPATH
+        run: echo "PYTHONPATH=.:helpers" >> $GITHUB_ENV
+
+      - name: Install dependencies
+        run: |
+          python -m pip install --upgrade pip
+          pip install -r .github/gh_requirements.txt
+
+      - name: Pull image from GHCR
+        run: docker pull ghcr.io/${{ github.repository }}:dev
+
+      # Only on scheduled runs, capture ISO weekday (1=Mon … 7=Sun).
+      - name: Set DAY_OF_WEEK
+        if: github.event_name == 'schedule'
+        run: echo "DAY_OF_WEEK=$(date -u +'%u')" >> $GITHUB_ENV
+
+      # This step is used to trigger the fast test coverage generation using the invoke task.
+      - name: Run Fast test and generate report
+        id: run_fast
+        continue-on-error: true
+        env:
+          GH_ACTION_ACCESS_TOKEN: ${{ secrets.GH_ACTION_ACCESS_TOKEN }}
+          CSFY_AWS_ACCESS_KEY_ID: ${{ env.AWS_ACCESS_KEY_ID }}
+          CSFY_AWS_SECRET_ACCESS_KEY: ${{ env.AWS_SECRET_ACCESS_KEY }}
+          CSFY_AWS_SESSION_TOKEN: ${{ env.AWS_SESSION_TOKEN }}
+          CSFY_AWS_DEFAULT_REGION: ${{ env.AWS_DEFAULT_REGION }}
+          CSFY_ECR_BASE_PATH: ghcr.io/${{ github.repository_owner }}
+          CSFY_AWS_S3_BUCKET: ${{ vars.CSFY_AWS_S3_BUCKET }}
+        run: invoke run_coverage --suite fast
+
+      - name: Upload Fast Test Coverage to Codecov
+        id: upload_fast
+        # Upload report if the fast test run successfully and report is generated.
+        if: steps.run_fast.outcome == 'success'
+        continue-on-error: true
+        uses: codecov/codecov-action@v5
+        with:
+          token: ${{ secrets.CODECOV_TOKEN }}
+          files: ./coverage.xml
+          # Specify the Codecov flag name associated with this test suite.
+          # Required to separate coverage reports by type (e.g., fast, slow, superslow) inside the Codecov UI.
+          flags: fast
+          name: fast-test-coverage
+
+      - name: Run Slow test and generate report
+        id: run_slow
+        continue-on-error: true
+        env:
+          GH_ACTION_ACCESS_TOKEN: ${{ secrets.GH_ACTION_ACCESS_TOKEN }}
+          CSFY_AWS_ACCESS_KEY_ID: ${{ env.AWS_ACCESS_KEY_ID }}
+          CSFY_AWS_SECRET_ACCESS_KEY: ${{ env.AWS_SECRET_ACCESS_KEY }}
+          CSFY_AWS_SESSION_TOKEN: ${{ env.AWS_SESSION_TOKEN }}
+          CSFY_AWS_DEFAULT_REGION: ${{ env.AWS_DEFAULT_REGION }}
+          CSFY_ECR_BASE_PATH: ghcr.io/${{ github.repository_owner }}
+          CSFY_AWS_S3_BUCKET: ${{ vars.CSFY_AWS_S3_BUCKET }}
+        run: invoke run_coverage --suite slow
+
+      - name: Upload Slow Test Coverage to Codecov
+        id: upload_slow
+        if: steps.run_slow.outcome == 'success'
+        continue-on-error: true
+        uses: codecov/codecov-action@v5
+        with:
+          token: ${{ secrets.CODECOV_TOKEN }}
+          files: ./coverage.xml
+          flags: slow
+          name: slow-test-coverage
+
+      - name: Run Superslow test and generate report
+        # Scheduled for Monday (i.e Day 1) or when manually triggered.
+        if: (github.event_name == 'schedule' && env.DAY_OF_WEEK == '1') || github.event_name == 'workflow_dispatch'
+        env:
+          GH_ACTION_ACCESS_TOKEN: ${{ secrets.GH_ACTION_ACCESS_TOKEN }}
+          CSFY_AWS_ACCESS_KEY_ID: ${{ env.AWS_ACCESS_KEY_ID }}
+          CSFY_AWS_SECRET_ACCESS_KEY: ${{ env.AWS_SECRET_ACCESS_KEY }}
+          CSFY_AWS_SESSION_TOKEN: ${{ env.AWS_SESSION_TOKEN }}
+          CSFY_AWS_DEFAULT_REGION: ${{ env.AWS_DEFAULT_REGION }}
+          CSFY_ECR_BASE_PATH: ghcr.io/${{ github.repository_owner }}
+          CSFY_AWS_S3_BUCKET: ${{ vars.CSFY_AWS_S3_BUCKET }}
+        run: invoke run_coverage --suite superslow
+
+      - name: Upload Superslow Test Coverage to Codecov
+        #TODO(Shaunak): Consider removing it when we turn this workflow into a reusable one.
+        if: steps.run_superslow.outcome == 'success'
+        uses: codecov/codecov-action@v5
+        with:
+          token: ${{ secrets.CODECOV_TOKEN }}
+          files: ./coverage.xml
+          flags: superslow
+          name: superslow-test-coverage
+
+        # Fail the job in CI if any of the fast/ slow run/ upload steps above failed.
+      - name: Fail if fast/slow test or upload failed
path: .github/workflows/coverage_tests.yml
user: login: Shaunak01
body: ![image](https://github.com/user-attachments/assets/fcc03746-478d-463a-ad10-914674b0a3d9)
Yess works nicee
created_at: 2025-05-01T15:17:57Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,146 @@
+name: Test coverage
+
+on:
+  workflow_dispatch: {}
+  # every day at 00:00 UTC.
+  schedule:
+    - cron: '0 0 * * *'
+
+env:
+  CSFY_CI: true
+
+permissions:
+  # Required to authenticate and retrieve temporary AWS credentials via OIDC.
+  id-token: write
+  # Required to fetch and check out code from the repository.
+  contents: read
+  # Required to authenticate and pull Docker images from GitHub Container Registry (GHCR).
+  packages: read
+
+
+jobs:
+  run_test_coverage:
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Configure AWS credentials
+        uses: aws-actions/configure-aws-credentials@v1
+        with:
+          role-to-assume: ${{ vars.GH_ACTION_AWS_ROLE_ARN }}
+          role-session-name: ${{ vars.GH_ACTION_AWS_SESSION_NAME }}
+          aws-region: ${{ vars.CSFY_AWS_DEFAULT_REGION }}
+
+      - name: Login to GHCR
+        run: docker login ghcr.io -u gpsaggese -p ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Cleanup
+        run: sudo chmod 777 -R .
+
+      - name: Checkout code
+        uses: actions/checkout@v3
+        with:
+          submodules: recursive
+          fetch-depth: 0
+          token: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Update PYTHONPATH
+        run: echo "PYTHONPATH=.:helpers" >> $GITHUB_ENV
+
+      - name: Install dependencies
+        run: |
+          python -m pip install --upgrade pip
+          pip install -r .github/gh_requirements.txt
+
+      - name: Pull image from GHCR
+        run: docker pull ghcr.io/${{ github.repository }}:dev
+
+      # Only on scheduled runs, capture ISO weekday (1=Mon … 7=Sun).
+      - name: Set DAY_OF_WEEK
+        if: github.event_name == 'schedule'
+        run: echo "DAY_OF_WEEK=$(date -u +'%u')" >> $GITHUB_ENV
+
+      # This step is used to trigger the fast test coverage generation using the invoke task.
+      - name: Run Fast test and generate report
+        id: run_fast
+        continue-on-error: true
+        env:
+          GH_ACTION_ACCESS_TOKEN: ${{ secrets.GH_ACTION_ACCESS_TOKEN }}
+          CSFY_AWS_ACCESS_KEY_ID: ${{ env.AWS_ACCESS_KEY_ID }}
+          CSFY_AWS_SECRET_ACCESS_KEY: ${{ env.AWS_SECRET_ACCESS_KEY }}
+          CSFY_AWS_SESSION_TOKEN: ${{ env.AWS_SESSION_TOKEN }}
+          CSFY_AWS_DEFAULT_REGION: ${{ env.AWS_DEFAULT_REGION }}
+          CSFY_ECR_BASE_PATH: ghcr.io/${{ github.repository_owner }}
+          CSFY_AWS_S3_BUCKET: ${{ vars.CSFY_AWS_S3_BUCKET }}
+        run: invoke run_coverage --suite fast
+
+      - name: Upload Fast Test Coverage to Codecov
+        id: upload_fast
+        # Upload report if the fast test run successfully and report is generated.
+        if: steps.run_fast.outcome == 'success'
+        continue-on-error: true
+        uses: codecov/codecov-action@v5
+        with:
+          token: ${{ secrets.CODECOV_TOKEN }}
+          files: ./coverage.xml
+          # Specify the Codecov flag name associated with this test suite.
+          # Required to separate coverage reports by type (e.g., fast, slow, superslow) inside the Codecov UI.
+          flags: fast
+          name: fast-test-coverage
+
+      - name: Run Slow test and generate report
+        id: run_slow
+        continue-on-error: true
+        env:
+          GH_ACTION_ACCESS_TOKEN: ${{ secrets.GH_ACTION_ACCESS_TOKEN }}
+          CSFY_AWS_ACCESS_KEY_ID: ${{ env.AWS_ACCESS_KEY_ID }}
+          CSFY_AWS_SECRET_ACCESS_KEY: ${{ env.AWS_SECRET_ACCESS_KEY }}
+          CSFY_AWS_SESSION_TOKEN: ${{ env.AWS_SESSION_TOKEN }}
+          CSFY_AWS_DEFAULT_REGION: ${{ env.AWS_DEFAULT_REGION }}
+          CSFY_ECR_BASE_PATH: ghcr.io/${{ github.repository_owner }}
+          CSFY_AWS_S3_BUCKET: ${{ vars.CSFY_AWS_S3_BUCKET }}
+        run: invoke run_coverage --suite slow
+
+      - name: Upload Slow Test Coverage to Codecov
+        id: upload_slow
+        if: steps.run_slow.outcome == 'success'
+        continue-on-error: true
+        uses: codecov/codecov-action@v5
+        with:
+          token: ${{ secrets.CODECOV_TOKEN }}
+          files: ./coverage.xml
+          flags: slow
+          name: slow-test-coverage
+
+      - name: Run Superslow test and generate report
+        # Scheduled for Monday (i.e Day 1) or when manually triggered.
+        if: (github.event_name == 'schedule' && env.DAY_OF_WEEK == '1') || github.event_name == 'workflow_dispatch'
+        env:
+          GH_ACTION_ACCESS_TOKEN: ${{ secrets.GH_ACTION_ACCESS_TOKEN }}
+          CSFY_AWS_ACCESS_KEY_ID: ${{ env.AWS_ACCESS_KEY_ID }}
+          CSFY_AWS_SECRET_ACCESS_KEY: ${{ env.AWS_SECRET_ACCESS_KEY }}
+          CSFY_AWS_SESSION_TOKEN: ${{ env.AWS_SESSION_TOKEN }}
+          CSFY_AWS_DEFAULT_REGION: ${{ env.AWS_DEFAULT_REGION }}
+          CSFY_ECR_BASE_PATH: ghcr.io/${{ github.repository_owner }}
+          CSFY_AWS_S3_BUCKET: ${{ vars.CSFY_AWS_S3_BUCKET }}
+        run: invoke run_coverage --suite superslow
+
+      - name: Upload Superslow Test Coverage to Codecov
+        #TODO(Shaunak): Consider removing it when we turn this workflow into a reusable one.
+        if: steps.run_superslow.outcome == 'success'
+        uses: codecov/codecov-action@v5
+        with:
+          token: ${{ secrets.CODECOV_TOKEN }}
+          files: ./coverage.xml
+          flags: superslow
+          name: superslow-test-coverage
+
+        # Fail the job in CI if any of the fast/ slow run/ upload steps above failed.
+      - name: Fail if fast/slow test or upload failed
path: .github/workflows/coverage_tests.yml
user: login: Shaunak01
body: Test:
    - Fast test step failed
    - Upload skipped as intended
    - Other tests ran successfully (not superslow as that runs on Monday only but checked by triggering also)
    - Job failed at end because fast test step and upload step did not run/failed.
created_at: 2025-05-01T15:20:31Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,173 @@
+name: Test coverage
+
+on:
+  workflow_dispatch: {}
+  # every day at 00:00 UTC.
+  schedule:
+    - cron: '0 0 * * *'
+
+env:
+  CSFY_CI: true
+
+permissions:
+  # Required to authenticate and retrieve temporary AWS credentials via OIDC.
+  id-token: write
+  # Required to fetch and check out code from the repository.
+  contents: read
+  # Required to authenticate and pull Docker images from GitHub Container Registry (GHCR).
+  packages: read
+
+
+jobs:
+  run_test_coverage:
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Configure AWS credentials
+        uses: aws-actions/configure-aws-credentials@v1
+        with:
+          role-to-assume: ${{ vars.GH_ACTION_AWS_ROLE_ARN }}
+          role-session-name: ${{ vars.GH_ACTION_AWS_SESSION_NAME }}
+          aws-region: ${{ vars.CSFY_AWS_DEFAULT_REGION }}
+
+      - name: Login to GHCR
+        run: docker login ghcr.io -u gpsaggese -p ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Cleanup
+        run: sudo chmod 777 -R .
+
+      - name: Checkout code
+        uses: actions/checkout@v3
+        with:
+          submodules: recursive
+          fetch-depth: 0
+          token: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Update PYTHONPATH
+        run: echo "PYTHONPATH=.:helpers" >> $GITHUB_ENV
+
+      - name: Install dependencies
+        run: |
+          python -m pip install --upgrade pip
+          pip install -r .github/gh_requirements.txt
+
+      - name: Pull image from GHCR
+        run: docker pull ghcr.io/${{ github.repository }}:dev
+
+      # Only on scheduled runs, capture ISO weekday (1=Mon … 7=Sun).
+      - name: Set DAY_OF_WEEK
+        if: github.event_name == 'schedule'
+        run: echo "DAY_OF_WEEK=$(date -u +'%u')" >> $GITHUB_ENV
+
+      # This step is used to trigger the fast test coverage generation using the invoke task.
+      - name: Run Fast test and generate report
+        id: run_fast
+        continue-on-error: true
+        run: |
+          echo "Simulating fast test failure"
+          exit 1
+        # env:
+        #   GH_ACTION_ACCESS_TOKEN: ${{ secrets.GH_ACTION_ACCESS_TOKEN }}
+        #   CSFY_AWS_ACCESS_KEY_ID: ${{ env.AWS_ACCESS_KEY_ID }}
+        #   CSFY_AWS_SECRET_ACCESS_KEY: ${{ env.AWS_SECRET_ACCESS_KEY }}
+        #   CSFY_AWS_SESSION_TOKEN: ${{ env.AWS_SESSION_TOKEN }}
+        #   CSFY_AWS_DEFAULT_REGION: ${{ env.AWS_DEFAULT_REGION }}
+        #   CSFY_ECR_BASE_PATH: ghcr.io/${{ github.repository_owner }}
+        #   CSFY_AWS_S3_BUCKET: ${{ vars.CSFY_AWS_S3_BUCKET }}
+        # run: invoke run_coverage --suite fast
+
+      - name: Upload Fast Test Coverage to Codecov
+        id: upload_fast
+        # Only upload if the previous fast test run step succeeded (i.r report generated).
+        # failed step don’t generate a coverage report, so there's nothing to upload.
+        if: steps.run_fast.outcome == 'success'
+        continue-on-error: true
+        uses: codecov/codecov-action@v5
+        with:
+          token: ${{ secrets.CODECOV_TOKEN }}
+          files: ./coverage.xml
+          # Specify the Codecov flag name associated with this test suite.
+          # Required to separate coverage reports by type (e.g., fast, slow, superslow) inside the Codecov UI.
+          flags: fast
+          name: fast-test-coverage
+
+      - name: Run Slow test and generate report
+        id: run_slow
+        continue-on-error: true
+        env:
+          GH_ACTION_ACCESS_TOKEN: ${{ secrets.GH_ACTION_ACCESS_TOKEN }}
+          CSFY_AWS_ACCESS_KEY_ID: ${{ env.AWS_ACCESS_KEY_ID }}
+          CSFY_AWS_SECRET_ACCESS_KEY: ${{ env.AWS_SECRET_ACCESS_KEY }}
+          CSFY_AWS_SESSION_TOKEN: ${{ env.AWS_SESSION_TOKEN }}
+          CSFY_AWS_DEFAULT_REGION: ${{ env.AWS_DEFAULT_REGION }}
+          CSFY_ECR_BASE_PATH: ghcr.io/${{ github.repository_owner }}
+          CSFY_AWS_S3_BUCKET: ${{ vars.CSFY_AWS_S3_BUCKET }}
+        run: invoke run_coverage --suite slow
+
+      - name: Upload Slow Test Coverage to Codecov
+        id: upload_slow
+        # Only upload if the previous slow test run step succeeded (i.r report generated).
+        if: steps.run_slow.outcome == 'success'
+        continue-on-error: true
+        uses: codecov/codecov-action@v5
+        with:
+          token: ${{ secrets.CODECOV_TOKEN }}
+          files: ./coverage.xml
+          flags: slow
+          name: slow-test-coverage
+
+      - name: Run Superslow test and generate report
+        # Run only on scheduled jobs or manual trigger
+        if: github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'
path: .github/workflows/coverage_tests.yml
user: login: dremdem
body: I'd say it's outdated. 
created_at: 2025-05-02T08:31:17Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,173 @@
+name: Test coverage
+
+on:
+  workflow_dispatch: {}
path: .github/workflows/coverage_tests.yml
user: login: dremdem
body: Let's add a comment above: 
```
# Run manually.
```
created_at: 2025-05-02T08:34:42Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,173 @@
+name: Test coverage
+
+on:
+  workflow_dispatch: {}
+  # every day at 00:00 UTC.
+  schedule:
+    - cron: '0 0 * * *'
+
+env:
+  CSFY_CI: true
+
+permissions:
+  # Required to authenticate and retrieve temporary AWS credentials via OIDC.
+  id-token: write
+  # Required to fetch and check out code from the repository.
+  contents: read
+  # Required to authenticate and pull Docker images from GitHub Container Registry (GHCR).
+  packages: read
+
+
+jobs:
+  run_test_coverage:
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Configure AWS credentials
+        uses: aws-actions/configure-aws-credentials@v1
+        with:
+          role-to-assume: ${{ vars.GH_ACTION_AWS_ROLE_ARN }}
+          role-session-name: ${{ vars.GH_ACTION_AWS_SESSION_NAME }}
+          aws-region: ${{ vars.CSFY_AWS_DEFAULT_REGION }}
+
+      - name: Login to GHCR
+        run: docker login ghcr.io -u gpsaggese -p ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Cleanup
+        run: sudo chmod 777 -R .
+
+      - name: Checkout code
+        uses: actions/checkout@v3
+        with:
+          submodules: recursive
+          fetch-depth: 0
+          token: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Update PYTHONPATH
+        run: echo "PYTHONPATH=.:helpers" >> $GITHUB_ENV
+
+      - name: Install dependencies
+        run: |
+          python -m pip install --upgrade pip
+          pip install -r .github/gh_requirements.txt
+
+      - name: Pull image from GHCR
+        run: docker pull ghcr.io/${{ github.repository }}:dev
+
+      # Only on scheduled runs, capture ISO weekday (1=Mon … 7=Sun).
+      - name: Set DAY_OF_WEEK
+        if: github.event_name == 'schedule'
+        run: echo "DAY_OF_WEEK=$(date -u +'%u')" >> $GITHUB_ENV
+
+      # This step is used to trigger the fast test coverage generation using the invoke task.
+      - name: Run Fast test and generate report
+        id: run_fast
+        continue-on-error: true
+        run: |
+          echo "Simulating fast test failure"
path: .github/workflows/coverage_tests.yml
user: login: dremdem
body: Let's revert it before merging. 
created_at: 2025-05-02T08:37:02Z
author_association: CONTRIBUTOR

PR comments:
user: login: codecov[bot]
created_at: 2025-04-21T23:53:21Z
body: ## [Codecov](https://app.codecov.io/gh/causify-ai/helpers/pull/598?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=causify-ai) Report
Attention: Patch coverage is `12.50000%` with `28 lines` in your changes missing coverage. Please review.
| [Files with missing lines](https://app.codecov.io/gh/causify-ai/helpers/pull/598?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=causify-ai) | Patch % | Lines |
|---|---|---|
| [helpers/lib\_tasks\_pytest.py](https://app.codecov.io/gh/causify-ai/helpers/pull/598?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=causify-ai#diff-aGVscGVycy9saWJfdGFza3NfcHl0ZXN0LnB5) | 12.50% | [26 Missing and 2 partials :warning: ](https://app.codecov.io/gh/causify-ai/helpers/pull/598?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=causify-ai) |

:loudspeaker: Thoughts on this report? [Let us know!](https://github.com/codecov/feedback/issues/255)
author_association: NONE
user: login: dremdem
created_at: 2025-05-01T12:56:14Z
body: @Shaunak01 
Can we run linter? 
author_association: CONTRIBUTOR
user: login: Shaunak01
created_at: 2025-05-01T18:04:06Z
body: @dremdem I have run the linter not sure abt the error. The files are linted

```
################################################################################
* Failed assertion *
cond=False
The Git client is not clean:
dev_scripts_helpers/update_devops_packages/notebooks/Master_buildmeister_dashboard.ipynb
################################################################################
```
author_association: CONTRIBUTOR
user: login: dremdem
created_at: 2025-05-01T19:00:18Z
body: > @dremdem I have run the linter not sure abt the error. The files are linted

I ran the `Linter` on this file locally and got: 

```
RuntimeError: /venv/lib/python3.12/site-packages/nbformat/__init__.py:96: MissingIDFieldWarning: Cell is missing an id field, this will become a hard error in future nbformat versions. You may want to use `normalize()` on your notebooks before validations (available since nbformat 5.1.4). Previous versions of nbformat are fixing this issue transparently, and will stop doing so in the future. [process_jupytext]
Traceback (most recent call last): [process_jupytext]
[jupytext] Reading dev_scripts_helpers/update_devops_packages/notebooks/Master_buildmeister_dashboard.ipynb in format ipynb [process_jupytext]
dev_scripts_helpers/update_devops_packages/notebooks/Master_buildmeister_dashboard.ipynb: Cell content differ on code cell #7: --- expected [process_jupytext]
process_jupytext failed with command `$(find -wholename '*dev_scripts_helpers/notebooks/process_jupytext.py') -f dev_scripts_helpers/update_devops_packages/notebooks/Master_buildmeister_dashboard.ipynb --action test`
 [process_jupytext]
```

Found some info
https://stackoverflow.com/questions/75632445/anaconda-jupyter-notebook-and-missingidfieldwarning

Tried that approach: 

```
import nbformat

with open("problematic_notebook.ipynb", "r") as file:
    nb_corrupted = nbformat.reader.read(file)

nbformat.validator.validate(nb_corrupted)
# <stdin>:1: MissingIDFieldWarning: Code cell is missing an id field, 
# this will become a hard error in future nbformat versions. 
# You may want to use `normalize()` on your notebooks before validations (available since nbformat 5.1.4). 
# Previous versions of nbformat are fixing this issue transparently, and will stop doing so in the future.

nb_fixed = nbformat.validator.normalize(nb_corrupted)
nbformat.validator.validate(nb_fixed[1])
# Produces no warnings or errors.

with open("fixed_notebook.ipynb", "w") as file:
    nbformat.write(nb_fixed[1], file)

```

But no luck. 

Feel free to file a bug. 
FYI @sonniki
author_association: CONTRIBUTOR
user: login: gpsaggese
created_at: 2025-05-02T14:47:14Z
body: Let's merge this and then we do another iteration. We can't have PRs in review for 2 weeks. We want to maintain high tempo
author_association: CONTRIBUTOR