Repository: streamlit/streamlit
Pull request #11054: Add automated PR comment for frontend coverage
Author: login: lukasmasuch
Labels: name: change:other
description: PR contains other type of change
name: impact:internal
description: PR changes only affect internal code
Created: 2025-04-08T12:43:03Z | Merged: 2025-04-08T18:02:09Z

## Describe your changes

Add an automated PR comment for singificant changes in frontend coverage:

<img width="599" alt="image" src="https://github.com/user-attachments/assets/61df7aea-2aae-4bf8-9a20-0aaf1975ab20" />

The same as with: https://github.com/streamlit/streamlit/pull/10951

---

**Contribution License Agreement**

By submitting this pull request you agree that all contributions to this project are made under the Apache 2.0 license.



Diff:
diff --git a/.github/workflows/js-tests.yml b/.github/workflows/js-tests.yml
index d8b4008a6ea..bbc2fe6f930 100644
--- a/.github/workflows/js-tests.yml
+++ b/.github/workflows/js-tests.yml
@@ -118,3 +118,148 @@ jobs:
       - name: Build package
         working-directory: component-lib
         run: yarn build
+
+  js-coverage-comment:
+    needs:
+      - js-unit-tests
+    runs-on: ubuntu-latest
+    permissions:
+      # Required for downloading artifacts
+      actions: read
+      # Required for creating/updating PR comments
+      pull-requests: write
+    if: github.event_name == 'pull_request'
+
+    steps:
+      - name: Checkout Streamlit code
+        uses: actions/checkout@v4
+        with:
+          ref: ${{ inputs.ref }}
+          persist-credentials: false
+          submodules: "recursive"
+
+      - name: Download coverage JSON
+        uses: actions/download-artifact@v4
+        with:
+          name: vitest_coverage_json
+          path: /tmp/current_coverage
+
+      - name: Get latest develop run ID
+        id: get-latest-run
+        uses: actions/github-script@v7
+        with:
+          script: |
+            const { data: runs } = await github.rest.actions.listWorkflowRuns({
+              owner: context.repo.owner,
+              repo: context.repo.repo,
+              workflow_id: 'js-tests.yml',
+              branch: 'develop',
+              status: 'success',
+              per_page: 1
+            });
+
+            if (runs.workflow_runs.length === 0) {
+              throw new Error('No successful workflow runs found on develop branch');
+            }
+
+            core.setOutput('run_id', runs.workflow_runs[0].id);
+
+      - name: Download develop coverage
+        uses: actions/download-artifact@v4
+        with:
+          repository: streamlit/streamlit
+          run-id: ${{ steps.get-latest-run.outputs.run_id }}
+          name: vitest_coverage_json
+          path: /tmp/develop_coverage
+          # Needs a token to download the artifact from another
+          # workflow run:
+          github-token: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Check coverage changes and comment
+        uses: actions/github-script@v7
+        with:
+          script: |
+            const COVERAGE_CHANGE_THRESHOLD = 0.03; // 0.03%
+            const fs = require('fs');
+
+            try {
+              // Read current coverage from the downloaded artifact
+              const currentCoverageData = JSON.parse(fs.readFileSync('/tmp/current_coverage/coverage-summary.json', 'utf8'));
+
+              // Read develop coverage from downloaded artifact
+              const developCoverageData = JSON.parse(fs.readFileSync('/tmp/develop_coverage/coverage-summary.json', 'utf8'));
+
+              // Calculate total coverage (focusing on lines as specified)
+              const currentTotal = currentCoverageData.total.lines.pct;
+              const developTotal = developCoverageData.total.lines.pct;
+
+              // Calculate percentage difference
+              const diffPercent = currentTotal - developTotal;
+              const absDiffPercent = Math.abs(diffPercent);
+
+              console.log(`Current frontend coverage: ${currentTotal.toFixed(4)}%`);
+              console.log(`Develop frontend coverage: ${developTotal.toFixed(4)}%`);
+              console.log(`Coverage difference: ${diffPercent.toFixed(4)}%`);
+
+              const changeType = diffPercent > 0 ? 'increased' : 'decreased';
+              const emoji = diffPercent > 0 ? '📈' : '📉';
+
+              // Get current coverage details
+              const currentLines = currentCoverageData.total.lines;
+              const developLines = developCoverageData.total.lines;
+
+              const commentBody = `### ${emoji} Frontend coverage change detected
+
+              The frontend unit test (vitest) coverage has **${changeType} by ${Math.abs(diffPercent).toFixed(4)}%**
+
+              - Current PR: ${currentTotal.toFixed(4)}% (${currentLines.total} lines, ${currentLines.total - currentLines.covered} missed)
+              - Latest develop: ${developTotal.toFixed(4)}% (${developLines.total} lines, ${developLines.total - developLines.covered} missed)
+
+              ${absDiffPercent >= COVERAGE_CHANGE_THRESHOLD
+                ? (diffPercent > 0
+                    ? '🎉 Great job on improving test coverage!'
+                    : '💡 Consider adding more unit tests to maintain or improve coverage.'
+                  )
+                : '✅ Coverage change is within normal range.'
+              }
+
+              📊 [View detailed coverage comparison](https://issues.streamlit.app/Test_Coverage_(Frontend)?pr=${context.issue.number})`;
+
+              // Unique identifier for our comment
+              const commentIdentifier = '<!-- STREAMLIT-JAVASCRIPT-COVERAGE-CHECK -->';
+              const fullCommentBody = `${commentIdentifier}\n${commentBody}`;
+
+              // Get all comments on the PR
+              const { data: comments } = await github.rest.issues.listComments({
+                owner: context.repo.owner,
+                repo: context.repo.repo,
+                issue_number: context.issue.number,
+              });
+
+              // Look for our previous coverage comment
+              const coverageComment = comments.find(comment => comment.body.includes(commentIdentifier));
+
+              if (coverageComment) {
+                // Update existing comment
+                await github.rest.issues.updateComment({
+                  owner: context.repo.owner,
+                  repo: context.repo.repo,
+                  comment_id: coverageComment.id,
+                  body: fullCommentBody
+                });
+                console.log('Updated existing coverage comment');
+              } else if (absDiffPercent >= COVERAGE_CHANGE_THRESHOLD) {
+                // Only create new comment for significant changes
+                await github.rest.issues.createComment({
+                  owner: context.repo.owner,
+                  repo: context.repo.repo,
+                  issue_number: context.issue.number,
+                  body: fullCommentBody
+                });
+                console.log('Created new coverage comment');
+              } else {
+                console.log('Coverage change is not significant and no existing comment found, skipping comment');
+              }
+            } catch (error) {
+              console.error(`Error checking coverage changes: ${error.message}`);
+            }
diff --git a/.github/workflows/nightly.yml b/.github/workflows/nightly.yml
index 6bec82cbbb8..11de7a0421b 100644
--- a/.github/workflows/nightly.yml
+++ b/.github/workflows/nightly.yml
@@ -89,6 +89,10 @@ jobs:
     uses: ./.github/workflows/js-tests.yml
     with:
       ref: ${{needs.create-nightly-tag.outputs.tag}}
+    permissions:
+      # Additional permissions for coverage-comment
+      actions: read
+      pull-requests: write
 
   run-playwright-tests:
     uses: ./.github/workflows/playwright.yml
diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml
index b00eb58ef18..8b32d746519 100644
--- a/.github/workflows/release.yml
+++ b/.github/workflows/release.yml
@@ -23,6 +23,10 @@ jobs:
     uses: ./.github/workflows/js-tests.yml
     with:
       ref: ${{ github.ref_name }}
+    permissions:
+      # Additional permissions for coverage-comment
+      actions: read
+      pull-requests: write
 
   run-playwright-tests:
     uses: ./.github/workflows/playwright.yml


Reviews:
user: login: copilot-pull-request-reviewer[bot]
body: 

Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.


<details>
<summary>Comments suppressed due to low confidence (1)</summary>

**.github/workflows/js-tests.yml:182**
* [nitpick] The constant COVERAGE_CHANGE_THRESHOLD is set to 0.03 and documented as 0.03%, which might be misinterpreted. Consider renaming it to clarify that the value represents percentage points or adjusting the value if a different threshold is intended.
```
const COVERAGE_CHANGE_THRESHOLD = 0.03; // 0.03%
```
</details>


state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-08T13:31:41Z
user: login: sfc-gh-kjavadyan
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-04-08T17:06:16Z
user: login: sfc-gh-kjavadyan
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-08T17:08:10Z
user: login: lukasmasuch
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-04-08T17:17:17Z
user: login: sfc-gh-kjavadyan
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-08T17:45:08Z
user: login: lukasmasuch
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-04-08T18:01:55Z

Review comments:
diff_hunk: @@ -118,3 +118,148 @@ jobs:
       - name: Build package
         working-directory: component-lib
         run: yarn build
+
+  js-coverage-comment:
+    needs:
+      - js-unit-tests
+    runs-on: ubuntu-latest
+    permissions:
+      # Required for downloading artifacts
+      actions: read
+      # Required for creating/updating PR comments
+      pull-requests: write
+    if: github.event_name == 'pull_request'
+
+    steps:
+      - name: Checkout Streamlit code
+        uses: actions/checkout@v4
+        with:
+          ref: ${{ inputs.ref }}
+          persist-credentials: false
+          submodules: "recursive"
+
+      - name: Download coverage JSON
+        uses: actions/download-artifact@v4
+        with:
+          name: vitest_coverage_json
+          path: /tmp/current_coverage
+
+      - name: Get latest develop run ID
+        id: get-latest-run
+        uses: actions/github-script@v7
+        with:
+          script: |
+            const { data: runs } = await github.rest.actions.listWorkflowRuns({
+              owner: context.repo.owner,
+              repo: context.repo.repo,
+              workflow_id: 'js-tests.yml',
+              branch: 'develop',
+              status: 'success',
+              per_page: 1
+            });
+
+            if (runs.workflow_runs.length === 0) {
+              throw new Error('No successful workflow runs found on develop branch');
+            }
+
+            core.setOutput('run_id', runs.workflow_runs[0].id);
+
+      - name: Download develop coverage
+        uses: actions/download-artifact@v4
+        with:
+          repository: streamlit/streamlit
+          run-id: ${{ steps.get-latest-run.outputs.run_id }}
+          name: vitest_coverage_json
+          path: /tmp/develop_coverage
+          # Needs a token to download the artifact from another
+          # workflow run:
+          github-token: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Check coverage changes and comment
+        uses: actions/github-script@v7
+        with:
+          script: |
+            const COVERAGE_CHANGE_THRESHOLD = 0.03; // 0.03%
path: .github/workflows/js-tests.yml
user: login: sfc-gh-kjavadyan
body: Just want to note, that `0.03` looks small number, but in reality since we already have good coverage, it would be really not easy to achieve 😄 
created_at: 2025-04-08T17:08:10Z
author_association: CONTRIBUTOR
diff_hunk: @@ -118,3 +118,148 @@ jobs:
       - name: Build package
         working-directory: component-lib
         run: yarn build
+
+  js-coverage-comment:
+    needs:
+      - js-unit-tests
+    runs-on: ubuntu-latest
+    permissions:
+      # Required for downloading artifacts
+      actions: read
+      # Required for creating/updating PR comments
+      pull-requests: write
+    if: github.event_name == 'pull_request'
+
+    steps:
+      - name: Checkout Streamlit code
+        uses: actions/checkout@v4
+        with:
+          ref: ${{ inputs.ref }}
+          persist-credentials: false
+          submodules: "recursive"
+
+      - name: Download coverage JSON
+        uses: actions/download-artifact@v4
+        with:
+          name: vitest_coverage_json
+          path: /tmp/current_coverage
+
+      - name: Get latest develop run ID
+        id: get-latest-run
+        uses: actions/github-script@v7
+        with:
+          script: |
+            const { data: runs } = await github.rest.actions.listWorkflowRuns({
+              owner: context.repo.owner,
+              repo: context.repo.repo,
+              workflow_id: 'js-tests.yml',
+              branch: 'develop',
+              status: 'success',
+              per_page: 1
+            });
+
+            if (runs.workflow_runs.length === 0) {
+              throw new Error('No successful workflow runs found on develop branch');
+            }
+
+            core.setOutput('run_id', runs.workflow_runs[0].id);
+
+      - name: Download develop coverage
+        uses: actions/download-artifact@v4
+        with:
+          repository: streamlit/streamlit
+          run-id: ${{ steps.get-latest-run.outputs.run_id }}
+          name: vitest_coverage_json
+          path: /tmp/develop_coverage
+          # Needs a token to download the artifact from another
+          # workflow run:
+          github-token: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Check coverage changes and comment
+        uses: actions/github-script@v7
+        with:
+          script: |
+            const COVERAGE_CHANGE_THRESHOLD = 0.03; // 0.03%
path: .github/workflows/js-tests.yml
user: login: lukasmasuch
body: yeah, I think we can also go a bit lower here, maybe 0.02? I just don't want this to become too annoying :) 
created_at: 2025-04-08T17:17:17Z
author_association: COLLABORATOR
diff_hunk: @@ -118,3 +118,148 @@ jobs:
       - name: Build package
         working-directory: component-lib
         run: yarn build
+
+  js-coverage-comment:
+    needs:
+      - js-unit-tests
+    runs-on: ubuntu-latest
+    permissions:
+      # Required for downloading artifacts
+      actions: read
+      # Required for creating/updating PR comments
+      pull-requests: write
+    if: github.event_name == 'pull_request'
+
+    steps:
+      - name: Checkout Streamlit code
+        uses: actions/checkout@v4
+        with:
+          ref: ${{ inputs.ref }}
+          persist-credentials: false
+          submodules: "recursive"
+
+      - name: Download coverage JSON
+        uses: actions/download-artifact@v4
+        with:
+          name: vitest_coverage_json
+          path: /tmp/current_coverage
+
+      - name: Get latest develop run ID
+        id: get-latest-run
+        uses: actions/github-script@v7
+        with:
+          script: |
+            const { data: runs } = await github.rest.actions.listWorkflowRuns({
+              owner: context.repo.owner,
+              repo: context.repo.repo,
+              workflow_id: 'js-tests.yml',
+              branch: 'develop',
+              status: 'success',
+              per_page: 1
+            });
+
+            if (runs.workflow_runs.length === 0) {
+              throw new Error('No successful workflow runs found on develop branch');
+            }
+
+            core.setOutput('run_id', runs.workflow_runs[0].id);
+
+      - name: Download develop coverage
+        uses: actions/download-artifact@v4
+        with:
+          repository: streamlit/streamlit
+          run-id: ${{ steps.get-latest-run.outputs.run_id }}
+          name: vitest_coverage_json
+          path: /tmp/develop_coverage
+          # Needs a token to download the artifact from another
+          # workflow run:
+          github-token: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Check coverage changes and comment
+        uses: actions/github-script@v7
+        with:
+          script: |
+            const COVERAGE_CHANGE_THRESHOLD = 0.03; // 0.03%
path: .github/workflows/js-tests.yml
user: login: sfc-gh-kjavadyan
body: `0.03` sounds good to me!
created_at: 2025-04-08T17:45:08Z
author_association: CONTRIBUTOR
diff_hunk: @@ -118,3 +118,148 @@ jobs:
       - name: Build package
         working-directory: component-lib
         run: yarn build
+
+  js-coverage-comment:
+    needs:
+      - js-unit-tests
+    runs-on: ubuntu-latest
+    permissions:
+      # Required for downloading artifacts
+      actions: read
+      # Required for creating/updating PR comments
+      pull-requests: write
+    if: github.event_name == 'pull_request'
+
+    steps:
+      - name: Checkout Streamlit code
+        uses: actions/checkout@v4
+        with:
+          ref: ${{ inputs.ref }}
+          persist-credentials: false
+          submodules: "recursive"
+
+      - name: Download coverage JSON
+        uses: actions/download-artifact@v4
+        with:
+          name: vitest_coverage_json
+          path: /tmp/current_coverage
+
+      - name: Get latest develop run ID
+        id: get-latest-run
+        uses: actions/github-script@v7
+        with:
+          script: |
+            const { data: runs } = await github.rest.actions.listWorkflowRuns({
+              owner: context.repo.owner,
+              repo: context.repo.repo,
+              workflow_id: 'js-tests.yml',
+              branch: 'develop',
+              status: 'success',
+              per_page: 1
+            });
+
+            if (runs.workflow_runs.length === 0) {
+              throw new Error('No successful workflow runs found on develop branch');
+            }
+
+            core.setOutput('run_id', runs.workflow_runs[0].id);
+
+      - name: Download develop coverage
+        uses: actions/download-artifact@v4
+        with:
+          repository: streamlit/streamlit
+          run-id: ${{ steps.get-latest-run.outputs.run_id }}
+          name: vitest_coverage_json
+          path: /tmp/develop_coverage
+          # Needs a token to download the artifact from another
+          # workflow run:
+          github-token: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Check coverage changes and comment
+        uses: actions/github-script@v7
+        with:
+          script: |
+            const COVERAGE_CHANGE_THRESHOLD = 0.03; // 0.03%
path: .github/workflows/js-tests.yml
user: login: lukasmasuch
body: ok, let's merge and see how the coverage moves over time 👍 
created_at: 2025-04-08T18:01:55Z
author_association: COLLABORATOR

PR comments:
user: login: snyk-io[bot]
created_at: 2025-04-08T12:43:12Z
body: ![](https://res.cloudinary.com/snyk/image/upload/r-d/scm-platform/snyk-pull-requests/pr-banner-default.svg)

### :tada: **Snyk checks have passed. No issues have been found so far.**

:white_check_mark: **security/snyk** check is complete. No issues have been found. [(View Details)](https://app.snyk.io/org/streamlit-open-source/pr-checks/283e2070-ef3a-43dc-89cd-62f48f971313?refs=de6d5c5def6707cfd4ab92627c7aabf62fdc8fb9&source=prChecksComment)

:white_check_mark: **license/snyk** check is complete. No issues have been found. [(View Details)](https://app.snyk.io/org/streamlit-open-source/pr-checks/283e2070-ef3a-43dc-89cd-62f48f971313/license?refs=de6d5c5def6707cfd4ab92627c7aabf62fdc8fb9&source=prChecksComment)



author_association: CONTRIBUTOR
user: login: github-actions[bot]
created_at: 2025-04-08T12:43:25Z
body: ### ✅ PR preview is ready!

| Name | Link |
| :----: | ---- |
| 📦 Wheel file | https://core-previews.s3-us-west-2.amazonaws.com/pr-11054/streamlit-1.44.1-py3-none-any.whl |
| 🕹️ Preview app | [pr-11054.streamlit.app](https://pr-11054.streamlit.app) (☁️ Deploy [here](https://share.streamlit.io/deploy?repository=streamlit/core-previews&branch=pr-11054&mainModule=E2E_Tester_🧪.py&subdomain=pr-11054) if not accessible)| 
author_association: CONTRIBUTOR
user: login: github-actions[bot]
created_at: 2025-04-08T12:55:17Z
body: <!-- STREAMLIT-JAVASCRIPT-COVERAGE-CHECK -->
### 📉 Frontend coverage change detected

  The frontend unit test (vitest) coverage has **decreased by 0.0000%**

  - Current PR: 85.7200% (41382 lines, 5906 missed)
  - Latest develop: 85.7200% (41382 lines, 5906 missed)

  ✅ Coverage change is within normal range.

  📊 [View detailed coverage comparison](https://issues.streamlit.app/Test_Coverage_(Frontend)?pr=11054)
author_association: CONTRIBUTOR