Repository: bearcove/loona (language: rust)

Issue:
h2: Track h2spec regressions
Right now there's no code in CI that compares the number of passed/failed tests with the number of passed/failed tests in main.

This seems mildly annoying to do, but someone should do it.


Discussion / hints:
Note that regressions happen fairly often, as large portions of the code are rewritten: for example #127 broke 4 tests (we went from 74 passed to 70 passed for http2).

Downloading h2spec-results for main, and comparing the XML files with what we have, should let us fail the workflow if some tests passed but don't pass anymore (and we should be able to say which ones broke).

Fix patch:
diff --git a/.github/workflows/report-regressions.yml b/.github/workflows/report-regressions.yml
new file mode 100644
index 00000000..ade53fca
--- /dev/null
+++ b/.github/workflows/report-regressions.yml
@@ -0,0 +1,26 @@
+name: report-regressions
+
+on:
+  workflow_run:
+    workflows: [test]
+    types: [completed]
+    
+permissions:
+  checks: write
+  issues: write
+
+jobs:
+  checks:
+    runs-on: self-hosted
+    steps:
+      - name: Download h2spec Report
+        uses: dawidd6/action-download-artifact@v2
+        with:
+          name: h2spec-results
+          workflow: ${{ github.event.workflow.id }}
+          run_id: ${{ github.event.workflow_run.id }}
+      - name: Report h2spec regressions      
+        uses: hapsoc/report-h2spec-regressions@61fee0f19a82d79427b38dfb094d259f43208fc2
+        with:
+          github-token: ${{ secrets.GITHUB_TOKEN }}
+          suites: "h2spec-generic,h2spec-hpack,h2spec-http2"
\ No newline at end of file
diff --git a/scripts/.gitignore b/scripts/.gitignore
new file mode 100644
index 00000000..b512c09d
--- /dev/null
+++ b/scripts/.gitignore
@@ -0,0 +1,1 @@
+node_modules
\ No newline at end of file
diff --git a/scripts/cov.sh b/scripts/cov.sh
index 447649b2..ae2e255b 100755
--- a/scripts/cov.sh
+++ b/scripts/cov.sh
@@ -12,9 +12,10 @@ mkdir -p "${COVERAGE_DIR}/raw"
 # must be set before building
 export LLVM_PROFILE_FILE="${COVERAGE_DIR}/raw/fluke-%p-%12m.profraw"
 
-RUSTC_PATH=$(rustup which rustc)
-# replace `bin/rustc` with `lib/rustlib/x86_64-unknown-linux-gnu/bin/llvm-cov`
-LLVM_TOOLS_PATH=${RUSTC_PATH%/*/*}/lib/rustlib/x86_64-unknown-linux-gnu/bin
+RUSTC_SYSROOT=$(rustc --print sysroot)
+# extract target triple. this is wrong in case of cross-compilation but ah-well.
+RUSTC_TARGET_TRIPLE=$(rustc --version --verbose | grep host | cut -d' ' -f2)
+LLVM_TOOLS_PATH=${RUSTC_SYSROOT}/lib/rustlib/${RUSTC_TARGET_TRIPLE}/bin
 LLVM_PROFDATA="${LLVM_TOOLS_PATH}/llvm-profdata"
 "${LLVM_PROFDATA}" --version
 LLVM_COV="${LLVM_TOOLS_PATH}/llvm-cov"


Test patch: