Repository: zarr-developers/zarr-python

Issue:
[TST] Bring back Codecov?
While we were developing Zarr-Python 3, test coverage checks stopped getting published to [Codecov](https://app.codecov.io/gh/zarr-developers/zarr-python).

Do we want to bring Codecov back? If so, what needs to be done to enable this in hatch / pytest / github actions?



Discussion:
codecov[bot]: ## [Codecov](https://app.codecov.io/gh/zarr-developers/zarr-python/pull/2682?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=zarr-developers) Report
All modified and coverable lines are covered by tests :white_check_mark:
> Project coverage is 88.37%. Comparing base [(`eaf5d7a`)](https://app.codecov.io/gh/zarr-developers/zarr-python/commit/eaf5d7ab4c5a791ce112e0ced45bca2288eb9d69?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=zarr-developers) to head [(`caf3a7d`)](https://app.codecov.io/gh/zarr-developers/zarr-python/commit/caf3a7dcf454974657dc741bbf2734718ff84b14?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=zarr-developers).
> Report is 422 commits behind head on main.

> :exclamation:  There is a different number of reports uploaded between BASE (eaf5d7a) and HEAD (caf3a7d). Click for more details.
> 
> <details><summary>HEAD has 8 uploads less than BASE</summary>
>
>| Flag | BASE (eaf5d7a) | HEAD (caf3a7d) |
>|------|------|------|
>||10|2|
></details>

<details><summary>Additional details and impacted files</summary>


```diff
@@             Coverage Diff             @@
##             main    #2682       +/-   ##
===========================================
- Coverage   99.98%   88.37%   -11.62%     
===========================================
  Files          38       59       +21     
  Lines       14718     6648     -8070     
===========================================
- Hits        14716     5875     -8841     
- Misses          2      773      +771     
```

[see 97 files with indirect coverage changes](https://app.codecov.io/gh/zarr-developers/zarr-python/pull/2682/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=zarr-developers)

</details>
jhamman: This is working. We may need to merge with codecov check failing because it is comparing against eaf5d7a which is way back in v2 land. Going forward, subsequent checks should compare against main and the behavior should be that the check will only fail if the coverage goes down.

Patch:
diff --git a/codecov.yml b/codecov.yml
new file mode 100644
index 00000000..83274aed
--- /dev/null
+++ b/codecov.yml
@@ -0,0 +1,10 @@
+coverage:
+  status:
+    patch:
+      default:
+        target: auto
+    project:
+      default:
+        target: auto
+        threshold: 0.1
+comment: false
diff --git a/pyproject.toml b/pyproject.toml
index 05db0860..96b7ead7 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -103,13 +103,13 @@ Homepage = "https://github.com/zarr-developers/zarr-python"
 [tool.coverage.report]
 exclude_lines = [
     "pragma: no cover",
+    "if TYPE_CHECKING:",
     "pragma: ${PY_MAJOR_VERSION} no cover",
     '.*\.\.\.' # Ignore "..." lines
 ]
 
 [tool.coverage.run]
 omit = [
-    "src/zarr/meta_v1.py",
     "bench/compress_normal.py",
 ]
 
@@ -140,8 +140,8 @@ numpy = ["1.25", "2.1"]
 features = ["gpu"]
 
 [tool.hatch.envs.test.scripts]
-run-coverage = "pytest --cov-config=pyproject.toml --cov=pkg --cov=src"
-run-coverage-gpu = "pip install cupy-cuda12x && pytest -m gpu --cov-config=pyproject.toml --cov=pkg --cov=src"
+run-coverage = "pytest --cov-config=pyproject.toml --cov=pkg --cov-report xml --cov=src --junitxml=junit.xml -o junit_family=legacy"
+run-coverage-gpu = "pip install cupy-cuda12x && pytest -m gpu --cov-config=pyproject.toml --cov=pkg --cov-report xml --cov=src --junitxml=junit.xml -o junit_family=legacy"
 run = "run-coverage --no-cov"
 run-verbose = "run-coverage --verbose"
 run-mypy = "mypy src"
@@ -170,7 +170,7 @@ numpy = ["1.25", "2.1"]
 version = ["minimal"]
 
 [tool.hatch.envs.gputest.scripts]
-run-coverage = "pytest -m gpu --cov-config=pyproject.toml --cov=pkg --cov=src"
+run-coverage = "pytest -m gpu --cov-config=pyproject.toml --cov=pkg --cov-report xml --cov=src --junitxml=junit.xml -o junit_family=legacy"
 run = "run-coverage --no-cov"
 run-verbose = "run-coverage --verbose"
 run-mypy = "mypy src"


Test patch:
diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml
index 5309ea45..ea65c3f0 100644
--- a/.github/workflows/test.yml
+++ b/.github/workflows/test.yml
@@ -60,7 +60,12 @@ jobs:
         hatch env run -e test.py${{ matrix.python-version }}-${{ matrix.numpy-version }}-${{ matrix.dependency-set }} list-env
     - name: Run Tests
       run: |
-        hatch env run --env test.py${{ matrix.python-version }}-${{ matrix.numpy-version }}-${{ matrix.dependency-set }} run
+        hatch env run --env test.py${{ matrix.python-version }}-${{ matrix.numpy-version }}-${{ matrix.dependency-set }} run-coverage
+    - name: Upload coverage
+      uses: codecov/codecov-action@v5
+      with:
+        token: ${{ secrets.CODECOV_TOKEN }}
+        verbose: true # optional (default = false)
 
   test-upstream-and-min-deps:
     name: py=${{ matrix.python-version }}-${{ matrix.dependency-set }}