Repository: codecov/umbrella
Pull request #129: Consolidate tests to use a unified `config/codecov.yml`
Author: login: Swatinem
Created: 2025-05-08T12:46:33Z | Merged: 2025-05-09T06:23:32Z

This makes sure all the various docker images are using the same `config/codecov.yml`.

Alongside, this does fix up a few tests that were depending on some weird config behavior, as well as improve usage of mock configs.


Diff:
diff --git a/apps/codecov-api/docker/test.yml b/apps/codecov-api/docker/test.yml
index 155f06646a..786370cfb3 100644
--- a/apps/codecov-api/docker/test.yml
+++ b/apps/codecov-api/docker/test.yml
@@ -1,3 +1,7 @@
+# !!! ATTENTION !!!
+# This file is a copy of the root `tools/devenv/config/test.yml`.
+# Make sure to keep it in sync!
+
 setup:
   codecov_url: https://codecov.io
   debug: no
@@ -5,6 +9,8 @@ setup:
   encryption_secret: "zp^P9*i8aR3"
   timeseries:
     enabled: true
+  ta_timeseries:
+    enabled: true
   okta:
     iss: "https://example.okta.com"
   http:
@@ -26,6 +32,9 @@ services:
     bucket: archive
     port: 9000
     host: minio
+  smtp:
+    host: mailhog
+    port: 1025
 
 github:
   bot:
diff --git a/apps/worker/docker-compose.yml b/apps/worker/docker-compose.yml
index 91ab79a60c..203bc5f017 100644
--- a/apps/worker/docker-compose.yml
+++ b/apps/worker/docker-compose.yml
@@ -10,7 +10,7 @@ services:
       - timescale
     volumes:
       - ./:/app/apps/worker
-      - ./docker/test_codecov_config.yml:/config/codecov.yml
+      - ./docker/test.yml:/config/codecov.yml
     environment:
       # Improves pytest-cov performance in python 3.12
       # https://github.com/nedbat/coveragepy/issues/1665#issuecomment-1937075835
diff --git a/apps/worker/docker/test.yml b/apps/worker/docker/test.yml
new file mode 100644
index 0000000000..786370cfb3
--- /dev/null
+++ b/apps/worker/docker/test.yml
@@ -0,0 +1,84 @@
+# !!! ATTENTION !!!
+# This file is a copy of the root `tools/devenv/config/test.yml`.
+# Make sure to keep it in sync!
+
+setup:
+  codecov_url: https://codecov.io
+  debug: no
+  loglvl: INFO
+  encryption_secret: "zp^P9*i8aR3"
+  timeseries:
+    enabled: true
+  ta_timeseries:
+    enabled: true
+  okta:
+    iss: "https://example.okta.com"
+  http:
+    force_https: yes
+    cookie_secret: abc123
+    timeouts:
+      connect: 10
+      receive: 15
+
+services:
+  database_url: postgres://postgres:password@postgres:5432/postgres
+  timeseries_database_url: postgres://postgres:password@timescale:5432/postgres
+  redis_url: redis://redis:6379
+  minio:
+    hash_key: testixik8qdauiab1yiffydimvi72ekq # never change this
+    access_key_id: codecov-default-key
+    secret_access_key: codecov-default-secret
+    verify_ssl: false
+    bucket: archive
+    port: 9000
+    host: minio
+  smtp:
+    host: mailhog
+    port: 1025
+
+github:
+  bot:
+    username: codecov-io
+  integration:
+    id: 254
+    pem: src/certs/github.pem
+
+bitbucket:
+  bot:
+    username: codecov-io
+
+gitlab:
+  bot:
+    username: codecov-io
+
+site:
+  codecov:
+    require_ci_to_pass: yes
+
+  coverage:
+    precision: 2
+    round: down
+    range: "70...100"
+
+    status:
+      project: yes
+      patch: yes
+      changes: no
+
+  parsers:
+    gcov:
+      branch_detection:
+        conditional: yes
+        loop: yes
+        method: no
+        macro: no
+
+    javascript:
+      enable_partials: no
+
+  comment:
+    layout: "reach, diff, flags, files, footer"
+    behavior: default
+    require_changes: no
+    require_base: no
+    require_head: yes
diff --git a/apps/worker/services/test_analytics/tests/conftest.py b/apps/worker/services/test_analytics/tests/conftest.py
index 73c1a3ee46..8aeb6e5d22 100644
--- a/apps/worker/services/test_analytics/tests/conftest.py
+++ b/apps/worker/services/test_analytics/tests/conftest.py
@@ -1,43 +1,14 @@
-import os
-from typing import Any
-
 import pytest
-import yaml
 
-from shared.config import _get_config_instance, get_config
 from shared.storage import get_appropriate_storage_service
 from shared.storage.exceptions import BucketAlreadyExistsError
 
 
 @pytest.fixture
-def storage(mock_configuration):
+def storage():
     storage_service = get_appropriate_storage_service()
     try:
-        storage_service.create_root_storage(get_config("services", "minio", "bucket"))
+        storage_service.create_root_storage("archive")
     except BucketAlreadyExistsError:
         pass
     return storage_service
-
-
-@pytest.fixture()
-def custom_config(tmp_path):
-    config_instance = _get_config_instance()
-    saved_config = config_instance._params
-
-    file_path = tmp_path / "codecov.yml"
-    os.environ["CODECOV_YML"] = str(file_path)
-
-    _conf = config_instance._params or {}
-
-    def set(custom_config: dict[Any, Any]):
-        # clear cache
-        config_instance._params = None
-
-        # for overwrites
-        _conf.update(custom_config)
-        file_path.write_text(yaml.dump(_conf))
-
-    yield set
-
-    os.environ.pop("CODECOV_YML")
-    config_instance._params = saved_config
diff --git a/apps/worker/services/test_analytics/tests/test_ta_cache_rollups.py b/apps/worker/services/test_analytics/tests/test_ta_cache_rollups.py
index 214338ed14..8ba18dc2e5 100644
--- a/apps/worker/services/test_analytics/tests/test_ta_cache_rollups.py
+++ b/apps/worker/services/test_analytics/tests/test_ta_cache_rollups.py
@@ -1,12 +1,10 @@
 import datetime as dt
-from typing import cast
 
 import polars as pl
 import pytest
 
 from services.test_analytics.ta_cache_rollups import VERSION
 from services.test_analytics.utils import calc_test_id
-from shared.config import get_config
 from shared.django_apps.ta_timeseries.models import (
     Testrun,
     TestrunBranchSummary,
@@ -22,9 +20,7 @@ def read_table(
     meta_container: dict[str, str] | None = None,
 ):
     decompressed_table: bytes = storage.read_file(
-        cast(str, get_config("services", "minio", "bucket", default="archive")),
-        storage_path,
-        metadata_container=meta_container,
+        "archive", storage_path, metadata_container=meta_container
     )
     return pl.read_ipc(decompressed_table)
 
diff --git a/apps/worker/services/test_analytics/tests/test_ta_processing.py b/apps/worker/services/test_analytics/tests/test_ta_processing.py
index 8a876d8ca0..f4eeb824f4 100644
--- a/apps/worker/services/test_analytics/tests/test_ta_processing.py
+++ b/apps/worker/services/test_analytics/tests/test_ta_processing.py
@@ -17,27 +17,7 @@
 from shared.django_apps.reports.models import UploadError
 from shared.django_apps.reports.tests.factories import UploadFactory
 from shared.django_apps.ta_timeseries.models import Testrun
-from shared.storage import get_appropriate_storage_service
-from shared.storage.exceptions import BucketAlreadyExistsError, FileNotInStorageError
-
-
-@pytest.fixture(autouse=True)
-def minio_service(custom_config):
-    conf = {
-        "services": {
-            "minio": {
-                "port": 9000,
-            },
-        }
-    }
-
-    custom_config(conf)
-
-    storage = get_appropriate_storage_service(1)
-    try:
-        storage.create_root_storage()
-    except BucketAlreadyExistsError:
-        pass
+from shared.storage.exceptions import FileNotInStorageError
 
 
 @pytest.mark.django_db
@@ -76,33 +56,24 @@ def test_parsing_error():
         (None, False, True),
     ],
 )
-def test_should_delete_archive(expire_raw, uploads, result, custom_config):
-    custom_config(
-        {
-            "services": {
-                "minio": {"expire_raw_after_n_days": expire_raw},
-            }
-        }
+def test_should_delete_archive(
+    expire_raw, uploads, result, mock_configuration, mock_storage
+):
+    mock_configuration.set_params(
+        {"services": {"minio": {"expire_raw_after_n_days": expire_raw}}}
     )
-
     fake_yaml = UserYaml.from_dict(
         {"codecov": {"archive": {"uploads": uploads}}} if uploads is not None else {}
     )
+
     assert should_delete_archive_settings(fake_yaml) == result
 
 
 @pytest.mark.django_db
-def test_rewrite_or_delete_upload_deletes(custom_config):
-    conf = {
-        "services": {
-            "minio": {
-                "port": 9000,
-                "expire_raw_after_n_days": 1,
-            },
-        }
-    }
-
-    custom_config(conf)
+def test_rewrite_or_delete_upload_deletes(mock_configuration, mock_storage):
+    mock_configuration.set_params(
+        {"services": {"minio": {"expire_raw_after_n_days": 1}}}
+    )
 
     upload = UploadFactory(storage_path="url")
     archive_service = ArchiveService(upload.report.commit.repository)
@@ -118,17 +89,10 @@ def test_rewrite_or_delete_upload_deletes(custom_config):
 
 
 @pytest.mark.django_db
-def test_rewrite_or_delete_upload_does_not_delete(custom_config):
-    conf = {
-        "services": {
-            "minio": {
-                "port": 9000,
-                "expire_raw_after_n_days": 1,
-            },
-        }
-    }
-
-    custom_config(conf)
+def test_rewrite_or_delete_upload_does_not_delete(mock_configuration, mock_storage):
+    mock_configuration.set_params(
+        {"services": {"minio": {"expire_raw_after_n_days": 1}}}
+    )
 
     upload = UploadFactory(storage_path="http_url")
     archive_service = ArchiveService(upload.report.commit.repository)
@@ -143,17 +107,7 @@ def test_rewrite_or_delete_upload_does_not_delete(custom_config):
 
 
 @pytest.mark.django_db
-def test_rewrite_or_delete_upload_rewrites(custom_config):
-    conf = {
-        "services": {
-            "minio": {
-                "port": 9000,
-            },
-        }
-    }
-
-    custom_config(conf)
-
+def test_rewrite_or_delete_upload_rewrites(mock_storage):
     upload = UploadFactory(storage_path="url")
     archive_service = ArchiveService(upload.report.commit.repository)
 
diff --git a/libs/shared/docker-compose.yml b/libs/shared/docker-compose.yml
index 1500a4b16d..9709f67bed 100644
--- a/libs/shared/docker-compose.yml
+++ b/libs/shared/docker-compose.yml
@@ -13,6 +13,7 @@ services:
       - ./shared/:/app/libs/shared/shared
       - ./tests/:/app/libs/shared/tests
       - ./.coveragerc:/app/libs/shared/.coveragerc
+      - ./docker/test.yml:/config/codecov.yml
 
   postgres:
     image: postgres:14-alpine
diff --git a/libs/shared/docker/test.yml b/libs/shared/docker/test.yml
new file mode 100644
index 0000000000..786370cfb3
--- /dev/null
+++ b/libs/shared/docker/test.yml
@@ -0,0 +1,84 @@
+# !!! ATTENTION !!!
+# This file is a copy of the root `tools/devenv/config/test.yml`.
+# Make sure to keep it in sync!
+
+setup:
+  codecov_url: https://codecov.io
+  debug: no
+  loglvl: INFO
+  encryption_secret: "zp^P9*i8aR3"
+  timeseries:
+    enabled: true
+  ta_timeseries:
+    enabled: true
+  okta:
+    iss: "https://example.okta.com"
+  http:
+    force_https: yes
+    cookie_secret: abc123
+    timeouts:
+      connect: 10
+      receive: 15
+
+services:
+  database_url: postgres://postgres:password@postgres:5432/postgres
+  timeseries_database_url: postgres://postgres:password@timescale:5432/postgres
+  redis_url: redis://redis:6379
+  minio:
+    hash_key: testixik8qdauiab1yiffydimvi72ekq # never change this
+    access_key_id: codecov-default-key
+    secret_access_key: codecov-default-secret
+    verify_ssl: false
+    bucket: archive
+    port: 9000
+    host: minio
+  smtp:
+    host: mailhog
+    port: 1025
+
+github:
+  bot:
+    username: codecov-io
+  integration:
+    id: 254
+    pem: src/certs/github.pem
+
+bitbucket:
+  bot:
+    username: codecov-io
+
+gitlab:
+  bot:
+    username: codecov-io
+
+site:
+  codecov:
+    require_ci_to_pass: yes
+
+  coverage:
+    precision: 2
+    round: down
+    range: "70...100"
+
+    status:
+      project: yes
+      patch: yes
+      changes: no
+
+  parsers:
+    gcov:
+      branch_detection:
+        conditional: yes
+        loop: yes
+        method: no
+        macro: no
+
+    javascript:
+      enable_partials: no
+
+  comment:
+    layout: "reach, diff, flags, files, footer"
+    behavior: default
+    require_changes: no
+    require_base: no
+    require_head: yes
diff --git a/libs/shared/tests/unit/yaml/test_user_yaml.py b/libs/shared/tests/unit/yaml/test_user_yaml.py
index 21c3d8968f..4204eec297 100644
--- a/libs/shared/tests/unit/yaml/test_user_yaml.py
+++ b/libs/shared/tests/unit/yaml/test_user_yaml.py
@@ -7,6 +7,7 @@
     LEGACY_DEFAULT_SITE_CONFIG,
     PATCH_CENTRIC_DEFAULT_CONFIG,
     PATCH_CENTRIC_DEFAULT_TIME_START,
+    ConfigHelper,
 )
 from shared.yaml import UserYaml, merge_yamls
 from shared.yaml.user_yaml import (
@@ -364,7 +365,10 @@ def test_default_yaml_behavior_change(self):
         assert patch_centric != current_yaml
         assert patch_centric == PATCH_CENTRIC_DEFAULT_CONFIG
 
-    def test_get_final_yaml_default_based_on_owner_context(self):
+    def test_get_final_yaml_default_based_on_owner_context(self, mocker):
+        mocker.patch("shared.config._get_config_instance", return_value=ConfigHelper())
+        mocker.patch("shared.config.ConfigHelper.yaml_content", return_value={})
+
         day_timedelta = datetime.timedelta(days=1)
         patch_centric_expected_onboarding_date = (
             PATCH_CENTRIC_DEFAULT_TIME_START + day_timedelta
diff --git a/apps/worker/docker/test_codecov_config.yml b/tools/devenv/config/test.yml
similarity index 89%
rename from apps/worker/docker/test_codecov_config.yml
rename to tools/devenv/config/test.yml
index 81487a89ff..d49df7c49a 100644
--- a/apps/worker/docker/test_codecov_config.yml
+++ b/tools/devenv/config/test.yml
@@ -7,6 +7,14 @@ setup:
     enabled: true
   ta_timeseries:
     enabled: true
+  okta:
+    iss: "https://example.okta.com"
+  http:
+    force_https: yes
+    cookie_secret: abc123
+    timeouts:
+      connect: 10
+      receive: 15
 
 services:
   database_url: postgres://postgres:password@postgres:5432/postgres
@@ -14,10 +22,10 @@ services:
   redis_url: redis://redis:6379
   minio:
     hash_key: testixik8qdauiab1yiffydimvi72ekq # never change this
-    bucket: archive
     access_key_id: codecov-default-key
     secret_access_key: codecov-default-secret
     verify_ssl: false
+    bucket: archive
     port: 9000
     host: minio
   smtp:


Reviews:
user: login: matt-codecov
body: when we run tests locally we use the already-running containers which have mounted the 1password `codecov.yml`. any thoughts on how we can get this `test.yml` into those containers and shim it into the test invocations?
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-05-08T16:09:06Z
user: login: Swatinem
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-09T06:16:23Z

Review comments:
diff_hunk: @@ -1,10 +1,16 @@
+# !!! ATTENTION !!!
path: apps/codecov-api/docker/test.yml
user: login: matt-codecov
body: you could commit a symlink?
created_at: 2025-05-08T16:04:57Z
author_association: CONTRIBUTOR
diff_hunk: @@ -1,10 +1,16 @@
+# !!! ATTENTION !!!
path: apps/codecov-api/docker/test.yml
user: login: Swatinem
body: I haven’t actually tried, but all the docs are suggestion that docker *really* doesn’t like stuff being loaded/mounted from outside its `context`, also symlinks.
created_at: 2025-05-09T06:16:23Z
author_association: CONTRIBUTOR

PR comments:
user: login: seer-by-sentry[bot]
created_at: 2025-05-08T12:47:41Z
body: ### :white_check_mark: Sentry found no issues in your recent changes :white_check_mark:

author_association: CONTRIBUTOR
user: login: codecov[bot]
created_at: 2025-05-08T12:49:48Z
body: ## [Codecov](https://app.codecov.io/gh/codecov/umbrella/pull/129?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=codecov) Report
All modified and coverable lines are covered by tests :white_check_mark:
> Project coverage is 94.24%. Comparing base [(`885bdf5`)](https://app.codecov.io/gh/codecov/umbrella/commit/885bdf5fd74874d5f1ae7af2028108bf7bd5f3a5?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=codecov) to head [(`17e5f37`)](https://app.codecov.io/gh/codecov/umbrella/commit/17e5f3742e6893ec360612f72dfa5f984285e4a3?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=codecov).
> Report is 5 commits behind head on main.

:white_check_mark: All tests successful. No failed tests found.

:x: Your patch status has failed because the patch coverage (50.00%) is below the target coverage (90.00%). You can increase the patch coverage or adjust the [target](https://docs.codecov.com/docs/commit-status#target) coverage.

<details><summary>Additional details and impacted files</summary>


```diff
@@            Coverage Diff             @@
##             main     #129      +/-   ##
==========================================
+ Coverage   94.20%   94.24%   +0.04%     
==========================================
  Files        1200     1200              
  Lines       44734    44716      -18     
  Branches     1425     1425              
==========================================
+ Hits        42140    42144       +4     
+ Misses       2288     2266      -22     
  Partials      306      306              
```

| [Flag](https://app.codecov.io/gh/codecov/umbrella/pull/129/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=codecov) | Coverage Δ | |
|---|---|---|
| [apiunit](https://app.codecov.io/gh/codecov/umbrella/pull/129/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=codecov) | `96.35% <ø> (ø)` | |
| [shared-docker-uploader](https://app.codecov.io/gh/codecov/umbrella/pull/129/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=codecov) | `89.02% <ø> (+0.17%)` | :arrow_up: |
| [workerintegration](https://app.codecov.io/gh/codecov/umbrella/pull/129/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=codecov) | `61.82% <50.00%> (+0.03%)` | :arrow_up: |
| [workerunit](https://app.codecov.io/gh/codecov/umbrella/pull/129/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=codecov) | `91.03% <100.00%> (-0.02%)` | :arrow_down: |

Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=codecov#carryforward-flags-in-the-pull-request-comment) to find out more.

</details>

[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/codecov/umbrella/pull/129?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=codecov).   
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=codecov).

author_association: NONE
user: login: codecov-notifications[bot]
created_at: 2025-05-08T12:49:54Z
body: ## [Codecov](https://stage-app.codecov.dev/gh/codecov/umbrella/pull/129?dropdown=coverage&src=pr&el=h1) Report
All modified and coverable lines are covered by tests :white_check_mark:

:white_check_mark: All tests successful. No failed tests found.

:x: Your patch status has failed because the patch coverage (50.00%) is below the target coverage (90.00%). You can increase the patch coverage or adjust the [target](https://docs.codecov.com/docs/commit-status#target) coverage.


:loudspeaker: Thoughts on this report? [Let us know!](https://about.codecov.io/pull-request-comment-report/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=codecov)

author_association: NONE
user: login: codspeed-hq[bot]
created_at: 2025-05-08T12:53:21Z
body: <!-- __CODSPEED_PERFORMANCE_REPORT_COMMENT__ -->
<!-- __CODSPEED_INSTRUMENTATION_PERFORMANCE_REPORT_COMMENT__ -->

## [CodSpeed Performance Report](https://codspeed.io/codecov/umbrella/branches/swatinem%2Fone-test-yaml)

### Merging #129 will **not alter performance**

<sub>Comparing <code>swatinem/one-test-yaml</code> (17e5f37) with <code>main</code> (1fe5818)</sub>



### Summary

`✅ 9` untouched benchmarks  




author_association: NONE