Repository: iterative/dvc
Pull request #9153 (2023-03-10 15:38:45)

config: ignore unspecified keys from `feature` section
A warning is shown to the user for the unsupported/ignored keys, and then ignored. The warning is raised only once per interpreter session.



codecov[bot]: ## [Codecov](https://codecov.io/gh/iterative/dvc/pull/9153?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=iterative) Report
Patch coverage: **`100.00`**% and project coverage change: **`-0.31`** :warning:
> Comparison is base [(`1cfdc22`)](https://codecov.io/gh/iterative/dvc/commit/1cfdc22549af4cf8a75d670d27456ca74bc9161c?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=iterative) 92.93% compared to head [(`eeeb421`)](https://codecov.io/gh/iterative/dvc/pull/9153?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=iterative) 92.62%.

<details><summary>Additional details and impacted files</summary>


```diff
@@            Coverage Diff             @@
##             main    #9153      +/-   ##
==========================================
- Coverage   92.93%   92.62%   -0.31%     
==========================================
  Files         456      456              
  Lines       36865    36892      +27     
  Branches     5315     5322       +7     
==========================================
- Hits        34261    34173      -88     
- Misses       2081     2177      +96     
- Partials      523      542      +19     
```


| [Impacted Files](https://codecov.io/gh/iterative/dvc/pull/9153?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=iterative) | Coverage Δ | |
|---|---|---|
| [dvc/config\_schema.py](https://codecov.io/gh/iterative/dvc/pull/9153?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=iterative#diff-ZHZjL2NvbmZpZ19zY2hlbWEucHk=) | `92.72% <100.00%> (-1.87%)` | :arrow_down: |
| [dvc/fs/\_\_init\_\_.py](https://codecov.io/gh/iterative/dvc/pull/9153?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=iterative#diff-ZHZjL2ZzL19faW5pdF9fLnB5) | `100.00% <100.00%> (ø)` | |
| [tests/unit/test\_config.py](https://codecov.io/gh/iterative/dvc/pull/9153?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=iterative#diff-dGVzdHMvdW5pdC90ZXN0X2NvbmZpZy5weQ==) | `96.15% <100.00%> (-3.85%)` | :arrow_down: |

... and [30 files with indirect coverage changes](https://codecov.io/gh/iterative/dvc/pull/9153/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=iterative)

Help us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=iterative). Have a feature suggestion? [Share it here.](https://app.codecov.io/gh/feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=iterative)

</details>

[:umbrella: View full report in Codecov by Sentry](https://codecov.io/gh/iterative/dvc/pull/9153?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=iterative).   
:loudspeaker: Do you have feedback about the report comment? [Let us know in this issue](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=iterative).


Patch:
diff --git a/dvc/config_schema.py b/dvc/config_schema.py
index bdfeed003..5f7d15926 100644
--- a/dvc/config_schema.py
+++ b/dvc/config_schema.py
@@ -1,8 +1,21 @@
+import logging
 import os
 from urllib.parse import urlparse
 
-from funcy import walk_values
-from voluptuous import All, Any, Coerce, Invalid, Lower, Optional, Range, Schema
+from funcy import once, walk_values
+from voluptuous import (
+    REMOVE_EXTRA,
+    All,
+    Any,
+    Coerce,
+    Invalid,
+    Lower,
+    Optional,
+    Range,
+    Schema,
+)
+
+logger = logging.getLogger(__name__)
 
 Bool = All(
     Lower,
@@ -65,6 +78,27 @@ class RelPath(str):
     pass
 
 
+class FeatureSchema(Schema):
+    def __init__(self, schema, required=False):
+        super().__init__(schema, required=required, extra=REMOVE_EXTRA)
+
+    @staticmethod
+    @once
+    def _log_deprecated(keys):
+        # only run this once per session
+        message = "%s config option%s unsupported"
+        paths = ", ".join(f"'feature.{key}'" for key in keys)
+        pluralize = " is" if len(keys) == 1 else "s are"
+        logger.warning(message, paths, pluralize)
+
+    def __call__(self, data):
+        ret = super().__call__(data)
+        extra_keys = data.keys() - ret.keys()
+        if extra_keys:
+            self._log_deprecated(sorted(extra_keys))
+        return ret
+
+
 REMOTE_COMMON = {
     "url": str,
     "checksum_jobs": All(Coerce(int), Range(1)),
@@ -262,12 +296,12 @@ SCHEMA = {
         },
     },
     # section for experimental features
-    "feature": {
-        Optional("machine", default=False): Bool,
-        # enabled by default. It's of no use, kept for backward compatibility.
-        Optional("data_index_cache", default=False): Bool,
-        Optional("parametrization", default=True): Bool,
-    },
+    # only specified keys are validated, others get logged and then ignored/removed
+    "feature": FeatureSchema(
+        {
+            Optional("machine", default=False): Bool,
+        },
+    ),
     "plots": {
         "html_template": str,
         Optional("auto_open", default=False): Bool,
diff --git a/dvc/fs/__init__.py b/dvc/fs/__init__.py
index ae2544de1..59ad0e5a1 100644
--- a/dvc/fs/__init__.py
+++ b/dvc/fs/__init__.py
@@ -121,7 +121,7 @@ def _get_cloud_fs(repo_config, **kwargs):
 
     remote_conf = get_fs_config(repo_config, **kwargs)
     try:
-        remote_conf = SCHEMA["remote"][str](remote_conf)
+        remote_conf = SCHEMA["remote"][str](remote_conf)  # type: ignore[index]
     except Invalid as exc:
         raise RepoConfigError(str(exc)) from None
 


Test patch:
diff --git a/tests/unit/test_config.py b/tests/unit/test_config.py
index d0badc22f..c81efa7fc 100644
--- a/tests/unit/test_config.py
+++ b/tests/unit/test_config.py
@@ -1,3 +1,4 @@
+import logging
 import os
 import textwrap
 
@@ -94,3 +95,22 @@ def test_load_configob_error(tmp_dir, dvc, mocker):
     with pytest.raises(ConfigError):
         with config.edit():
             pass
+
+
+def test_feature_section_supports_arbitrary_values(caplog):
+    with caplog.at_level(logging.WARNING, logger="dvc.config_schema"):
+        data = Config.validate(
+            {
+                "feature": {
+                    "random_key_1": "random_value_1",
+                    "random_key_2": 42,
+                }
+            }
+        )
+
+    assert "random_key_1" not in data
+    assert "random_key_2" not in data
+    assert (
+        "'feature.random_key_1', 'feature.random_key_2' "
+        "config options are unsupported"
+    ) in caplog.text