Repository: equinor/ert
Pull request #10272: Have DESIGN_MATRIX  not require design_sheet and default_sheet
Author: login: jonathan-eq
Labels: name: release-notes:improvement
description: Automatically categorise as improvement in release notes
Created: 2025-03-11T14:23:30Z | Merged: 2025-03-20T14:32:41Z

**Issue**
Resolves #10270


**Approach**
This commit makes the DESIGN_MATRIX keyword not require arguments for design and default sheet. Now DESIGN_SHEET defaults to `DesignSheet` and `DEFAULT_SHEET` defaults to "DefaultSheet"

(Screenshot of new behavior in GUI if applicable)


- [x] PR title captures the intent of the changes, and is fitting for release notes.
- [x] Added appropriate release note label
- [x] Commit history is consistent and clean, in line with the [contribution guidelines](https://github.com/equinor/ert/blob/main/CONTRIBUTING.md).
- [ ] Make sure unit tests pass locally after every commit (`git rebase -i main
      --exec 'just rapid-tests'`)

## When applicable
- [ ] **When there are user facing changes**: Updated documentation
- [ ] **New behavior or changes to existing untested code**: Ensured that unit tests are added (See [Ground Rules](https://github.com/equinor/ert/blob/main/CONTRIBUTING.md#ground-rules)).
- [ ] **Large PR**: Prepare changes in small commits for more convenient review
- [ ] **Bug fix**: Add regression test for the bug
- [ ] **Bug fix**: Create Backport PR to latest release

<!--
Adding labels helps the maintainers when writing release notes. This is the [list of release note labels](https://github.com/equinor/ert/labels?q=release-notes).
-->


Linked issue #10270: Make design and default sheet not required arguments for the DESIGN_MATRIX keyword
Currently all arguments are required; ie.
`DESIGN_MATRIX poly_design.xlsx DESIGN_SHEET:DesignSheet01 DEFAULT_SHEET:DefaultSheet`

but both named arguments should get default value and thus making them optional
Linked issue comments: user: login: jonathan-eq
created_at: 2025-03-11T14:01:40Z
body: I will set DESIGN_SHEET default value to "DesignSheet01" and DEFAULT_SHEET default value to "DefaultSheet"
author_association: CONTRIBUTOR

Diff:
diff --git a/src/ert/config/design_matrix.py b/src/ert/config/design_matrix.py
index 859324bc4a4..72a6126065f 100644
--- a/src/ert/config/design_matrix.py
+++ b/src/ert/config/design_matrix.py
@@ -43,8 +43,8 @@ def __post_init__(self) -> None:
     def from_config_list(cls, config_list: list[str | dict[str, str]]) -> DesignMatrix:
         filename = Path(cast(str, config_list[0]))
         options = cast(dict[str, str], config_list[1])
-        design_sheet = options.get("DESIGN_SHEET")
-        default_sheet = options.get("DEFAULT_SHEET")
+        design_sheet = options.get("DESIGN_SHEET", "DesignSheet")
+        default_sheet = options.get("DEFAULT_SHEET", "DefaultSheet")
         errors = []
         if filename.suffix not in {
             ".xlsx",
@@ -55,14 +55,6 @@ def from_config_list(cls, config_list: list[str | dict[str, str]]) -> DesignMatr
                     f"DESIGN_MATRIX must be of format .xls or .xlsx; is '{filename}'"
                 ).set_context(config_list)
             )
-        if design_sheet is None:
-            errors.append(
-                ErrorInfo("Missing required DESIGN_SHEET").set_context(config_list)
-            )
-        if default_sheet is None:
-            errors.append(
-                ErrorInfo("Missing required DEFAULT_SHEET").set_context(config_list)
-            )
         if design_sheet is not None and design_sheet == default_sheet:
             errors.append(
                 ErrorInfo(
diff --git a/src/ert/config/parsing/config_schema.py b/src/ert/config/parsing/config_schema.py
index 9a9d026982f..329841bf097 100644
--- a/src/ert/config/parsing/config_schema.py
+++ b/src/ert/config/parsing/config_schema.py
@@ -261,8 +261,8 @@ def install_job_directory_keyword() -> SchemaItem:
 def design_matrix_keyword() -> SchemaItem:
     return SchemaItem(
         kw=ConfigKeys.DESIGN_MATRIX,
-        argc_min=2,
-        argc_max=2,
+        argc_min=1,
+        argc_max=3,
         type_map=[
             SchemaItemType.EXISTING_PATH,
             SchemaItemType.STRING,
diff --git a/tests/ert/ui_tests/cli/analysis/test_design_matrix.py b/tests/ert/ui_tests/cli/analysis/test_design_matrix.py
index 7e52ef403d5..6eb284c4f3d 100644
--- a/tests/ert/ui_tests/cli/analysis/test_design_matrix.py
+++ b/tests/ert/ui_tests/cli/analysis/test_design_matrix.py
@@ -22,7 +22,7 @@
 
 def _create_design_matrix(filename, design_sheet_df, default_sheet_df=None):
     with pd.ExcelWriter(filename) as xl_write:
-        design_sheet_df.to_excel(xl_write, index=False, sheet_name="DesignSheet01")
+        design_sheet_df.to_excel(xl_write, index=False, sheet_name="DesignSheet")
         if default_sheet_df is not None:
             default_sheet_df.to_excel(
                 xl_write, index=False, sheet_name="DefaultSheet", header=False
@@ -54,7 +54,7 @@ def test_run_poly_example_with_design_matrix():
                 NUM_REALIZATIONS 10
                 MIN_REALIZATIONS 1
                 GEN_DATA POLY_RES RESULT_FILE:poly.out
-                DESIGN_MATRIX poly_design.xlsx DESIGN_SHEET:DesignSheet01 DEFAULT_SHEET:DefaultSheet
+                DESIGN_MATRIX poly_design.xlsx
                 INSTALL_JOB poly_eval POLY_EVAL
                 FORWARD_MODEL poly_eval
                 """
@@ -156,7 +156,7 @@ def test_run_poly_example_with_design_matrix_and_genkw_merge(default_values, err
                 MIN_REALIZATIONS 1
                 GEN_DATA POLY_RES RESULT_FILE:poly.out
                 GEN_KW COEFFS my_template my_output coeff_priors
-                DESIGN_MATRIX poly_design.xlsx DESIGN_SHEET:DesignSheet01 DEFAULT_SHEET:DefaultSheet
+                DESIGN_MATRIX poly_design.xlsx DESIGN_SHEET:DesignSheet DEFAULT_SHEET:DefaultSheet
                 INSTALL_JOB poly_eval POLY_EVAL
                 FORWARD_MODEL poly_eval
                 """
@@ -268,8 +268,8 @@ def test_run_poly_example_with_multiple_design_matrix_instances():
                 NUM_REALIZATIONS 10
                 MIN_REALIZATIONS 1
                 GEN_DATA POLY_RES RESULT_FILE:poly.out
-                DESIGN_MATRIX poly_design_1.xlsx DESIGN_SHEET:DesignSheet01 DEFAULT_SHEET:DefaultSheet
-                DESIGN_MATRIX poly_design_2.xlsx DESIGN_SHEET:DesignSheet01 DEFAULT_SHEET:DefaultSheet
+                DESIGN_MATRIX poly_design_1.xlsx
+                DESIGN_MATRIX poly_design_2.xlsx
                 INSTALL_JOB poly_eval POLY_EVAL
                 FORWARD_MODEL poly_eval
                 """
@@ -358,7 +358,7 @@ def test_design_matrix_on_esmda(experiment_mode, ensemble_name, iterations):
                 GEN_KW COEFFS_B coeff_priors_b
                 GEN_KW COEFFS_C coeff_priors_c
                 GEN_DATA POLY_RES RESULT_FILE:poly.out
-                DESIGN_MATRIX design_matrix.xlsx DESIGN_SHEET:DesignSheet01 DEFAULT_SHEET:DefaultSheet
+                DESIGN_MATRIX design_matrix.xlsx
                 INSTALL_JOB poly_eval POLY_EVAL
                 FORWARD_MODEL poly_eval
                 """
diff --git a/tests/ert/unit_tests/config/test_analysis_config.py b/tests/ert/unit_tests/config/test_analysis_config.py
index 7abaf145a78..bc416ba4439 100644
--- a/tests/ert/unit_tests/config/test_analysis_config.py
+++ b/tests/ert/unit_tests/config/test_analysis_config.py
@@ -126,40 +126,6 @@ def test_invalid_design_matrix_format_raises_validation_error():
         )
 
 
-def test_design_matrix_without_design_sheet_raises_validation_error():
-    with pytest.raises(ConfigValidationError, match="Missing required DESIGN_SHEET"):
-        AnalysisConfig.from_dict(
-            {
-                ConfigKeys.DESIGN_MATRIX: [
-                    [
-                        "my_matrix.xlsx",
-                        {
-                            "DESIGN_": "design",
-                            "DEFAULT_SHEET": "default",
-                        },
-                    ]
-                ],
-            }
-        )
-
-
-def test_design_matrix_without_default_sheet_raises_validation_error():
-    with pytest.raises(ConfigValidationError, match="Missing required DEFAULT_SHEET"):
-        AnalysisConfig.from_dict(
-            {
-                ConfigKeys.DESIGN_MATRIX: [
-                    [
-                        "my_matrix.xlsx",
-                        {
-                            "DESIGN_SHEET": "design",
-                            "DEFAULT_": "default",
-                        },
-                    ]
-                ],
-            }
-        )
-
-
 def test_invalid_min_realization_percentage_raises_config_validation_error():
     with pytest.raises(
         ConfigValidationError,
diff --git a/tests/ert/unit_tests/config/test_ert_config.py b/tests/ert/unit_tests/config/test_ert_config.py
index ef27bae01a3..cfa75ee4eb8 100644
--- a/tests/ert/unit_tests/config/test_ert_config.py
+++ b/tests/ert/unit_tests/config/test_ert_config.py
@@ -2020,7 +2020,7 @@ def test_design2params_also_validates_design_matrix(tmp_path, caplog, monkeypatc
         {"DESIGN2PARAMS": mock_design2params},
     )
     ErtConfig.from_file_contents(
-        f"NUM_REALIZATIONS 1\nFORWARD_MODEL DESIGN2PARAMS(<xls_filename>={design_matrix_file}, <designsheet>=DesignSheet01,<defaultssheet>=DefaultSheet)"
+        f"NUM_REALIZATIONS 1\nFORWARD_MODEL DESIGN2PARAMS(<xls_filename>={design_matrix_file}, <designsheet>=DesignSheet,<defaultssheet>=DefaultSheet)"
     )
     assert "DESIGN_MATRIX validation of DESIGN2PARAMS" in caplog.text
 
@@ -2058,8 +2058,8 @@ def test_two_design2params_validates_design_matrix_merging(
     ErtConfig.from_file_contents(
         f"""\
             NUM_REALIZATIONS 1
-            FORWARD_MODEL DESIGN2PARAMS(<xls_filename>={design_matrix_file}, <designsheet>=DesignSheet01,<defaultssheet>=DefaultSheet)
-            FORWARD_MODEL DESIGN2PARAMS(<xls_filename>={design_matrix_file2}, <designsheet>=DesignSheet01,<defaultssheet>=DefaultSheet)
+            FORWARD_MODEL DESIGN2PARAMS(<xls_filename>={design_matrix_file}, <designsheet>=DesignSheet,<defaultssheet>=DefaultSheet)
+            FORWARD_MODEL DESIGN2PARAMS(<xls_filename>={design_matrix_file2}, <designsheet>=DesignSheet,<defaultssheet>=DefaultSheet)
             """
     )
     assert (
@@ -2112,12 +2112,34 @@ def test_three_design2params_validates_design_matrix_merging(
     ErtConfig.from_file_contents(
         f"""\
             NUM_REALIZATIONS 1
-            FORWARD_MODEL DESIGN2PARAMS(<xls_filename>={design_matrix_file}, <designsheet>=DesignSheet01,<defaultssheet>=DefaultSheet)
-            FORWARD_MODEL DESIGN2PARAMS(<xls_filename>={design_matrix_file2}, <designsheet>=DesignSheet01,<defaultssheet>=DefaultSheet)
-            FORWARD_MODEL DESIGN2PARAMS(<xls_filename>={design_matrix_file3}, <designsheet>=DesignSheet01,<defaultssheet>=DefaultSheet)
+            FORWARD_MODEL DESIGN2PARAMS(<xls_filename>={design_matrix_file}, <designsheet>=DesignSheet,<defaultssheet>=DefaultSheet)
+            FORWARD_MODEL DESIGN2PARAMS(<xls_filename>={design_matrix_file2}, <designsheet>=DesignSheet,<defaultssheet>=DefaultSheet)
+            FORWARD_MODEL DESIGN2PARAMS(<xls_filename>={design_matrix_file3}, <designsheet>=DesignSheet,<defaultssheet>=DefaultSheet)
             """
     )
     assert (
         "Design matrix merging would have failed due to: Design Matrices don't have the same active realizations"
         in caplog.text
     )
+
+
+def test_design_matrix_default_argument(tmp_path):
+    design_matrix_file = tmp_path / "my_design_matrix.xlsx"
+    _create_design_matrix(
+        design_matrix_file,
+        pd.DataFrame(
+            {
+                "REAL": [1],
+                "a": [1],
+                "category": ["cat1"],
+            }
+        ),
+        pd.DataFrame([["b", 1], ["c", 2]]),
+    )
+
+    config = ErtConfig.from_file_contents(
+        f"NUM_REALIZATIONS 1\nDESIGN_MATRIX {design_matrix_file}"
+    )
+    assert config.analysis_config.design_matrix
+    assert config.analysis_config.design_matrix.design_sheet == "DesignSheet"
+    assert config.analysis_config.design_matrix.default_sheet == "DefaultSheet"
diff --git a/tests/ert/unit_tests/gui/simulation/test_run_dialog.py b/tests/ert/unit_tests/gui/simulation/test_run_dialog.py
index 0301bf2ad87..3599de7f146 100644
--- a/tests/ert/unit_tests/gui/simulation/test_run_dialog.py
+++ b/tests/ert/unit_tests/gui/simulation/test_run_dialog.py
@@ -684,7 +684,7 @@ def test_that_design_matrix_show_parameters_button_is_visible(
     )
     default_sheet_df = pd.DataFrame([["b", 1], ["c", 2]])
     with pd.ExcelWriter(xls_filename) as xl_write:
-        design_matrix_df.to_excel(xl_write, index=False, sheet_name="DesignSheet01")
+        design_matrix_df.to_excel(xl_write, index=False, sheet_name="DesignSheet")
         default_sheet_df.to_excel(
             xl_write, index=False, sheet_name="DefaultSheet", header=False
         )
@@ -693,9 +693,7 @@ def test_that_design_matrix_show_parameters_button_is_visible(
     with open(config_file, "w", encoding="utf-8") as f:
         f.write("NUM_REALIZATIONS 1")
         if design_matrix_entry:
-            f.write(
-                f"\nDESIGN_MATRIX {xls_filename} DESIGN_SHEET:DesignSheet01 DEFAULT_SHEET:DefaultSheet"
-            )
+            f.write(f"\nDESIGN_MATRIX {xls_filename}")
 
     args_mock = Mock()
     args_mock.config = config_file
diff --git a/tests/ert/unit_tests/sensitivity_analysis/test_design_matrix.py b/tests/ert/unit_tests/sensitivity_analysis/test_design_matrix.py
index 68605b52833..37db2bc3885 100644
--- a/tests/ert/unit_tests/sensitivity_analysis/test_design_matrix.py
+++ b/tests/ert/unit_tests/sensitivity_analysis/test_design_matrix.py
@@ -12,11 +12,11 @@
 
 def _create_design_matrix(xls_path, design_matrix_df, default_sheet_df) -> DesignMatrix:
     with pd.ExcelWriter(xls_path) as xl_write:
-        design_matrix_df.to_excel(xl_write, index=False, sheet_name="DesignSheet01")
+        design_matrix_df.to_excel(xl_write, index=False, sheet_name="DesignSheet")
         default_sheet_df.to_excel(
-            xl_write, index=False, sheet_name="DefaultValues", header=False
+            xl_write, index=False, sheet_name="DefaultSheet", header=False
         )
-    return DesignMatrix(xls_path, "DesignSheet01", "DefaultValues")
+    return DesignMatrix(xls_path, "DesignSheet", "DefaultSheet")
 
 
 @pytest.mark.parametrize(
@@ -145,11 +145,11 @@ def test_read_and_merge_with_existing_parameters(tmp_path, parameters, error_msg
     )
     default_sheet_df = pd.DataFrame([["a", 1], ["b", 4]])
     with pd.ExcelWriter(design_path) as xl_write:
-        design_matrix_df.to_excel(xl_write, index=False, sheet_name="DesignSheet01")
+        design_matrix_df.to_excel(xl_write, index=False, sheet_name="DesignSheet")
         default_sheet_df.to_excel(
-            xl_write, index=False, sheet_name="DefaultValues", header=False
+            xl_write, index=False, sheet_name="DefaultSheet", header=False
         )
-    design_matrix = DesignMatrix(design_path, "DesignSheet01", "DefaultValues")
+    design_matrix = DesignMatrix(design_path, "DesignSheet", "DefaultSheet")
     if error_msg:
         with pytest.raises(ValueError, match=error_msg):
             design_matrix.merge_with_existing_parameters(extra_genkw_config)
@@ -180,11 +180,11 @@ def test_reading_design_matrix(tmp_path):
     )
     default_sheet_df = pd.DataFrame([["one", 1], ["b", 4], ["d", "case_name"]])
     with pd.ExcelWriter(design_path) as xl_write:
-        design_matrix_df.to_excel(xl_write, index=False, sheet_name="DesignSheet01")
+        design_matrix_df.to_excel(xl_write, index=False, sheet_name="DesignSheet")
         default_sheet_df.to_excel(
-            xl_write, index=False, sheet_name="DefaultValues", header=False
+            xl_write, index=False, sheet_name="DefaultSheet", header=False
         )
-    design_matrix = DesignMatrix(design_path, "DesignSheet01", "DefaultValues")
+    design_matrix = DesignMatrix(design_path, "DesignSheet", "DefaultSheet")
     design_params = design_matrix.parameter_configuration
     assert all(param in design_params for param in ("a", "b", "c", "one", "d"))
     assert design_matrix.active_realizations == [True, True, False, False, True]
@@ -218,13 +218,13 @@ def test_reading_design_matrix_validate_reals(tmp_path, real_column, error_msg):
     )
     default_sheet_df = pd.DataFrame()
     with pd.ExcelWriter(design_path) as xl_write:
-        design_matrix_df.to_excel(xl_write, index=False, sheet_name="DesignSheet01")
+        design_matrix_df.to_excel(xl_write, index=False, sheet_name="DesignSheet")
         default_sheet_df.to_excel(
-            xl_write, index=False, sheet_name="DefaultValues", header=False
+            xl_write, index=False, sheet_name="DefaultSheet", header=False
         )
 
     with pytest.raises(ValueError, match=error_msg):
-        DesignMatrix(design_path, "DesignSheet01", "DefaultValues")
+        DesignMatrix(design_path, "DesignSheet", "DefaultSheet")
 
 
 @pytest.mark.parametrize(
@@ -270,13 +270,13 @@ def test_reading_design_matrix_validate_headers(tmp_path, column_names, error_ms
     )
     default_sheet_df = pd.DataFrame([["one", 1], ["b", 4], ["d", 6]])
     with pd.ExcelWriter(design_path) as xl_write:
-        design_matrix_df.to_excel(xl_write, index=False, sheet_name="DesignSheet01")
+        design_matrix_df.to_excel(xl_write, index=False, sheet_name="DesignSheet")
         default_sheet_df.to_excel(
-            xl_write, index=False, sheet_name="DefaultValues", header=False
+            xl_write, index=False, sheet_name="DefaultSheet", header=False
         )
 
     with pytest.raises(ValueError, match=error_msg):
-        DesignMatrix(design_path, "DesignSheet01", "DefaultValues")
+        DesignMatrix(design_path, "DesignSheet", "DefaultSheet")
 
 
 @pytest.mark.parametrize(
@@ -306,13 +306,13 @@ def test_reading_design_matrix_validate_cells(tmp_path, values, error_msg):
     )
     default_sheet_df = pd.DataFrame()
     with pd.ExcelWriter(design_path) as xl_write:
-        design_matrix_df.to_excel(xl_write, index=False, sheet_name="DesignSheet01")
+        design_matrix_df.to_excel(xl_write, index=False, sheet_name="DesignSheet")
         default_sheet_df.to_excel(
-            xl_write, index=False, sheet_name="DefaultValues", header=False
+            xl_write, index=False, sheet_name="DefaultSheet", header=False
         )
 
     with pytest.raises(ValueError, match=error_msg):
-        DesignMatrix(design_path, "DesignSheet01", "DefaultValues")
+        DesignMatrix(design_path, "DesignSheet", "DefaultSheet")
 
 
 @pytest.mark.parametrize(
@@ -352,13 +352,13 @@ def test_reading_default_sheet_validation(tmp_path, data, error_msg):
     )
     default_sheet_df = pd.DataFrame(data)
     with pd.ExcelWriter(design_path) as xl_write:
-        design_matrix_df.to_excel(xl_write, index=False, sheet_name="DesignSheet01")
+        design_matrix_df.to_excel(xl_write, index=False, sheet_name="DesignSheet")
         default_sheet_df.to_excel(
-            xl_write, index=False, sheet_name="DefaultValues", header=False
+            xl_write, index=False, sheet_name="DefaultSheet", header=False
         )
 
     with pytest.raises(ValueError, match=error_msg):
-        DesignMatrix(design_path, "DesignSheet01", "DefaultValues")
+        DesignMatrix(design_path, "DesignSheet", "DefaultSheet")
 
 
 def test_default_values_used(tmp_path):
@@ -373,11 +373,11 @@ def test_default_values_used(tmp_path):
     )
     default_sheet_df = pd.DataFrame([["one", 1], ["b", 4], ["d", "case_name"]])
     with pd.ExcelWriter(design_path) as xl_write:
-        design_matrix_df.to_excel(xl_write, index=False, sheet_name="DesignSheet01")
+        design_matrix_df.to_excel(xl_write, index=False, sheet_name="DesignSheet")
         default_sheet_df.to_excel(
-            xl_write, index=False, sheet_name="DefaultValues", header=False
+            xl_write, index=False, sheet_name="DefaultSheet", header=False
         )
-    design_matrix = DesignMatrix(design_path, "DesignSheet01", "DefaultValues")
+    design_matrix = DesignMatrix(design_path, "DesignSheet", "DefaultSheet")
     df = design_matrix.design_matrix_df
     np.testing.assert_equal(df["one"], np.array([1, 1, 1, 1]))
     np.testing.assert_equal(df["b"], np.array([0, 2, 0, 1]))
diff --git a/tests/ert/unit_tests/test_libres_facade.py b/tests/ert/unit_tests/test_libres_facade.py
index 1f8646a7ca4..4a8c45a021e 100644
--- a/tests/ert/unit_tests/test_libres_facade.py
+++ b/tests/ert/unit_tests/test_libres_facade.py
@@ -276,11 +276,11 @@ def test_save_parameters_to_storage_from_design_dataframe(
     c_values = np.random.default_rng().uniform(-5, 5, 10)
     design_matrix_df = DataFrame({"a": a_values, "b": b_values, "c": c_values})
     with ExcelWriter(design_path) as xl_write:
-        design_matrix_df.to_excel(xl_write, index=False, sheet_name="DesignSheet01")
+        design_matrix_df.to_excel(xl_write, index=False, sheet_name="DesignSheet")
         DataFrame().to_excel(
-            xl_write, index=False, sheet_name="DefaultValues", header=False
+            xl_write, index=False, sheet_name="DefaultSheet", header=False
         )
-    design_matrix = DesignMatrix(design_path, "DesignSheet01", "DefaultValues")
+    design_matrix = DesignMatrix(design_path, "DesignSheet", "DefaultSheet")
     with open_storage(tmp_path / "storage", mode="w") as storage:
         experiment_id = storage.create_experiment(
             parameters=[design_matrix.parameter_configuration]


Reviews:
user: login: larsevj
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-12T09:08:33Z
user: login: larsevj
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-12T09:10:16Z
user: login: jonathan-eq
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-12T10:52:19Z
user: login: jonathan-eq
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-12T10:58:53Z
user: login: jonathan-eq
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-12T11:23:32Z
user: login: larsevj
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-13T08:47:49Z
user: login: larsevj
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-13T08:48:10Z
user: login: larsevj
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-18T11:13:43Z
user: login: jonathan-eq
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-18T11:25:43Z
user: login: xjules
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-18T12:28:29Z
user: login: xjules
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-19T11:30:17Z
user: login: larsevj
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-19T11:40:53Z
user: login: jonathan-eq
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-19T11:46:38Z
user: login: larsevj
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-19T11:51:52Z
user: login: jonathan-eq
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-19T12:00:31Z
user: login: larsevj
body: Looks good! :rocket: Just remember to squash.
state: APPROVED
author_association: COLLABORATOR
submitted_at: 2025-03-20T08:46:15Z

Review comments:
diff_hunk: @@ -103,55 +103,6 @@ def test_invalid_min_realization_raises_config_validation_error():
         )
 
 
-def test_invalid_design_matrix_format_raises_validation_error():
-    with pytest.raises(
-        ConfigValidationError,
-        match="DESIGN_MATRIX must be of format \\.xls or \\.xlsx; is 'my_matrix\\.txt'",
path: tests/ert/unit_tests/config/test_analysis_config.py
user: login: larsevj
body: Why is this test removed?
created_at: 2025-03-12T09:08:33Z
author_association: COLLABORATOR
diff_hunk: @@ -103,55 +103,6 @@ def test_invalid_min_realization_raises_config_validation_error():
         )
 
 
-def test_invalid_design_matrix_format_raises_validation_error():
-    with pytest.raises(
-        ConfigValidationError,
-        match="DESIGN_MATRIX must be of format \\.xls or \\.xlsx; is 'my_matrix\\.txt'",
-    ):
-        AnalysisConfig.from_dict(
-            {
-                ConfigKeys.NUM_REALIZATIONS: 1,
-                ConfigKeys.DESIGN_MATRIX: [
-                    [
-                        "my_matrix.txt",
-                        "DESIGN_SHEET:sheet1",
-                        "DEFAULT_SHEET:sheet2",
-                    ],
-                ],
-            }
-        )
-
-
-def test_design_matrix_without_design_sheet_raises_validation_error():
-    with pytest.raises(ConfigValidationError, match="Missing required DESIGN_SHEET"):
-        AnalysisConfig.from_dict(
-            {
-                ConfigKeys.DESIGN_MATRIX: [
-                    [
-                        "my_matrix.xlsx",
-                        "DESIGN_:design",
-                        "DEFAULT_SHEET:default",
-                    ]
path: tests/ert/unit_tests/config/test_analysis_config.py
user: login: larsevj
body: What does happen if you input this? Or if you input for instance `DESIGN_MATRIX some_file.xlsx design_sheet default` or something else that does not really match the intended input?
created_at: 2025-03-12T09:10:16Z
author_association: COLLABORATOR
diff_hunk: @@ -103,55 +103,6 @@ def test_invalid_min_realization_raises_config_validation_error():
         )
 
 
-def test_invalid_design_matrix_format_raises_validation_error():
-    with pytest.raises(
-        ConfigValidationError,
-        match="DESIGN_MATRIX must be of format \\.xls or \\.xlsx; is 'my_matrix\\.txt'",
path: tests/ert/unit_tests/config/test_analysis_config.py
user: login: jonathan-eq
body: Oops, it was probably caught in the crossfire 🤔 
created_at: 2025-03-12T10:52:19Z
author_association: CONTRIBUTOR
diff_hunk: @@ -103,55 +103,6 @@ def test_invalid_min_realization_raises_config_validation_error():
         )
 
 
-def test_invalid_design_matrix_format_raises_validation_error():
-    with pytest.raises(
-        ConfigValidationError,
-        match="DESIGN_MATRIX must be of format \\.xls or \\.xlsx; is 'my_matrix\\.txt'",
-    ):
-        AnalysisConfig.from_dict(
-            {
-                ConfigKeys.NUM_REALIZATIONS: 1,
-                ConfigKeys.DESIGN_MATRIX: [
-                    [
-                        "my_matrix.txt",
-                        "DESIGN_SHEET:sheet1",
-                        "DEFAULT_SHEET:sheet2",
-                    ],
-                ],
-            }
-        )
-
-
-def test_design_matrix_without_design_sheet_raises_validation_error():
-    with pytest.raises(ConfigValidationError, match="Missing required DESIGN_SHEET"):
-        AnalysisConfig.from_dict(
-            {
-                ConfigKeys.DESIGN_MATRIX: [
-                    [
-                        "my_matrix.xlsx",
-                        "DESIGN_:design",
-                        "DEFAULT_SHEET:default",
-                    ]
path: tests/ert/unit_tests/config/test_analysis_config.py
user: login: jonathan-eq
body: Then we use the default
created_at: 2025-03-12T10:58:53Z
author_association: CONTRIBUTOR
diff_hunk: @@ -103,55 +103,6 @@ def test_invalid_min_realization_raises_config_validation_error():
         )
 
 
-def test_invalid_design_matrix_format_raises_validation_error():
-    with pytest.raises(
-        ConfigValidationError,
-        match="DESIGN_MATRIX must be of format \\.xls or \\.xlsx; is 'my_matrix\\.txt'",
-    ):
-        AnalysisConfig.from_dict(
-            {
-                ConfigKeys.NUM_REALIZATIONS: 1,
-                ConfigKeys.DESIGN_MATRIX: [
-                    [
-                        "my_matrix.txt",
-                        "DESIGN_SHEET:sheet1",
-                        "DEFAULT_SHEET:sheet2",
-                    ],
-                ],
-            }
-        )
-
-
-def test_design_matrix_without_design_sheet_raises_validation_error():
-    with pytest.raises(ConfigValidationError, match="Missing required DESIGN_SHEET"):
-        AnalysisConfig.from_dict(
-            {
-                ConfigKeys.DESIGN_MATRIX: [
-                    [
-                        "my_matrix.xlsx",
-                        "DESIGN_:design",
-                        "DEFAULT_SHEET:default",
-                    ]
path: tests/ert/unit_tests/config/test_analysis_config.py
user: login: jonathan-eq
body: I will add a test^
created_at: 2025-03-12T11:23:32Z
author_association: CONTRIBUTOR
diff_hunk: @@ -103,55 +103,6 @@ def test_invalid_min_realization_raises_config_validation_error():
         )
 
 
-def test_invalid_design_matrix_format_raises_validation_error():
-    with pytest.raises(
-        ConfigValidationError,
-        match="DESIGN_MATRIX must be of format \\.xls or \\.xlsx; is 'my_matrix\\.txt'",
-    ):
-        AnalysisConfig.from_dict(
-            {
-                ConfigKeys.NUM_REALIZATIONS: 1,
-                ConfigKeys.DESIGN_MATRIX: [
-                    [
-                        "my_matrix.txt",
-                        "DESIGN_SHEET:sheet1",
-                        "DEFAULT_SHEET:sheet2",
-                    ],
-                ],
-            }
-        )
-
-
-def test_design_matrix_without_design_sheet_raises_validation_error():
-    with pytest.raises(ConfigValidationError, match="Missing required DESIGN_SHEET"):
-        AnalysisConfig.from_dict(
-            {
-                ConfigKeys.DESIGN_MATRIX: [
-                    [
-                        "my_matrix.xlsx",
-                        "DESIGN_:design",
-                        "DEFAULT_SHEET:default",
-                    ]
path: tests/ert/unit_tests/config/test_analysis_config.py
user: login: larsevj
body: Could you add a test if you try to have more than one positional argument? Or a mix?
created_at: 2025-03-13T08:47:49Z
author_association: COLLABORATOR
diff_hunk: @@ -103,55 +103,6 @@ def test_invalid_min_realization_raises_config_validation_error():
         )
 
 
-def test_invalid_design_matrix_format_raises_validation_error():
-    with pytest.raises(
-        ConfigValidationError,
-        match="DESIGN_MATRIX must be of format \\.xls or \\.xlsx; is 'my_matrix\\.txt'",
-    ):
-        AnalysisConfig.from_dict(
-            {
-                ConfigKeys.NUM_REALIZATIONS: 1,
-                ConfigKeys.DESIGN_MATRIX: [
-                    [
-                        "my_matrix.txt",
-                        "DESIGN_SHEET:sheet1",
-                        "DEFAULT_SHEET:sheet2",
-                    ],
-                ],
-            }
-        )
-
-
-def test_design_matrix_without_design_sheet_raises_validation_error():
-    with pytest.raises(ConfigValidationError, match="Missing required DESIGN_SHEET"):
-        AnalysisConfig.from_dict(
-            {
-                ConfigKeys.DESIGN_MATRIX: [
-                    [
-                        "my_matrix.xlsx",
-                        "DESIGN_:design",
-                        "DEFAULT_SHEET:default",
-                    ]
path: tests/ert/unit_tests/config/test_analysis_config.py
user: login: larsevj
body: I believe it will error in that case.
created_at: 2025-03-13T08:48:09Z
author_association: COLLABORATOR
diff_hunk: @@ -103,55 +103,6 @@ def test_invalid_min_realization_raises_config_validation_error():
         )
 
 
-def test_invalid_design_matrix_format_raises_validation_error():
-    with pytest.raises(
-        ConfigValidationError,
-        match="DESIGN_MATRIX must be of format \\.xls or \\.xlsx; is 'my_matrix\\.txt'",
-    ):
-        AnalysisConfig.from_dict(
-            {
-                ConfigKeys.NUM_REALIZATIONS: 1,
-                ConfigKeys.DESIGN_MATRIX: [
-                    [
-                        "my_matrix.txt",
-                        "DESIGN_SHEET:sheet1",
-                        "DEFAULT_SHEET:sheet2",
-                    ],
-                ],
-            }
-        )
-
-
-def test_design_matrix_without_design_sheet_raises_validation_error():
-    with pytest.raises(ConfigValidationError, match="Missing required DESIGN_SHEET"):
-        AnalysisConfig.from_dict(
-            {
-                ConfigKeys.DESIGN_MATRIX: [
-                    [
-                        "my_matrix.xlsx",
-                        "DESIGN_:design",
-                        "DEFAULT_SHEET:default",
-                    ]
path: tests/ert/unit_tests/config/test_analysis_config.py
user: login: larsevj
body: I believe the behavior of the current test should never occur. We should never just ignore a user input, and I think ERT might raise validation error if you tried to use the input above in the config.
created_at: 2025-03-18T11:13:43Z
author_association: COLLABORATOR
diff_hunk: @@ -103,55 +103,6 @@ def test_invalid_min_realization_raises_config_validation_error():
         )
 
 
-def test_invalid_design_matrix_format_raises_validation_error():
-    with pytest.raises(
-        ConfigValidationError,
-        match="DESIGN_MATRIX must be of format \\.xls or \\.xlsx; is 'my_matrix\\.txt'",
-    ):
-        AnalysisConfig.from_dict(
-            {
-                ConfigKeys.NUM_REALIZATIONS: 1,
-                ConfigKeys.DESIGN_MATRIX: [
-                    [
-                        "my_matrix.txt",
-                        "DESIGN_SHEET:sheet1",
-                        "DEFAULT_SHEET:sheet2",
-                    ],
-                ],
-            }
-        )
-
-
-def test_design_matrix_without_design_sheet_raises_validation_error():
-    with pytest.raises(ConfigValidationError, match="Missing required DESIGN_SHEET"):
-        AnalysisConfig.from_dict(
-            {
-                ConfigKeys.DESIGN_MATRIX: [
-                    [
-                        "my_matrix.xlsx",
-                        "DESIGN_:design",
-                        "DEFAULT_SHEET:default",
-                    ]
path: tests/ert/unit_tests/config/test_analysis_config.py
user: login: jonathan-eq
body: > Could you add a test if you try to have more than one positional argument? Or a mix?

I think I misunderstood what you meant here...
This is the responsibility of the ert config parser anyways and outside of the scope of this issue I think. I will drop testing this
created_at: 2025-03-18T11:25:43Z
author_association: CONTRIBUTOR
diff_hunk: @@ -54,7 +54,7 @@ def test_run_poly_example_with_design_matrix():
                 NUM_REALIZATIONS 10
                 MIN_REALIZATIONS 1
                 GEN_DATA POLY_RES RESULT_FILE:poly.out
-                DESIGN_MATRIX poly_design.xlsx DESIGN_SHEET:DesignSheet01 DEFAULT_SHEET:DefaultSheet
+                DESIGN_MATRIX poly_design.xlsx DESIGN_SHEET:DesignSheet DEFAULT_SHEET:DefaultSheet
path: tests/ert/ui_tests/cli/analysis/test_design_matrix.py
user: login: xjules
body: Maybe now you can remove all those params?
created_at: 2025-03-18T12:28:29Z
author_association: CONTRIBUTOR
diff_hunk: @@ -122,30 +122,63 @@ def test_invalid_design_matrix_format_raises_validation_error():
         )
 
 
-def test_design_matrix_without_design_sheet_raises_validation_error():
-    with pytest.raises(ConfigValidationError, match="Missing required DESIGN_SHEET"):
+def test_design_matrix_with_unknown_argument_uses_default(tmp_path, monkeypatch):
+    monkeypatch.chdir(tmp_path)
+    with pd.ExcelWriter("my_design_matrix.xlsx") as xl_write:
+        design_matrix_df = pd.DataFrame(
+            {
+                "REAL": [0, 1, 2],
+                "a": [1, 2, 3],
+                "b": [0, 2, 0],
+            }
+        )
+        default_sheet_df = pd.DataFrame([["a", 1], ["b", 4]])
+        design_matrix_df.to_excel(xl_write, index=False, sheet_name="DesignSheet")
+        default_sheet_df.to_excel(
+            xl_write, index=False, sheet_name="my_default_sheet", header=False
+        )
+    assert (
         AnalysisConfig.from_dict(
             {
+                ConfigKeys.NUM_REALIZATIONS: 3,
                 ConfigKeys.DESIGN_MATRIX: [
                     [
-                        "my_matrix.xlsx",
-                        "DESIGN_:design",
-                        "DEFAULT_SHEET:default",
+                        os.path.abspath("my_design_matrix.xlsx"),
+                        "foo_argument:bar",
+                        "DEFAULT_SHEET:my_default_sheet",
                     ]
                 ],
             }
-        )
+        ).design_matrix.design_sheet
+        == "DesignSheet"
+    )
 
 
-def test_design_matrix_without_default_sheet_raises_validation_error():
-    with pytest.raises(ConfigValidationError, match="Missing required DEFAULT_SHEET"):
+def test_design_matrix_with_too_many_arguments_fails(tmp_path, monkeypatch):
+    monkeypatch.chdir(tmp_path)
+    with pd.ExcelWriter("my_design_matrix.xlsx") as xl_write:
+        design_matrix_df = pd.DataFrame(
+            {
+                "REAL": [0, 1, 2],
+                "a": [1, 2, 3],
+                "b": [0, 2, 0],
+            }
+        )
+        default_sheet_df = pd.DataFrame([["a", 1], ["b", 4]])
+        design_matrix_df.to_excel(xl_write, index=False, sheet_name="DesignSheet")
+        default_sheet_df.to_excel(
+            xl_write, index=False, sheet_name="my_default_sheet", header=False
+        )
+    with pytest.raises(ConfigValidationError, match="Invalid argument 'too_many'"):
         AnalysisConfig.from_dict(
             {
+                ConfigKeys.NUM_REALIZATIONS: 3,
                 ConfigKeys.DESIGN_MATRIX: [
                     [
-                        "my_matrix.xlsx",
-                        "DESIGN_SHEET:design",
-                        "DEFAULT_:default",
+                        os.path.abspath("my_design_matrix.xlsx"),
+                        "too_many",
path: tests/ert/unit_tests/config/test_analysis_config.py
user: login: xjules
body: ["too","many","arguments"] :smile: 
created_at: 2025-03-19T11:30:17Z
author_association: CONTRIBUTOR
diff_hunk: @@ -103,55 +103,6 @@ def test_invalid_min_realization_raises_config_validation_error():
         )
 
 
-def test_invalid_design_matrix_format_raises_validation_error():
-    with pytest.raises(
-        ConfigValidationError,
-        match="DESIGN_MATRIX must be of format \\.xls or \\.xlsx; is 'my_matrix\\.txt'",
-    ):
-        AnalysisConfig.from_dict(
-            {
-                ConfigKeys.NUM_REALIZATIONS: 1,
-                ConfigKeys.DESIGN_MATRIX: [
-                    [
-                        "my_matrix.txt",
-                        "DESIGN_SHEET:sheet1",
-                        "DEFAULT_SHEET:sheet2",
-                    ],
-                ],
-            }
-        )
-
-
-def test_design_matrix_without_design_sheet_raises_validation_error():
-    with pytest.raises(ConfigValidationError, match="Missing required DESIGN_SHEET"):
-        AnalysisConfig.from_dict(
-            {
-                ConfigKeys.DESIGN_MATRIX: [
-                    [
-                        "my_matrix.xlsx",
-                        "DESIGN_:design",
-                        "DEFAULT_SHEET:default",
-                    ]
path: tests/ert/unit_tests/config/test_analysis_config.py
user: login: larsevj
body: Will you remove this test then?
created_at: 2025-03-19T11:40:53Z
author_association: COLLABORATOR
diff_hunk: @@ -103,55 +103,6 @@ def test_invalid_min_realization_raises_config_validation_error():
         )
 
 
-def test_invalid_design_matrix_format_raises_validation_error():
-    with pytest.raises(
-        ConfigValidationError,
-        match="DESIGN_MATRIX must be of format \\.xls or \\.xlsx; is 'my_matrix\\.txt'",
-    ):
-        AnalysisConfig.from_dict(
-            {
-                ConfigKeys.NUM_REALIZATIONS: 1,
-                ConfigKeys.DESIGN_MATRIX: [
-                    [
-                        "my_matrix.txt",
-                        "DESIGN_SHEET:sheet1",
-                        "DEFAULT_SHEET:sheet2",
-                    ],
-                ],
-            }
-        )
-
-
-def test_design_matrix_without_design_sheet_raises_validation_error():
-    with pytest.raises(ConfigValidationError, match="Missing required DESIGN_SHEET"):
-        AnalysisConfig.from_dict(
-            {
-                ConfigKeys.DESIGN_MATRIX: [
-                    [
-                        "my_matrix.xlsx",
-                        "DESIGN_:design",
-                        "DEFAULT_SHEET:default",
-                    ]
path: tests/ert/unit_tests/config/test_analysis_config.py
user: login: jonathan-eq
body: I removed both tests. Neither of them were specific for design_matrix or this PR. There are ert_parser tests already testing this (i hope).
created_at: 2025-03-19T11:46:38Z
author_association: CONTRIBUTOR
diff_hunk: @@ -358,7 +358,7 @@ def test_design_matrix_on_esmda(experiment_mode, ensemble_name, iterations):
                 GEN_KW COEFFS_B coeff_priors_b
                 GEN_KW COEFFS_C coeff_priors_c
                 GEN_DATA POLY_RES RESULT_FILE:poly.out
-                DESIGN_MATRIX design_matrix.xlsx DESIGN_SHEET:DesignSheet01 DEFAULT_SHEET:DefaultSheet
path: tests/ert/ui_tests/cli/analysis/test_design_matrix.py
user: login: larsevj
body: Do we have any tests left at all that do not use the default values?
created_at: 2025-03-19T11:51:51Z
author_association: COLLABORATOR
diff_hunk: @@ -358,7 +358,7 @@ def test_design_matrix_on_esmda(experiment_mode, ensemble_name, iterations):
                 GEN_KW COEFFS_B coeff_priors_b
                 GEN_KW COEFFS_C coeff_priors_c
                 GEN_DATA POLY_RES RESULT_FILE:poly.out
-                DESIGN_MATRIX design_matrix.xlsx DESIGN_SHEET:DesignSheet01 DEFAULT_SHEET:DefaultSheet
path: tests/ert/ui_tests/cli/analysis/test_design_matrix.py
user: login: jonathan-eq
body: No, but maybe we should 🤔 
created_at: 2025-03-19T12:00:31Z
author_association: CONTRIBUTOR

PR comments:
user: login: codspeed-hq[bot]
created_at: 2025-03-11T15:02:22Z
body: <!-- __CODSPEED_PERFORMANCE_REPORT_COMMENT__ -->
<!-- __CODSPEED_INSTRUMENTATION_PERFORMANCE_REPORT_COMMENT__ -->

## [CodSpeed Performance Report](https://codspeed.io/equinor/ert/branches/jonathan-eq%3Adesign_matrix)

### Merging #10272 will **not alter performance**

<sub>Comparing <code>jonathan-eq:design_matrix</code> (71e7ac1) with <code>main</code> (34eb8f8)</sub>



### Summary

`✅ 25` untouched benchmarks  




author_association: NONE
user: login: larsevj
created_at: 2025-03-12T09:06:20Z
body: Would not `DesignSheet` be a more natural default than `DesignSheet01` ?
author_association: COLLABORATOR
user: login: jonathan-eq
created_at: 2025-03-12T09:08:05Z
body: > Would not `DesignSheet` be a more natural default than `DesignSheet01` ?

Maybe, but that was just the example in the issue. What do you think @JHolba @xjules ?
author_association: CONTRIBUTOR
user: login: jonathan-eq
created_at: 2025-03-12T10:32:44Z
body: > Would not `DesignSheet` be a more natural default than `DesignSheet01` ?

This is also the current default in design2params, but there the defaultsheet default is "DefaultValues"
author_association: CONTRIBUTOR
user: login: sondreso
created_at: 2025-03-12T14:15:29Z
body: > > Would not `DesignSheet` be a more natural default than `DesignSheet01` ?
> 
> This is also the current default in design2params, but there the defaultsheet default is "DefaultValues"

We should have the same defaults as DESIGN2PARAMS here, otherwise we might cause unnecessary confusion when transitioning users over 🙂 
author_association: CONTRIBUTOR
user: login: larsevj
created_at: 2025-03-13T08:39:38Z
body: > > Would not `DesignSheet` be a more natural default than `DesignSheet01` ?
> 
> This is also the current default in design2params, but there the defaultsheet default is "DefaultValues"

If I try to run without arguments for design_sheet and default_sheet I get the following error from design2params:
```
File /data/design_matrix.xlsx is probably not of correct type. Failed with exception 'Worksheet named '<designsheet>' not found'
```
author_association: COLLABORATOR
user: login: larsevj
created_at: 2025-03-13T08:54:33Z
body: I suspect the actual default values today are `<designsheet>` and `<defaultssheet>` ..

Which hopefully no one uses.
author_association: COLLABORATOR
user: login: jonathan-eq
created_at: 2025-03-14T07:54:07Z
body: > <designsheet

I am not sure, that seems to be the values expected to be substituted 
author_association: CONTRIBUTOR