Repository: great-expectations/great_expectations

Issue:
Custom inherited expectation class : can't override a required parameter
**Describe the bug**
Trying to create a custom expectation using custom parameters based on [this doc](https://docs.greatexpectations.io/docs/guides/expectations/creating_custom_expectations/how_to_create_custom_parameterized_expectations).
I want to check that a value (as a string) is a positive integer value, so instead of creating a `expect_column_values_to_match_regex` and set the regex everytime, I want to create a `expect_column_values_as_string_to_be_positive_integer` with a fixed regex `"^\\d+$"`
When I run it, it fails because the "regex" parameter is "not set" !

**To Reproduce**
My class code :

```
from great_expectations.core import ExpectationConfiguration
from great_expectations.expectations.core import ExpectColumnValuesToMatchRegex
from typing import Optional

class ExpectColumnValuesAsStringToBePositiveInteger(ExpectColumnValuesToMatchRegex):
    default_kwarg_values = {
        "regex": "^\\d+$",
    }

    def validate_configuration(self, configuration: Optional[ExpectationConfiguration]):
        super().validate_configuration(configuration)
        assert "regex" not in configuration.kwargs, "regex cannot be altered"

```

**Expected behavior**
Based on the documentation I expect this class to work, but I get an error:

_expect_column_values_as_string_to_be_positive_integer raised an exception: regex is required_

```
great_expectations/expectations/core/expect_column_values_to_match_regex.py", line 115, in validate_configuration
raise InvalidExpectationConfigurationError(str(e))
great_expectations.exceptions.exceptions.InvalidExpectationConfigurationError: regex is required
```

**Environment (please complete the following information):**
- Operating System: windows
- Great Expectations Version: 0.14.10

**Additional context**
Please fix this issue or remove the documentation, I did not succeed a single time in creating a custom expectation!! (see also this other issue that is almost 4 months old now: https://github.com/great-expectations/great_expectations/issues/3776)
Although this is a VERY useful feature! I would be able to create a ton of custom expectations for my users simply by overriding parameters like these, think of expectations like:
- expect column value to be valid currency (based on expect_column_values_to_be_in_set)
- expect column value to be valid country (based on expect_column_values_to_be_in_set)
- expect column as a string to be a valid float format ((based on expect_column_values_to_match_regex)
- .....

These are "public" examples but I have a lot of business expectations like these that would be very useful.


Discussion:
Also, if you want to maintain the feature of creating a custom expectation, this should be very useful to be able to override the result string that is generated after a run for the data docs....
Hey @pmariac ! Thank you for raising this; I have eyes on this issue as we speak. Stay tuned!
Seems like it is already "fixed" as the "custom parameterized" way of creating an expectation has been removed from the doc yesterday with the 0.14.11 release ??! Or maybe am I missing something here?

I'll have a look at this on next week.
Interesting timing! Take a look at #4455; there was a bug (thank you for identifying 🙇 ) preventing the workflow detailed in the guide. That guide should come live again soon, and the workflow you were following should be repaired. There may be similar issues with related parameterized workflows we're investigating currently. Thank you again for surfacing @pmariac !
Hey @pmariac! I'll be looking into this case over the next couple of days; in the meantime you may be interested in [this documentation](https://docs.greatexpectations.io/docs/guides/expectations/creating_custom_expectations/how_to_create_custom_regex_based_column_map_expectations); the pattern and workflow from the above PR for `RegexBasedColumnMapExpectations` (which I believe fits your use case) is now live!

Resolving patch:
diff --git a/great_expectations/expectations/core/expect_column_values_to_match_regex.py b/great_expectations/expectations/core/expect_column_values_to_match_regex.py
--- a/great_expectations/expectations/core/expect_column_values_to_match_regex.py
+++ b/great_expectations/expectations/core/expect_column_values_to_match_regex.py
@@ -90,6 +90,7 @@ class ExpectColumnValuesToMatchRegex(ColumnMapExpectation):
         "result_format": "BASIC",
         "include_config": True,
         "catch_exceptions": True,
+        "regex": "(?s).*",
     }
     args_keys = (
         "column",
@@ -102,14 +103,18 @@ def validate_configuration(
         super().validate_configuration(configuration)
         if configuration is None:
             configuration = self.configuration
+
+        # supports extensibility by allowing value_set to not be provided in config but captured via child-class default_kwarg_values, e.g. parameterized expectations
+        regex = configuration.kwargs.get("regex") or self.default_kwarg_values.get(
+            "regex"
+        )
+
         try:
-            assert "regex" in configuration.kwargs, "regex is required"
-            assert isinstance(
-                configuration.kwargs["regex"], (str, dict)
-            ), "regex must be a string"
-            if isinstance(configuration.kwargs["regex"], dict):
+            assert "regex" in configuration.kwargs or regex, "regex is required"
+            assert isinstance(regex, (str, dict)), "regex must be a string"
+            if isinstance(regex, dict):
                 assert (
-                    "$PARAMETER" in configuration.kwargs["regex"]
+                    "$PARAMETER" in regex
                 ), 'Evaluation Parameter dict for regex kwarg must have "$PARAMETER" key.'
         except AssertionError as e:
             raise InvalidExpectationConfigurationError(str(e))


Test patch:
diff --git a/tests/expectations/core/test_expect_column_values_to_match_regex_parameterized.py b/tests/expectations/core/test_expect_column_values_to_match_regex_parameterized.py
new file mode 100644
--- /dev/null
+++ b/tests/expectations/core/test_expect_column_values_to_match_regex_parameterized.py
@@ -0,0 +1,69 @@
+from typing import Optional
+
+from great_expectations.core import ExpectationConfiguration
+from great_expectations.expectations.core import ExpectColumnValuesToMatchRegex
+
+
+class ExpectColumnValuesAsStringToBePositiveInteger(ExpectColumnValuesToMatchRegex):
+    default_kwarg_values = {
+        "regex": "^\\d+$",
+    }
+
+    def validate_configuration(self, configuration: Optional[ExpectationConfiguration]):
+        super().validate_configuration(configuration)
+        assert "regex" not in configuration.kwargs, "regex cannot be altered"
+
+
+import pandas as pd
+import pytest
+
+from great_expectations.core.batch import RuntimeBatchRequest
+from great_expectations.data_context import DataContext
+
+
+def test_expect_column_values_as_string_to_be_positive_integers_pass(
+    data_context_with_datasource_pandas_engine,
+):
+    context: DataContext = data_context_with_datasource_pandas_engine
+
+    df = pd.DataFrame({"a": ["1", "2", "3", "4", "5"]})
+
+    batch_request = RuntimeBatchRequest(
+        datasource_name="my_datasource",
+        data_connector_name="default_runtime_data_connector_name",
+        data_asset_name="my_data_asset",
+        runtime_parameters={"batch_data": df},
+        batch_identifiers={"default_identifier_name": "my_identifier"},
+    )
+    validator = context.get_validator(
+        batch_request=batch_request,
+        create_expectation_suite_with_name="test",
+    )
+
+    assert validator.expect_column_values_as_string_to_be_positive_integer(
+        column="a"
+    ).success
+
+
+def test_expect_column_values_as_string_to_be_positive_integers_fail(
+    data_context_with_datasource_pandas_engine,
+):
+    context: DataContext = data_context_with_datasource_pandas_engine
+
+    df = pd.DataFrame({"a": ["1", "2", "3", "4", "a"]})
+
+    batch_request = RuntimeBatchRequest(
+        datasource_name="my_datasource",
+        data_connector_name="default_runtime_data_connector_name",
+        data_asset_name="my_data_asset",
+        runtime_parameters={"batch_data": df},
+        batch_identifiers={"default_identifier_name": "my_identifier"},
+    )
+    validator = context.get_validator(
+        batch_request=batch_request,
+        create_expectation_suite_with_name="test",
+    )
+
+    assert not validator.expect_column_values_as_string_to_be_positive_integer(
+        column="a"
+    ).success