Repository: great-expectations/great_expectations

Issue:
V2 Custom Expectation Custom Rendering No Longer Works
**Describe the bug**
V2 custom expectation custom rendering (as documented [here](https://docs.greatexpectations.io/en/latest/reference/spare_parts/data_docs_reference.html#customizing-data-docs)) no longer works.

**To Reproduce**
Steps to reproduce the behavior:
1. `git clone --branch jdimatteo/reproduce-custom-expectation-rendering-bug git@github.com:jdimatteo/great_expectations.git && cd great_expectations/tests/integration/docusaurus/connecting_to_your_data/database/`
2. `python3.9 -m venv venv0.13.25 && source venv0.13.25/bin/activate`
3. `pip install -r requirements-0.13.25.txt`
4. `python sqlite_python_example_with_v2_api.py`

This should run without error as it does with GE version 0.12.7 (see details below), however instead fails as shown in this AssertionError:

```
Traceback (most recent call last):
  File "/home/jdimatteo/dev/great_expectations_fork_1/tests/integration/docusaurus/connecting_to_your_data/database/sqlite_python_example_with_v2_api.py", line 161, in <module>
    main()
  File "/home/jdimatteo/dev/great_expectations_fork_1/tests/integration/docusaurus/connecting_to_your_data/database/sqlite_python_example_with_v2_api.py", line 158, in main
    assert_custom_rendering_present_in_html()
  File "/home/jdimatteo/dev/great_expectations_fork_1/tests/integration/docusaurus/connecting_to_your_data/database/sqlite_python_example_with_v2_api.py", line 141, in assert_custom_rendering_present_in_html
    assert "**{" not in html_file_contents
AssertionError
```

This `AssertionError` corresponds to the missing custom rendering as shown here:

![image](https://user-images.githubusercontent.com/3752732/128114821-208db9d3-741b-46eb-bbd2-b4973bc62095.png)

**Expected behavior**

Custom rendering should work as it did in prior GE version 0.12.7:

1. `git clone --branch jdimatteo/reproduce-custom-expectation-rendering-bug git@github.com:jdimatteo/great_expectations.git && cd great_expectations/tests/integration/docusaurus/connecting_to_your_data/database/`
2. `python3.9 -m venv venv0.12.7 && source venv0.12.7/bin/activate`
3. `pip install -r requirements-0.12.7.txt`
4. `python sqlite_python_example_with_v2_api.py`

This runs without error and produces expected custom rendering:

![image](https://user-images.githubusercontent.com/3752732/128114904-f6118a7d-931a-4bf9-aa3c-ad247b3c0f9a.png)

**Environment (please complete the following information):**
 - Operating System: Linux
 - Great Expectations Version: 0.13.25

**Additional context**

I understand that the V2 API is going to be deprecated at some point, however it isn't deprecated yet and the V3 API isn't ready for all use cases because it is documented as experimental [here](https://docs.greatexpectations.io/en/latest/guides/how_to_guides/migrating_versions.html) and there are several bugs preventing usage that might not be fixed for several weeks (e.g. https://github.com/great-expectations/great_expectations/issues/3095). Using an old version of Great Expectations is also not an option, because doing so requires using older versions of other libraries as well such as urllib3 with security vulnerabilities including https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-33503.

See also discussion at https://greatexpectationstalk.slack.com/archives/CUTCNHN82/p1628046306055600.


Discussion:
I'm working on a PR to fix this
Hi @jdimatteo - thanks for raising the issue, and thanks for working on the PR! I'll keep an eye out for it over the next few days. Please let me know if I can be of any assistance.
Do V3 style custom expectations work when using the V2 API?

https://docs.greatexpectations.io/en/latest/guides/how_to_guides/creating_and_editing_expectations/how_to_create_custom_expectations.html

I'll probably test this soon if nobody knows off hand.
Hi @jdimatteo - no, custom expectations were a major impetus for the V3 API and are mutually exclusive from V2 expectations. Expectations in V3 are each in their own file, whereas in V2 they all reside in backend-specific dataset modules (e.g. `pandas_dataset.py`)
Thanks @talagluck for the info! I've got this partially working locally and will submit a PR soon hopefully.

Resolving patch:
diff --git a/great_expectations/render/renderer/content_block/validation_results_table_content_block.py b/great_expectations/render/renderer/content_block/validation_results_table_content_block.py
--- a/great_expectations/render/renderer/content_block/validation_results_table_content_block.py
+++ b/great_expectations/render/renderer/content_block/validation_results_table_content_block.py
@@ -1,7 +1,9 @@
 import logging
 import traceback
+import warnings
 from copy import deepcopy
 
+from great_expectations.core.expectation_configuration import ExpectationConfiguration
 from great_expectations.expectations.core.expect_column_kl_divergence_to_be_less_than import (
     ExpectColumnKlDivergenceToBeLessThan,
 )
@@ -66,6 +68,10 @@ def _get_content_block_fn(cls, expectation_type):
         expectation_string_fn = (
             expectation_string_fn[1] if expectation_string_fn else None
         )
+        if expectation_string_fn is None:
+            expectation_string_fn = cls._get_legacy_v2_api_style_expectation_string_fn(
+                expectation_type
+            )
         if expectation_string_fn is None:
             expectation_string_fn = getattr(cls, "_missing_content_block_fn")
 
@@ -147,7 +153,12 @@ def row_generator_fn(
                 observed_value = [
                     observed_value_renderer[1](result=result)
                     if observed_value_renderer
-                    else "--"
+                    else (
+                        cls._get_legacy_v2_api_observed_value(
+                            expectation_string_fn, result
+                        )
+                        or "--"
+                    )
                 ]
             except Exception as e:
                 exception_traceback = traceback.format_exc()
@@ -168,3 +179,46 @@ def row_generator_fn(
                 return [status_cell + expectation_string_cell + observed_value]
 
         return row_generator_fn
+
+    @classmethod
+    def _get_legacy_v2_api_style_expectation_string_fn(cls, expectation_type):
+        legacy_expectation_string_fn = getattr(cls, expectation_type, None)
+        if legacy_expectation_string_fn is None:
+            # With the V2 API, expectation rendering was implemented by defining a method with the same name as the expectation.
+            # If no legacy rendering is present, return None.
+            return None
+
+        warnings.warn(
+            "V2 API style custom rendering is deprecated and is not fully supported anymore; please switch to V3 API and associated rendering style",
+            DeprecationWarning,
+        )
+
+        def expectation_string_fn_with_legacy_translation(
+            configuration: ExpectationConfiguration, runtime_configuration: dict
+        ):
+            if runtime_configuration is None:
+                runtime_configuration = {}
+
+            # With the V2 API, the expectation string function had a different signature; the below translates from the new signature to the legacy signature.
+            return legacy_expectation_string_fn(
+                expectation=configuration,
+                styling=runtime_configuration.get("styling", None),
+                include_column_name=runtime_configuration.get(
+                    "include_column_name", True
+                ),
+            )
+
+        return expectation_string_fn_with_legacy_translation
+
+    @staticmethod
+    def _get_legacy_v2_api_observed_value(expectation_string_fn, result):
+        if (
+            expectation_string_fn.__name__
+            != "expectation_string_fn_with_legacy_translation"
+        ):
+            # If legacy V2 API style rendering is used, "expectation_string_fn" will be the method defined in the above "_get_legacy_v2_api_style_expectation_string_fn".
+            # If this isn't the case, return None, so we don't do any legacy logic.
+            return None
+
+        # With V2 API style rendering, the result had an "observed_value" entry that could be rendered.
+        return result["result"].get("observed_value")


Test patch:
diff --git a/tests/render/test_render_ValidationResultsTableContentBlockRenderer.py b/tests/render/test_render_ValidationResultsTableContentBlockRenderer.py
--- a/tests/render/test_render_ValidationResultsTableContentBlockRenderer.py
+++ b/tests/render/test_render_ValidationResultsTableContentBlockRenderer.py
@@ -188,6 +188,100 @@ def test_ValidationResultsTableContentBlockRenderer_get_content_block_fn(evr_suc
     assert content_block_fn_output == content_block_fn_expected_output
 
 
+def test_ValidationResultsTableContentBlockRenderer_get_content_block_fn_with_v2_api_style_custom_rendering():
+    """Test backwards support for custom expectation rendering with the V2 API as described at
+    https://docs.greatexpectations.io/en/latest/reference/spare_parts/data_docs_reference.html#customizing-data-docs.
+    """
+
+    custom_expectation_template = "custom_expectation_template"
+    custom_expectation_observed_value = "custom_expectation_observed_value"
+
+    class ValidationResultsTableContentBlockRendererWithV2ApiStyleCustomExpectations(
+        ValidationResultsTableContentBlockRenderer
+    ):
+        @classmethod
+        def expect_custom_expectation_written_in_v2_api_style(
+            cls, expectation, styling=None, include_column_name: bool = True
+        ):
+            return [
+                RenderedStringTemplateContent(
+                    content_block_type="string_template",
+                    string_template={
+                        "template": custom_expectation_template,
+                        "params": expectation.kwargs,
+                        "styling": styling,
+                    },
+                )
+            ]
+
+    evr = ExpectationValidationResult(
+        success=True,
+        result={
+            "observed_value": custom_expectation_observed_value,
+        },
+        exception_info={
+            "raised_exception": False,
+            "exception_message": None,
+            "exception_traceback": None,
+        },
+        expectation_config=ExpectationConfiguration(
+            expectation_type="expect_custom_expectation",
+            kwargs={"column": "a_column_name", "result_format": "SUMMARY"},
+        ),
+    )
+
+    content_block_fn = ValidationResultsTableContentBlockRendererWithV2ApiStyleCustomExpectations._get_content_block_fn(
+        "expect_custom_expectation_written_in_v2_api_style"
+    )
+    content_block_fn_output = content_block_fn(result=evr)
+
+    content_block_fn_expected_output = [
+        [
+            RenderedStringTemplateContent(
+                **{
+                    "content_block_type": "string_template",
+                    "string_template": {
+                        "template": "$icon",
+                        "params": {"icon": "", "markdown_status_icon": "✅"},
+                        "styling": {
+                            "params": {
+                                "icon": {
+                                    "classes": [
+                                        "fas",
+                                        "fa-check-circle",
+                                        "text-success",
+                                    ],
+                                    "tag": "i",
+                                }
+                            }
+                        },
+                    },
+                    "styling": {
+                        "parent": {
+                            "classes": ["hide-succeeded-validation-target-child"]
+                        }
+                    },
+                }
+            ),
+            RenderedStringTemplateContent(
+                **{
+                    "content_block_type": "string_template",
+                    "string_template": {
+                        "template": custom_expectation_template,
+                        "params": {
+                            "column": "a_column_name",
+                            "result_format": "SUMMARY",
+                        },
+                        "styling": None,
+                    },
+                }
+            ),
+            custom_expectation_observed_value,
+        ]
+    ]
+    assert content_block_fn_output == content_block_fn_expected_output
+
+
 def test_ValidationResultsTableContentBlockRenderer_get_observed_value(evr_success):
     evr_no_result_key = ExpectationValidationResult(
         success=True,