Repository: great-expectations/great_expectations

Issue:
GE can't parse a row condition in a conditional expectation
**Describe the bug**
GE can't parse a row condition in a conditional expectation if there is a comparison between strings and the string in the condition contains a whitespace.

**To Reproduce**
**First Case: (ExpectationConfiguration in json format)**
{
    "expectation_type": "expect_column_values_to_not_be_null",
    "column": column_A,	
    "condition_parser": "great_expectations__experimental__",
    "mostly": 1.0,
    "row_condition": "col(\\"communication_type_desc_orig\\") == \\"Email\\""
}
Result: the expectation executes without any issues.


**Second Case: (ExpectationConfiguration in json format)**
{
    "expectation_type": "expect_column_values_to_not_be_null",
    "column": column_B,
    "condition_parser": "great_expectations__experimental__",
    "mostly": 1.0,
    "row_condition": "col(\\"communication_type_desc_orig\\")==\\"Secure Message\\""
}

Result: The expectation fails to execute. Error: unable to parse condition: col("communication_type_desc_orig")=="Secure Message"
`Stacktrace:`
 `File "/usr/local/lib/python3.9/site-packages/great_expectations/expectations/row_conditions.py", line 111, in _parse_great_expectations_condition`
 `return condition.parseString(row_condition)`
 `File "/usr/local/lib/python3.9/site-packages/pyparsing/core.py", line 1141, in parse_string
 raise exc.with_traceback(None)`
 `pyparsing.exceptions.ParseException: Expected {{Combine:({Suppress:('col(\\\"') W:(A-Za-z, .0-9A-Z_a-z) Suppress:('\\\")')}) '.NOTNULL()'} ^ {Combine:({Suppress:('col(\\\"') W:(A-Za-z, .0-9A-Z_a-z) Suppress:('\\\")')}) {'>' ^ '<' ^ '>=' ^ '<=' ^ '=='} {Re:('[+-]?\\\\d+(?:\\\\.\\\\d*)?(?:[eE][+-]?\\\\d+)?') ^ {Suppress:('\\\"') W:(.0-9A-Z_a-z) Suppress:('\\\"')} ^ {Suppress:(\\\"'\\\") W:(.0-9A-Z_a-z) Suppress:(\\\"'\\\")}}}}, found 'Message'  (at char 45), (line:1, col:46)
`

**Expected behavior**
The expectation executes without any errors.

**Environment (please complete the following information):**
 - Operating System: Linux
 - Great Expectations Version: 0.15.31
 - Execution Engine: SqlAlchemyExecutionEngine



Discussion:
Hi,
I am having a similar issue. I am trying similar approach for redshift database to use row_condition with condition_parser as per below, however I am keep getting an error `unable to parse condition:`

## expectation
```
    {
      "expectation_type": "expect_column_values_to_be_between",
      "kwargs": {
        "column": "current lvr",
        "max_value": 100,
        "min_value": 1,
        "row_condition": "col(\"number of days in advance\")<=0",
        "condition_parser": "great_expectations__experimental__"
      },
      "meta": {}
    }
```

## Error:
```
"exception_message": "unable to parse condition: col(\"number of days in advance\")<=0",
"exception_traceback": "Traceback (most recent call last):
File \"/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/great_expectations/expectations/row_conditions.py\", line 111, in _parse_great_expectations_condition
return condition.parseString(row_condition)
File \"/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pyparsing/core.py\", line 1141, in parse_string
raise exc.with_traceback(None)
pyparsing.exceptions.ParseException: Expected {{Combine:({Suppress:('col(\"') W:(A-Za-z, .0-9A-Z_a-z) Suppress:('\")')}) '.NOTNULL()'} ^ {Combine:({Suppress:('col(\"') W:(A-Za-z, .0-9A-Z_a-z) Suppress:('\")')}) {'>' ^ '<' ^ '>=' ^ '<=' ^ '=='} {Re:('[+-]?\\d+(?:\\.\\d*)?(?:[eE][+-]?\\d+)?') ^ {Suppress:('\"') W:(.0-9A-Z_a-z) Suppress:('\"')} ^ {Suppress:(\"'\") W:(.0-9A-Z_a-z) Suppress:(\"'\")}}}}, found ' '  (at char 11), (line:1, col:12)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File \"/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/great_expectations/execution_engine/execution_engine.py\", line 421, in resolve_metrics
] = self.resolve_metric_bundle(metric_fn_bundle)
File \"/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/great_expectations/execution_engine/sqlalchemy_execution_engine.py\", line 996, in resolve_metric_bundle
selectable: Any = self.get_domain_records(
File \"/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/great_expectations/execution_engine/sqlalchemy_execution_engine.py\", line 616, in get_domain_records
parsed_condition = parse_condition_to_sqlalchemy(
File \"/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/great_expectations/expectations/row_conditions.py\", line 152, in parse_condition_to_sqlalchemy
parsed = _parse_great_expectations_condition(row_condition)
File \"/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/great_expectations/expectations/row_conditions.py\", line 113, in _parse_great_expectations_condition
raise ConditionParserError(f\"unable to parse condition: {row_condition}\")
great_expectations.expectations.row_conditions.ConditionParserError: unable to parse condition: col(\"number of days in advance\")<=0
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File \"/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/great_expectations/validator/validation_graph.py\", line 177, in resolve_validation_graph
self._execution_engine.resolve_metrics(
File \"/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/great_expectations/execution_engine/execution_engine.py\", line 424, in resolve_metrics
raise ge_exceptions.MetricResolutionError(
great_expectations.exceptions.exceptions.MetricResolutionError: unable to parse condition: col(\"number of days in advance\")<=0
",
```
Issue occurs when column name or the value to compare contains a space. If I try column name without a space (e.g. `current_id`), then it works fine. However, column name with space in it fails with an error.

Could someone please help me with this issue?
Hey @Vitalis-Tk & @zvpanchal ! Thanks for reaching out. I've been able to reproduce the behavior you're experiencing here, and will be discussing this functionality internally with the team in the coming week.
Hi @austiezr - Is there any update on this issue?

Using `<>` operator in row_condition throws parsing error. Is there any workaround to use not equal operator in the row_conditions? 

```
        "row_condition": "col(\"account_state\")<>'CLOSED'",
        "condition_parser": "great_expectations__experimental__"
```
Error that I get:

```
File "/.pyenv/versions/3.11.0/lib/python3.11/site-packages/pyparsing/core.py", line 1141, in parse_string
raise exc.with_traceback(None)
pyparsing.exceptions.ParseException: Expected {{Combine:({Suppress:('col("') W:(A-Za-z, .0-9A-Z_a-z) Suppress:('")')}) '.NOTNULL()'} ^ {Combine:({Suppress:('col("') W:(A-Za-z, .0-9A-Z_a-z) Suppress:('")')}) {'>' ^ '<' ^ '>=' ^ '<=' ^ '=='} {Re:('[+-]?\d+(?:\.\d*)?(?:[eE][+-]?\d+)?') ^ {Suppress:('"') W:(.0-9A-Z_a-z) Suppress:('"')} ^ {Suppress:("'") W:(.0-9A-Z_a-z) Suppress:("'")}}}}, found '>' (at char 21), (line:1, col:22)
```

I am using great_expectations v0.15.43 (v3 API)
Hi @zvpanchal and @Vitalis-Tk I wanted to provide an update that the Core Engineering team has picked up this issue. More updates to come very soon. Thank you for your patience and for your follow up :) 
> Hi @zvpanchal and @Vitalis-Tk I wanted to provide an update that the Core Engineering team has picked up this issue. More updates to come very soon. Thank you for your patience and for your follow up :)



Resolving patch:
diff --git a/great_expectations/expectations/row_conditions.py b/great_expectations/expectations/row_conditions.py
--- a/great_expectations/expectations/row_conditions.py
+++ b/great_expectations/expectations/row_conditions.py
@@ -42,9 +42,10 @@ def _set_notnull(s, l, t) -> None:  # noqa: E741 # ambiguous name `l`
     t["notnull"] = True
 
 
+SPACE_CHARS = " "
 column_name = Combine(
     Suppress(Literal('col("'))
-    + Word(alphas, f"{alphanums}_.").setResultsName("column")
+    + Word(alphas, f"{alphanums}_-.").setResultsName("column")
     + Suppress(Literal('")'))
 )
 gt = Literal(">")
@@ -56,7 +57,7 @@ def _set_notnull(s, l, t) -> None:  # noqa: E741 # ambiguous name `l`
 ops = (gt ^ lt ^ ge ^ le ^ eq ^ ne).setResultsName("op")
 fnumber = Regex(r"[+-]?\d+(?:\.\d*)?(?:[eE][+-]?\d+)?").setResultsName("fnumber")
 punctuation_without_apostrophe = punctuation.replace('"', "").replace("'", "")
-condition_value_chars = alphanums + punctuation_without_apostrophe
+condition_value_chars = alphanums + punctuation_without_apostrophe + SPACE_CHARS
 condition_value = Suppress('"') + Word(f"{condition_value_chars}._").setResultsName(
     "condition_value"
 ) + Suppress('"') ^ Suppress("'") + Word(f"{condition_value_chars}._").setResultsName(


Test patch:
diff --git a/tests/expectations/test_row_conditions.py b/tests/expectations/test_row_conditions.py
--- a/tests/expectations/test_row_conditions.py
+++ b/tests/expectations/test_row_conditions.py
@@ -22,6 +22,27 @@ def test_condition_parser():
     assert res["fnumber"] == "5"
 
 
+def test_condition_parser_with_underscore_in_column_name():
+    res = _parse_great_expectations_condition('col("pk_2") == "Two"')
+    assert res["column"] == "pk_2"
+    assert res["op"] == "=="
+    assert res["condition_value"] == "Two"
+
+
+def test_condition_parser_with_dash_in_column_name():
+    res = _parse_great_expectations_condition('col("pk-2") == "Two"')
+    assert res["column"] == "pk-2"
+    assert res["op"] == "=="
+    assert res["condition_value"] == "Two"
+
+
+def test_condition_parser_with_space_in_condition_value():
+    res = _parse_great_expectations_condition('col("pk_2") == "Two Two"')
+    assert res["column"] == "pk_2"
+    assert res["op"] == "=="
+    assert res["condition_value"] == "Two Two"
+
+
 def test_parse_condition_to_spark(spark_session):
     res = parse_condition_to_spark('col("foo") > 5')
     # This is mostly a demonstrative test; it may be brittle. I do not know how to test