Repository: great-expectations/great_expectations

Issue:
GE errors when trying to return a result when contains a pandas NA
**Describe the bug**

If asking GE for a complete result when `expect_column_values_to_not_be_null`. GE will try to return a result which contains samples of the values which failed the expectation (if result is set to `COMPLETE`). It fails when one of the values is the new pandas `NA` type (GE internals fails to convert it).

**To Reproduce**
```python
import great_expectations as ge
import pandas as pd

df = pd.DataFrame({"a": [1, pd.NA, 3]})
dfe = ge.dataset.PandasDataset(df)
dfe.expect_column_values_to_not_be_null("a", result_format="COMPLETE") # Will error
```

Error message:
```
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "env/lib/python3.8/site-packages/great_expectations/core/__init__.py", line 2000, in __repr__
    return json.dumps(self.to_json_dict(), indent=2)
  File "env/lib/python3.8/site-packages/great_expectations/core/__init__.py", line 2026, in to_json_dict
    myself = expectationValidationResultSchema.dump(self)
  File "env/lib/python3.8/site-packages/great_expectations/marshmallow__shade/schema.py", line 556, in dump
    processed_obj = self._invoke_dump_processors(
  File "env/lib/python3.8/site-packages/great_expectations/marshmallow__shade/schema.py", line 1075, in _invoke_dump_processors
    data = self._invoke_processors(
  File "env/lib/python3.8/site-packages/great_expectations/marshmallow__shade/schema.py", line 1234, in _invoke_processors
    data = processor(data, many=many, **kwargs)
  File "env/lib/python3.8/site-packages/great_expectations/core/__init__.py", line 2096, in convert_result_to_serializable
    data.result = convert_to_json_serializable(data.result)
  File "env/lib/python3.8/site-packages/great_expectations/core/__init__.py", line 132, in convert_to_json_serializable
    new_dict[str(key)] = convert_to_json_serializable(data[key])
  File "env/lib/python3.8/site-packages/great_expectations/core/__init__.py", line 139, in convert_to_json_serializable
    new_list.append(convert_to_json_serializable(val))
  File "env/lib/python3.8/site-packages/great_expectations/core/__init__.py", line 195, in convert_to_json_serializable
    raise TypeError(
TypeError: <NA> is of type NAType which cannot be serialized.
```

**Expected behavior**
To return a result and not fail. Maybe with pd.NA or just replace it with a string interpretation of `pd.NA` or a `None` as the returned list of values (which failed).

**Environment (please complete the following information):**
 - OS: [Mac]
 - GE Version: 0.12.6
 - Pandas Version: 1.1.3




Discussion:
@isichei Thank you for reporting this! If you would like to make a PR (the fix must be very small), it would be super welcome! Please let us know.
Yeah can try and have a look over the next couple of weeks. When you say `the fix must be very small` do you mean it will be a small fix or just that you try to keep PRs small? 

If it is a bigger fix would you want me to make several small PRs?
@isichei I meant "must" not as in "we require the fix to be small", but as in "I think it will not take much code to fix this" :)

Looking forward to your PR!

Resolving patch:
diff --git a/great_expectations/core/__init__.py b/great_expectations/core/__init__.py
--- a/great_expectations/core/__init__.py
+++ b/great_expectations/core/__init__.py
@@ -112,8 +112,8 @@ def convert_to_json_serializable(data):
         return data.to_json_dict()
 
     try:
-        if not isinstance(data, list) and np.isnan(data):
-            # np.isnan is functionally vectorized, but we only want to apply this to single objects
+        if not isinstance(data, list) and pd.isna(data):
+            # pd.isna is functionally vectorized, but we only want to apply this to single objects
             # Hence, why we test for `not isinstance(list))`
             return None
     except TypeError:
@@ -233,8 +233,8 @@ def ensure_json_serializable(data):
         return
 
     try:
-        if not isinstance(data, list) and np.isnan(data):
-            # np.isnan is functionally vectorized, but we only want to apply this to single objects
+        if not isinstance(data, list) and pd.isna(data):
+            # pd.isna is functionally vectorized, but we only want to apply this to single objects
             # Hence, why we test for `not isinstance(list))`
             return
     except TypeError:


Test patch: