Repository: deepchecks/deepchecks

Issue:
[FEAT] Mixed Nulls - catch mixed nulls on non string columns (np.NaN/pd.NaN/Null/None/etc.)
**Docs**

API Reference should include what are the DEFAULT NULLS that are checks (currently this can be found only by going into source)
Example notebook can also print out these nulls for convenience.

**Null Types**
1. NaT nulls not caught @chelseatroy can you elaborate?
3. Seems that list currently includes only strings (and null character). Does this catch also null objects? (e.g. the python None. Numpy and pandas nulls. or any other null that is likely to find it's way due to multiple feature engineering backends)




Discussion:
I can elaborate!

`NaT` is what you get when you convert a sequence of values that contains timestamps, with some `None`s, to a pandas column. It stands for "not a timestamp". When I have a column with some `NaT`s, some timestamps, and some `None`s, the "Mixed Nulls" check still passes:

<img width="977" alt="Screen Shot 2022-03-21 at 6 03 54 PM" src="https://user-images.githubusercontent.com/5744164/159377231-54531852-2136-4234-af51-4bf0c3b4a844.png">

<img width="975" alt="Screen Shot 2022-03-21 at 6 04 24 PM" src="https://user-images.githubusercontent.com/5744164/159377252-1f0d74b7-1e27-4e4d-b982-1765809bd2b2.png">


I have confirmed that having the string 'null' and `None` also doesn't trigger Mixed Nulls:
<img width="941" alt="Screen Shot 2022-03-21 at 6 16 36 PM" src="https://user-images.githubusercontent.com/5744164/159383066-ae17b867-1654-48c8-b553-365a6e7e1d4c.png">


Updating here about this issue after some further investigation:

Mixed Nulls catches only **string nulls** ('none', 'nan', etc...). Also the documentation states that.
@noamzbr is there a specific reason that we implemented it only for those type of "nulls"?
The check can be more useful if it would catch various types of null representations... (e.g. python None, nump/pandas nulls, other types of null formats as they may arrive and be formatted from various data sources, etc.)

It also does catch np.NaN, but that's it. Agree that we should support also the various pandas nans and python None. 
We also do limit this check to columns of dtype pd.StringDtype, and I can't recall why. Probably can drop this requirement.

Only thing I would like to clarify - what do you mean by "other types of null formats as they may arrive and be formatted from various data sources, etc.)"? How can we catch other types of nulls we don't know about (except if they get casted earlier into numpy/pandas/python types)?
@noamzbr good question, and we might have to find those by dint of folks running datasets through and seeing what doesn't catch. I do think it would be a good start to make this catch the nulls and Python `None` that we know about, and expand the check also to include other column types than the string ones, and that might cover a lot of cases.
Agree with both of your comments...

(btw what I meant in "other types" is to check existing nulls that we know of, not those that we don't know of :). for example - if we query with SQLAlchemy and there is a NULL value there, then how will it appear)

Resolving patch:
diff --git a/deepchecks/tabular/checks/integrity/mixed_nulls.py b/deepchecks/tabular/checks/integrity/mixed_nulls.py
--- a/deepchecks/tabular/checks/integrity/mixed_nulls.py
+++ b/deepchecks/tabular/checks/integrity/mixed_nulls.py
@@ -9,10 +9,9 @@
 # ----------------------------------------------------------------------------
 #
 """Module contains Mixed Nulls check."""
-from collections import defaultdict
+from collections import defaultdict, Counter
 from typing import Union, Dict, List, Iterable
 
-import numpy as np
 import pandas as pd
 
 from deepchecks.tabular import Context, SingleDatasetCheck
@@ -22,6 +21,7 @@
 from deepchecks.utils.features import N_TOP_MESSAGE, column_importance_sorter_df
 from deepchecks.utils.strings import string_baseform, format_percent
 from deepchecks.utils.typing import Hashable
+from pkg_resources import parse_version
 
 
 __all__ = ['MixedNulls']
@@ -31,7 +31,7 @@
 
 
 class MixedNulls(SingleDatasetCheck):
-    """Search for various types of null values in a string column(s), including string representations of null.
+    """Search for various types of null values, including string representations of null.
 
     Parameters
     ----------
@@ -79,7 +79,7 @@ def run_logic(self, context: Context, dataset_type: str = 'train') -> CheckResul
         df = dataset.data
 
         df = select_from_dataframe(df, self.columns, self.ignore_columns)
-        null_string_list: set = self._validate_null_string_list(self.null_string_list, self.check_nan)
+        null_string_list: set = self._validate_null_string_list(self.null_string_list)
 
         # Result value
         display_array = []
@@ -87,14 +87,17 @@ def run_logic(self, context: Context, dataset_type: str = 'train') -> CheckResul
 
         for column_name in list(df.columns):
             column_data = df[column_name]
-            # TODO: Modify this once Dataset type casting mechanism is done
-            if column_data.dtype != pd.StringDtype:
-                continue
-            # Get counts of all values in series including NaNs, in sorted order of count
-            column_counts: pd.Series = column_data.value_counts(dropna=False)
+            # Pandas version 1.3.X and lower doesn't support counting separate NaN values in value_counts
+            if parse_version(pd.__version__) < parse_version('1.4.0'):
+                column_counts = Counter(column_data)
+            else:
+                # Get counts of all values in series including NaNs
+                column_counts: pd.Series = column_data.value_counts(dropna=False)
+
             # Filter out values not in the nulls list
             null_counts = {value: count for value, count in column_counts.items()
-                           if string_baseform(value) in null_string_list}
+                           if (self.check_nan and pd.isnull(value)) or (string_baseform(value) in null_string_list)}
+
             if len(null_counts) < 2:
                 continue
             # Save the column info
@@ -115,15 +118,14 @@ def run_logic(self, context: Context, dataset_type: str = 'train') -> CheckResul
 
         return CheckResult(result_dict, display=display)
 
-    def _validate_null_string_list(self, nsl, check_nan: bool) -> set:
+    def _validate_null_string_list(self, nsl) -> set:
         """Validate the object given is a list of strings. If null is given return default list of null values.
 
         Parameters
         ----------
         nsl
             Object to validate
-        check_nan : bool
-            Whether to add to null list to check also NaN values
+
         Returns
         -------
         set
@@ -141,8 +143,6 @@ def _validate_null_string_list(self, nsl, check_nan: bool) -> set:
         else:
             # Default values
             result = set(DEFAULT_NULL_VALUES)
-        if check_nan is None or check_nan is True:
-            result.add(np.NaN)
 
         return result
 


Test patch:
diff --git a/tests/checks/integrity/mixed_nulls_test.py b/tests/checks/integrity/mixed_nulls_test.py
--- a/tests/checks/integrity/mixed_nulls_test.py
+++ b/tests/checks/integrity/mixed_nulls_test.py
@@ -12,7 +12,7 @@
 import numpy as np
 import pandas as pd
 
-from hamcrest import assert_that, has_length, has_entry, has_items, calling, raises
+from hamcrest import assert_that, has_length, has_entry, has_items, calling, raises, is_
 
 from deepchecks.tabular.dataset import Dataset
 from deepchecks.tabular.checks.integrity.mixed_nulls import MixedNulls
@@ -56,7 +56,7 @@ def test_different_null_types():
     # Act
     result = MixedNulls().run(dataframe)
     # Assert
-    assert_that(result.value, has_entry('col1', has_length(3)))
+    assert_that(result.value, has_entry('col1', has_length(4)))
 
 
 def test_null_list_param():
@@ -99,6 +99,37 @@ def test_single_column_different_case_is_count_separately():
     assert_that(result.value, has_entry('col1', has_length(3)))
 
 
+def test_numeric_column_nulls():
+    # Arrange
+    data = {'col1': [1, 2, np.NaN, pd.NA, pd.NaT]}
+    dataframe = pd.DataFrame(data=data)
+    # Act
+    result = MixedNulls().run(dataframe)
+    # Assert
+    assert_that(result.value, has_entry('col1', has_length(3)))
+
+
+def test_numeric_column_nulls_with_none():
+    # Arrange
+    data = {'col1': [1, 2, np.NaN, pd.NA, pd.NaT, None]}
+    dataframe = pd.DataFrame(data=data)
+    # Act
+    result = MixedNulls().run(dataframe)
+    # Assert
+    assert_that(result.value, has_entry('col1', has_length(4)))
+
+
+def test_mix_value_columns():
+    # Arrange
+    data = {'col1': [1, 2, np.NaN, pd.NA, pd.NaT, 3], 'col2': ['foo', 'bar', 'Nan', 'nan', 'NaN', None]}
+    dataframe = pd.DataFrame(data=data)
+    # Act
+    result = MixedNulls().run(dataframe)
+    # Assert
+    assert_that(result.value, has_entry('col1', has_length(3)))
+    assert_that(result.value, has_entry('col2', has_length(4)))
+
+
 def test_single_column_nulls_with_special_characters():
     # Arrange
     data = {'col1': ['', '#@$', 'Nan!', '#nan', '<NaN>']}