Repository: modin-project/modin

Issue:
Remove redundant `.fillna(0)` at the end of `groupby.size()/.count()`
Currently, each `groupby.size()` call is finished with `.fillna(0)`:
https://github.com/modin-project/modin/blob/632d724adb013d6f4d18aa2a4f2faa9325d324e8/modin/pandas/groupby.py#L899

The behavior was originally introduced in #1802 in order to match pandas in the sense of not having NaN values in the result of `.size()/.count()`. The NaNs appeared at the reduction phase of groupby due to some specific pandas behavior:
```python
>>> import pandas as pd # pandas 1.0.5
>>> df = pd.DataFrame({"a": [1, 2], "b": [3, 4], "c": [4, 3]}).astype({"a": "category"})
>>> df.groupby(["a", "b"]).sum()
       c
a b
1 3  4.0
  4  NaN
2 3  NaN
  4  3.0
```
Running the same code on the latest pandas gives the expected behavior (not producing NaNs), meaning that we now can remove the finalizing `.fillna(0)` in our modin's code
```python
>>> import pandas as pd # pandas 1.5.3
>>> df = pd.DataFrame({"a": [1, 2], "b": [3, 4], "c": [4, 3]}).astype({"a": "category"})
>>> df.groupby(["a", "b"]).sum()
     c
a b
1 3  4
  4  0
2 3  0
  4  3
```


Discussion:

Resolving patch:
diff --git a/modin/pandas/groupby.py b/modin/pandas/groupby.py
--- a/modin/pandas/groupby.py
+++ b/modin/pandas/groupby.py
@@ -896,7 +896,7 @@ def size(self):
             result.name = self._df.name
         else:
             result.name = None
-        return result.fillna(0)
+        return result
 
     def sum(self, numeric_only=None, min_count=0):
         return self._wrap_aggregation(
@@ -1034,14 +1034,10 @@ def fillna(self, *args, **kwargs):
         )
 
     def count(self):
-        result = self._wrap_aggregation(
+        return self._wrap_aggregation(
             type(self._query_compiler).groupby_count,
             numeric_only=False,
         )
-        # pandas do it in case of Series
-        if isinstance(result, Series):
-            result = result.fillna(0)
-        return result
 
     def pipe(self, func, *args, **kwargs):
         return com.pipe(self, func, *args, **kwargs)


Test patch:
diff --git a/modin/pandas/test/dataframe/test_reduce.py b/modin/pandas/test/dataframe/test_reduce.py
--- a/modin/pandas/test/dataframe/test_reduce.py
+++ b/modin/pandas/test/dataframe/test_reduce.py
@@ -38,6 +38,7 @@
     df_equals_with_non_stable_indices,
     test_data_large_categorical_dataframe,
     default_to_pandas_ignore_string,
+    assert_dtypes_equal,
 )
 from modin.config import NPartitions, StorageFormat
 
@@ -444,7 +445,11 @@ def test_value_counts_categorical():
         # The order of HDK categories is different from Pandas
         # and, thus, index comparison fails.
         def comparator(df1, df2):
-            assert_series_equal(df1._to_pandas(), df2, check_index=False)
+            # Perform our own non-strict version of dtypes equality check
+            assert_dtypes_equal(df1, df2)
+            assert_series_equal(
+                df1._to_pandas(), df2, check_index=False, check_dtype=False
+            )
 
     else:
         comparator = df_equals
diff --git a/modin/pandas/test/test_series.py b/modin/pandas/test/test_series.py
--- a/modin/pandas/test/test_series.py
+++ b/modin/pandas/test/test_series.py
@@ -78,6 +78,7 @@
     default_to_pandas_ignore_string,
     CustomIntegerForAddition,
     NonCommutativeMultiplyInteger,
+    assert_dtypes_equal,
 )
 from modin.config import NPartitions, StorageFormat
 
@@ -3734,7 +3735,11 @@ def test_value_counts_categorical():
         # The order of HDK categories is different from Pandas
         # and, thus, index comparison fails.
         def comparator(df1, df2):
-            assert_series_equal(df1._to_pandas(), df2, check_index=False)
+            # Perform our own non-strict version of dtypes equality check
+            assert_dtypes_equal(df1, df2)
+            assert_series_equal(
+                df1._to_pandas(), df2, check_index=False, check_dtype=False
+            )
 
     else:
         comparator = df_equals