Repository: modin-project/modin

Issue:
`df.groupby` on empty data fails with `RecursionError` when experimental mode is enabled
Works perfectly fine:
```python
# import modin.config as cfg
# cfg.IsExperimental.put(True)

import modin.pandas as pd

df = pd.DataFrame({"a": [1, 1], "b": [1, 2]})
grp = df.groupby("a")
```
Fails:
```python
import modin.config as cfg
cfg.IsExperimental.put(True)

import modin.pandas as pd

df = pd.DataFrame({"a": [1, 1], "b": [1, 2]})
grp = df.groupby("a") # RecursionError
```
<details><summary>Traceback</summary>

```
Traceback (most recent call last):
  File "t2.py", line 10, in <module>
    grp = df.groupby("a")
  File "/localdisk/dchigare/repos/modin/modin/logging/logger_decorator.py", line 128, in run_and_log
    return obj(*args, **kwargs)
  File "/localdisk/dchigare/repos/modin/modin/pandas/dataframe.py", line 542, in groupby
    return DataFrameGroupBy(
  File "/localdisk/dchigare/repos/modin/modin/logging/logger_decorator.py", line 128, in run_and_log
    return obj(*args, **kwargs)
  File "/localdisk/dchigare/repos/modin/modin/pandas/groupby.py", line 98, in __init__
    self._query_compiler = self._df._query_compiler
  File "/localdisk/dchigare/repos/modin/modin/logging/logger_decorator.py", line 128, in run_and_log
    return obj(*args, **kwargs)
  File "/localdisk/dchigare/repos/modin/modin/pandas/groupby.py", line 154, in __getattribute__
    attr = super().__getattribute__(item)
  File "/localdisk/dchigare/repos/modin/modin/logging/logger_decorator.py", line 128, in run_and_log
    return obj(*args, **kwargs)
  File "/localdisk/dchigare/repos/modin/modin/pandas/groupby.py", line 154, in __getattribute__
    attr = super().__getattribute__(item)
  File "/localdisk/dchigare/repos/modin/modin/logging/logger_decorator.py", line 128, in run_and_log
    return obj(*args, **kwargs)
  File "/localdisk/dchigare/repos/modin/modin/pandas/groupby.py", line 154, in __getattribute__
    attr = super().__getattribute__(item)
  File "/localdisk/dchigare/repos/modin/modin/logging/logger_decorator.py", line 128, in run_and_log
    return obj(*args, **kwargs)
  File "/localdisk/dchigare/repos/modin/modin/pandas/groupby.py", line 154, in __getattribute__
    attr = super().__getattribute__(item)
  File "/localdisk/dchigare/repos/modin/modin/logging/logger_decorator.py", line 127, in run_and_log
    if LogMode.get() == "disable":
RecursionError: maximum recursion depth exceeded
```

</details>

The experimental-mode dependency of the failure is explained by that Modin wraps DataFrame, Series and Groupby classes in a special 'meta-magic' class, making them to work properly in a remote context.

https://github.com/modin-project/modin/blob/b2f7680fa2ad387ee0eb7de0633d0c86572e5f2b/modin/pandas/groupby.py#L1360-L1363

This recursion error started to appear once the [`.__getattribte__` method of DataframeGroupbyObj](https://github.com/modin-project/modin/blob/b2f7680fa2ad387ee0eb7de0633d0c86572e5f2b/modin/pandas/groupby.py#L153-L165) was overridden (so since [this commit in master](https://github.com/modin-project/modin/commit/4351b8b5b7cab8e3ff9b0f291df9f78d09d4f295)). The `.__getattribute__` method for the groupby class was copied from the `modin.pandas.DataFrame` class that is also wrapped by a 'meta-magic'.

I've tried to debug what's the crucial difference between the wrapped `DataframeGroupbyObj` and the `DataFrame`, but found nothing so far.

UPD1:
The reproducer works fine if I remove the [heritage of `ClassLoger` for the `DataframeGroupbyObj`](https://github.com/modin-project/modin/blob/b2f7680fa2ad387ee0eb7de0633d0c86572e5f2b/modin/pandas/groupby.py#L78). The `SeriesGroupby` though still fails with the same error (it inherits from `DataframeGroupbyObj`). Is 'magic-meta' experience difficulties with inheriting? Why it works fine then for `DataFrame` and `Series`?

@modin-project/modin-cloud (@vnlitvinov @anmyachev), any guess what could be the reason?



Discussion:
May be related to #1872
The problem is visible in [master's CI](https://github.com/modin-project/modin/actions/runs/3939267043/jobs/6738932576) but not visible in the PR's CI. This happens due to that master's CI explicitly specifies the HDK [tests to be in experimental mode](https://github.com/modin-project/modin/blob/b2f7680fa2ad387ee0eb7de0633d0c86572e5f2b/.github/workflows/push.yml#L114-L115) when [PR's CI doesn't](https://github.com/modin-project/modin/blob/b2f7680fa2ad387ee0eb7de0633d0c86572e5f2b/.github/workflows/ci.yml#L436-L437)
Fixed by https://github.com/modin-project/modin/pull/6408

Resolving patch:
diff --git a/modin/pandas/groupby.py b/modin/pandas/groupby.py
--- a/modin/pandas/groupby.py
+++ b/modin/pandas/groupby.py
@@ -37,7 +37,6 @@
 from modin.pandas.utils import cast_function_modin2pandas
 from modin.core.storage_formats.base.query_compiler import BaseQueryCompiler
 from modin.core.dataframe.algebra.default2pandas.groupby import GroupBy
-from modin.config import IsExperimental
 from .series import Series
 from .window import RollingGroupby
 from .utils import is_label
@@ -182,31 +181,19 @@ def __getattr__(self, key):
                 return self.__getitem__(key)
             raise err
 
-    # TODO: `.__getattribute__` overriding is broken in experimental mode. We should
-    # remove this branching one it's fixed:
-    # https://github.com/modin-project/modin/issues/5536
-    if not IsExperimental.get():
+    def __getattribute__(self, item):
+        attr = super().__getattribute__(item)
+        if item not in _DEFAULT_BEHAVIOUR and not self._query_compiler.lazy_execution:
+            # We default to pandas on empty DataFrames. This avoids a large amount of
+            # pain in underlying implementation and returns a result immediately rather
+            # than dealing with the edge cases that empty DataFrames have.
+            if callable(attr) and self._df.empty and hasattr(self._pandas_class, item):
 
-        def __getattribute__(self, item):
-            attr = super().__getattribute__(item)
-            if (
-                item not in _DEFAULT_BEHAVIOUR
-                and not self._query_compiler.lazy_execution
-            ):
-                # We default to pandas on empty DataFrames. This avoids a large amount of
-                # pain in underlying implementation and returns a result immediately rather
-                # than dealing with the edge cases that empty DataFrames have.
-                if (
-                    callable(attr)
-                    and self._df.empty
-                    and hasattr(self._pandas_class, item)
-                ):
-
-                    def default_handler(*args, **kwargs):
-                        return self._default_to_pandas(item, *args, **kwargs)
+                def default_handler(*args, **kwargs):
+                    return self._default_to_pandas(item, *args, **kwargs)
 
-                    return default_handler
-            return attr
+                return default_handler
+        return attr
 
     @property
     def ngroups(self):


Test patch: