Repository: modin-project/modin

Issue:
BUG: starting modin on ray 2.0.0 on mac raises long DeprecationWarning
### System information
- **OS Platform and Distribution (e.g., Linux Ubuntu 16.04)**: macOS Monterey
- **Modin version** (`modin.__version__`): cfcdf081e197d35e1fd1526f5585e361793dc2fa
- **Python version**: 3.10.4
- **Code we can use to reproduce**:

```python
import modin.pandas as pd
pd.DataFrame()
```

### Describe the problem
Starting modin on ray 2.0.0 on mac raises long DeprecationWarning because Modin accesses the private variable `MAC_DEGRADED_PERF_MMAP_SIZE_LIMIT`.

### Source code / logs
<details>

<summary>Stack trace</summary>

```
DeprecationWarning: `ray.ray_constants.MAC_DEGRADED_PERF_MMAP_SIZE_LIMIT` is a private attribute and access will be removed in a future Ray version.
  File "/Users/maheshvashishtha/opt/anaconda3/envs/modin-dev/bin/ipython", line 8, in <module>
    sys.exit(start_ipython())
  File "/Users/maheshvashishtha/opt/anaconda3/envs/modin-dev/lib/python3.10/site-packages/IPython/__init__.py", line 124, in start_ipython
    return launch_new_instance(argv=argv, **kwargs)
  File "/Users/maheshvashishtha/opt/anaconda3/envs/modin-dev/lib/python3.10/site-packages/traitlets/config/application.py", line 976, in launch_instance
    app.start()
  File "/Users/maheshvashishtha/opt/anaconda3/envs/modin-dev/lib/python3.10/site-packages/IPython/terminal/ipapp.py", line 318, in start
    self.shell.mainloop()
  File "/Users/maheshvashishtha/opt/anaconda3/envs/modin-dev/lib/python3.10/site-packages/IPython/terminal/interactiveshell.py", line 680, in mainloop
    self.interact()
  File "/Users/maheshvashishtha/opt/anaconda3/envs/modin-dev/lib/python3.10/site-packages/IPython/terminal/interactiveshell.py", line 673, in interact
    self.run_cell(code, store_history=True)
  File "/Users/maheshvashishtha/opt/anaconda3/envs/modin-dev/lib/python3.10/site-packages/IPython/core/interactiveshell.py", line 2881, in run_cell
    result = self._run_cell(
  File "/Users/maheshvashishtha/opt/anaconda3/envs/modin-dev/lib/python3.10/site-packages/IPython/core/interactiveshell.py", line 2936, in _run_cell
    return runner(coro)
  File "/Users/maheshvashishtha/opt/anaconda3/envs/modin-dev/lib/python3.10/site-packages/IPython/core/async_helpers.py", line 129, in _pseudo_sync_runner
    coro.send(None)
  File "/Users/maheshvashishtha/opt/anaconda3/envs/modin-dev/lib/python3.10/site-packages/IPython/core/interactiveshell.py", line 3135, in run_cell_async
    has_raised = await self.run_ast_nodes(code_ast.body, cell_name,
  File "/Users/maheshvashishtha/opt/anaconda3/envs/modin-dev/lib/python3.10/site-packages/IPython/core/interactiveshell.py", line 3338, in run_ast_nodes
    if await self.run_code(code, result, async_=asy):
  File "/Users/maheshvashishtha/opt/anaconda3/envs/modin-dev/lib/python3.10/site-packages/IPython/core/interactiveshell.py", line 3398, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-2-6006e405f520>", line 1, in <cell line: 1>
    pd.DataFrame()
  File "/Users/maheshvashishtha/software_sources/modin/modin/logging/logger_decorator.py", line 128, in run_and_log
    return obj(*args, **kwargs)
  File "/Users/maheshvashishtha/software_sources/modin/modin/_compat/pandas_api/latest/dataframe.py", line 50, in __init__
    self._init(
  File "/Users/maheshvashishtha/software_sources/modin/modin/logging/logger_decorator.py", line 128, in run_and_log
    return obj(*args, **kwargs)
  File "/Users/maheshvashishtha/software_sources/modin/modin/pandas/dataframe.py", line 122, in _init
    Engine.subscribe(_update_engine)
  File "/Users/maheshvashishtha/software_sources/modin/modin/config/pubsub.py", line 217, in subscribe
    callback(cls)
  File "/Users/maheshvashishtha/software_sources/modin/modin/pandas/__init__.py", line 135, in _update_engine
    initialize_ray()
  File "/Users/maheshvashishtha/software_sources/modin/modin/core/execution/ray/common/utils.py", line 104, in initialize_ray
    object_store_memory = _get_object_store_memory()
  File "/Users/maheshvashishtha/software_sources/modin/modin/core/execution/ray/common/utils.py", line 193, in _get_object_store_memory
    mac_size_limit = getattr(
  File "/Users/maheshvashishtha/opt/anaconda3/envs/modin-dev/lib/python3.10/site-packages/ray/__init__.py", line 196, in __getattr__
    traceback.print_stack()
2022-08-23 22:40:26,742	INFO worker.py:1518 -- Started a local Ray instance.
UserWarning: Distributing <class 'NoneType'> object. This may take some time.
```

```

```
</details>


Discussion:

Resolving patch:
diff --git a/modin/core/execution/ray/common/utils.py b/modin/core/execution/ray/common/utils.py
--- a/modin/core/execution/ray/common/utils.py
+++ b/modin/core/execution/ray/common/utils.py
@@ -36,6 +36,10 @@
 from modin.error_message import ErrorMessage
 
 _OBJECT_STORE_TO_SYSTEM_MEMORY_RATIO = 0.6
+# This constant should be in sync with the limit in ray, which is private,
+# not exposed to users, and not documented:
+# https://github.com/ray-project/ray/blob/4692e8d8023e789120d3f22b41ffb136b50f70ea/python/ray/_private/ray_constants.py#L57-L62
+_MAC_OBJECT_STORE_LIMIT_BYTES = 2 * 2**30
 
 ObjectIDType = ray.ObjectRef
 if version.parse(ray.__version__) >= version.parse("1.2.0"):
@@ -187,18 +191,19 @@ def _get_object_store_memory() -> Optional[int]:
     )
     if object_store_memory == 0:
         return None
-    # Versions of ray with MAC_DEGRADED_PERF_MMAP_SIZE_LIMIT don't allow us
-    # to initialize ray with object store size larger than that constant.
-    # For background see https://github.com/ray-project/ray/issues/20388
-    mac_size_limit = getattr(
-        ray.ray_constants, "MAC_DEGRADED_PERF_MMAP_SIZE_LIMIT", None
-    )
-    if (
-        sys.platform == "darwin"
-        and mac_size_limit is not None
-        and object_store_memory > mac_size_limit
+    # Newer versions of ray don't allow us to initialize ray with object store
+    # size larger than that _MAC_OBJECT_STORE_LIMIT_BYTES. It seems that
+    # object store > the limit is too slow even on ray 1.0.0. However, limiting
+    # the object store to _MAC_OBJECT_STORE_LIMIT_BYTES only seems to start
+    # helping at ray version 1.3.0. So if ray version is at least 1.3.0, cap
+    # the object store at _MAC_OBJECT_STORE_LIMIT_BYTES.
+    # For background on the ray bug see:
+    # - https://github.com/ray-project/ray/issues/20388
+    # - https://github.com/modin-project/modin/issues/4872
+    if sys.platform == "darwin" and version.parse(ray.__version__) >= version.parse(
+        "1.3.0"
     ):
-        object_store_memory = mac_size_limit
+        object_store_memory = min(object_store_memory, _MAC_OBJECT_STORE_LIMIT_BYTES)
     return object_store_memory
 
 


Test patch: