Repository: modin-project/modin

Issue:
`ValueError` during initialization of Ray 1.4 in Modin
### System information
- **OS Platform and Distribution (e.g., Linux Ubuntu 16.04)**: Linux Ubuntu
- **Modin version** (`modin.__version__`): 0.10.0
- **Python version**:  3.8.10
- **Code we can use to reproduce**:

```python
import modin.pandas as pd
df = pd.DataFrame([0])
```

<!--
You can obtain the Modin version with

python -c "import modin; print(modin.__version__)"
-->

### Describe the problem
<!-- Describe the problem clearly here. -->
During initialization of Ray in Modin `ValueError` is occurred:
```
ValueError: The configured object store size (486.0 GB) exceeds /dev/shm size (405.07611136 GB). This will harm performance. Consider deleting files in /dev/shm or increasing its size with --shm-size in Docker. To ignore this warning, set RAY_OBJECT_STORE_ALLOW_SLOW_STORAGE=1.
```



Discussion:
Ray verifies object-store size and raises such exception when [`plasma_directory` is not specified](https://github.com/ray-project/ray/blob/90805d302f5950244f346f55d253b926f1b5504f/python/ray/_private/services.py#L1751), in this case, ray assumes `/dev/shm` to be the default plasma directory (on Linux) and compares specified `object_store_size` with the available shared memory.

Modin at the ray initialization uses total available memory to compute default object store size, which appears to be greater than the available shared memory:
https://github.com/modin-project/modin/blob/b8dd3b2f1a1b416d425ecfe49227fefbe1774505/modin/engines/ray/utils.py#L139-L140 
Modin should use `ray._private.utils.get_shared_memory_bytes()` to fill `system_memory` value instead.

### Temporary workaround
Specify a valid `object_store_size` in bytes by `MODIN_MEMORY` environment variable or by explicit ray initialization before using Modin:
```python
import ray
ray.init(object_store_size=STORE_SIZE)

import modn.pandas as pd
...
```


Resolving patch:
diff --git a/modin/engines/ray/utils.py b/modin/engines/ray/utils.py
--- a/modin/engines/ray/utils.py
+++ b/modin/engines/ray/utils.py
@@ -15,6 +15,8 @@
 
 import os
 import sys
+import psutil
+import warnings
 
 from modin.config import (
     Backend,
@@ -135,9 +137,24 @@ def initialize_ray(
             object_store_memory = Memory.get()
             # In case anything failed above, we can still improve the memory for Modin.
             if object_store_memory is None:
-                # Round down to the nearest Gigabyte.
-                system_memory = ray._private.utils.get_system_memory()
-                object_store_memory = int(0.6 * system_memory // 10 ** 9 * 10 ** 9)
+                virtual_memory = psutil.virtual_memory().total
+                if sys.platform.startswith("linux"):
+                    shm_fd = os.open("/dev/shm", os.O_RDONLY)
+                    try:
+                        shm_stats = os.fstatvfs(shm_fd)
+                        system_memory = shm_stats.f_bsize * shm_stats.f_bavail
+                        if system_memory / (virtual_memory / 2) < 0.99:
+                            warnings.warn(
+                                f"The size of /dev/shm is too small ({system_memory} bytes). The required size "
+                                f"at least half of RAM ({virtual_memory // 2} bytes). Please, delete files in /dev/shm or "
+                                "increase size of /dev/shm with --shm-size in Docker. Also, you can set "
+                                "the required memory size for each Ray worker in bytes to MODIN_MEMORY environment variable."
+                            )
+                    finally:
+                        os.close(shm_fd)
+                else:
+                    system_memory = virtual_memory
+                object_store_memory = int(0.6 * system_memory // 1e9 * 1e9)
                 # If the memory pool is smaller than 2GB, just use the default in ray.
                 if object_store_memory == 0:
                     object_store_memory = None


Test patch: