Repository: modin-project/modin

Issue:
Cloud examples are outdated
### System information
- **OS Platform and Distribution (e.g., Linux Ubuntu 16.04)**: `Ubuntu 20.04`
- **Modin version** (`modin.__version__`): 0a642753af2cd2332ac71166bcddc89b0cace9d5
- **Python version**: `3.8.10`
- **Code we can use to reproduce**:

<!--
You can obtain the Modin version with

python -c "import modin; print(modin.__version__)"
-->

### Describe the problem
<!-- Describe the problem clearly here. -->

The image in use is no longer available from `AWS` provider, so all examples with the cloud do not work.

UPD: during testing, other types of errors were found, which will be discussed below.


Discussion:
Datasets need to be restored on s3 bucket (taxi, h2o)
Some cloud deps (for example, `rpyc`) do not come with conda packages, so we need to install them separately for now https://github.com/modin-project/modin/blob/e97e2e62ed821450a4c62611dfb9abac845fc5ff/setup.py#L42
The problem with `bootstrap_config` function that changes the config in an unexpected way: https://github.com/modin-project/modin/pull/3421#discussion_r714575841
At the time of testing, we have to install modin from various branches, where the development takes place (for now, manually).

This is especially important after major changes in the project like https://github.com/modin-project/modin/pull/3352 and https://github.com/modin-project/modin/pull/3210.

Need to automate.
The examples use code from Omniscripts, which is periodically updated regardless of cloud specifics, which leads to errors https://github.com/modin-project/modin/pull/3421/files#r714741862


Resolving patch:
diff --git a/examples/cluster/h2o-runner.py b/examples/cluster/h2o-runner.py
--- a/examples/cluster/h2o-runner.py
+++ b/examples/cluster/h2o-runner.py
@@ -28,9 +28,9 @@
     "aws",
     "aws_credentials",
     cluster_name="rayscale-test",
-    region="eu-north-1",
-    zone="eu-north-1b",
-    image="ami-00e1e82d7d4ca80d3",
+    region="eu-central-1",
+    zone="eu-central-1b",
+    image="ami-05f7491af5eef733a",
 )
 with test_cluster:
     parameters = {
@@ -47,5 +47,5 @@
     # - J1_1e6_1e3_0_0.csv
     # - J1_1e6_1e6_0_0.csv
     for data_file in ["G1_5e5_1e2_0_0.csv", "J1_1e6_NA_0_0.csv"]:
-        parameters["data_file"] = f"https://modin-datasets.s3.amazonaws.com/h2o/{data_file}"
+        parameters["data_file"] = f"s3://modin-datasets/cloud/h2o/{data_file}"
         run_benchmark(parameters)
diff --git a/examples/cluster/mortgage-runner.py b/examples/cluster/mortgage-runner.py
--- a/examples/cluster/mortgage-runner.py
+++ b/examples/cluster/mortgage-runner.py
@@ -16,6 +16,7 @@
 # pip install braceexpand
 
 # NOTE: expects https://github.com/intel-ai/omniscripts checked out and in PYTHONPATH
+# NOTE: deprecated example
 
 # the following import turns on experimental mode in Modin,
 # including enabling running things in remote cloud
diff --git a/examples/cluster/taxi-runner.py b/examples/cluster/taxi-runner.py
--- a/examples/cluster/taxi-runner.py
+++ b/examples/cluster/taxi-runner.py
@@ -35,20 +35,26 @@
     "aws",
     "aws_credentials",
     cluster_name="rayscale-test",
-    region="eu-north-1",
-    zone="eu-north-1b",
-    image="ami-00e1e82d7d4ca80d3",
+    region="eu-central-1",
+    zone="eu-central-1b",
+    image="ami-05f7491af5eef733a",
     **cluster_params,
 )
 with test_cluster:
-    data_file = "https://modin-datasets.s3.amazonaws.com/trips_data.csv"
     if USE_OMNISCI:
-        # Workaround for GH#2099
         from modin.experimental.cloud import get_connection
 
-        data_file, remote_data_file = "/tmp/trips_data.csv", data_file
-        get_connection().modules["subprocess"].check_call(
-            ["wget", remote_data_file, "-O", data_file]
+        # We should move omniscripts trigger in remote conext
+        # https://github.com/intel-ai/omniscripts/blob/7d4599bcacf51de876952c658048571d32275ac1/taxi/taxibench_pandas_ibis.py#L482
+        import modin.experimental.engines.omnisci_on_native.frame.omnisci_worker
+
+        OmnisciServer = (
+            get_connection()
+            .modules["modin.experimental.engines.omnisci_on_native.frame.omnisci_worker"]
+            .OmnisciServer
+        )
+        modin.experimental.engines.omnisci_on_native.frame.omnisci_worker.OmnisciServer = (
+            OmnisciServer
         )
 
         # Omniscripts check for files being present when given local file paths,
@@ -58,8 +64,7 @@
         utils.utils.glob = get_connection().modules["glob"]
 
     parameters = {
-        "data_file": data_file,
-        # "data_file": "s3://modin-datasets/trips_data.csv",
+        "data_file": "s3://modin-datasets/cloud/taxi/trips_xaa.csv",
         "dfiles_num": 1,
         "validation": False,
         "no_ibis": True,


Test patch: