Repository: modin-project/modin
Pull request: DOCS-#3863: Add usage guide to documentation
Signed-off-by: Igoshev, Yaroslav <yaroslav.igoshev@intel.com>

<!--
Thank you for your contribution!
Please review the contributing docs: https://modin.readthedocs.io/en/latest/CONTRIBUTING.html
if you have questions about contributing.
-->

## What do these changes do?

<!-- Please give a short brief about these changes. -->

- [x] commit message follows format outlined [here](https://modin.readthedocs.io/en/latest/contributing.html)
- [x] passes `flake8 modin/ asv_bench/benchmarks scripts/doc_checker.py`
- [x] passes `black --check modin/ asv_bench/benchmarks scripts/doc_checker.py`
- [x] signed commit with `git commit -s` <!-- you can amend your commit with a signature via `git commit -amend -s` -->
- [x] Resolves #3863  <!-- issue must be created for each patch -->
- [x] tests passing
- [x] module layout described at `docs/developer/architecture.rst` is up-to-date <!-- if you have added, renamed or removed files or directories please update the documentation accordingly -->

Built docs:
https://modin--3864.org.readthedocs.build/en/3864/index.html
Optimization guide section:
https://modin--3864.org.readthedocs.build/en/3864/usage_guide/optimization_guide/index.html

Problem statement:
Add usage guide to documentation
Our documentation is growing and put everything onto start page probably doesn't look good. We want to add comprehensive examples, which show Modin usage, performance against pandas, etc., as well as some optimization notes regarding improving/accelerating a generic query pipeline (like put less NPartitions number in case an operations is performed via map operator for a wide dataframe since there is excessive amount of remote partitions, which is much greater than the number of CPUs) and specific operations (like for inner join recommend using a larger dataframe as a left operand and a smaller dataframe as a right operand). Usage guide page looks as a good candidate where we would put all examples, tutorials, optimization notes/guide, etc.

Hints from issue discussion:

Commit message: DOCS-#3863: Add usage guide to documentaion

Signed-off-by: Igoshev, Yaroslav <yaroslav.igoshev@intel.com>

Patch to review:
diff --git a/docs/index.rst b/docs/index.rst
index a5f0e8066bf..c45ac4f4da0 100644
--- a/docs/index.rst
+++ b/docs/index.rst
@@ -7,11 +7,12 @@
 
 .. toctree::
    :hidden:
-   
+
    getting_started/index
    supported_apis/index
-   advanced_usage/index
+   advanced_usage/index # should probably go in usage_guide
    developer/index
+   usage_guide/index
    contact
 
 .. raw:: html
diff --git a/docs/usage_guide/examples/index.rst b/docs/usage_guide/examples/index.rst
new file mode 100644
index 00000000000..03eca0506d0
--- /dev/null
+++ b/docs/usage_guide/examples/index.rst
@@ -0,0 +1,21 @@
+Modin Usage Examples
+====================
+
+This section shows Modin usage examples in different scenarios like Modin on a local/remote cluster,
+Modin in the cloud, the use of Modin spreadsheet and whatnot. First of all you may want to look at the basic
+Modin usage examples in tutorial_ directory. Then, you can check the use of Modin in different real benchmarks:
+
+* `NYC Taxi`_
+* Census
+* Plasticc
+
+Next, you likely want to visit cluster_ folder to take a look at how to run Modin in the cloud.
+
+Finally, you could look at Modin experimental features usage.
+
+* spreadsheet_
+
+.. _tutorial: https://github.com/modin-project/modin/tree/master/examples/tutorial
+.. _cluster: https://github.com/modin-project/modin/tree/master/examples/cluster
+.. _spreadsheet: https://github.com/modin-project/modin/tree/master/examples/spreadsheet
+.. _NYC Taxi: https://github.com/modin-project/modin/blob/master/examples/jupyter/NYC_Taxi.ipynb
\ No newline at end of file
diff --git a/docs/usage_guide/index.rst b/docs/usage_guide/index.rst
new file mode 100644
index 00000000000..f78ed09c64d
--- /dev/null
+++ b/docs/usage_guide/index.rst
@@ -0,0 +1,15 @@
+Usage Guide
+===========
+
+Here you can find Modin usage examples and information related to optimizations both for general query pipeline
+and for specific operations.
+
+.. toctree::
+    :maxdepth: 4
+
+    examples/index
+    optimization_guide/index
+
+.. meta::
+    :description lang=en:
+        Usage-specific documentation.
\ No newline at end of file
diff --git a/docs/usage_guide/optimization_guide/index.rst b/docs/usage_guide/optimization_guide/index.rst
new file mode 100644
index 00000000000..a8bb09a7034
--- /dev/null
+++ b/docs/usage_guide/optimization_guide/index.rst
@@ -0,0 +1,58 @@
+Optimization Guide
+==================
+
+Here you can find information on Modin optimizations both for general query pipeline and for specific operations.
+
+Generic settings
+""""""""""""""""
+
+If you are working with a wide DataFrame and performing an operation that requires ``map`` Core Dataframe Algebra operator
+to be applied cell-wise, you may want to change the number of partitions DataFrame should partition into to avoid
+excessive amount of remote partitions, which is much greater than the number of CPUs.
+
+.. code-block:: python
+
+  from multiprocessing import cpu_count
+  import modin.config as cfg
+  import modin.pandas as pd
+  import numpy as np
+
+  data = np.random.randint(0, 100, size=(2**15, 2**15))
+
+  cfg.NPartitions.put(cpu_count())
+  df = pd.DataFrame(data)
+  %timeit df.abs()
+  1.9 s  49.8 ms per loop (mean  std. dev. of 7 runs, 1 loop each)
+
+  cfg.NPartitions.put(int(cpu_count() / 2))
+  df = pd.DataFrame(data)
+  %timeit df.abs()
+  506 ms  17.5 ms per loop (mean  std. dev. of 7 runs, 1 loop each)
+
+Operation-specific settings
+"""""""""""""""""""""""""""
+
+merge
+-----
+
+``merge`` operation in Modin uses broadcast join algorithm: combining a right Modin DataFrame into pandas DataFrame and
+broadcasting it to row partitions of the left Modin DataFrame. In order to minimize interprocess communication cost when doing
+inner join you may want to swap left and right DataFrames.
+
+.. code-block:: python
+
+  import modin.pandas as pd
+  import numpy as np
+
+  left_data = np.random.randint(0, 100, size=(2**8, 2**8))
+  right_data = np.random.randint(0, 100, size=(2**12, 2**12))
+
+  left_df = pd.DataFrame(left_data)
+  right_df = pd.DataFrame(right_data)
+  %timeit left_df.merge(right_df, how="inner", on=10)
+  3.59 s  107 ms per loop (mean  std. dev. of 7 runs, 1 loop each)
+
+  %timeit right_df.merge(left_df, how="inner", on=10)
+  1.22 s  40.1 ms per loop (mean  std. dev. of 7 runs, 1 loop each)
+
+Note that result columns order may differ for first and second ``merge``.
diff --git a/examples/jupyter/NYC_Taxi.ipynb b/examples/jupyter/NYC_Taxi.ipynb
index fc372d7068d..512459fc386 100644
--- a/examples/jupyter/NYC_Taxi.ipynb
+++ b/examples/jupyter/NYC_Taxi.ipynb
@@ -9,7 +9,9 @@
     "# This notebook expects that Modin and Ray are installed, e.g. by `pip install modin[ray]`.\n",
     "# For all ways to install Modin see official documentation at:\n",
     "# https://modin.readthedocs.io/en/latest/installation.html\n",
-    "import modin.pandas as pd"
+    "import modin.pandas as pd\n",
+    "import pandas\n",
+    "from timeit import default_timer as timer"
    ]
   },
   {
@@ -19,7 +21,8 @@
    "outputs": [],
    "source": [
     "columns_names = [\n",
-    "        \"trip_id\", \"vendor_id\", \"pickup_datetime\", \"dropoff_datetime\", \"store_and_fwd_flag\",\n",
+    "\n",
+    "    \"trip_id\", \"vendor_id\", \"pickup_datetime\", \"dropoff_datetime\", \"store_and_fwd_flag\",\n",
     "        \"rate_code_id\", \"pickup_longitude\", \"pickup_latitude\", \"dropoff_longitude\", \"dropoff_latitude\",\n",
     "        \"passenger_count\", \"trip_distance\", \"fare_amount\", \"extra\", \"mta_tax\", \"tip_amount\",\n",
     "        \"tolls_amount\", \"ehail_fee\", \"improvement_surcharge\", \"total_amount\", \"payment_type\",\n",
@@ -43,14 +46,34 @@
      "name": "stderr",
      "output_type": "stream",
      "text": [
-      "UserWarning: Parameters provided defaulting to pandas implementation.\n",
-      "To request implementation, send an email to feature_requests@modin.org.\n"
+      "UserWarning: Ray execution environment not yet initialized. Initializing...\n",
+      "To remove this warning, run the following python code before doing dataframe operations:\n",
+      "\n",
+      "    import ray\n",
+      "    ray.init()\n",
+      "\n",
+      "UserWarning: Not all Ray Dashboard dependencies were found. To use the dashboard please install Ray using `pip install ray[default]`. To disable this message, set RAY_DISABLE_IMPORT_WARNING env var to '1'.\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Modin time: 195.48647334426641\n",
+      "pandas time: 438.423446778208\n"
      ]
     }
    ],
    "source": [
-    "df = pd.read_csv('https://modin-datasets.s3.amazonaws.com/trips_data.csv', names=columns_names,\n",
-    "                header=None, parse_dates=parse_dates)"
+    "# We should provide a justification why Modin is faster/slower than pandas\n",
+    "t0 = timer()\n",
+    "modin_df = pd.read_csv('s3://modin-datasets/taxi/trips_xaa_5M.csv', names=columns_names, header=None, parse_dates=parse_dates)\n",
+    "t1 = timer()\n",
+    "print(\"Modin time:\", t1 - t0)\n",
+    "t0 = timer()\n",
+    "pandas_df = pandas.read_csv('s3://modin-datasets/taxi/trips_xaa_5M.csv', names=columns_names, header=None, parse_dates=parse_dates)\n",
+    "t1 = timer()\n",
+    "print(\"pandas time:\", t1 - t0)"
    ]
   },
   {
@@ -248,210 +271,210 @@
        "      <td>...</td>\n",
        "    </tr>\n",
        "    <tr>\n",
-       "      <th>9995</th>\n",
-       "      <td>9881</td>\n",
+       "      <th>4999995</th>\n",
+       "      <td>4999986</td>\n",
        "      <td>2</td>\n",
-       "      <td>2013-09-29 01:03:48</td>\n",
-       "      <td>2013-09-29 01:09:49</td>\n",
+       "      <td>2014-04-19 23:49:32</td>\n",
+       "      <td>2014-04-19 23:54:32</td>\n",
        "      <td>N</td>\n",
        "      <td>1</td>\n",
-       "      <td>-73.958336</td>\n",
-       "      <td>40.820820</td>\n",
-       "      <td>-73.953773</td>\n",
-       "      <td>40.825195</td>\n",
-       "      <td>...</td>\n",
-       "      <td>907.0</td>\n",
-       "      <td>225.0</td>\n",
-       "      <td>1.0</td>\n",
-       "      <td>Manhattan</td>\n",
-       "      <td>22500.0</td>\n",
-       "      <td>1022500.0</td>\n",
-       "      <td>E</td>\n",
-       "      <td>MN04</td>\n",
-       "      <td>Hamilton Heights</td>\n",
-       "      <td>3802.0</td>\n",
+       "      <td>-73.829720</td>\n",
+       "      <td>40.759655</td>\n",
+       "      <td>-73.819145</td>\n",
+       "      <td>40.766457</td>\n",
+       "      <td>...</td>\n",
+       "      <td>1087.0</td>\n",
+       "      <td>1159.0</td>\n",
+       "      <td>4.0</td>\n",
+       "      <td>Queens</td>\n",
+       "      <td>115900.0</td>\n",
+       "      <td>4115900.0</td>\n",
+       "      <td>I</td>\n",
+       "      <td>QN51</td>\n",
+       "      <td>Murray Hill</td>\n",
+       "      <td>4103.0</td>\n",
        "    </tr>\n",
        "    <tr>\n",
-       "      <th>9996</th>\n",
-       "      <td>9882</td>\n",
+       "      <th>4999996</th>\n",
+       "      <td>4999987</td>\n",
        "      <td>2</td>\n",
-       "      <td>2013-09-29 03:04:10</td>\n",
-       "      <td>2013-09-29 03:09:37</td>\n",
+       "      <td>2014-04-25 23:14:41</td>\n",
+       "      <td>2014-04-25 23:26:28</td>\n",
        "      <td>N</td>\n",
        "      <td>1</td>\n",
-       "      <td>-73.958824</td>\n",
-       "      <td>40.820251</td>\n",
-       "      <td>-73.934174</td>\n",
-       "      <td>40.853394</td>\n",
-       "      <td>...</td>\n",
-       "      <td>912.0</td>\n",
-       "      <td>271.0</td>\n",
-       "      <td>1.0</td>\n",
-       "      <td>Manhattan</td>\n",
-       "      <td>27100.0</td>\n",
-       "      <td>1027100.0</td>\n",
+       "      <td>-73.829750</td>\n",
+       "      <td>40.759586</td>\n",
+       "      <td>-73.842163</td>\n",
+       "      <td>40.783901</td>\n",
+       "      <td>...</td>\n",
+       "      <td>1035.0</td>\n",
+       "      <td>925.0</td>\n",
+       "      <td>4.0</td>\n",
+       "      <td>Queens</td>\n",
+       "      <td>92500.0</td>\n",
+       "      <td>4092500.0</td>\n",
        "      <td>E</td>\n",
-       "      <td>MN35</td>\n",
-       "      <td>Washington Heights North</td>\n",
-       "      <td>3801.0</td>\n",
+       "      <td>QN23</td>\n",
+       "      <td>College Point</td>\n",
+       "      <td>4103.0</td>\n",
        "    </tr>\n",
        "    <tr>\n",
-       "      <th>9997</th>\n",
-       "      <td>9883</td>\n",
+       "      <th>4999997</th>\n",
+       "      <td>4999988</td>\n",
        "      <td>2</td>\n",
-       "      <td>2013-09-30 16:28:12</td>\n",
-       "      <td>2013-09-30 16:56:03</td>\n",
+       "      <td>2014-04-14 15:03:51</td>\n",
+       "      <td>2014-04-14 15:27:21</td>\n",
        "      <td>N</td>\n",
        "      <td>1</td>\n",
-       "      <td>-73.956100</td>\n",
-       "      <td>40.818974</td>\n",
-       "      <td>-73.941055</td>\n",
-       "      <td>40.789993</td>\n",
-       "      <td>...</td>\n",
-       "      <td>1318.0</td>\n",
-       "      <td>170.0</td>\n",
-       "      <td>1.0</td>\n",
-       "      <td>Manhattan</td>\n",
-       "      <td>17000.0</td>\n",
-       "      <td>1017000.0</td>\n",
-       "      <td>E</td>\n",
-       "      <td>MN33</td>\n",
-       "      <td>East Harlem South</td>\n",
-       "      <td>3804.0</td>\n",
+       "      <td>-73.829262</td>\n",
+       "      <td>40.759766</td>\n",
+       "      <td>-73.913078</td>\n",
+       "      <td>40.747658</td>\n",
+       "      <td>...</td>\n",
+       "      <td>1349.0</td>\n",
+       "      <td>169.0</td>\n",
+       "      <td>4.0</td>\n",
+       "      <td>Queens</td>\n",
+       "      <td>16900.0</td>\n",
+       "      <td>4016900.0</td>\n",
+       "      <td>I</td>\n",
+       "      <td>QN31</td>\n",
+       "      <td>Hunters Point-Sunnyside-West Maspeth</td>\n",
+       "      <td>4109.0</td>\n",
        "    </tr>\n",
        "    <tr>\n",
-       "      <th>9998</th>\n",
-       "      <td>9884</td>\n",
+       "      <th>4999998</th>\n",
+       "      <td>4999989</td>\n",
        "      <td>2</td>\n",
-       "      <td>2013-09-01 13:15:15</td>\n",
-       "      <td>2013-09-01 13:23:10</td>\n",
+       "      <td>2014-04-03 01:35:17</td>\n",
+       "      <td>2014-04-03 01:41:35</td>\n",
        "      <td>N</td>\n",
        "      <td>1</td>\n",
-       "      <td>-73.955345</td>\n",
-       "      <td>40.820053</td>\n",
-       "      <td>-73.942444</td>\n",
-       "      <td>40.841507</td>\n",
-       "      <td>...</td>\n",
-       "      <td>911.0</td>\n",
-       "      <td>251.0</td>\n",
-       "      <td>1.0</td>\n",
-       "      <td>Manhattan</td>\n",
-       "      <td>25100.0</td>\n",
-       "      <td>1025100.0</td>\n",
+       "      <td>-73.829758</td>\n",
+       "      <td>40.759697</td>\n",
+       "      <td>-73.820633</td>\n",
+       "      <td>40.765450</td>\n",
+       "      <td>...</td>\n",
+       "      <td>1085.0</td>\n",
+       "      <td>1163.0</td>\n",
+       "      <td>4.0</td>\n",
+       "      <td>Queens</td>\n",
+       "      <td>116300.0</td>\n",
+       "      <td>4116300.0</td>\n",
        "      <td>E</td>\n",
-       "      <td>MN36</td>\n",
-       "      <td>Washington Heights South</td>\n",
-       "      <td>3801.0</td>\n",
+       "      <td>QN51</td>\n",
+       "      <td>Murray Hill</td>\n",
+       "      <td>4103.0</td>\n",
        "    </tr>\n",
        "    <tr>\n",
-       "      <th>9999</th>\n",
-       "      <td>9885</td>\n",
+       "      <th>4999999</th>\n",
+       "      <td>4999990</td>\n",
        "      <td>2</td>\n",
-       "      <td>2013-09-20 07:32:17</td>\n",
-       "      <td>2013-09-20 08:01:06</td>\n",
+       "      <td>2014-04-03 17:17:34</td>\n",
+       "      <td>2014-04-03 17:19:58</td>\n",
        "      <td>N</td>\n",
        "      <td>1</td>\n",
-       "      <td>-73.955353</td>\n",
-       "      <td>40.820213</td>\n",
-       "      <td>-73.957680</td>\n",
-       "      <td>40.765190</td>\n",
-       "      <td>...</td>\n",
-       "      <td>1758.0</td>\n",
-       "      <td>116.0</td>\n",
-       "      <td>1.0</td>\n",
-       "      <td>Manhattan</td>\n",
-       "      <td>11600.0</td>\n",
-       "      <td>1011600.0</td>\n",
-       "      <td>I</td>\n",
-       "      <td>MN31</td>\n",
-       "      <td>Lenox Hill-Roosevelt Island</td>\n",
-       "      <td>3805.0</td>\n",
+       "      <td>-73.829750</td>\n",
+       "      <td>40.759701</td>\n",
+       "      <td>-73.826752</td>\n",
+       "      <td>40.760365</td>\n",
+       "      <td>...</td>\n",
+       "      <td>760.0</td>\n",
+       "      <td>853.0</td>\n",
+       "      <td>4.0</td>\n",
+       "      <td>Queens</td>\n",
+       "      <td>85300.0</td>\n",
+       "      <td>4085300.0</td>\n",
+       "      <td>E</td>\n",
+       "      <td>QN22</td>\n",
+       "      <td>Flushing</td>\n",
+       "      <td>4103.0</td>\n",
        "    </tr>\n",
        "  </tbody>\n",
        "</table>\n",
-       "<p>10000 rows x 51 columns</p>\n",
+       "<p>5000000 rows x 51 columns</p>\n",
        "</div>"
       ],
       "text/plain": [
-       "      trip_id  vendor_id     pickup_datetime    dropoff_datetime  \\\n",
-       "0           1          2 2013-08-01 08:14:37 2013-08-01 09:09:06   \n",
-       "1           2          2 2013-08-01 09:13:00 2013-08-01 11:38:00   \n",
-       "2           3          2 2013-08-01 09:48:00 2013-08-01 09:49:00   \n",
-       "3           4          2 2013-08-01 10:38:35 2013-08-01 10:38:51   \n",
-       "4           5          2 2013-08-01 11:51:45 2013-08-01 12:03:52   \n",
-       "...       ...        ...                 ...                 ...   \n",
-       "9995     9881          2 2013-09-29 01:03:48 2013-09-29 01:09:49   \n",
-       "9996     9882          2 2013-09-29 03:04:10 2013-09-29 03:09:37   \n",
-       "9997     9883          2 2013-09-30 16:28:12 2013-09-30 16:56:03   \n",
-       "9998     9884          2 2013-09-01 13:15:15 2013-09-01 13:23:10   \n",
-       "9999     9885          2 2013-09-20 07:32:17 2013-09-20 08:01:06   \n",
-       "\n",
-       "     store_and_fwd_flag  rate_code_id  pickup_longitude  pickup_latitude  \\\n",
-       "0                     N             1               NaN              NaN   \n",
-       "1                     N             1               NaN              NaN   \n",
-       "2                     N             5               NaN              NaN   \n",
-       "3                     N             1               NaN              NaN   \n",
-       "4                     N             1               NaN              NaN   \n",
-       "...                 ...           ...               ...              ...   \n",
-       "9995                  N             1        -73.958336        40.820820   \n",
-       "9996                  N             1        -73.958824        40.820251   \n",
-       "9997                  N             1        -73.956100        40.818974   \n",
-       "9998                  N             1        -73.955345        40.820053   \n",
-       "9999                  N             1        -73.955353        40.820213   \n",
-       "\n",
-       "      dropoff_longitude  dropoff_latitude  ...  dropoff_nyct2010_gid  \\\n",
-       "0                   NaN               NaN  ...                   NaN   \n",
-       "1                   NaN               NaN  ...                   NaN   \n",
-       "2                   NaN               NaN  ...                   NaN   \n",
-       "3                   NaN               NaN  ...                   NaN   \n",
-       "4                   NaN               NaN  ...                   NaN   \n",
-       "...                 ...               ...  ...                   ...   \n",
-       "9995         -73.953773         40.825195  ...                 907.0   \n",
-       "9996         -73.934174         40.853394  ...                 912.0   \n",
-       "9997         -73.941055         40.789993  ...                1318.0   \n",
-       "9998         -73.942444         40.841507  ...                 911.0   \n",
-       "9999         -73.957680         40.765190  ...                1758.0   \n",
-       "\n",
-       "      dropoff_ctlabel  dropoff_borocode  dropoff_boroname  dropoff_ct2010  \\\n",
-       "0                 NaN               NaN               NaN             NaN   \n",
-       "1                 NaN               NaN               NaN             NaN   \n",
-       "2                 NaN               NaN               NaN             NaN   \n",
-       "3                 NaN               NaN               NaN             NaN   \n",
-       "4                 NaN               NaN               NaN             NaN   \n",
-       "...               ...               ...               ...             ...   \n",
-       "9995            225.0               1.0         Manhattan         22500.0   \n",
-       "9996            271.0               1.0         Manhattan         27100.0   \n",
-       "9997            170.0               1.0         Manhattan         17000.0   \n",
-       "9998            251.0               1.0         Manhattan         25100.0   \n",
-       "9999            116.0               1.0         Manhattan         11600.0   \n",
-       "\n",
-       "     dropoff_boroct2010  dropoff_cdeligibil  dropoff_ntacode  \\\n",
-       "0                   NaN                 NaN              NaN   \n",
-       "1                   NaN                 NaN              NaN   \n",
-       "2                   NaN                 NaN              NaN   \n",
-       "3                   NaN                 NaN              NaN   \n",
-       "4                   NaN                 NaN              NaN   \n",
-       "...                 ...                 ...              ...   \n",
-       "9995          1022500.0                   E             MN04   \n",
-       "9996          1027100.0                   E             MN35   \n",
-       "9997          1017000.0                   E             MN33   \n",
-       "9998          1025100.0                   E             MN36   \n",
-       "9999          1011600.0                   I             MN31   \n",
-       "\n",
-       "                  dropoff_ntaname dropoff_puma  \n",
-       "0                             NaN          NaN  \n",
-       "1                             NaN          NaN  \n",
-       "2                             NaN          NaN  \n",
-       "3                             NaN          NaN  \n",
-       "4                             NaN          NaN  \n",
-       "...                           ...          ...  \n",
-       "9995             Hamilton Heights       3802.0  \n",
-       "9996     Washington Heights North       3801.0  \n",
-       "9997            East Harlem South       3804.0  \n",
-       "9998     Washington Heights South       3801.0  \n",
-       "9999  Lenox Hill-Roosevelt Island       3805.0  \n",
-       "\n",
-       "[10000 rows x 51 columns]"
+       "         trip_id  vendor_id     pickup_datetime    dropoff_datetime  \\\n",
+       "0              1          2 2013-08-01 08:14:37 2013-08-01 09:09:06   \n",
+       "1              2          2 2013-08-01 09:13:00 2013-08-01 11:38:00   \n",
+       "2              3          2 2013-08-01 09:48:00 2013-08-01 09:49:00   \n",
+       "3              4          2 2013-08-01 10:38:35 2013-08-01 10:38:51   \n",
+       "4              5          2 2013-08-01 11:51:45 2013-08-01 12:03:52   \n",
+       "...          ...        ...                 ...                 ...   \n",
+       "4999995  4999986          2 2014-04-19 23:49:32 2014-04-19 23:54:32   \n",
+       "4999996  4999987          2 2014-04-25 23:14:41 2014-04-25 23:26:28   \n",
+       "4999997  4999988          2 2014-04-14 15:03:51 2014-04-14 15:27:21   \n",
+       "4999998  4999989          2 2014-04-03 01:35:17 2014-04-03 01:41:35   \n",
+       "4999999  4999990          2 2014-04-03 17:17:34 2014-04-03 17:19:58   \n",
+       "\n",
+       "        store_and_fwd_flag  rate_code_id  pickup_longitude  pickup_latitude  \\\n",
+       "0                        N             1               NaN              NaN   \n",
+       "1                        N             1               NaN              NaN   \n",
+       "2                        N             5               NaN              NaN   \n",
+       "3                        N             1               NaN              NaN   \n",
+       "4                        N             1               NaN              NaN   \n",
+       "...                    ...           ...               ...              ...   \n",
+       "4999995                  N             1        -73.829720        40.759655   \n",
+       "4999996                  N             1        -73.829750        40.759586   \n",
+       "4999997                  N             1        -73.829262        40.759766   \n",
+       "4999998                  N             1        -73.829758        40.759697   \n",
+       "4999999                  N             1        -73.829750        40.759701   \n",
+       "\n",
+       "         dropoff_longitude  dropoff_latitude  ...  dropoff_nyct2010_gid  \\\n",
+       "0                      NaN               NaN  ...                   NaN   \n",
+       "1                      NaN               NaN  ...                   NaN   \n",
+       "2                      NaN               NaN  ...                   NaN   \n",
+       "3                      NaN               NaN  ...                   NaN   \n",
+       "4                      NaN               NaN  ...                   NaN   \n",
+       "...                    ...               ...  ...                   ...   \n",
+       "4999995         -73.819145         40.766457  ...                1087.0   \n",
+       "4999996         -73.842163         40.783901  ...                1035.0   \n",
+       "4999997         -73.913078         40.747658  ...                1349.0   \n",
+       "4999998         -73.820633         40.765450  ...                1085.0   \n",
+       "4999999         -73.826752         40.760365  ...                 760.0   \n",
+       "\n",
+       "         dropoff_ctlabel  dropoff_borocode  dropoff_boroname  dropoff_ct2010  \\\n",
+       "0                    NaN               NaN               NaN             NaN   \n",
+       "1                    NaN               NaN               NaN             NaN   \n",
+       "2                    NaN               NaN               NaN             NaN   \n",
+       "3                    NaN               NaN               NaN             NaN   \n",
+       "4                    NaN               NaN               NaN             NaN   \n",
+       "...                  ...               ...               ...             ...   \n",
+       "4999995           1159.0               4.0            Queens        115900.0   \n",
+       "4999996            925.0               4.0            Queens         92500.0   \n",
+       "4999997            169.0               4.0            Queens         16900.0   \n",
+       "4999998           1163.0               4.0            Queens        116300.0   \n",
+       "4999999            853.0               4.0            Queens         85300.0   \n",
+       "\n",
+       "        dropoff_boroct2010  dropoff_cdeligibil  dropoff_ntacode  \\\n",
+       "0                      NaN                 NaN              NaN   \n",
+       "1                      NaN                 NaN              NaN   \n",
+       "2                      NaN                 NaN              NaN   \n",
+       "3                      NaN                 NaN              NaN   \n",
+       "4                      NaN                 NaN              NaN   \n",
+       "...                    ...                 ...              ...   \n",
+       "4999995          4115900.0                   I             QN51   \n",
+       "4999996          4092500.0                   E             QN23   \n",
+       "4999997          4016900.0                   I             QN31   \n",
+       "4999998          4116300.0                   E             QN51   \n",
+       "4999999          4085300.0                   E             QN22   \n",
+       "\n",
+       "                              dropoff_ntaname dropoff_puma  \n",
+       "0                                         NaN          NaN  \n",
+       "1                                         NaN          NaN  \n",
+       "2                                         NaN          NaN  \n",
+       "3                                         NaN          NaN  \n",
+       "4                                         NaN          NaN  \n",
+       "...                                       ...          ...  \n",
+       "4999995                           Murray Hill       4103.0  \n",
+       "4999996                         College Point       4103.0  \n",
+       "4999997  Hunters Point-Sunnyside-West Maspeth       4109.0  \n",
+       "4999998                           Murray Hill       4103.0  \n",
+       "4999999                              Flushing       4103.0  \n",
+       "\n",
+       "[5000000 rows x 51 columns]"
       ]
      },
      "execution_count": 4,
@@ -460,55 +483,13 @@
     }
    ],
    "source": [
-    "df"
+    "modin_df"
    ]
   },
   {
    "cell_type": "code",
    "execution_count": 5,
    "metadata": {},
-   "outputs": [],
-   "source": [
-    "def q1(df):\n",
-    "    return df.groupby(\"cab_type\")[\"cab_type\"].count()\n",
-    "def q2(df):\n",
-    "    return df.groupby(\"passenger_count\", as_index=False).mean()[[\"passenger_count\", \"total_amount\"]]\n",
-    "def q3(df):\n",
-    "    return df.groupby([\"passenger_count\", \"pickup_datetime\"]).size().reset_index()\n",
-    "def q4(df):\n",
-    "    transformed = pd.DataFrame({\n",
-    "        \"passenger_count\": df[\"passenger_count\"],\n",
-    "        \"pickup_datetime\": df[\"pickup_datetime\"].dt.year,\n",
-    "        \"trip_distance\": df[\"trip_distance\"].astype(\"int64\"),\n",
-    "    })\n",
-    "    return transformed.groupby([\"passenger_count\", \"pickup_datetime\", \"trip_distance\"])  \\\n",
-    "            .size().reset_index().sort_values(by=[\"pickup_datetime\", 0], ascending=[True, False])"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 6,
-   "metadata": {},
-   "outputs": [
-    {
-     "data": {
-      "text/plain": [
-       "10000"
-      ]
-     },
-     "execution_count": 6,
-     "metadata": {},
-     "output_type": "execute_result"
-    }
-   ],
-   "source": [
-    "q1(df)"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 7,
-   "metadata": {},
    "outputs": [
     {
      "data": {
@@ -531,74 +512,889 @@
        "  <thead>\n",
        "    <tr style=\"text-align: right;\">\n",
        "      <th></th>\n",
-       "      <th>passenger_count</th>\n",
-       "      <th>total_amount</th>\n",
+       "      <th>trip_id</th>\n",
+       "      <th>vendor_id</th>\n",
+       "      <th>pickup_datetime</th>\n",
+       "      <th>dropoff_datetime</th>\n",
+       "      <th>store_and_fwd_flag</th>\n",
+       "      <th>rate_code_id</th>\n",
+       "      <th>pickup_longitude</th>\n",
+       "      <th>pickup_latitude</th>\n",
+       "      <th>dropoff_longitude</th>\n",
+       "      <th>dropoff_latitude</th>\n",
+       "      <th>...</th>\n",
+       "      <th>dropoff_nyct2010_gid</th>\n",
+       "      <th>dropoff_ctlabel</th>\n",
+       "      <th>dropoff_borocode</th>\n",
+       "      <th>dropoff_boroname</th>\n",
+       "      <th>dropoff_ct2010</th>\n",
+       "      <th>dropoff_boroct2010</th>\n",
+       "      <th>dropoff_cdeligibil</th>\n",
+       "      <th>dropoff_ntacode</th>\n",
+       "      <th>dropoff_ntaname</th>\n",
+       "      <th>dropoff_puma</th>\n",
        "    </tr>\n",
        "  </thead>\n",
        "  <tbody>\n",
        "    <tr>\n",
        "      <th>0</th>\n",
-       "      <td>0</td>\n",
-       "      <td>18.333333</td>\n",
+       "      <td>1</td>\n",
+       "      <td>2</td>\n",
+       "      <td>2013-08-01 08:14:37</td>\n",
+       "      <td>2013-08-01 09:09:06</td>\n",
+       "      <td>N</td>\n",
+       "      <td>1</td>\n",
+       "      <td>NaN</td>\n",
+       "      <td>NaN</td>\n",
+       "      <td>NaN</td>\n",
+       "      <td>NaN</td>\n",
+       "      <td>...</td>\n",
+       "      <td>NaN</td>\n",
+       "      <td>NaN</td>\n",
+       "      <td>NaN</td>\n",
+       "      <td>NaN</td>\n",
+       "      <td>NaN</td>\n",
+       "      <td>NaN</td>\n",
+       "      <td>NaN</td>\n",
+       "      <td>NaN</td>\n",
+       "      <td>NaN</td>\n",
+       "      <td>NaN</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>1</th>\n",
+       "      <td>2</td>\n",
+       "      <td>2</td>\n",
+       "      <td>2013-08-01 09:13:00</td>\n",
+       "      <td>2013-08-01 11:38:00</td>\n",
+       "      <td>N</td>\n",
        "      <td>1</td>\n",
-       "      <td>15.258850</td>\n",
+       "      <td>NaN</td>\n",
+       "      <td>NaN</td>\n",
+       "      <td>NaN</td>\n",
+       "      <td>NaN</td>\n",
+       "      <td>...</td>\n",
+       "      <td>NaN</td>\n",
+       "      <td>NaN</td>\n",
+       "      <td>NaN</td>\n",
+       "      <td>NaN</td>\n",
+       "      <td>NaN</td>\n",
+       "      <td>NaN</td>\n",
+       "      <td>NaN</td>\n",
+       "      <td>NaN</td>\n",
+       "      <td>NaN</td>\n",
+       "      <td>NaN</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>2</th>\n",
+       "      <td>3</td>\n",
        "      <td>2</td>\n",
-       "      <td>20.332356</td>\n",
+       "      <td>2013-08-01 09:48:00</td>\n",
+       "      <td>2013-08-01 09:49:00</td>\n",
+       "      <td>N</td>\n",
+       "      <td>5</td>\n",
+       "      <td>NaN</td>\n",
+       "      <td>NaN</td>\n",
+       "      <td>NaN</td>\n",
+       "      <td>NaN</td>\n",
+       "      <td>...</td>\n",
+       "      <td>NaN</td>\n",
+       "      <td>NaN</td>\n",
+       "      <td>NaN</td>\n",
+       "      <td>NaN</td>\n",
+       "      <td>NaN</td>\n",
+       "      <td>NaN</td>\n",
+       "      <td>NaN</td>\n",
+       "      <td>NaN</td>\n",
+       "      <td>NaN</td>\n",
+       "      <td>NaN</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>3</th>\n",
-       "      <td>3</td>\n",
-       "      <td>13.748845</td>\n",
+       "      <td>4</td>\n",
+       "      <td>2</td>\n",
+       "      <td>2013-08-01 10:38:35</td>\n",
+       "      <td>2013-08-01 10:38:51</td>\n",
+       "      <td>N</td>\n",
+       "      <td>1</td>\n",
+       "      <td>NaN</td>\n",
+       "      <td>NaN</td>\n",
+       "      <td>NaN</td>\n",
+       "      <td>NaN</td>\n",
+       "      <td>...</td>\n",
+       "      <td>NaN</td>\n",
+       "      <td>NaN</td>\n",
+       "      <td>NaN</td>\n",
+       "      <td>NaN</td>\n",
+       "      <td>NaN</td>\n",
+       "      <td>NaN</td>\n",
+       "      <td>NaN</td>\n",
+       "      <td>NaN</td>\n",
+       "      <td>NaN</td>\n",
+       "      <td>NaN</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>4</th>\n",
-       "      <td>4</td>\n",
-       "      <td>19.742688</td>\n",
+       "      <td>5</td>\n",
+       "      <td>2</td>\n",
+       "      <td>2013-08-01 11:51:45</td>\n",
+       "      <td>2013-08-01 12:03:52</td>\n",
+       "      <td>N</td>\n",
+       "      <td>1</td>\n",
+       "      <td>NaN</td>\n",
+       "      <td>NaN</td>\n",
+       "      <td>NaN</td>\n",
+       "      <td>NaN</td>\n",
+       "      <td>...</td>\n",
+       "      <td>NaN</td>\n",
+       "      <td>NaN</td>\n",
+       "      <td>NaN</td>\n",
+       "      <td>NaN</td>\n",
+       "      <td>NaN</td>\n",
+       "      <td>NaN</td>\n",
+       "      <td>NaN</td>\n",
+       "      <td>NaN</td>\n",
+       "      <td>NaN</td>\n",
+       "      <td>NaN</td>\n",
        "    </tr>\n",
        "    <tr>\n",
-       "      <th>5</th>\n",
-       "      <td>5</td>\n",
-       "      <td>14.786221</td>\n",
+       "      <th>...</th>\n",
+       "      <td>...</td>\n",
+       "      <td>...</td>\n",
+       "      <td>...</td>\n",
+       "      <td>...</td>\n",
+       "      <td>...</td>\n",
+       "      <td>...</td>\n",
+       "      <td>...</td>\n",
+       "      <td>...</td>\n",
+       "      <td>...</td>\n",
+       "      <td>...</td>\n",
+       "      <td>...</td>\n",
+       "      <td>...</td>\n",
+       "      <td>...</td>\n",
+       "      <td>...</td>\n",
+       "      <td>...</td>\n",
+       "      <td>...</td>\n",
+       "      <td>...</td>\n",
+       "      <td>...</td>\n",
+       "      <td>...</td>\n",
+       "      <td>...</td>\n",
+       "      <td>...</td>\n",
        "    </tr>\n",
        "    <tr>\n",
-       "      <th>6</th>\n",
-       "      <td>6</td>\n",
-       "      <td>15.400085</td>\n",
+       "      <th>4999995</th>\n",
+       "      <td>4999986</td>\n",
+       "      <td>2</td>\n",
+       "      <td>2014-04-19 23:49:32</td>\n",
+       "      <td>2014-04-19 23:54:32</td>\n",
+       "      <td>N</td>\n",
+       "      <td>1</td>\n",
+       "      <td>-73.829720</td>\n",
+       "      <td>40.759655</td>\n",
+       "      <td>-73.819145</td>\n",
+       "      <td>40.766457</td>\n",
+       "      <td>...</td>\n",
+       "      <td>1087.0</td>\n",
+       "      <td>1159.0</td>\n",
+       "      <td>4.0</td>\n",
+       "      <td>Queens</td>\n",
+       "      <td>115900.0</td>\n",
+       "      <td>4115900.0</td>\n",
+       "      <td>I</td>\n",
+       "      <td>QN51</td>\n",
+       "      <td>Murray Hill</td>\n",
+       "      <td>4103.0</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>4999996</th>\n",
+       "      <td>4999987</td>\n",
+       "      <td>2</td>\n",
+       "      <td>2014-04-25 23:14:41</td>\n",
+       "      <td>2014-04-25 23:26:28</td>\n",
+       "      <td>N</td>\n",
+       "      <td>1</td>\n",
+       "      <td>-73.829750</td>\n",
+       "      <td>40.759586</td>\n",
+       "      <td>-73.842163</td>\n",
+       "      <td>40.783901</td>\n",
+       "      <td>...</td>\n",
+       "      <td>1035.0</td>\n",
+       "      <td>925.0</td>\n",
+       "      <td>4.0</td>\n",
+       "      <td>Queens</td>\n",
+       "      <td>92500.0</td>\n",
+       "      <td>4092500.0</td>\n",
+       "      <td>E</td>\n",
+       "      <td>QN23</td>\n",
+       "      <td>College Point</td>\n",
+       "      <td>4103.0</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>4999997</th>\n",
+       "      <td>4999988</td>\n",
+       "      <td>2</td>\n",
+       "      <td>2014-04-14 15:03:51</td>\n",
+       "      <td>2014-04-14 15:27:21</td>\n",
+       "      <td>N</td>\n",
+       "      <td>1</td>\n",
+       "      <td>-73.829262</td>\n",
+       "      <td>40.759766</td>\n",
+       "      <td>-73.913078</td>\n",
+       "      <td>40.747658</td>\n",
+       "      <td>...</td>\n",
+       "      <td>1349.0</td>\n",
+       "      <td>169.0</td>\n",
+       "      <td>4.0</td>\n",
+       "      <td>Queens</td>\n",
+       "      <td>16900.0</td>\n",
+       "      <td>4016900.0</td>\n",
+       "      <td>I</td>\n",
+       "      <td>QN31</td>\n",
+       "      <td>Hunters Point-Sunnyside-West Maspeth</td>\n",
+       "      <td>4109.0</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>4999998</th>\n",
+       "      <td>4999989</td>\n",
+       "      <td>2</td>\n",
+       "      <td>2014-04-03 01:35:17</td>\n",
+       "      <td>2014-04-03 01:41:35</td>\n",
+       "      <td>N</td>\n",
+       "      <td>1</td>\n",
+       "      <td>-73.829758</td>\n",
+       "      <td>40.759697</td>\n",
+       "      <td>-73.820633</td>\n",
+       "      <td>40.765450</td>\n",
+       "      <td>...</td>\n",
+       "      <td>1085.0</td>\n",
+       "      <td>1163.0</td>\n",
+       "      <td>4.0</td>\n",
+       "      <td>Queens</td>\n",
+       "      <td>116300.0</td>\n",
+       "      <td>4116300.0</td>\n",
+       "      <td>E</td>\n",
+       "      <td>QN51</td>\n",
+       "      <td>Murray Hill</td>\n",
+       "      <td>4103.0</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>4999999</th>\n",
+       "      <td>4999990</td>\n",
+       "      <td>2</td>\n",
+       "      <td>2014-04-03 17:17:34</td>\n",
+       "      <td>2014-04-03 17:19:58</td>\n",
+       "      <td>N</td>\n",
+       "      <td>1</td>\n",
+       "      <td>-73.829750</td>\n",
+       "      <td>40.759701</td>\n",
+       "      <td>-73.826752</td>\n",
+       "      <td>40.760365</td>\n",
+       "      <td>...</td>\n",
+       "      <td>760.0</td>\n",
+       "      <td>853.0</td>\n",
+       "      <td>4.0</td>\n",
+       "      <td>Queens</td>\n",
+       "      <td>85300.0</td>\n",
+       "      <td>4085300.0</td>\n",
+       "      <td>E</td>\n",
+       "      <td>QN22</td>\n",
+       "      <td>Flushing</td>\n",
+       "      <td>4103.0</td>\n",
        "    </tr>\n",
        "  </tbody>\n",
        "</table>\n",
+       "<p>5000000 rows × 51 columns</p>\n",
        "</div>"
       ],
       "text/plain": [
-       "   passenger_count  total_amount\n",
-       "0                0     18.333333\n",
-       "1                1     15.258850\n",
-       "2                2     20.332356\n",
-       "3                3     13.748845\n",
-       "4                4     19.742688\n",
-       "5                5     14.786221\n",
-       "6                6     15.400085"
+       "         trip_id  vendor_id     pickup_datetime    dropoff_datetime  \\\n",
+       "0              1          2 2013-08-01 08:14:37 2013-08-01 09:09:06   \n",
+       "1              2          2 2013-08-01 09:13:00 2013-08-01 11:38:00   \n",
+       "2              3          2 2013-08-01 09:48:00 2013-08-01 09:49:00   \n",
+       "3              4          2 2013-08-01 10:38:35 2013-08-01 10:38:51   \n",
+       "4              5          2 2013-08-01 11:51:45 2013-08-01 12:03:52   \n",
+       "...          ...        ...                 ...                 ...   \n",
+       "4999995  4999986          2 2014-04-19 23:49:32 2014-04-19 23:54:32   \n",
+       "4999996  4999987          2 2014-04-25 23:14:41 2014-04-25 23:26:28   \n",
+       "4999997  4999988          2 2014-04-14 15:03:51 2014-04-14 15:27:21   \n",
+       "4999998  4999989          2 2014-04-03 01:35:17 2014-04-03 01:41:35   \n",
+       "4999999  4999990          2 2014-04-03 17:17:34 2014-04-03 17:19:58   \n",
+       "\n",
+       "        store_and_fwd_flag  rate_code_id  pickup_longitude  pickup_latitude  \\\n",
+       "0                        N             1               NaN              NaN   \n",
+       "1                        N             1               NaN              NaN   \n",
+       "2                        N             5               NaN              NaN   \n",
+       "3                        N             1               NaN              NaN   \n",
+       "4                        N             1               NaN              NaN   \n",
+       "...                    ...           ...               ...              ...   \n",
+       "4999995                  N             1        -73.829720        40.759655   \n",
+       "4999996                  N             1        -73.829750        40.759586   \n",
+       "4999997                  N             1        -73.829262        40.759766   \n",
+       "4999998                  N             1        -73.829758        40.759697   \n",
+       "4999999                  N             1        -73.829750        40.759701   \n",
+       "\n",
+       "         dropoff_longitude  dropoff_latitude  ...  dropoff_nyct2010_gid  \\\n",
+       "0                      NaN               NaN  ...                   NaN   \n",
+       "1                      NaN               NaN  ...                   NaN   \n",
+       "2                      NaN               NaN  ...                   NaN   \n",
+       "3                      NaN               NaN  ...                   NaN   \n",
+       "4                      NaN               NaN  ...                   NaN   \n",
+       "...                    ...               ...  ...                   ...   \n",
+       "4999995         -73.819145         40.766457  ...                1087.0   \n",
+       "4999996         -73.842163         40.783901  ...                1035.0   \n",
+       "4999997         -73.913078         40.747658  ...                1349.0   \n",
+       "4999998         -73.820633         40.765450  ...                1085.0   \n",
+       "4999999         -73.826752         40.760365  ...                 760.0   \n",
+       "\n",
+       "         dropoff_ctlabel  dropoff_borocode  dropoff_boroname  dropoff_ct2010  \\\n",
+       "0                    NaN               NaN               NaN             NaN   \n",
+       "1                    NaN               NaN               NaN             NaN   \n",
+       "2                    NaN               NaN               NaN             NaN   \n",
+       "3                    NaN               NaN               NaN             NaN   \n",
+       "4                    NaN               NaN               NaN             NaN   \n",
+       "...                  ...               ...               ...             ...   \n",
+       "4999995           1159.0               4.0            Queens        115900.0   \n",
+       "4999996            925.0               4.0            Queens         92500.0   \n",
+       "4999997            169.0               4.0            Queens         16900.0   \n",
+       "4999998           1163.0               4.0            Queens        116300.0   \n",
+       "4999999            853.0               4.0            Queens         85300.0   \n",
+       "\n",
+       "         dropoff_boroct2010  dropoff_cdeligibil  dropoff_ntacode  \\\n",
+       "0                       NaN                 NaN              NaN   \n",
+       "1                       NaN                 NaN              NaN   \n",
+       "2                       NaN                 NaN              NaN   \n",
+       "3                       NaN                 NaN              NaN   \n",
+       "4                       NaN                 NaN              NaN   \n",
+       "...                     ...                 ...              ...   \n",
+       "4999995           4115900.0                   I             QN51   \n",
+       "4999996           4092500.0                   E             QN23   \n",
+       "4999997           4016900.0                   I             QN31   \n",
+       "4999998           4116300.0                   E             QN51   \n",
+       "4999999           4085300.0                   E             QN22   \n",
+       "\n",
+       "                              dropoff_ntaname  dropoff_puma  \n",
+       "0                                         NaN           NaN  \n",
+       "1                                         NaN           NaN  \n",
+       "2                                         NaN           NaN  \n",
+       "3                                         NaN           NaN  \n",
+       "4                                         NaN           NaN  \n",
+       "...                                       ...           ...  \n",
+       "4999995                           Murray Hill        4103.0  \n",
+       "4999996                         College Point        4103.0  \n",
+       "4999997  Hunters Point-Sunnyside-West Maspeth        4109.0  \n",
+       "4999998                           Murray Hill        4103.0  \n",
+       "4999999                              Flushing        4103.0  \n",
+       "\n",
+       "[5000000 rows x 51 columns]"
       ]
      },
-     "execution_count": 7,
+     "execution_count": 5,
      "metadata": {},
      "output_type": "execute_result"
     }
    ],
    "source": [
-    "q2(df)"
+    "pandas_df"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "def q1(df):\n",
+    "    return df.groupby(\"cab_type\")[\"cab_type\"].count()\n",
+    "def q2(df):\n",
+    "    return df.groupby(\"passenger_count\", as_index=False).mean()[[\"passenger_count\", \"total_amount\"]]\n",
+    "def q3(df):\n",
+    "    return df.groupby([\"passenger_count\", \"pickup_datetime\"]).size().reset_index()\n",
+    "def q4(df):\n",
+    "    transformed = pd.DataFrame({\n",
+    "        \"passenger_count\": df[\"passenger_count\"],\n",
+    "        \"pickup_datetime\": df[\"pickup_datetime\"].dt.year,\n",
+    "        \"trip_distance\": df[\"trip_distance\"].astype(\"int64\"),\n",
+    "    })\n",
+    "    return transformed.groupby([\"passenger_count\", \"pickup_datetime\", \"trip_distance\"])  \\\n",
+    "            .size().reset_index().sort_values(by=[\"pickup_datetime\", 0], ascending=[True, False])"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 7,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# We should provide a justification why Modin is faster/slower than pandas"
    ]
   },
   {
    "cell_type": "code",
    "execution_count": 8,
    "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Modin time: 0.7333072200417519\n",
+      "pandas time: 0.6914568543434143\n"
+     ]
+    }
+   ],
+   "source": [
+    "t0 = timer()\n",
+    "modin_q1_result = q1(modin_df)\n",
+    "t1 = timer()\n",
+    "print(\"Modin time:\", t1 - t0)\n",
+    "t0 = timer()\n",
+    "pandas_q1_result = q1(pandas_df)\n",
+    "t1 = timer()\n",
+    "print(\"pandas time:\", t1 - t0)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 9,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "cab_type\n",
+       "green    5000000\n",
+       "Name: cab_type, dtype: int64"
+      ]
+     },
+     "execution_count": 9,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "modin_q1_result"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 10,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "cab_type\n",
+       "green    5000000\n",
+       "Name: cab_type, dtype: int64"
+      ]
+     },
+     "execution_count": 10,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "pandas_q1_result"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 11,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Modin time: 9.099235692992806\n",
+      "pandas time: 3.116944821551442\n"
+     ]
+    }
+   ],
+   "source": [
+    "t0 = timer()\n",
+    "modin_q2_result = q2(modin_df)\n",
+    "t1 = timer()\n",
+    "print(\"Modin time:\", t1 - t0)\n",
+    "t0 = timer()\n",
+    "pandas_q2_result = q2(pandas_df)\n",
+    "t1 = timer()\n",
+    "print(\"pandas time:\", t1 - t0)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 12,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/html": [
+       "<div>\n",
+       "<style scoped>\n",
+       "    .dataframe tbody tr th:only-of-type {\n",
+       "        vertical-align: middle;\n",
+       "    }\n",
+       "\n",
+       "    .dataframe tbody tr th {\n",
+       "        vertical-align: top;\n",
+       "    }\n",
+       "\n",
+       "    .dataframe thead th {\n",
+       "        text-align: right;\n",
+       "    }\n",
+       "</style>\n",
+       "<table border=\"1\" class=\"dataframe\">\n",
+       "  <thead>\n",
+       "    <tr style=\"text-align: right;\">\n",
+       "      <th></th>\n",
+       "      <th>passenger_count</th>\n",
+       "      <th>total_amount</th>\n",
+       "    </tr>\n",
+       "  </thead>\n",
+       "  <tbody>\n",
+       "    <tr>\n",
+       "      <th>0</th>\n",
+       "      <td>0</td>\n",
+       "      <td>8.046854</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>1</th>\n",
+       "      <td>1</td>\n",
+       "      <td>13.923278</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>2</th>\n",
+       "      <td>2</td>\n",
+       "      <td>14.612774</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>3</th>\n",
+       "      <td>3</td>\n",
+       "      <td>14.683111</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>4</th>\n",
+       "      <td>4</td>\n",
+       "      <td>14.793963</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>5</th>\n",
+       "      <td>5</td>\n",
+       "      <td>14.104099</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>6</th>\n",
+       "      <td>6</td>\n",
+       "      <td>15.083793</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>7</th>\n",
+       "      <td>7</td>\n",
+       "      <td>15.922059</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>8</th>\n",
+       "      <td>8</td>\n",
+       "      <td>13.881477</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>9</th>\n",
+       "      <td>9</td>\n",
+       "      <td>27.346216</td>\n",
+       "    </tr>\n",
+       "  </tbody>\n",
+       "</table>\n",
+       "</div>"
+      ],
+      "text/plain": [
+       "   passenger_count  total_amount\n",
+       "0                0      8.046854\n",
+       "1                1     13.923278\n",
+       "2                2     14.612774\n",
+       "3                3     14.683111\n",
+       "4                4     14.793963\n",
+       "5                5     14.104099\n",
+       "6                6     15.083793\n",
+       "7                7     15.922059\n",
+       "8                8     13.881477\n",
+       "9                9     27.346216"
+      ]
+     },
+     "execution_count": 12,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "modin_q2_result"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 13,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/html": [
+       "<div>\n",
+       "<style scoped>\n",
+       "    .dataframe tbody tr th:only-of-type {\n",
+       "        vertical-align: middle;\n",
+       "    }\n",
+       "\n",
+       "    .dataframe tbody tr th {\n",
+       "        vertical-align: top;\n",
+       "    }\n",
+       "\n",
+       "    .dataframe thead th {\n",
+       "        text-align: right;\n",
+       "    }\n",
+       "</style>\n",
+       "<table border=\"1\" class=\"dataframe\">\n",
+       "  <thead>\n",
+       "    <tr style=\"text-align: right;\">\n",
+       "      <th></th>\n",
+       "      <th>passenger_count</th>\n",
+       "      <th>total_amount</th>\n",
+       "    </tr>\n",
+       "  </thead>\n",
+       "  <tbody>\n",
+       "    <tr>\n",
+       "      <th>0</th>\n",
+       "      <td>0</td>\n",
+       "      <td>8.046854</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>1</th>\n",
+       "      <td>1</td>\n",
+       "      <td>13.923278</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>2</th>\n",
+       "      <td>2</td>\n",
+       "      <td>14.612774</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>3</th>\n",
+       "      <td>3</td>\n",
+       "      <td>14.683111</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>4</th>\n",
+       "      <td>4</td>\n",
+       "      <td>14.793963</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>5</th>\n",
+       "      <td>5</td>\n",
+       "      <td>14.104099</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>6</th>\n",
+       "      <td>6</td>\n",
+       "      <td>15.083793</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>7</th>\n",
+       "      <td>7</td>\n",
+       "      <td>15.922059</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>8</th>\n",
+       "      <td>8</td>\n",
+       "      <td>13.881477</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>9</th>\n",
+       "      <td>9</td>\n",
+       "      <td>27.346216</td>\n",
+       "    </tr>\n",
+       "  </tbody>\n",
+       "</table>\n",
+       "</div>"
+      ],
+      "text/plain": [
+       "   passenger_count  total_amount\n",
+       "0                0      8.046854\n",
+       "1                1     13.923278\n",
+       "2                2     14.612774\n",
+       "3                3     14.683111\n",
+       "4                4     14.793963\n",
+       "5                5     14.104099\n",
+       "6                6     15.083793\n",
+       "7                7     15.922059\n",
+       "8                8     13.881477\n",
+       "9                9     27.346216"
+      ]
+     },
+     "execution_count": 13,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "pandas_q2_result"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 14,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Modin time: 71.6060786191374\n",
+      "pandas time: 7.657089374959469\n"
+     ]
+    }
+   ],
+   "source": [
+    "t0 = timer()\n",
+    "modin_q3_result = q3(modin_df)\n",
+    "t1 = timer()\n",
+    "print(\"Modin time:\", t1 - t0)\n",
+    "t0 = timer()\n",
+    "pandas_q3_result = q3(pandas_df)\n",
+    "t1 = timer()\n",
+    "print(\"pandas time:\", t1 - t0)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 15,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/html": [
+       "<div>\n",
+       "<style scoped>\n",
+       "    .dataframe tbody tr th:only-of-type {\n",
+       "        vertical-align: middle;\n",
+       "    }\n",
+       "\n",
+       "    .dataframe tbody tr th {\n",
+       "        vertical-align: top;\n",
+       "    }\n",
+       "\n",
+       "    .dataframe thead th {\n",
+       "        text-align: right;\n",
+       "    }\n",
+       "</style>\n",
+       "<table border=\"1\" class=\"dataframe\">\n",
+       "  <thead>\n",
+       "    <tr style=\"text-align: right;\">\n",
+       "      <th></th>\n",
+       "      <th>passenger_count</th>\n",
+       "      <th>pickup_datetime</th>\n",
+       "      <th>0</th>\n",
+       "    </tr>\n",
+       "  </thead>\n",
+       "  <tbody>\n",
+       "    <tr>\n",
+       "      <th>0</th>\n",
+       "      <td>0</td>\n",
+       "      <td>2013-08-14 12:07:00</td>\n",
+       "      <td>1</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>1</th>\n",
+       "      <td>0</td>\n",
+       "      <td>2013-08-14 12:37:00</td>\n",
+       "      <td>1</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>2</th>\n",
+       "      <td>0</td>\n",
+       "      <td>2013-08-15 00:00:00</td>\n",
+       "      <td>1</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>3</th>\n",
+       "      <td>0</td>\n",
+       "      <td>2013-09-17 13:30:00</td>\n",
+       "      <td>1</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>4</th>\n",
+       "      <td>0</td>\n",
+       "      <td>2013-09-25 17:32:46</td>\n",
+       "      <td>1</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>...</th>\n",
+       "      <td>...</td>\n",
+       "      <td>...</td>\n",
+       "      <td>...</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>4334692</th>\n",
+       "      <td>9</td>\n",
+       "      <td>2014-03-18 00:10:46</td>\n",
+       "      <td>1</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>4334693</th>\n",
+       "      <td>9</td>\n",
+       "      <td>2014-04-14 16:18:00</td>\n",
+       "      <td>1</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>4334694</th>\n",
+       "      <td>9</td>\n",
+       "      <td>2014-04-22 18:26:00</td>\n",
+       "      <td>1</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>4334695</th>\n",
+       "      <td>9</td>\n",
+       "      <td>2014-04-30 12:35:37</td>\n",
+       "      <td>1</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>4334696</th>\n",
+       "      <td>9</td>\n",
+       "      <td>2014-04-30 17:47:05</td>\n",
+       "      <td>1</td>\n",
+       "    </tr>\n",
+       "  </tbody>\n",
+       "</table>\n",
+       "<p>4334697 rows x 3 columns</p>\n",
+       "</div>"
+      ],
+      "text/plain": [
+       "         passenger_count     pickup_datetime  0\n",
+       "0                      0 2013-08-14 12:07:00  1\n",
+       "1                      0 2013-08-14 12:37:00  1\n",
+       "2                      0 2013-08-15 00:00:00  1\n",
+       "3                      0 2013-09-17 13:30:00  1\n",
+       "4                      0 2013-09-25 17:32:46  1\n",
+       "...                  ...                 ... ..\n",
+       "4334692                9 2014-03-18 00:10:46  1\n",
+       "4334693                9 2014-04-14 16:18:00  1\n",
+       "4334694                9 2014-04-22 18:26:00  1\n",
+       "4334695                9 2014-04-30 12:35:37  1\n",
+       "4334696                9 2014-04-30 17:47:05  1\n",
+       "\n",
+       "[4334697 rows x 3 columns]"
+      ]
+     },
+     "execution_count": 15,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "modin_q3_result"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 16,
+   "metadata": {},
    "outputs": [
     {
      "data": {
@@ -647,14 +1443,14 @@
        "    </tr>\n",
        "    <tr>\n",
        "      <th>3</th>\n",
-       "      <td>1</td>\n",
-       "      <td>2013-08-01 08:14:37</td>\n",
+       "      <td>0</td>\n",
+       "      <td>2013-09-17 13:30:00</td>\n",
        "      <td>1</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>4</th>\n",
-       "      <td>1</td>\n",
-       "      <td>2013-08-01 09:48:00</td>\n",
+       "      <td>0</td>\n",
+       "      <td>2013-09-25 17:32:46</td>\n",
        "      <td>1</td>\n",
        "    </tr>\n",
        "    <tr>\n",
@@ -664,69 +1460,116 @@
        "      <td>...</td>\n",
        "    </tr>\n",
        "    <tr>\n",
-       "      <th>9909</th>\n",
-       "      <td>6</td>\n",
-       "      <td>2013-09-28 18:30:15</td>\n",
+       "      <th>4334692</th>\n",
+       "      <td>9</td>\n",
+       "      <td>2014-03-18 00:10:46</td>\n",
        "      <td>1</td>\n",
        "    </tr>\n",
        "    <tr>\n",
-       "      <th>9910</th>\n",
-       "      <td>6</td>\n",
-       "      <td>2013-09-28 19:57:22</td>\n",
+       "      <th>4334693</th>\n",
+       "      <td>9</td>\n",
+       "      <td>2014-04-14 16:18:00</td>\n",
        "      <td>1</td>\n",
        "    </tr>\n",
        "    <tr>\n",
-       "      <th>9911</th>\n",
-       "      <td>6</td>\n",
-       "      <td>2013-09-29 18:47:29</td>\n",
+       "      <th>4334694</th>\n",
+       "      <td>9</td>\n",
+       "      <td>2014-04-22 18:26:00</td>\n",
        "      <td>1</td>\n",
        "    </tr>\n",
        "    <tr>\n",
-       "      <th>9912</th>\n",
-       "      <td>6</td>\n",
-       "      <td>2013-09-30 02:27:33</td>\n",
+       "      <th>4334695</th>\n",
+       "      <td>9</td>\n",
+       "      <td>2014-04-30 12:35:37</td>\n",
        "      <td>1</td>\n",
        "    </tr>\n",
        "    <tr>\n",
-       "      <th>9913</th>\n",
-       "      <td>6</td>\n",
-       "      <td>2013-09-30 21:31:06</td>\n",
+       "      <th>4334696</th>\n",
+       "      <td>9</td>\n",
+       "      <td>2014-04-30 17:47:05</td>\n",
        "      <td>1</td>\n",
        "    </tr>\n",
        "  </tbody>\n",
        "</table>\n",
-       "<p>9914 rows x 3 columns</p>\n",
+       "<p>4334697 rows × 3 columns</p>\n",
        "</div>"
       ],
       "text/plain": [
-       "      passenger_count     pickup_datetime  0\n",
-       "0                   0 2013-08-14 12:07:00  1\n",
-       "1                   0 2013-08-14 12:37:00  1\n",
-       "2                   0 2013-08-15 00:00:00  1\n",
-       "3                   1 2013-08-01 08:14:37  1\n",
-       "4                   1 2013-08-01 09:48:00  1\n",
-       "...               ...                 ... ..\n",
-       "9909                6 2013-09-28 18:30:15  1\n",
-       "9910                6 2013-09-28 19:57:22  1\n",
-       "9911                6 2013-09-29 18:47:29  1\n",
-       "9912                6 2013-09-30 02:27:33  1\n",
-       "9913                6 2013-09-30 21:31:06  1\n",
-       "\n",
-       "[9914 rows x 3 columns]"
+       "         passenger_count     pickup_datetime  0\n",
+       "0                      0 2013-08-14 12:07:00  1\n",
+       "1                      0 2013-08-14 12:37:00  1\n",
+       "2                      0 2013-08-15 00:00:00  1\n",
+       "3                      0 2013-09-17 13:30:00  1\n",
+       "4                      0 2013-09-25 17:32:46  1\n",
+       "...                  ...                 ... ..\n",
+       "4334692                9 2014-03-18 00:10:46  1\n",
+       "4334693                9 2014-04-14 16:18:00  1\n",
+       "4334694                9 2014-04-22 18:26:00  1\n",
+       "4334695                9 2014-04-30 12:35:37  1\n",
+       "4334696                9 2014-04-30 17:47:05  1\n",
+       "\n",
+       "[4334697 rows x 3 columns]"
       ]
      },
-     "execution_count": 8,
+     "execution_count": 16,
      "metadata": {},
      "output_type": "execute_result"
     }
    ],
    "source": [
-    "q3(df)"
+    "pandas_q3_result"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 9,
+   "execution_count": 17,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "UserWarning: Distributing <class 'dict'> object. This may take some time.\n",
+      "UserWarning: sort_values defaulting to pandas implementation.\n",
+      "To request implementation, send an email to feature_requests@modin.org.\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Modin time: 4.687446435913444\n"
+     ]
+    },
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "UserWarning: sort_values defaulting to pandas implementation.\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "pandas time: 1.4840463008731604\n"
+     ]
+    }
+   ],
+   "source": [
+    "t0 = timer()\n",
+    "modin_q4_result = q4(modin_df)\n",
+    "t1 = timer()\n",
+    "print(\"Modin time:\", t1 - t0)\n",
+    "t0 = timer()\n",
+    "pandas_q4_result = q4(pandas_df)\n",
+    "t1 = timer()\n",
+    "print(\"pandas time:\", t1 - t0)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 18,
    "metadata": {},
    "outputs": [
     {
@@ -758,39 +1601,39 @@
        "  </thead>\n",
        "  <tbody>\n",
        "    <tr>\n",
-       "      <th>2</th>\n",
+       "      <th>51</th>\n",
        "      <td>1</td>\n",
        "      <td>2013</td>\n",
-       "      <td>0</td>\n",
-       "      <td>1991</td>\n",
+       "      <td>1</td>\n",
+       "      <td>274599</td>\n",
        "    </tr>\n",
        "    <tr>\n",
-       "      <th>3</th>\n",
+       "      <th>50</th>\n",
        "      <td>1</td>\n",
        "      <td>2013</td>\n",
-       "      <td>1</td>\n",
-       "      <td>1270</td>\n",
+       "      <td>0</td>\n",
+       "      <td>210759</td>\n",
        "    </tr>\n",
        "    <tr>\n",
-       "      <th>4</th>\n",
+       "      <th>52</th>\n",
        "      <td>1</td>\n",
        "      <td>2013</td>\n",
        "      <td>2</td>\n",
-       "      <td>853</td>\n",
+       "      <td>152696</td>\n",
        "    </tr>\n",
        "    <tr>\n",
-       "      <th>80</th>\n",
-       "      <td>5</td>\n",
+       "      <th>53</th>\n",
+       "      <td>1</td>\n",
        "      <td>2013</td>\n",
-       "      <td>0</td>\n",
-       "      <td>551</td>\n",
+       "      <td>3</td>\n",
+       "      <td>96007</td>\n",
        "    </tr>\n",
        "    <tr>\n",
-       "      <th>81</th>\n",
-       "      <td>5</td>\n",
-       "      <td>2013</td>\n",
+       "      <th>54</th>\n",
        "      <td>1</td>\n",
-       "      <td>537</td>\n",
+       "      <td>2013</td>\n",
+       "      <td>4</td>\n",
+       "      <td>62700</td>\n",
        "    </tr>\n",
        "    <tr>\n",
        "      <th>...</th>\n",
@@ -800,133 +1643,216 @@
        "      <td>...</td>\n",
        "    </tr>\n",
        "    <tr>\n",
-       "      <th>77</th>\n",
-       "      <td>4</td>\n",
-       "      <td>2013</td>\n",
+       "      <th>632</th>\n",
+       "      <td>8</td>\n",
+       "      <td>2014</td>\n",
        "      <td>10</td>\n",
        "      <td>1</td>\n",
        "    </tr>\n",
        "    <tr>\n",
-       "      <th>78</th>\n",
-       "      <td>4</td>\n",
-       "      <td>2013</td>\n",
-       "      <td>11</td>\n",
+       "      <th>633</th>\n",
+       "      <td>8</td>\n",
+       "      <td>2014</td>\n",
+       "      <td>35</td>\n",
        "      <td>1</td>\n",
        "    </tr>\n",
        "    <tr>\n",
-       "      <th>79</th>\n",
-       "      <td>4</td>\n",
-       "      <td>2013</td>\n",
-       "      <td>14</td>\n",
+       "      <th>642</th>\n",
+       "      <td>9</td>\n",
+       "      <td>2014</td>\n",
+       "      <td>5</td>\n",
        "      <td>1</td>\n",
        "    </tr>\n",
        "    <tr>\n",
-       "      <th>102</th>\n",
-       "      <td>5</td>\n",
-       "      <td>2013</td>\n",
-       "      <td>28</td>\n",
+       "      <th>644</th>\n",
+       "      <td>9</td>\n",
+       "      <td>2014</td>\n",
+       "      <td>7</td>\n",
        "      <td>1</td>\n",
        "    </tr>\n",
        "    <tr>\n",
-       "      <th>115</th>\n",
-       "      <td>6</td>\n",
-       "      <td>2013</td>\n",
-       "      <td>14</td>\n",
+       "      <th>645</th>\n",
+       "      <td>9</td>\n",
+       "      <td>2014</td>\n",
+       "      <td>18</td>\n",
        "      <td>1</td>\n",
        "    </tr>\n",
        "  </tbody>\n",
        "</table>\n",
-       "<p>116 rows x 4 columns</p>\n",
+       "<p>646 rows x 4 columns</p>\n",
        "</div>"
       ],
       "text/plain": [
-       "     passenger_count  pickup_datetime  trip_distance     0\n",
-       "2                  1             2013              0  1991\n",
-       "3                  1             2013              1  1270\n",
-       "4                  1             2013              2   853\n",
-       "80                 5             2013              0   551\n",
-       "81                 5             2013              1   537\n",
-       "..               ...              ...            ...   ...\n",
-       "77                 4             2013             10     1\n",
-       "78                 4             2013             11     1\n",
-       "79                 4             2013             14     1\n",
-       "102                5             2013             28     1\n",
-       "115                6             2013             14     1\n",
-       "\n",
-       "[116 rows x 4 columns]"
+       "     passenger_count  pickup_datetime  trip_distance       0\n",
+       "51                 1             2013              1  274599\n",
+       "50                 1             2013              0  210759\n",
+       "52                 1             2013              2  152696\n",
+       "53                 1             2013              3   96007\n",
+       "54                 1             2013              4   62700\n",
+       "..               ...              ...            ...     ...\n",
+       "632                8             2014             10       1\n",
+       "633                8             2014             35       1\n",
+       "642                9             2014              5       1\n",
+       "644                9             2014              7       1\n",
+       "645                9             2014             18       1\n",
+       "\n",
+       "[646 rows x 4 columns]"
       ]
      },
-     "execution_count": 9,
+     "execution_count": 18,
      "metadata": {},
      "output_type": "execute_result"
     }
    ],
    "source": [
-    "q4(df)"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "%timeit q1(df)"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "%timeit q2(df)"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "%timeit q3(df)"
+    "modin_q4_result"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": null,
+   "execution_count": 19,
    "metadata": {},
-   "outputs": [],
+   "outputs": [
+    {
+     "data": {
+      "text/html": [
+       "<div>\n",
+       "<style scoped>\n",
+       "    .dataframe tbody tr th:only-of-type {\n",
+       "        vertical-align: middle;\n",
+       "    }\n",
+       "\n",
+       "    .dataframe tbody tr th {\n",
+       "        vertical-align: top;\n",
+       "    }\n",
+       "\n",
+       "    .dataframe thead th {\n",
+       "        text-align: right;\n",
+       "    }\n",
+       "</style>\n",
+       "<table border=\"1\" class=\"dataframe\">\n",
+       "  <thead>\n",
+       "    <tr style=\"text-align: right;\">\n",
+       "      <th></th>\n",
+       "      <th>passenger_count</th>\n",
+       "      <th>pickup_datetime</th>\n",
+       "      <th>trip_distance</th>\n",
+       "      <th>0</th>\n",
+       "    </tr>\n",
+       "  </thead>\n",
+       "  <tbody>\n",
+       "    <tr>\n",
+       "      <th>51</th>\n",
+       "      <td>1</td>\n",
+       "      <td>2013</td>\n",
+       "      <td>1</td>\n",
+       "      <td>274599</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>50</th>\n",
+       "      <td>1</td>\n",
+       "      <td>2013</td>\n",
+       "      <td>0</td>\n",
+       "      <td>210759</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>52</th>\n",
+       "      <td>1</td>\n",
+       "      <td>2013</td>\n",
+       "      <td>2</td>\n",
+       "      <td>152696</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>53</th>\n",
+       "      <td>1</td>\n",
+       "      <td>2013</td>\n",
+       "      <td>3</td>\n",
+       "      <td>96007</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>54</th>\n",
+       "      <td>1</td>\n",
+       "      <td>2013</td>\n",
+       "      <td>4</td>\n",
+       "      <td>62700</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>...</th>\n",
+       "      <td>...</td>\n",
+       "      <td>...</td>\n",
+       "      <td>...</td>\n",
+       "      <td>...</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>632</th>\n",
+       "      <td>8</td>\n",
+       "      <td>2014</td>\n",
+       "      <td>10</td>\n",
+       "      <td>1</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>633</th>\n",
+       "      <td>8</td>\n",
+       "      <td>2014</td>\n",
+       "      <td>35</td>\n",
+       "      <td>1</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>642</th>\n",
+       "      <td>9</td>\n",
+       "      <td>2014</td>\n",
+       "      <td>5</td>\n",
+       "      <td>1</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>644</th>\n",
+       "      <td>9</td>\n",
+       "      <td>2014</td>\n",
+       "      <td>7</td>\n",
+       "      <td>1</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>645</th>\n",
+       "      <td>9</td>\n",
+       "      <td>2014</td>\n",
+       "      <td>18</td>\n",
+       "      <td>1</td>\n",
+       "    </tr>\n",
+       "  </tbody>\n",
+       "</table>\n",
+       "<p>646 rows x 4 columns</p>\n",
+       "</div>"
+      ],
+      "text/plain": [
+       "     passenger_count  pickup_datetime  trip_distance       0\n",
+       "51                 1             2013              1  274599\n",
+       "50                 1             2013              0  210759\n",
+       "52                 1             2013              2  152696\n",
+       "53                 1             2013              3   96007\n",
+       "54                 1             2013              4   62700\n",
+       "..               ...              ...            ...     ...\n",
+       "632                8             2014             10       1\n",
+       "633                8             2014             35       1\n",
+       "642                9             2014              5       1\n",
+       "644                9             2014              7       1\n",
+       "645                9             2014             18       1\n",
+       "\n",
+       "[646 rows x 4 columns]"
+      ]
+     },
+     "execution_count": 19,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
    "source": [
-    "%timeit q4(df)"
+    "pandas_q4_result"
    ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": []
   }
  ],
  "metadata": {
-  "license": "http://www.apache.org/licenses/LICENSE-2.0",
-  "license_notice": [
-   "Licensed to Modin Development Team under one or more contributor license agreements.",
-   "See the NOTICE file distributed with this work for additional information regarding",
-   "copyright ownership.  The Modin Development Team licenses this file to you under the",
-   "Apache License, Version 2.0 (the \"License\"); you may not use this file except in",
-   "compliance with the License.  You may obtain a copy of the License at",
-   "",
-   "    http://www.apache.org/licenses/LICENSE-2.0",
-   "",
-   "Unless required by applicable law or agreed to in writing, software distributed under",
-   "the License is distributed on an \"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF",
-   "ANY KIND, either express or implied. See the License for the specific language",
-   "governing permissions and limitations under the License."
-  ],
   "kernelspec": {
-   "display_name": "Python 3",
+   "display_name": "Python 3 (ipykernel)",
    "language": "python",
    "name": "python3"
   },
@@ -940,8 +1866,23 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.7.8"
-  }
+   "version": "3.8.12"
+  },
+  "license": "http://www.apache.org/licenses/LICENSE-2.0",
+  "license_notice": [
+   "Licensed to Modin Development Team under one or more contributor license agreements.",
+   "See the NOTICE file distributed with this work for additional information regarding",
+   "copyright ownership.  The Modin Development Team licenses this file to you under the",
+   "Apache License, Version 2.0 (the \"License\"); you may not use this file except in",
+   "compliance with the License.  You may obtain a copy of the License at",
+   "",
+   "    http://www.apache.org/licenses/LICENSE-2.0",
+   "",
+   "Unless required by applicable law or agreed to in writing, software distributed under",
+   "the License is distributed on an \"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF",
+   "ANY KIND, either express or implied. See the License for the specific language",
+   "governing permissions and limitations under the License."
+  ]
  },
  "nbformat": 4,
  "nbformat_minor": 4


Code review comments:
diff_hunk: @@ -0,0 +1,58 @@
+Optimization Guide
+==================
+
+Here you can find information on Modin optimizations both for general query pipeline and for specific operations.
+
+Generic settings
+""""""""""""""""
+
+If you are working with a wide DataFrame and performing an operation that requires ``map`` Core Dataframe Algebra operator
+to be applied cell-wise, you may want to change the number of partitions DataFrame should partition into to avoid
+excessive amount of remote partitions, which is much greater than the number of CPUs.
+
+.. code-block:: python
+
+  from multiprocessing import cpu_count
+  import modin.config as cfg
+  import modin.pandas as pd
+  import numpy as np
+
+  data = np.random.randint(0, 100, size=(2**15, 2**15))
+
+  cfg.NPartitions.put(cpu_count())
+  df = pd.DataFrame(data)
+  %timeit df.abs()
+  1.9 s  49.8 ms per loop (mean  std. dev. of 7 runs, 1 loop each)
+
+  cfg.NPartitions.put(int(cpu_count() / 2))
+  df = pd.DataFrame(data)
+  %timeit df.abs()
+  506 ms  17.5 ms per loop (mean  std. dev. of 7 runs, 1 loop each)
+
+Operation-specific settings
+"""""""""""""""""""""""""""
+
+merge
+-----
+
+``merge`` operation in Modin uses broadcast join algorithm: combining a right Modin DataFrame into pandas DataFrame and
+broadcasting it to row partitions of the left Modin DataFrame. In order to minimize interprocess communication cost when doing
+inner join you may want to swap left and right DataFrames.
path: docs/usage_guide/optimization_guide/index.rst
text: @user1:
It's actually strange, that the user should worry about this, I think we have to treat it as an issue and fix it on the Modin side.

@author:
Swapping left and right DataFrames may lead different columns order. Not sure we want to optimize this case and then align columns order that might be expensive.

diff_hunk: @@ -0,0 +1,58 @@
+Optimization Guide
+==================
+
+Here you can find information on Modin optimizations both for general query pipeline and for specific operations.
+
+Generic settings
+""""""""""""""""
+
+If you are working with a wide DataFrame and performing an operation that requires ``map`` Core Dataframe Algebra operator
+to be applied cell-wise, you may want to change the number of partitions DataFrame should partition into to avoid
+excessive amount of remote partitions, which is much greater than the number of CPUs.
+
+.. code-block:: python
+
+  from multiprocessing import cpu_count
+  import modin.config as cfg
+  import modin.pandas as pd
+  import numpy as np
+
+  data = np.random.randint(0, 100, size=(2**15, 2**15))
+
+  cfg.NPartitions.put(cpu_count())
+  df = pd.DataFrame(data)
+  %timeit df.abs()
+  1.9 s  49.8 ms per loop (mean  std. dev. of 7 runs, 1 loop each)
+
+  cfg.NPartitions.put(int(cpu_count() / 2))
+  df = pd.DataFrame(data)
+  %timeit df.abs()
+  506 ms  17.5 ms per loop (mean  std. dev. of 7 runs, 1 loop each)
+
+Operation-specific settings
+"""""""""""""""""""""""""""
+
+merge
+-----
+
+``merge`` operation in Modin uses broadcast join algorithm: combining a right Modin DataFrame into pandas DataFrame and
+broadcasting it to row partitions of the left Modin DataFrame. In order to minimize interprocess communication cost when doing
+inner join you may want to swap left and right DataFrames.
path: docs/usage_guide/optimization_guide/index.rst
text: @user1:
```suggestion
an inner join you may want to swap left and right DataFrames.
```

diff_hunk: @@ -0,0 +1,21 @@
+Modin Usage Examples
+====================
+
+This section shows Modin usage examples in different scenarios like Modin on a local/remote cluster,
+Modin in the cloud, the use of Modin spreadsheet and whatnot. First of all you may want to look at the basic
path: docs/usage_guide/examples/index.rst
text: @user1:
Might be useful to add links to the spreadsheet API doc here, and if we have other links in the doc for installation guides.

@user2:
The links for the spreadsheet API and other topics are not presented here is because they're located in different directories for some reason. Spreadsheet API and all the other stuff is stored at the [`/docs/advanced_usage`](https://github.com/YarShev/modin/tree/dev/yigoshev-issue3863/docs/advanced_usage) when the getting started guide is located at the [`/docs/usage_guide`](https://github.com/YarShev/modin/tree/dev/yigoshev-issue3863/docs/usage_guide). For me it seems like duplication and I would combine these two sections. 

@author was this section split intentional or we can combine them to reduce duplication with no losses?

@author:
I suggest moving advanced_usage in usage_guide.

@user2:
combined the sections