Repository: modin-project/modin
Pull request: TEST-#4201: Add read_parquet, explode, tail, and various arithmetic t…
Signed-off-by: Naren Krishna <naren@ponder.io>

<!--
Thank you for your contribution!
Please review the contributing docs: https://modin.readthedocs.io/en/latest/development/contributing.html
if you have questions about contributing.
-->

## What do these changes do?

Linked to #4201 

- [x] commit message follows format outlined [here](https://modin.readthedocs.io/en/latest/development/contributing.html#commit-message-formatting)
- [x] passes `flake8 modin/ asv_bench/benchmarks scripts/doc_checker.py`
- [x] passes `black --check modin/ asv_bench/benchmarks scripts/doc_checker.py`
- [x] signed commit with `git commit -s` <!-- you can amend your commit with a signature via `git commit -amend -s` -->
- [x] Resolves #4201 ? <!-- issue must be created for each patch -->
- [x] tests added and passing
- [x] module layout described at `docs/development/architecture.rst` is up-to-date <!-- if you have added, renamed or removed files or directories please update the documentation accordingly -->
- [x] added (Issue Number: PR title (PR Number)) and github username to release notes for next major release <!-- e.g. DOCS-#4077: Add release notes template to docs folder (#4078) -->


Problem statement:
TEST: Add more functions to asv_bench
**Is your feature request related to a problem? Please describe.**

We want to ensure that the most commonly used Pandas functions are a part of `asv_bench` and that they are running benchmarks on different dataset sizes across different numbers of CPUs comparing Modin vs. Pandas.

A clear and concise description of what the problem is. What kind of performance improvements would you like to see with this new API?

Hints from issue discussion:

Commit message: address comments

Signed-off-by: Naren Krishna <naren@ponder.io>

Patch to review:
diff --git a/asv_bench/benchmarks/benchmarks.py b/asv_bench/benchmarks/benchmarks.py
index e991575b53c..445c9d8f77d 100644
--- a/asv_bench/benchmarks/benchmarks.py
+++ b/asv_bench/benchmarks/benchmarks.py
@@ -302,6 +302,9 @@ def setup(self, shape, axis):
     def time_sum(self, shape, axis):
         execute(self.df.sum(axis=axis))
 
+    def time_count(self, shape, axis):
+        execute(self.df.count(axis=axis))
+
     def time_median(self, shape, axis):
         execute(self.df.median(axis=axis))
 
@@ -314,6 +317,30 @@ def time_apply(self, shape, axis):
     def time_mean(self, shape, axis):
         execute(self.df.mean(axis=axis))
 
+    def time_mode(self, shape, axis):
+        execute(self.df.mode(axis=axis))
+
+    def time_add(self, shape, axis):
+        execute(self.df.add(2, axis=axis))
+
+    def time_mul(self, shape, axis):
+        execute(self.df.mul(2, axis=axis))
+
+    def time_mod(self, shape, axis):
+        execute(self.df.mod(2, axis=axis))
+
+    def time_abs(self, shape, axis):
+        execute(self.df.abs())
+
+    def time_aggregate(self, shape, axis):
+        execute(self.df.aggregate(lambda df: df.sum(), axis=axis))
+
+    def time_is_in(self, shape, axis):
+        execute(self.df.isin([0, 2]))
+
+    def time_transpose(self, shape, axis):
+        execute(self.df.transpose())
+
 
 class TimeSortValues:
     param_names = ["shape", "columns_number", "ascending_list"]
@@ -376,6 +403,40 @@ def time_head(self, shape, head_count):
         execute(self.df.head(self.head_count))
 
 
+class TimeTail:
+    param_names = ["shape", "tail_count"]
+    params = [
+        get_benchmark_shapes("TimeTail"),
+        [5, 0.8],
+    ]
+
+    def setup(self, shape, tail_count):
+        self.df = generate_dataframe(ASV_USE_IMPL, "int", *shape, RAND_LOW, RAND_HIGH)
+        self.tail_count = (
+            int(tail_count * len(self.df.index))
+            if isinstance(tail_count, float)
+            else tail_count
+        )
+
+    def time_tail(self, shape, tail_count):
+        execute(self.df.tail(self.tail_count))
+
+
+class TimeExplode:
+    param_names = ["shape"]
+    params = [
+        get_benchmark_shapes("TimeExplode"),
+    ]
+
+    def setup(self, shape):
+        self.df = generate_dataframe(
+            ASV_USE_IMPL, "int", *shape, RAND_LOW, RAND_HIGH, gen_unique_key=True
+        )
+
+    def time_explode(self, shape):
+        execute(self.df.explode("col1"))
+
+
 class TimeFillnaSeries:
     param_names = ["value_type", "shape", "limit"]
     params = [
diff --git a/asv_bench/benchmarks/io/parquet.py b/asv_bench/benchmarks/io/parquet.py
new file mode 100644
index 00000000000..98e7aa46ef2
--- /dev/null
+++ b/asv_bench/benchmarks/io/parquet.py
@@ -0,0 +1,58 @@
+# Licensed to Modin Development Team under one or more contributor license agreements.
+# See the NOTICE file distributed with this work for additional information regarding
+# copyright ownership.  The Modin Development Team licenses this file to you under the
+# Apache License, Version 2.0 (the "License"); you may not use this file except in
+# compliance with the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software distributed under
+# the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF
+# ANY KIND, either express or implied. See the License for the specific language
+# governing permissions and limitations under the License.
+
+import modin.pandas as pd
+
+from ..utils import (
+    ASV_USE_IMPL,
+    IMPL,
+    execute,
+    get_shape_id,
+    prepare_io_data_parquet,
+    get_benchmark_shapes,
+)
+
+
+class BaseReadParquet:
+    # test data file should be created only once
+    def setup_cache(self, test_filename="io_test_file"):
+        test_filenames = prepare_io_data_parquet(
+            test_filename, self.data_type, get_benchmark_shapes(self.__class__.__name__)
+        )
+        return test_filenames
+
+    def setup(self, test_filenames, shape, *args, **kwargs):
+        # ray init
+        if ASV_USE_IMPL == "modin":
+            pd.DataFrame([])
+        self.shape_id = get_shape_id(shape)
+
+
+class TimeReadParquet(BaseReadParquet):
+    shapes = get_benchmark_shapes("TimeReadParquet")
+    data_type = "str_int"
+
+    param_names = ["shape"]
+    params = [
+        shapes,
+    ]
+
+    def setup(self, test_filenames, shape):
+        super().setup(test_filenames, shape)
+
+    def time_read_parquet(self, test_filenames, shape):
+        execute(
+            IMPL[ASV_USE_IMPL].read_parquet(
+                test_filenames[self.shape_id],
+            )
+        )
diff --git a/asv_bench/benchmarks/utils/__init__.py b/asv_bench/benchmarks/utils/__init__.py
index 1a2b5330575..4a2e1be7c39 100644
--- a/asv_bench/benchmarks/utils/__init__.py
+++ b/asv_bench/benchmarks/utils/__init__.py
@@ -26,6 +26,7 @@
     gen_nan_data,
     generate_dataframe,
     prepare_io_data,
+    prepare_io_data_parquet,
     random_string,
     random_columns,
     random_booleans,
@@ -47,6 +48,7 @@
     "gen_nan_data",
     "generate_dataframe",
     "prepare_io_data",
+    "prepare_io_data_parquet",
     "random_string",
     "random_columns",
     "random_booleans",
diff --git a/asv_bench/benchmarks/utils/common.py b/asv_bench/benchmarks/utils/common.py
index b745552caa2..44f3758f209 100644
--- a/asv_bench/benchmarks/utils/common.py
+++ b/asv_bench/benchmarks/utils/common.py
@@ -563,8 +563,37 @@ def prepare_io_data(test_filename: str, data_type: str, shapes: list):
     test_filenames = {}
     for shape in shapes:
         shape_id = get_shape_id(shape)
-        test_filenames[shape_id] = f"{test_filename}_{shape_id}.csv"
+        test_filenames[shape_id] = f"{test_filename}_{shape_id}_{data_type}.csv"
         df = generate_dataframe("pandas", data_type, *shape, RAND_LOW, RAND_HIGH)
         df.to_csv(test_filenames[shape_id], index=False)
 
     return test_filenames
+
+
+def prepare_io_data_parquet(test_filename: str, data_type: str, shapes: list):
+    """
+    Prepare data for IO tests with caching.
+
+    Parameters
+    ----------
+    test_filename : str
+        Unique file identifier that is used to distinguish data
+        for different tests.
+    data_type : "str_int"
+        Type of data generation.
+    shapes : list
+        Data shapes to prepare.
+
+    Returns
+    -------
+    test_filenames : dict
+        Dictionary that maps dataset shape to the file on disk.
+    """
+    test_filenames = {}
+    for shape in shapes:
+        shape_id = get_shape_id(shape)
+        test_filenames[shape_id] = f"{test_filename}_{shape_id}_{data_type}.parquet"
+        df = generate_dataframe("pandas", data_type, *shape, RAND_LOW, RAND_HIGH)
+        df.to_parquet(test_filenames[shape_id], index=False)
+
+    return test_filenames
diff --git a/asv_bench/benchmarks/utils/data_shapes.py b/asv_bench/benchmarks/utils/data_shapes.py
index ae8918646c9..7ca03ccc5fb 100644
--- a/asv_bench/benchmarks/utils/data_shapes.py
+++ b/asv_bench/benchmarks/utils/data_shapes.py
@@ -78,6 +78,8 @@
             "TimeSortValues",
             "TimeDrop",
             "TimeHead",
+            "TimeTail",
+            "TimeExplode",
             "TimeFillna",
             "TimeFillnaDataFrame",
             "TimeValueCountsFrame",
@@ -92,6 +94,7 @@
             "TimeReadCsvSkiprows",
             "TimeReadCsvTrueFalseValues",
             "TimeReadCsvNamesDtype",
+            "TimeReadParquet",
             # Scalability benchmarks
             "TimeFromPandas",
             "TimeToPandas",
diff --git a/docs/release_notes/release_notes-0.14.0.rst b/docs/release_notes/release_notes-0.14.0.rst
index 394fe1543e1..b9e613ecd3a 100644
--- a/docs/release_notes/release_notes-0.14.0.rst
+++ b/docs/release_notes/release_notes-0.14.0.rst
@@ -41,6 +41,7 @@ Key Features and Updates
   * TEST-#3628: Report coverage data for `test-internals` CI job (#4198)
   * TEST-#3938: Test tutorial notebooks in CI (#4145)
   * TEST-#4153: Fix condition of running lint-commit and set of CI triggers (#4156)
+  * TEST-#4201: Add read_parquet, explode, tail, and various arithmetic functions to asv_bench (#4203)
 * Documentation improvements
   * DOCS-#4077: Add release notes template to docs folder (#4078)
   * DOCS-#4082: Add pdf/epub/htmlzip formats for doc builds (#4083)


Code review comments:
diff_hunk: @@ -0,0 +1,58 @@
+# Licensed to Modin Development Team under one or more contributor license agreements.
+# See the NOTICE file distributed with this work for additional information regarding
+# copyright ownership.  The Modin Development Team licenses this file to you under the
+# Apache License, Version 2.0 (the "License"); you may not use this file except in
+# compliance with the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software distributed under
+# the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF
+# ANY KIND, either express or implied. See the License for the specific language
+# governing permissions and limitations under the License.
+
+import modin.pandas as pd
+
+from ..utils import (
+    ASV_USE_IMPL,
+    IMPL,
+    execute,
+    get_shape_id,
+    prepare_io_data_parquet,
+    get_benchmark_shapes,
+)
+
+
+class BaseReadParquet:
path: asv_bench/benchmarks/io/parquet.py
text: @user1:
I don't think we should add a base class until we have multiple subclasses.

@author:
We do this with read_csv, so wanted some consistency in case in future we add other data types for parquet.

@user1:
I generally prefer in cases like this to assume [YAGNI](https://en.wikipedia.org/wiki/You_aren%27t_gonna_need_it) and write the simpler code. However, I won't insist.

@user2:
I agree with @user1, let's keep it simple.