Repository: spiraldb/vortex
Pull request #2823: Continuous profiling of benchmarks
Author: login: AdamGS
Created: 2025-03-27T22:17:26Z | Merged: 2025-03-28T14:35:51Z

We should probably follow this up with splitting benchmarks into per-format iteration (maybe every query?), changing the labels every time so we can really slice and dice the data. That'll probably require changes to the JSON processing we do, so I'm leaving it to a future PR.


Diff:
diff --git a/.github/workflows/bench-pr.yml b/.github/workflows/bench-pr.yml
index 817a2224bb0..9eb350e27cf 100644
--- a/.github/workflows/bench-pr.yml
+++ b/.github/workflows/bench-pr.yml
@@ -52,12 +52,24 @@ jobs:
         with:
           version: v1.2.1
 
-      - name: Run ${{ matrix.benchmark.name }} benchmark
+      - name: Build binary
         shell: bash
         env:
-          RUSTFLAGS: '-C target-cpu=native'
+          RUSTFLAGS: '-C target-cpu=native -C force-frame-pointers=yes'
+        run: |
+          cargo build --bin ${{ matrix.benchmark.id }} --package bench-vortex --profile release_debug
+
+      - name: Setup Polar Signals
+        uses: polarsignals/gh-actions-ps-profiling@v0.2.0
+        with:
+          polarsignals_cloud_token: ${{ secrets.POLAR_SIGNALS_API_KEY }}
+          labels: 'branch=${{ github.ref_name }};gh_run_id=${{ github.run_id }};benchmark=${{ matrix.benchmark.id }}'
+          profiling_frequency: 200
+
+      - name: Run ${{ matrix.benchmark.name }} benchmark
+        shell: bash
         run: |
-          cargo run --bin ${{ matrix.benchmark.id }} --package bench-vortex --release -- -d gh-json | tee ${{ matrix.benchmark.id }}.json
+          target/release_debug/${{ matrix.benchmark.id }} -d gh-json | tee ${{ matrix.benchmark.id }}.json
 
       - name: Setup AWS CLI
         uses: aws-actions/configure-aws-credentials@v4
diff --git a/.github/workflows/sql-benchmarks.yml b/.github/workflows/sql-benchmarks.yml
index 778f1971c20..22f356a2bc4 100644
--- a/.github/workflows/sql-benchmarks.yml
+++ b/.github/workflows/sql-benchmarks.yml
@@ -37,7 +37,7 @@ jobs:
       - family=c7i.8xlarge
       - image=ubuntu24-full-x64
       - disk=large
-      - tag=${{ matrix.include.id }}
+      - tag=${{ matrix.id }}
 
     steps:
       - uses: runs-on/action@v1
@@ -58,22 +58,46 @@ jobs:
           role-to-assume: arn:aws:iam::375504701696:role/GitHubBenchmarkRole
           aws-region: us-east-1
 
+      - name: Build binary
+        shell: bash
+        env:
+          RUSTFLAGS: '-C target-cpu=native -C force-frame-pointers=yes'
+        run: |
+          cargo build --bin ${{ matrix.binary_name }} --package bench-vortex --profile release_debug
+
+      - name: Generate data
+        shell: bash
+        run:
+          # Generate data, running each query once to make sure they don't panic.
+          target/release_debug/${{ matrix.binary_name }} --formats parquet,vortex -i1
+
+      - name: Upload data
+        if: matrix.remote_storage != null
+        shell: bash
+        env:
+          AWS_REGION: 'eu-west-1'
+        run: |
+          aws s3 rm --recursive ${{ matrix.remote_storage }}
+          aws s3 cp --recursive ${{matrix.local_dir}} ${{ matrix.remote_storage }}
+
+      - name: Setup Polar Signals
+        uses: polarsignals/gh-actions-ps-profiling@v0.2.0
+        with:
+          polarsignals_cloud_token: ${{ secrets.POLAR_SIGNALS_API_KEY }}
+          labels: 'branch=${{ github.ref_name }};gh_run_id=${{ github.run_id }};benchmark=${{ matrix.id }}'
+          profiling_frequency: 200
+
       - name: Run ${{ matrix.name }} benchmark
         if: matrix.remote_storage == null
         shell: bash
         env:
-          RUSTFLAGS: '-C target-cpu=native'
           OTEL_SERVICE_NAME: 'vortex-bench'
           OTEL_EXPORTER_OTLP_PROTOCOL: 'http/protobuf'
           OTEL_EXPORTER_OTLP_ENDPOINT: '${{ secrets.OTEL_EXPORTER_OTLP_ENDPOINT }}'
           OTEL_EXPORTER_OTLP_HEADERS: '${{ secrets.OTEL_EXPORTER_OTLP_HEADERS }}'
           OTEL_RESOURCE_ATTRIBUTES: 'bench-name=${{ matrix.id }}'
         run: |
-          cargo run \
-              --bin ${{ matrix.binary_name }} \
-              --package bench-vortex \
-              --release \
-              -- \
+            target/release_debug/${{ matrix.binary_name }} \
               -d gh-json \
               --export-spans \
             | tee results.json
@@ -82,7 +106,6 @@ jobs:
         if: matrix.remote_storage != null
         shell: bash
         env:
-          RUSTFLAGS: '-C target-cpu=native'
           AWS_REGION: 'eu-west-1'
           OTEL_SERVICE_NAME: 'vortex-bench'
           OTEL_EXPORTER_OTLP_PROTOCOL: 'http/protobuf'
@@ -90,16 +113,7 @@ jobs:
           OTEL_EXPORTER_OTLP_HEADERS: '${{ secrets.OTEL_EXPORTER_OTLP_HEADERS }}'
           OTEL_RESOURCE_ATTRIBUTES: 'bench-name=${{ matrix.id }}'
         run: |
-          # Generate data, running each query once to make sure they don't panic.
-          cargo run --release --bin ${{ matrix.binary_name }} --package bench-vortex -- --formats parquet,vortex -i1
-          aws s3 rm --recursive ${{ matrix.remote_storage }}
-          aws s3 cp --recursive ${{matrix.local_dir}} ${{ matrix.remote_storage }}
-
-          cargo run \
-              --package bench-vortex \
-              --bin ${{ matrix.binary_name }} \
-              --release \
-              -- \
+          target/release_debug/${{ matrix.binary_name }} \
               --use-remote-data-dir ${{ matrix.remote_storage }} \
               --formats parquet,vortex \
               --export-spans \


Reviews:
user: login: a10y
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-28T00:32:35Z
user: login: AdamGS
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-28T10:14:46Z
user: login: robert3005
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-03-28T14:30:23Z

Review comments:
diff_hunk: @@ -52,12 +52,28 @@ jobs:
         with:
           version: v1.2.1
 
-      - name: Run ${{ matrix.benchmark.name }} benchmark
+      - name: Build binary
         shell: bash
         env:
           RUSTFLAGS: '-C target-cpu=native'
         run: |
-          cargo run --bin ${{ matrix.benchmark.id }} --package bench-vortex --release -- -d gh-json | tee ${{ matrix.benchmark.id }}.json
+          cargo build --bin ${{ matrix.benchmark.id }} --package bench-vortex --profile release_debug
+
+      - name: Extract branch name
+        shell: bash
+        run: echo "branch=$(echo ${GITHUB_REF#refs/heads/})" >>$GITHUB_OUTPUT
+        id: extract_branch
+
+      - name: Setup Polar Signals
path: .github/workflows/bench-pr.yml
user: login: a10y
body: 👀❤️
created_at: 2025-03-28T00:32:35Z
author_association: CONTRIBUTOR
diff_hunk: @@ -52,12 +52,28 @@ jobs:
         with:
           version: v1.2.1
 
-      - name: Run ${{ matrix.benchmark.name }} benchmark
+      - name: Build binary
path: .github/workflows/bench-pr.yml
user: login: AdamGS
body: We build the binary before we start running the profiling agent, because we don't care about these traces
created_at: 2025-03-28T10:14:46Z
author_association: CONTRIBUTOR

PR comments:
user: login: github-actions[bot]
created_at: 2025-03-27T22:25:53Z
body: # Benchmarks: TPC-H on NVME
<details>
<summary>Table of Results</summary>

| name                            |   PR fa0fa2ae |   base b60fd71e |   ratio (PR/base) | unit   |
|:--------------------------------|--------------:|----------------:|------------------:|:-------|
| tpch_q01/arrow                  |      44815504 |     4.38434e+07 |          1.02217  | ns     |
| tpch_q02/arrow                  |      45455545 |     4.6041e+07  |          0.987285 | ns     |
| tpch_q03/arrow                  |      29165035 |     2.93138e+07 |          0.994924 | ns     |
| tpch_q04/arrow                  |      21348315 |     2.1404e+07  |          0.9974   | ns     |
| tpch_q05/arrow                  |      44751877 |     4.47048e+07 |          1.00105  | ns     |
| tpch_q06/arrow                  |       8095150 |     9.40137e+06 |          0.861061 | ns     |
| tpch_q07/arrow                  |      74727091 |     7.19272e+07 |          1.03893  | ns     |
| tpch_q08/arrow                  |      54026194 |     5.25178e+07 |          1.02872  | ns     |
| tpch_q09/arrow                  |      68721408 |     6.86173e+07 |          1.00152  | ns     |
| tpch_q10/arrow                  |      43305533 |     4.35453e+07 |          0.994494 | ns     |
| tpch_q11/arrow                  |      21749119 |     2.13924e+07 |          1.01668  | ns     |
| tpch_q12/arrow                  |      22496976 |     2.91615e+07 |          0.771461 | ns     |
| tpch_q13/arrow                  |      14751091 |     1.52904e+07 |          0.964729 | ns     |
| tpch_q14/arrow                  |       9286906 |     1.30755e+07 |          0.710254 | ns     |
| tpch_q15/arrow                  |      21831987 |     2.55802e+07 |          0.853471 | ns     |
| tpch_q16/arrow                  |      21085309 |     2.05542e+07 |          1.02584  | ns     |
| tpch_q17/arrow                  |      61190478 |     6.20575e+07 |          0.986028 | ns     |
| tpch_q18/arrow                  |     110814566 |     9.55891e+07 |          1.15928  | ns     |
| tpch_q19/arrow                  |      19602178 |     2.85507e+07 |          0.686575 | ns     |
| tpch_q20/arrow                  |      28788592 |     3.27374e+07 |          0.87938  | ns     |
| tpch_q21/arrow                  |     113990046 |     1.10806e+08 |          1.02873  | ns     |
| tpch_q22/arrow                  |      13604863 |     1.46495e+07 |          0.928694 | ns     |
| tpch_q01/parquet                |     121791650 |     1.14427e+08 |          1.06436  | ns     |
| tpch_q02/parquet                |     114571867 |     1.0973e+08  |          1.04412  | ns     |
| tpch_q03/parquet                |     107515553 |     1.04543e+08 |          1.02844  | ns     |
| tpch_q04/parquet                |      61249898 |     6.06845e+07 |          1.00932  | ns     |
| tpch_q05/parquet                |     115456857 |     1.18272e+08 |          0.976195 | ns     |
| tpch_q06/parquet                |      27704449 |     3.56059e+07 |          0.778086 | ns     |
| tpch_q07/parquet                |     136532326 |     1.27343e+08 |          1.07216  | ns     |
| tpch_q08/parquet                |     166706221 |     1.60012e+08 |          1.04183  | ns     |
| tpch_q09/parquet                |     223421572 |     2.068e+08   |          1.08038  | ns     |
| tpch_q10/parquet                |     136591285 |     1.32624e+08 |          1.02991  | ns     |
| tpch_q11/parquet                |      53718808 |     4.98133e+07 |          1.0784   | ns     |
| tpch_q12/parquet                |     109246067 |     9.12879e+07 |          1.19672  | ns     |
| tpch_q13/parquet                |     158506825 |     1.49863e+08 |          1.05768  | ns     |
| tpch_q14/parquet                |      46189141 |     4.59029e+07 |          1.00624  | ns     |
| tpch_q15/parquet                |      74808762 |     6.59187e+07 |          1.13486  | ns     |
| tpch_q16/parquet                |      50627071 |     5.00026e+07 |          1.01249  | ns     |
| tpch_q17/parquet                |     135768594 |     1.30788e+08 |          1.03808  | ns     |
| tpch_q18/parquet                |     197351569 |     1.9661e+08  |          1.00377  | ns     |
| tpch_q19/parquet                |      77528913 |     7.37826e+07 |          1.05078  | ns     |
| tpch_q20/parquet                |      98138105 |     9.59849e+07 |          1.02243  | ns     |
| tpch_q21/parquet                |     182147648 |     1.76835e+08 |          1.03004  | ns     |
| tpch_q22/parquet                |      51324634 |     5.02654e+07 |          1.02107  | ns     |
| tpch_q01/vortex-file-compressed |      37594367 |     3.49896e+07 |          1.07444  | ns     |
| tpch_q02/vortex-file-compressed |      48640186 |     5.21059e+07 |          0.933488 | ns     |
| tpch_q03/vortex-file-compressed |      26090359 |     3.01625e+07 |          0.864993 | ns     |
| tpch_q04/vortex-file-compressed |      17043799 |     1.78729e+07 |          0.953614 | ns     |
| tpch_q05/vortex-file-compressed |      41302611 |     4.51428e+07 |          0.914933 | ns     |
| tpch_q06/vortex-file-compressed |      10752753 |     1.00955e+07 |          1.0651   | ns     |
| tpch_q07/vortex-file-compressed |      64115970 |     6.48693e+07 |          0.988387 | ns     |
| tpch_q08/vortex-file-compressed |      50488729 |     5.01639e+07 |          1.00648  | ns     |
| tpch_q09/vortex-file-compressed |      67731696 |     6.52632e+07 |          1.03782  | ns     |
| tpch_q10/vortex-file-compressed |      49081272 |     5.03698e+07 |          0.974419 | ns     |
| tpch_q11/vortex-file-compressed |      23822315 |     2.54808e+07 |          0.934914 | ns     |
| tpch_q12/vortex-file-compressed |      23355326 |     2.39726e+07 |          0.97425  | ns     |
| tpch_q13/vortex-file-compressed |      23057069 |     2.46314e+07 |          0.936083 | ns     |
| tpch_q14/vortex-file-compressed |      12628443 |     1.31787e+07 |          0.958245 | ns     |
| tpch_q15/vortex-file-compressed |      22817330 |     2.56725e+07 |          0.888783 | ns     |
| tpch_q16/vortex-file-compressed |      25942660 |     2.65795e+07 |          0.976041 | ns     |
| tpch_q17/vortex-file-compressed |      57204116 |     5.72502e+07 |          0.999195 | ns     |
| tpch_q18/vortex-file-compressed |      89419969 |     8.53604e+07 |          1.04756  | ns     |
| tpch_q19/vortex-file-compressed |      28296512 |     3.00387e+07 |          0.942002 | ns     |
| tpch_q20/vortex-file-compressed |      32254523 |     3.47961e+07 |          0.926957 | ns     |
| tpch_q21/vortex-file-compressed |      83816648 |     8.48642e+07 |          0.987656 | ns     |
| tpch_q22/vortex-file-compressed |      27913713 |     2.77533e+07 |          1.00578  | ns     |
</details>

<!-- thollander/actions-comment-pull-request "bench-pr-comment-tpch-nvme" -->
author_association: CONTRIBUTOR
user: login: github-actions[bot]
created_at: 2025-03-27T22:27:54Z
body: # Benchmarks: TPC-H on S3
<details>
<summary>Table of Results</summary>

| name                            |   PR fa0fa2ae |   base b60fd71e |   ratio (PR/base) | unit   |
|:--------------------------------|--------------:|----------------:|------------------:|:-------|
| tpch_q01/parquet                |     258647334 |     2.61572e+08 |          0.988821 | ns     |
| tpch_q02/parquet                |     681026924 |     6.77955e+08 |          1.00453  | ns     |
| tpch_q03/parquet                |     407287382 |     4.27422e+08 |          0.952894 | ns     |
| tpch_q04/parquet                |     241630595 |     2.34429e+08 |          1.03072  | ns     |
| tpch_q05/parquet                |     571837294 |     5.76629e+08 |          0.991691 | ns     |
| tpch_q06/parquet                |     177878507 |     1.74373e+08 |          1.02011  | ns     |
| tpch_q07/parquet                |     604419288 |     6.68012e+08 |          0.904803 | ns     |
| tpch_q08/parquet                |     786374430 |     7.82191e+08 |          1.00535  | ns     |
| tpch_q09/parquet                |     720988339 |     7.13776e+08 |          1.0101   | ns     |
| tpch_q10/parquet                |     528070337 |     5.38521e+08 |          0.980593 | ns     |
| tpch_q11/parquet                |     270878053 |     2.71442e+08 |          0.997922 | ns     |
| tpch_q12/parquet                |     272286890 |     2.72076e+08 |          1.00077  | ns     |
| tpch_q13/parquet                |     380483010 |     3.78496e+08 |          1.00525  | ns     |
| tpch_q14/parquet                |     257554733 |     2.55376e+08 |          1.00853  | ns     |
| tpch_q15/parquet                |     458497652 |     4.56499e+08 |          1.00438  | ns     |
| tpch_q16/parquet                |     253918253 |     2.49934e+08 |          1.01594  | ns     |
| tpch_q17/parquet                |     400082948 |     3.93639e+08 |          1.01637  | ns     |
| tpch_q18/parquet                |     538554255 |     5.39779e+08 |          0.997731 | ns     |
| tpch_q19/parquet                |     278925257 |     2.72451e+08 |          1.02376  | ns     |
| tpch_q20/parquet                |     544368277 |     5.12626e+08 |          1.06192  | ns     |
| tpch_q21/parquet                |     643569997 |     6.26473e+08 |          1.02729  | ns     |
| tpch_q22/parquet                |     262876408 |     2.56698e+08 |          1.02407  | ns     |
| tpch_q01/vortex-file-compressed |     122460217 |     1.21451e+08 |          1.00831  | ns     |
| tpch_q02/vortex-file-compressed |     324441379 |     3.18094e+08 |          1.01995  | ns     |
| tpch_q03/vortex-file-compressed |     261832439 |     2.65291e+08 |          0.986962 | ns     |
| tpch_q04/vortex-file-compressed |     197867711 |     1.95953e+08 |          1.00977  | ns     |
| tpch_q05/vortex-file-compressed |     290054945 |     2.79352e+08 |          1.03831  | ns     |
| tpch_q06/vortex-file-compressed |      87537072 |     8.82209e+07 |          0.992249 | ns     |
| tpch_q07/vortex-file-compressed |     297564588 |     2.90323e+08 |          1.02494  | ns     |
| tpch_q08/vortex-file-compressed |     388175017 |     3.87737e+08 |          1.00113  | ns     |
| tpch_q09/vortex-file-compressed |     408062333 |     4.07129e+08 |          1.00229  | ns     |
| tpch_q10/vortex-file-compressed |     382087042 |     3.84292e+08 |          0.994264 | ns     |
| tpch_q11/vortex-file-compressed |     125000833 |     1.29075e+08 |          0.968434 | ns     |
| tpch_q12/vortex-file-compressed |     152568419 |     1.53634e+08 |          0.993063 | ns     |
| tpch_q13/vortex-file-compressed |     217287621 |     2.1317e+08  |          1.01931  | ns     |
| tpch_q14/vortex-file-compressed |     168644507 |     1.72657e+08 |          0.976762 | ns     |
| tpch_q15/vortex-file-compressed |     303566565 |     3.11411e+08 |          0.974811 | ns     |
| tpch_q16/vortex-file-compressed |     116462417 |     1.0054e+08  |          1.15837  | ns     |
| tpch_q17/vortex-file-compressed |     309231656 |     3.09683e+08 |          0.998541 | ns     |
| tpch_q18/vortex-file-compressed |     398235141 |     3.98427e+08 |          0.999519 | ns     |
| tpch_q19/vortex-file-compressed |     117708936 |     1.15004e+08 |          1.02352  | ns     |
| tpch_q20/vortex-file-compressed |     244789941 |     2.44399e+08 |          1.0016   | ns     |
| tpch_q21/vortex-file-compressed |     489829452 |     4.7757e+08  |          1.02567  | ns     |
| tpch_q22/vortex-file-compressed |     126639468 |     1.26195e+08 |          1.00352  | ns     |
</details>

<!-- thollander/actions-comment-pull-request "bench-pr-comment-tpch-s3" -->
author_association: CONTRIBUTOR
user: login: github-actions[bot]
created_at: 2025-03-27T22:28:54Z
body: # Benchmarks: Clickbench on NVME
<details>
<summary>Table of Results</summary>

| name                                  |   PR fa0fa2ae |   base b60fd71e |   ratio (PR/base) | unit   |
|:--------------------------------------|--------------:|----------------:|------------------:|:-------|
| clickbench_q00/parquet                |       2808465 |     2.43531e+06 |          1.15323  | ns     |
| clickbench_q01/parquet                |      38723785 |     3.34655e+07 |          1.15712  | ns     |
| clickbench_q02/parquet                |      72747218 |     6.39837e+07 |          1.13696  | ns     |
| clickbench_q03/parquet                |      59683080 |     5.32022e+07 |          1.12182  | ns     |
| clickbench_q04/parquet                |     368859391 |     3.20408e+08 |          1.15122  | ns     |
| clickbench_q05/parquet                |     373702885 |     3.18059e+08 |          1.17495  | ns     |
| clickbench_q06/parquet                |       2711710 |     2.37464e+06 |          1.14194  | ns     |
| clickbench_q07/parquet                |      39765861 |     3.2846e+07  |          1.21068  | ns     |
| clickbench_q08/parquet                |     444701296 |     3.86549e+08 |          1.15044  | ns     |
| clickbench_q09/parquet                |     671814103 |     5.76113e+08 |          1.16611  | ns     |
| clickbench_q10/parquet                |     137732864 |     1.20996e+08 |          1.13832  | ns     |
| clickbench_q11/parquet                |     161179072 |     1.42579e+08 |          1.13045  | ns     |
| clickbench_q12/parquet                |     368701929 |     3.1111e+08  |          1.18512  | ns     |
| clickbench_q13/parquet                |     565019466 |     4.85999e+08 |          1.16259  | ns     |
| clickbench_q14/parquet                |     368585930 |     3.21105e+08 |          1.14787  | ns     |
| clickbench_q15/parquet                |     407949923 |     3.52643e+08 |          1.15684  | ns     |
| clickbench_q16/parquet                |     887568095 |     7.84304e+08 |          1.13166  | ns     |
| clickbench_q17/parquet                |     784299536 |     6.94799e+08 |          1.12881  | ns     |
| clickbench_q18/parquet                |    1743967875 |     1.54947e+09 |          1.12553  | ns     |
| clickbench_q19/parquet                |      45075815 |     4.49099e+07 |          1.00369  | ns     |
| clickbench_q20/parquet                |     630904844 |     5.73432e+08 |          1.10023  | ns     |
| clickbench_q21/parquet                |     717651339 |     6.21517e+08 |          1.15468  | ns     |
| clickbench_q22/parquet                |    1126221672 |     9.71574e+08 |          1.15917  | ns     |
| clickbench_q23/parquet                |    4599576813 |     3.96194e+09 |          1.16094  | ns     |
| clickbench_q24/parquet                |     228572220 |     2.01643e+08 |          1.13355  | ns     |
| clickbench_q25/parquet                |     196083245 |     1.7481e+08  |          1.1217   | ns     |
| clickbench_q26/parquet                |     259463179 |     2.26389e+08 |          1.1461   | ns     |
| clickbench_q27/parquet                |     893616567 |     7.5759e+08  |          1.17955  | ns     |
| clickbench_q28/parquet                |    5018161867 |     4.41666e+09 |          1.13619  | ns     |
| clickbench_q29/parquet                |     287449210 |     2.67934e+08 |          1.07284  | ns     |
| clickbench_q30/parquet                |     366588311 |     3.2771e+08  |          1.11864  | ns     |
| clickbench_q31/parquet                |     429310012 |     3.77638e+08 |          1.13683  | ns     |
| clickbench_q32/parquet                |    2165454621 |     1.80211e+09 |          1.20162  | ns     |
| clickbench_q33/parquet                |    1783204247 |     1.55775e+09 |          1.14473  | ns     |
| clickbench_q34/parquet                |    1770873155 |     1.52405e+09 |          1.16195  | ns     |
| clickbench_q35/parquet                |     581168269 |     5.10162e+08 |          1.13918  | ns     |
| clickbench_q36/parquet                |     166630309 |     1.50053e+08 |          1.11047  | ns     |
| clickbench_q37/parquet                |      79182937 |     6.71107e+07 |          1.17989  | ns     |
| clickbench_q38/parquet                |     105147605 |     9.79051e+07 |          1.07397  | ns     |
| clickbench_q39/parquet                |     319811865 |     2.9535e+08  |          1.08282  | ns     |
| clickbench_q40/parquet                |      52781685 |     4.9135e+07  |          1.07422  | ns     |
| clickbench_q41/parquet                |      47485137 |     4.3932e+07  |          1.08088  | ns     |
| clickbench_q42/parquet                |      59836123 |     5.75947e+07 |          1.03892  | ns     |
| clickbench_q00/vortex-file-compressed |       5888415 |     4.6466e+06  |          1.26725  | ns     |
| clickbench_q01/vortex-file-compressed |      27063717 |     2.22113e+07 |          1.21847  | ns     |
| clickbench_q02/vortex-file-compressed |      37065276 |     3.82408e+07 |          0.969259 | ns     |
| clickbench_q03/vortex-file-compressed |      52479932 |     4.47485e+07 |          1.17278  | ns     |
| clickbench_q04/vortex-file-compressed |     460254692 |     3.81349e+08 |          1.20691  | ns     |
| clickbench_q05/vortex-file-compressed |     409964208 |     3.43155e+08 |          1.19469  | ns     |
| clickbench_q06/vortex-file-compressed |       5908991 |     4.86543e+06 |          1.21448  | ns     |
| clickbench_q07/vortex-file-compressed |      24904718 |     2.53248e+07 |          0.983413 | ns     |
| clickbench_q08/vortex-file-compressed |     503183313 |     4.39616e+08 |          1.1446   | ns     |
| clickbench_q09/vortex-file-compressed |     595909658 |     5.17072e+08 |          1.15247  | ns     |
| clickbench_q10/vortex-file-compressed |      75116320 |     7.12902e+07 |          1.05367  | ns     |
| clickbench_q11/vortex-file-compressed |      79589440 |     7.16898e+07 |          1.11019  | ns     |
| clickbench_q12/vortex-file-compressed |     320561195 |     2.58834e+08 |          1.23848  | ns     |
| clickbench_q13/vortex-file-compressed |     437739143 |     3.86559e+08 |          1.1324   | ns     |
| clickbench_q14/vortex-file-compressed |     313280130 |     2.59465e+08 |          1.20741  | ns     |
| clickbench_q15/vortex-file-compressed |     514010386 |     4.2781e+08  |          1.20149  | ns     |
| clickbench_q16/vortex-file-compressed |    1011368596 |     8.20785e+08 |          1.2322   | ns     |
| clickbench_q17/vortex-file-compressed |     944358118 |     8.03519e+08 |          1.17528  | ns     |
| clickbench_q18/vortex-file-compressed |    1688083132 |     1.32416e+09 |          1.27484  | ns     |
| clickbench_q19/vortex-file-compressed |      41120520 |     3.2714e+07  |          1.25697  | ns     |
| clickbench_q20/vortex-file-compressed |     296652629 |     2.66801e+08 |          1.11189  | ns     |
| clickbench_q21/vortex-file-compressed |     356757244 |     3.02793e+08 |          1.17822  | ns     |
| clickbench_q22/vortex-file-compressed |     632008499 |     5.33665e+08 |          1.18428  | ns     |
| clickbench_q23/vortex-file-compressed |    1366427583 |     9.14429e+08 |          1.4943   | ns     |
| clickbench_q24/vortex-file-compressed |     106057401 |     9.21493e+07 |          1.15093  | ns     |
| clickbench_q25/vortex-file-compressed |     115212116 |     9.92456e+07 |          1.16088  | ns     |
| clickbench_q26/vortex-file-compressed |     140472663 |     1.23612e+08 |          1.1364   | ns     |
| clickbench_q27/vortex-file-compressed |     674353606 |     5.73748e+08 |          1.17535  | ns     |
| clickbench_q28/vortex-file-compressed |    6117656132 |     5.0397e+09  |          1.21389  | ns     |
| clickbench_q29/vortex-file-compressed |     289699106 |     2.66786e+08 |          1.08589  | ns     |
| clickbench_q30/vortex-file-compressed |     270181929 |     2.21054e+08 |          1.22225  | ns     |
| clickbench_q31/vortex-file-compressed |     276047623 |     2.43454e+08 |          1.13388  | ns     |
| clickbench_q32/vortex-file-compressed |    1570969297 |     1.34286e+09 |          1.16987  | ns     |
| clickbench_q33/vortex-file-compressed |    1555768248 |     1.27048e+09 |          1.22455  | ns     |
| clickbench_q34/vortex-file-compressed |    1536951369 |     1.2602e+09  |          1.21961  | ns     |
| clickbench_q35/vortex-file-compressed |     739267132 |     6.64481e+08 |          1.11255  | ns     |
| clickbench_q36/vortex-file-compressed |     130169581 |     1.11409e+08 |          1.16839  | ns     |
| clickbench_q37/vortex-file-compressed |      78020761 |     7.83752e+07 |          0.995477 | ns     |
| clickbench_q38/vortex-file-compressed |      98256395 |     8.82546e+07 |          1.11333  | ns     |
| clickbench_q39/vortex-file-compressed |     198181567 |     1.78286e+08 |          1.11159  | ns     |
| clickbench_q40/vortex-file-compressed |      47987033 |     4.39889e+07 |          1.09089  | ns     |
| clickbench_q41/vortex-file-compressed |      42963047 |     3.96787e+07 |          1.08277  | ns     |
| clickbench_q42/vortex-file-compressed |      40812726 |     3.22656e+07 |          1.2649   | ns     |
</details>

<!-- thollander/actions-comment-pull-request "bench-pr-comment-clickbench-nvme" -->
author_association: CONTRIBUTOR
user: login: github-actions[bot]
created_at: 2025-03-27T23:01:01Z
body: # Benchmarks: random_access
<details>
<summary>Table of Results</summary>

| name                                   |   PR fa0fa2ae |   base b60fd71e |   ratio (PR/base) | unit   |
|:---------------------------------------|--------------:|----------------:|------------------:|:-------|
| random-access/vortex-tokio-local-disk  |       1873844 |     2.23031e+06 |          0.840173 | ns     |
| random-access/parquet-tokio-local-disk |     235728272 |     2.29961e+08 |          1.02508  | ns     |
</details>

<!-- thollander/actions-comment-pull-request "bench-pr-comment-random_access" -->
author_association: CONTRIBUTOR
user: login: github-actions[bot]
created_at: 2025-03-27T23:17:28Z
body: # Benchmarks: compress
<details>
<summary>Table of Results</summary>

| name                                                                                |      PR fa0fa2ae |    base b60fd71e |   ratio (PR/base) | unit     |
|:------------------------------------------------------------------------------------|-----------------:|-----------------:|------------------:|:---------|
| compress time/taxi throughput                                                       |      0.335048    |      0.371985    |          0.900702 | bytes/ns |
| parquet_rs-zstd compress time/taxi throughput                                       |      0.342229    |      0.34914     |          0.980206 | bytes/ns |
| decompress time/taxi throughput                                                     |      2.00335     |      2.26177     |          0.885744 | bytes/ns |
| parquet_rs-zstd decompress time/taxi throughput                                     |      1.86852     |      1.92893     |          0.968686 | bytes/ns |
| compress time/AirlineSentiment throughput                                           |      0.00354504  |      0.0038302   |          0.92555  | bytes/ns |
| parquet_rs-zstd compress time/AirlineSentiment throughput                           |      0.0595877   |      0.0499007   |          1.19413  | bytes/ns |
| decompress time/AirlineSentiment throughput                                         |      0.0250841   |      0.0253479   |          0.989593 | bytes/ns |
| parquet_rs-zstd decompress time/AirlineSentiment throughput                         |      0.0934745   |      0.0960475   |          0.973211 | bytes/ns |
| compress time/Arade throughput                                                      |      0.397361    |      0.409738    |          0.969792 | bytes/ns |
| parquet_rs-zstd compress time/Arade throughput                                      |      0.484671    |      0.507355    |          0.955289 | bytes/ns |
| decompress time/Arade throughput                                                    |      2.36797     |      2.36014     |          1.00332  | bytes/ns |
| parquet_rs-zstd decompress time/Arade throughput                                    |      1.99196     |      2.05106     |          0.971185 | bytes/ns |
| compress time/Bimbo throughput                                                      |      0.755572    |      0.793043    |          0.95275  | bytes/ns |
| parquet_rs-zstd compress time/Bimbo throughput                                      |      0.431444    |      0.446545    |          0.966183 | bytes/ns |
| decompress time/Bimbo throughput                                                    |      2.57079     |      2.55762     |          1.00515  | bytes/ns |
| parquet_rs-zstd decompress time/Bimbo throughput                                    |      3.14938     |      3.19066     |          0.987063 | bytes/ns |
| compress time/CMSprovider throughput                                                |      0.198747    |      0.2036      |          0.976162 | bytes/ns |
| parquet_rs-zstd compress time/CMSprovider throughput                                |      0.434718    |      0.442585    |          0.982226 | bytes/ns |
| decompress time/CMSprovider throughput                                              |      4.2517      |      4.35155     |          0.977055 | bytes/ns |
| parquet_rs-zstd decompress time/CMSprovider throughput                              |      2.1471      |      2.23977     |          0.958628 | bytes/ns |
| compress time/Euro2016 throughput                                                   |      0.248601    |      0.25771     |          0.964654 | bytes/ns |
| parquet_rs-zstd compress time/Euro2016 throughput                                   |      0.355365    |      0.366979    |          0.968354 | bytes/ns |
| decompress time/Euro2016 throughput                                                 |      2.73771     |      2.83373     |          0.966116 | bytes/ns |
| parquet_rs-zstd decompress time/Euro2016 throughput                                 |      1.21337     |      1.25422     |          0.967426 | bytes/ns |
| compress time/Food throughput                                                       |      0.305276    |      0.316513    |          0.964498 | bytes/ns |
| parquet_rs-zstd compress time/Food throughput                                       |      0.402146    |      0.418521    |          0.960873 | bytes/ns |
| decompress time/Food throughput                                                     |      6.07313     |      6.50344     |          0.933834 | bytes/ns |
| parquet_rs-zstd decompress time/Food throughput                                     |      1.97617     |      2.0525      |          0.96281  | bytes/ns |
| compress time/HashTags throughput                                                   |      0.409691    |      0.422289    |          0.970167 | bytes/ns |
| parquet_rs-zstd compress time/HashTags throughput                                   |      0.964392    |      0.988113    |          0.975994 | bytes/ns |
| decompress time/HashTags throughput                                                 |      6.16877     |      6.42714     |          0.959799 | bytes/ns |
| parquet_rs-zstd decompress time/HashTags throughput                                 |      3.21342     |      3.45211     |          0.930857 | bytes/ns |
| compress time/TPC-H l_comment chunked throughput                                    |      0.250217    |      0.259281    |          0.965043 | bytes/ns |
| parquet_rs-zstd compress time/TPC-H l_comment chunked throughput                    |      0.320991    |      0.334051    |          0.960906 | bytes/ns |
| decompress time/TPC-H l_comment chunked throughput                                  |      3.10165     |      3.17476     |          0.976971 | bytes/ns |
| parquet_rs-zstd decompress time/TPC-H l_comment chunked throughput                  |      1.41433     |      1.47493     |          0.958908 | bytes/ns |
| compress time/TPC-H l_comment canonical throughput                                  |      0.131966    |      0.13474     |          0.979405 | bytes/ns |
| parquet_rs-zstd compress time/TPC-H l_comment canonical throughput                  |      0.320098    |      0.328579    |          0.974189 | bytes/ns |
| decompress time/TPC-H l_comment canonical throughput                                |      3.13124     |      3.20394     |          0.97731  | bytes/ns |
| parquet_rs-zstd decompress time/TPC-H l_comment canonical throughput                |      1.42106     |      1.44724     |          0.981907 | bytes/ns |
| compress time/wide table cols=10 chunks=1 rows=1000 throughput                      |      0.155809    |      0.158331    |          0.984071 | bytes/ns |
| parquet_rs-zstd compress time/wide table cols=10 chunks=1 rows=1000 throughput      |      0.240212    |      0.250434    |          0.959185 | bytes/ns |
| decompress time/wide table cols=10 chunks=1 rows=1000 throughput                    |      0.465697    |      0.436732    |          1.06632  | bytes/ns |
| parquet_rs-zstd decompress time/wide table cols=10 chunks=1 rows=1000 throughput    |      0.559305    |      0.551357    |          1.01441  | bytes/ns |
| compress time/wide table cols=100 chunks=1 rows=1000 throughput                     |      0.14807     |      0.151342    |          0.978377 | bytes/ns |
| parquet_rs-zstd compress time/wide table cols=100 chunks=1 rows=1000 throughput     |      0.235226    |      0.24194     |          0.972251 | bytes/ns |
| decompress time/wide table cols=100 chunks=1 rows=1000 throughput                   |      1.18886     |      1.17752     |          1.00964  | bytes/ns |
| parquet_rs-zstd decompress time/wide table cols=100 chunks=1 rows=1000 throughput   |      0.53483     |      0.548227    |          0.975561 | bytes/ns |
| compress time/wide table cols=1000 chunks=1 rows=1000 throughput                    |      0.136349    |      0.14493     |          0.940795 | bytes/ns |
| parquet_rs-zstd compress time/wide table cols=1000 chunks=1 rows=1000 throughput    |      0.20901     |      0.230723    |          0.905889 | bytes/ns |
| decompress time/wide table cols=1000 chunks=1 rows=1000 throughput                  |      0.877098    |      0.875079    |          1.00231  | bytes/ns |
| parquet_rs-zstd decompress time/wide table cols=1000 chunks=1 rows=1000 throughput  |      0.490816    |      0.535261    |          0.916966 | bytes/ns |
| compress time/wide table cols=10 chunks=50 rows=1000 throughput                     |      0.0827419   |      0.0853806   |          0.969094 | bytes/ns |
| parquet_rs-zstd compress time/wide table cols=10 chunks=50 rows=1000 throughput     |      0.169491    |      0.172183    |          0.984366 | bytes/ns |
| decompress time/wide table cols=10 chunks=50 rows=1000 throughput                   |      0.659034    |      0.467641    |          1.40927  | bytes/ns |
| parquet_rs-zstd decompress time/wide table cols=10 chunks=50 rows=1000 throughput   |      0.588274    |      0.576986    |          1.01957  | bytes/ns |
| compress time/wide table cols=100 chunks=50 rows=1000 throughput                    |      0.0767196   |      0.0800505   |          0.95839  | bytes/ns |
| parquet_rs-zstd compress time/wide table cols=100 chunks=50 rows=1000 throughput    |      0.156728    |      0.161731    |          0.969063 | bytes/ns |
| decompress time/wide table cols=100 chunks=50 rows=1000 throughput                  |      1.19287     |      1.16133     |          1.02716  | bytes/ns |
| parquet_rs-zstd decompress time/wide table cols=100 chunks=50 rows=1000 throughput  |      0.535974    |      0.548968    |          0.976331 | bytes/ns |
| compress time/wide table cols=1000 chunks=50 rows=1000 throughput                   |      0.0573546   |      0.0664036   |          0.863728 | bytes/ns |
| parquet_rs-zstd compress time/wide table cols=1000 chunks=50 rows=1000 throughput   |      0.112221    |      0.122994    |          0.912406 | bytes/ns |
| decompress time/wide table cols=1000 chunks=50 rows=1000 throughput                 |      0.868171    |      0.866901    |          1.00147  | bytes/ns |
| parquet_rs-zstd decompress time/wide table cols=1000 chunks=50 rows=1000 throughput |      0.418246    |      0.477349    |          0.876186 | bytes/ns |
| vortex:raw size/taxi                                                                |      0.11663     |      0.11663     |          1        | ratio    |
| vortex size/taxi                                                                    |      5.76615e+07 |      5.76615e+07 |          1        | bytes    |
| vortex:parquet-zstd size/taxi                                                       |      1.03039     |      1.03039     |          1        | ratio    |
| vortex:raw size/AirlineSentiment                                                    |      1.28843     |      1.28843     |          1        | ratio    |
| vortex size/AirlineSentiment                                                        |   4208           |   4208           |          1        | bytes    |
| vortex:parquet-zstd size/AirlineSentiment                                           |      4.3516      |      4.3516      |          1        | ratio    |
| vortex:raw size/Arade                                                               |      0.123496    |      0.123496    |          1        | ratio    |
| vortex size/Arade                                                                   |      1.46551e+08 |      1.46551e+08 |          1        | bytes    |
| vortex:parquet-zstd size/Arade                                                      |      0.479876    |      0.479876    |          1        | ratio    |
| vortex:raw size/Bimbo                                                               |      0.121296    |      0.121296    |          1        | ratio    |
| vortex size/Bimbo                                                                   |      8.63791e+08 |      8.63791e+08 |          1        | bytes    |
| vortex:parquet-zstd size/Bimbo                                                      |      2.2254      |      2.2254      |          1        | ratio    |
| vortex:raw size/CMSprovider                                                         |      0.212072    |      0.212072    |          1        | ratio    |
| vortex size/CMSprovider                                                             |      1.33284e+09 |      1.33284e+09 |          1        | bytes    |
| vortex:parquet-zstd size/CMSprovider                                                |      1.73204     |      1.73204     |          1        | ratio    |
| vortex:raw size/Euro2016                                                            |      0.407544    |      0.407544    |          1        | ratio    |
| vortex size/Euro2016                                                                |      1.85442e+08 |      1.85442e+08 |          1        | bytes    |
| vortex:parquet-zstd size/Euro2016                                                   |      1.55978     |      1.55978     |          1        | ratio    |
| vortex:raw size/Food                                                                |      0.180192    |      0.180192    |          1        | ratio    |
| vortex size/Food                                                                    |      6.04616e+07 |      6.04616e+07 |          1        | bytes    |
| vortex:parquet-zstd size/Food                                                       |      1.66883     |      1.66883     |          1        | ratio    |
| vortex:raw size/HashTags                                                            |      0.159158    |      0.159158    |          1        | ratio    |
| vortex size/HashTags                                                                |      3.03782e+08 |      3.03782e+08 |          1        | bytes    |
| vortex:parquet-zstd size/HashTags                                                   |      2.26762     |      2.26762     |          1        | ratio    |
| vortex:raw size/TPC-H l_comment chunked                                             |      0.416714    |      0.423377    |          0.984261 | ratio    |
| vortex size/TPC-H l_comment chunked                                                 |      1.03844e+08 |      1.05504e+08 |          0.984261 | bytes    |
| vortex:parquet-zstd size/TPC-H l_comment chunked                                    |      1.82396     |      1.85304     |          0.984305 | ratio    |
| vortex:raw size/TPC-H l_comment canonical                                           |      0.420658    |      0.426113    |          0.987198 | ratio    |
| vortex size/TPC-H l_comment canonical                                               |      1.04824e+08 |      1.06184e+08 |          0.987198 | bytes    |
| vortex:parquet-zstd size/TPC-H l_comment canonical                                  |      1.84116     |      1.86495     |          0.987246 | ratio    |
| vortex:raw size/wide table cols=10 chunks=1 rows=1000                               |      0.630222    |      0.630222    |          1        | ratio    |
| vortex size/wide table cols=10 chunks=1 rows=1000                                   | 100896           | 100896           |          1        | bytes    |
| vortex:parquet-zstd size/wide table cols=10 chunks=1 rows=1000                      |      1.07929     |      1.07929     |          1        | ratio    |
| vortex:raw size/wide table cols=100 chunks=1 rows=1000                              |      0.627265    |      0.627265    |          1        | ratio    |
| vortex size/wide table cols=100 chunks=1 rows=1000                                  |      1.00414e+06 |      1.00414e+06 |          1        | bytes    |
| vortex:parquet-zstd size/wide table cols=100 chunks=1 rows=1000                     |      1.07417     |      1.07417     |          1        | ratio    |
| vortex:raw size/wide table cols=1000 chunks=1 rows=1000                             |      0.626969    |      0.626969    |          1        | ratio    |
| vortex size/wide table cols=1000 chunks=1 rows=1000                                 |      1.00365e+07 |      1.00365e+07 |          1        | bytes    |
| vortex:parquet-zstd size/wide table cols=1000 chunks=1 rows=1000                    |      1.07366     |      1.07366     |          1        | ratio    |
| vortex:raw size/wide table cols=10 chunks=50 rows=1000                              |      0.613693    |      0.613693    |          1        | ratio    |
| vortex size/wide table cols=10 chunks=50 rows=1000                                  | 100896           | 100896           |          1        | bytes    |
| vortex:parquet-zstd size/wide table cols=10 chunks=50 rows=1000                     |      1.07929     |      1.07929     |          1        | ratio    |
| vortex:raw size/wide table cols=100 chunks=50 rows=1000                             |      0.612126    |      0.612126    |          1        | ratio    |
| vortex size/wide table cols=100 chunks=50 rows=1000                                 |      1.00414e+06 |      1.00414e+06 |          1        | bytes    |
| vortex:parquet-zstd size/wide table cols=100 chunks=50 rows=1000                    |      1.07417     |      1.07417     |          1        | ratio    |
| vortex:raw size/wide table cols=1000 chunks=50 rows=1000                            |      0.611969    |      0.611969    |          1        | ratio    |
| vortex size/wide table cols=1000 chunks=50 rows=1000                                |      1.00365e+07 |      1.00365e+07 |          1        | bytes    |
| vortex:parquet-zstd size/wide table cols=1000 chunks=50 rows=1000                   |      1.07366     |      1.07366     |          1        | ratio    |
</details>

<!-- thollander/actions-comment-pull-request "bench-pr-comment-compress" -->
author_association: CONTRIBUTOR