Repository: ClickHouse/ClickHouse
Pull request #77789: Optimize order by single nullable or low-cardinality columns
Author: login: taiyang-li
Labels: name: pr-performance
description: Pull request with some performance improvements
name: pr-synced-to-cloud
description: The PR is synced to the cloud repo
Created: 2025-03-18T08:35:22Z | Merged: 2025-03-20T21:18:22Z

### Changelog category (leave one):
- Performance Improvement


### Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):
Optimize order by single nullable or low-cardinality columns


Opt1: devirtualize ColumnNullable::compareAt to improve order by single nullable column
``` sql
alter table hits_v1 add column nWatchId Nullable(UInt64) materialized toNullable(WatchID)
alter table hits_v1 materialize column nWatchId;
select nWatchId from datasets.hits_v1 order by nWatchId format Null settings compile_sort_description = 0; 

Before:
0 rows in set. Elapsed: 1.638 sec. Processed 8.87 million rows, 79.87 MB (5.42 million rows/s., 48.75 MB/s.)
0 rows in set. Elapsed: 1.602 sec. Processed 8.87 million rows, 79.87 MB (5.54 million rows/s., 49.85 MB/s.)
0 rows in set. Elapsed: 1.537 sec. Processed 8.87 million rows, 79.87 MB (5.77 million rows/s., 51.96 MB/s.) 

After
0 rows in set. Elapsed: 1.404 sec. Processed 8.87 million rows, 79.87 MB (6.32 million rows/s., 56.90 MB/s.)
0 rows in set. Elapsed: 1.396 sec. Processed 8.87 million rows, 79.87 MB (6.36 million rows/s., 57.21 MB/s.)
0 rows in set. Elapsed: 1.360 sec. Processed 8.87 million rows, 79.87 MB (6.53 million rows/s., 58.73 MB/s.)
```

Opt2: inroduce binary search in `SortingQueueImpl::updateBatchSize` in case batch_size large. It happens when sort columns have low cardinality. 
``` sql 
CREATE TABLE IF NOT EXISTS t (a UInt64, b Nullable(UInt64)) ENGINE = Memory() ;

-- cardinality: 100
truncate table t; 
insert into t select a%100, b%100 from  generateRandom('a UInt64, b Nullable(UInt64)') limit 10000000;  
before:
0 rows in set. Elapsed: 0.155 sec. Processed 10.00 million rows, 90.00 MB (64.55 million rows/s., 580.93 MB/s.)
0 rows in set. Elapsed: 0.151 sec. Processed 10.00 million rows, 90.00 MB (66.31 million rows/s., 596.78 MB/s.)
after:
0 rows in set. Elapsed: 0.111 sec. Processed 10.00 million rows, 90.00 MB (90.43 million rows/s., 813.86 MB/s.)
0 rows in set. Elapsed: 0.113 sec. Processed 10.00 million rows, 90.00 MB (88.47 million rows/s., 796.22 MB/s.)

-- cardinality: 1000
truncate table t; 
insert into t select a%1000, b%1000 from  generateRandom('a UInt64, b Nullable(UInt64)') limit 10000000;  
before
0 rows in set. Elapsed: 0.166 sec. Processed 10.00 million rows, 90.00 MB (60.36 million rows/s., 543.24 MB/s.)
0 rows in set. Elapsed: 0.165 sec. Processed 10.00 million rows, 90.00 MB (60.47 million rows/s., 544.21 MB/s.)
after
0 rows in set. Elapsed: 0.136 sec. Processed 10.00 million rows, 90.00 MB (73.74 million rows/s., 663.65 MB/s.)
0 rows in set. Elapsed: 0.131 sec. Processed 10.00 million rows, 90.00 MB (76.48 million rows/s., 688.32 MB/s.)

-- cardinality: 10000
truncate table t; 
insert into t select a%10000, b%10000 from  generateRandom('a UInt64, b Nullable(UInt64)') limit 10000000;  
before
0 rows in set. Elapsed: 0.203 sec. Processed 10.00 million rows, 90.00 MB (49.30 million rows/s., 443.71 MB/s.)
0 rows in set. Elapsed: 0.197 sec. Processed 10.00 million rows, 90.00 MB (50.77 million rows/s., 456.89 MB/s.)
after
0 rows in set. Elapsed: 0.250 sec. Processed 10.00 million rows, 90.00 MB (40.06 million rows/s., 360.54 MB/s.)
0 rows in set. Elapsed: 0.257 sec. Processed 10.00 million rows, 90.00 MB (38.95 million rows/s., 350.55 MB/s.)

-- cardinality: 100000
truncate table t; 
insert into t select a%100000, b%100000 from  generateRandom('a UInt64, b Nullable(UInt64)') limit 10000000;  
before
0 rows in set. Elapsed: 0.529 sec. Processed 10.00 million rows, 90.00 MB (18.89 million rows/s., 169.97 MB/s.)
0 rows in set. Elapsed: 0.496 sec. Processed 10.00 million rows, 90.00 MB (20.14 million rows/s., 181.29 MB/s.)
after
0 rows in set. Elapsed: 0.571 sec. Processed 10.00 million rows, 90.00 MB (17.51 million rows/s., 157.61 MB/s.)
0 rows in set. Elapsed: 0.541 sec. Processed 10.00 million rows, 90.00 MB (18.49 million rows/s., 166.37 MB/s.)

-- cardinality: 1000000
truncate table t; 
insert into t select a%1000000, b%1000000 from  generateRandom('a UInt64, b Nullable(UInt64)') limit 10000000;  
before
0 rows in set. Elapsed: 1.501 sec. Processed 10.00 million rows, 90.00 MB (6.66 million rows/s., 59.98 MB/s.)
0 rows in set. Elapsed: 1.517 sec. Processed 10.00 million rows, 90.00 MB (6.59 million rows/s., 59.31 MB/s.)
after
0 rows in set. Elapsed: 1.466 sec. Processed 10.00 million rows, 90.00 MB (6.82 million rows/s., 61.41 MB/s.)
0 rows in set. Elapsed: 1.380 sec. Processed 10.00 million rows, 90.00 MB (7.25 million rows/s., 65.23 MB/s.)

-- random distribution
truncate table t; 
insert into t select a, b from  generateRandom('a UInt64, b Nullable(UInt64)') limit 10000000;  
before:
0 rows in set. Elapsed: 1.500 sec. Processed 10.00 million rows, 90.00 MB (6.67 million rows/s., 60.01 MB/s.)
0 rows in set. Elapsed: 1.505 sec. Processed 10.00 million rows, 90.00 MB (6.64 million rows/s., 59.80 MB/s.)
after:
0 rows in set. Elapsed: 1.507 sec. Processed 10.00 million rows, 90.00 MB (6.64 million rows/s., 59.73 MB/s.)
0 rows in set. Elapsed: 1.463 sec. Processed 10.00 million rows, 90.00 MB (6.84 million rows/s., 61.53 MB/s.)
```

### Documentation entry for user-facing changes

- [ ] Documentation is written (mandatory for new features)

<!---
Directly edit documentation source files in the "docs" folder with the same pull-request as code changes

or

Add a user-readable short description of the changes that should be added to docs.clickhouse.com below.

At a minimum, the following information should be added (but add more as needed).
- Motivation: Why is this function, table engine, etc. useful to ClickHouse users?

- Parameters: If the feature being added takes arguments, options or is influenced by settings, please list them below with a brief explanation.

- Example use: A query or command.
-->



Diff:
diff --git a/src/Core/SortCursor.h b/src/Core/SortCursor.h
index 45d035251a36..7ecfd7f3f458 100644
--- a/src/Core/SortCursor.h
+++ b/src/Core/SortCursor.h
@@ -4,25 +4,29 @@
 #include <vector>
 #include <algorithm>
 
-#include <Common/typeid_cast.h>
-#include <Common/assert_cast.h>
-#include <Core/callOnTypeIndex.h>
-#include <Core/SortDescription.h>
+#include <Columns/ColumnDecimal.h>
+#include <Columns/ColumnFixedString.h>
+#include <Columns/ColumnNullable.h>
+#include <Columns/ColumnString.h>
+#include <Columns/IColumn.h>
+#include <Core/Block.h>
 #include <Core/ColumnNumbers.h>
-#include <DataTypes/DataTypesNumber.h>
-#include <DataTypes/DataTypesDecimal.h>
-#include <DataTypes/DataTypeString.h>
-#include <DataTypes/DataTypeFixedString.h>
+#include <Core/SortDescription.h>
+#include <Core/callOnTypeIndex.h>
 #include <DataTypes/DataTypeDate.h>
 #include <DataTypes/DataTypeDate32.h>
 #include <DataTypes/DataTypeDateTime.h>
 #include <DataTypes/DataTypeDateTime64.h>
 #include <DataTypes/DataTypeEnum.h>
-#include <DataTypes/DataTypeUUID.h>
+#include <DataTypes/DataTypeFixedString.h>
 #include <DataTypes/DataTypeIPv4andIPv6.h>
-#include <Columns/ColumnDecimal.h>
-#include <Columns/ColumnString.h>
-#include <Columns/ColumnFixedString.h>
+#include <DataTypes/DataTypeNullable.h>
+#include <DataTypes/DataTypeString.h>
+#include <DataTypes/DataTypeUUID.h>
+#include <DataTypes/DataTypesDecimal.h>
+#include <DataTypes/DataTypesNumber.h>
+#include <Common/assert_cast.h>
+#include <Common/typeid_cast.h>
 
 #include "config.h"
 
@@ -255,15 +259,10 @@ struct SimpleSortCursor : SortCursorHelper<SimpleSortCursor>
             res = direction * impl->sort_columns[0]->compareAt(lhs_pos, rhs_pos, *(rhs.impl->sort_columns[0]), nulls_direction);
         }
 
-        if (res > 0)
-            return true;
-        if (res < 0)
-            return false;
-
         if constexpr (consider_order)
-            return impl->order > rhs.impl->order;
+            return res ? res > 0 : impl->order > rhs.impl->order;
         else
-            return false;
+            return res > 0;
     }
 };
 
@@ -290,18 +289,63 @@ struct SpecializedSingleColumnSortCursor : SortCursorHelper<SpecializedSingleCol
 
         int res = desc.direction * lhs_column.compareAt(lhs_pos, rhs_pos, rhs_column, desc.nulls_direction);
 
-        if (res > 0)
-            return true;
-        if (res < 0)
-            return false;
+        if constexpr (consider_order)
+            return res ? res > 0 : this_impl->order > rhs.impl->order;
+        else
+            return res > 0;
+    }
+};
 
+template <typename ColumnType>
+struct SpecializedSingleNullableColumnSortCursor : SortCursorHelper<SpecializedSingleNullableColumnSortCursor<ColumnType>>
+{
+    using SortCursorHelper<SpecializedSingleNullableColumnSortCursor>::SortCursorHelper;
+
+    template <bool consider_order = true>
+    bool ALWAYS_INLINE greaterAt(const SortCursorHelper<SpecializedSingleNullableColumnSortCursor> & rhs, size_t lhs_pos, size_t rhs_pos) const
+    {
+        auto & this_impl = this->impl;
+
+        auto & lhs_columns = this_impl->sort_columns;
+        auto & rhs_columns = rhs.impl->sort_columns;
+
+        assert(lhs_columns.size() == 1);
+        assert(rhs_columns.size() == 1);
+
+        const auto & lhs_column = assert_cast<const ColumnNullable &>(*lhs_columns[0]);
+        const auto & rhs_column = assert_cast<const ColumnNullable &>(*rhs_columns[0]);
+        const auto & lhs_nullmap = lhs_column.getNullMapData();
+        const auto & rhs_nullmap = rhs_column.getNullMapData();
+        const auto & denull_lhs_column = assert_cast<const ColumnType &>(lhs_column.getNestedColumn());
+        const auto & denull_rhs_column = assert_cast<const ColumnType &>(rhs_column.getNestedColumn());
+
+        const auto & desc = this->impl->desc[0];
+
+        auto get_compare_result = [&]() -> int
+        {
+            bool lval_is_null = lhs_nullmap[lhs_pos];
+            bool rval_is_null = rhs_nullmap[rhs_pos];
+
+            if (unlikely(lval_is_null || rval_is_null))
+            {
+                if (lval_is_null && rval_is_null)
+                    return 0;
+                return lval_is_null ? desc.nulls_direction : -desc.nulls_direction;
+            }
+
+            return denull_lhs_column.compareAt(lhs_pos, rhs_pos, denull_rhs_column, desc.nulls_direction);
+        };
+
+
+        int res = desc.direction * get_compare_result();
         if constexpr (consider_order)
-            return this_impl->order > rhs.impl->order;
+            return res ? res > 0 : this_impl->order > rhs.impl->order;
         else
-            return false;
+            return res > 0;
     }
 };
 
+
 /// Separate comparator for locale-sensitive string comparisons
 struct SortCursorWithCollation : SortCursorHelper<SortCursorWithCollation>
 {
@@ -557,14 +601,32 @@ class SortingQueueImpl
         else
             return;
 
-        if (unlikely(begin_cursor.totallyLessOrEquals(next_child_cursor)))
+        /// Linear detection at most 16 elements to quickly find a small batch size.
+        /// This heuristic helps to avoid the overhead of binary search for small batches.
+        constexpr size_t max_linear_detection = 16;
+        size_t i = 0;
+        while (i < max_linear_detection && min_cursor_pos + batch_size < min_cursor_size
+               && next_child_cursor.greaterWithOffset(begin_cursor, 0, batch_size))
         {
-            batch_size = min_cursor_size - min_cursor_pos;
-            return;
+            ++batch_size;
+            ++i;
         }
 
-        while (min_cursor_pos + batch_size < min_cursor_size && next_child_cursor.greaterWithOffset(begin_cursor, 0, batch_size))
-            ++batch_size;
+        if (i < max_linear_detection)
+            return;
+
+        /// Binary search for the rest of elements in case of large batch size especially when sort columns have low cardinality.
+        size_t start_offset = batch_size;
+        size_t end_offset = min_cursor_size - min_cursor_pos;
+        while (start_offset < end_offset)
+        {
+            size_t mid_offset = start_offset + (end_offset - start_offset) / 2;
+            if (next_child_cursor.greaterWithOffset(begin_cursor, 0, mid_offset))
+                start_offset = mid_offset + 1;
+            else
+                end_offset = mid_offset;
+        }
+        batch_size = start_offset;
     }
 };
 
@@ -602,19 +664,38 @@ class SortQueueVariants
         }
         if (sort_description.size() == 1)
         {
-            TypeIndex column_type_index = sort_description_types[0]->getTypeId();
-
-            bool result = callOnIndexAndDataType<void>(
-                column_type_index,
-                [&](const auto & types)
-                {
-                    using Types = std::decay_t<decltype(types)>;
-                    using ColumnDataType = typename Types::LeftType;
-                    using ColumnType = typename ColumnDataType::ColumnType;
-
-                    initializeQueues<SpecializedSingleColumnSortCursor<ColumnType>>();
-                    return true;
-                });
+            bool result = false;
+            if (!sort_description_types[0]->isNullable())
+            {
+                TypeIndex column_type_index = sort_description_types[0]->getTypeId();
+                result = callOnIndexAndDataType<void>(
+                    column_type_index,
+                    [&](const auto & types)
+                    {
+                        using Types = std::decay_t<decltype(types)>;
+                        using ColumnDataType = typename Types::LeftType;
+                        using ColumnType = typename ColumnDataType::ColumnType;
+
+                        initializeQueues<SpecializedSingleColumnSortCursor<ColumnType>>();
+                        return true;
+                    });
+            }
+            else
+            {
+                DataTypePtr denull_type = removeNullable(sort_description_types[0]);
+                TypeIndex column_type_index = denull_type->getTypeId();
+                result = callOnIndexAndDataType<void>(
+                    column_type_index,
+                    [&](const auto & types)
+                    {
+                        using Types = std::decay_t<decltype(types)>;
+                        using ColumnDataType = typename Types::LeftType;
+                        using ColumnType = typename ColumnDataType::ColumnType;
+
+                        initializeQueues<SpecializedSingleNullableColumnSortCursor<ColumnType>>();
+                        return true;
+                    });
+            }
 
             if (!result)
                 initializeQueues<SimpleSortCursor>();
@@ -692,6 +773,36 @@ class SortQueueVariants
         SortingQueueImpl<SpecializedSingleColumnSortCursor<ColumnString>, strategy>,
         SortingQueueImpl<SpecializedSingleColumnSortCursor<ColumnFixedString>, strategy>,
 
+        SortingQueueImpl<SpecializedSingleNullableColumnSortCursor<ColumnVector<UInt8>>, strategy>,
+        SortingQueueImpl<SpecializedSingleNullableColumnSortCursor<ColumnVector<UInt16>>, strategy>,
+        SortingQueueImpl<SpecializedSingleNullableColumnSortCursor<ColumnVector<UInt32>>, strategy>,
+        SortingQueueImpl<SpecializedSingleNullableColumnSortCursor<ColumnVector<UInt64>>, strategy>,
+        SortingQueueImpl<SpecializedSingleNullableColumnSortCursor<ColumnVector<UInt128>>, strategy>,
+        SortingQueueImpl<SpecializedSingleNullableColumnSortCursor<ColumnVector<UInt256>>, strategy>,
+        SortingQueueImpl<SpecializedSingleNullableColumnSortCursor<ColumnVector<Int8>>, strategy>,
+        SortingQueueImpl<SpecializedSingleNullableColumnSortCursor<ColumnVector<Int16>>, strategy>,
+        SortingQueueImpl<SpecializedSingleNullableColumnSortCursor<ColumnVector<Int32>>, strategy>,
+        SortingQueueImpl<SpecializedSingleNullableColumnSortCursor<ColumnVector<Int64>>, strategy>,
+        SortingQueueImpl<SpecializedSingleNullableColumnSortCursor<ColumnVector<Int128>>, strategy>,
+        SortingQueueImpl<SpecializedSingleNullableColumnSortCursor<ColumnVector<Int256>>, strategy>,
+
+        SortingQueueImpl<SpecializedSingleNullableColumnSortCursor<ColumnVector<BFloat16>>, strategy>,
+        SortingQueueImpl<SpecializedSingleNullableColumnSortCursor<ColumnVector<Float32>>, strategy>,
+        SortingQueueImpl<SpecializedSingleNullableColumnSortCursor<ColumnVector<Float64>>, strategy>,
+
+        SortingQueueImpl<SpecializedSingleNullableColumnSortCursor<ColumnDecimal<Decimal32>>, strategy>,
+        SortingQueueImpl<SpecializedSingleNullableColumnSortCursor<ColumnDecimal<Decimal64>>, strategy>,
+        SortingQueueImpl<SpecializedSingleNullableColumnSortCursor<ColumnDecimal<Decimal128>>, strategy>,
+        SortingQueueImpl<SpecializedSingleNullableColumnSortCursor<ColumnDecimal<Decimal256>>, strategy>,
+        SortingQueueImpl<SpecializedSingleNullableColumnSortCursor<ColumnDecimal<DateTime64>>, strategy>,
+
+        SortingQueueImpl<SpecializedSingleNullableColumnSortCursor<ColumnVector<UUID>>, strategy>,
+        SortingQueueImpl<SpecializedSingleNullableColumnSortCursor<ColumnVector<IPv4>>, strategy>,
+        SortingQueueImpl<SpecializedSingleNullableColumnSortCursor<ColumnVector<IPv6>>, strategy>,
+
+        SortingQueueImpl<SpecializedSingleNullableColumnSortCursor<ColumnString>, strategy>,
+        SortingQueueImpl<SpecializedSingleNullableColumnSortCursor<ColumnFixedString>, strategy>,
+
         SortingQueueImpl<SimpleSortCursor, strategy>,
         SortingQueueImpl<SortCursor, strategy>,
         SortingQueueImpl<SortCursorWithCollation, strategy>>;
diff --git a/tests/performance/order_by_single_nullable.xml b/tests/performance/order_by_single_nullable.xml
new file mode 100644
index 000000000000..04da642641cd
--- /dev/null
+++ b/tests/performance/order_by_single_nullable.xml
@@ -0,0 +1,14 @@
+<test>
+    <create_query>CREATE TABLE IF NOT EXISTS t_opt_orderby (a UInt64, b Nullable(UInt64)) ENGINE = Memory()</create_query>
+    <fill_query>insert into t_opt_orderby select * from  generateRandom('a UInt64, b Nullable(UInt64)') limit 10000000</fill_query>
+
+    <settings>
+        <max_threads>1</max_threads>
+        <compile_sort_description>0</compile_sort_description>
+    </settings>
+
+    <query>select a from t_opt_orderby order by a</query>
+    <query>select b from t_opt_orderby order by b</query>
+
+    <drop_query>DROP TABLE IF EXISTS t_opt_orderby</drop_query>
+</test>


Reviews:
user: login: Algunenano
body: Looks good. Just need to review some sync failures that look unrelated and I'll merge it
state: APPROVED
author_association: MEMBER
submitted_at: 2025-03-18T18:25:40Z
user: login: rschu1ze
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-24T18:28:21Z
user: login: taiyang-li
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-25T01:49:26Z

Review comments:
diff_hunk: @@ -0,0 +1,14 @@
+<test>
+    <create_query>CREATE TABLE IF NOT EXISTS t_opt_orderby (a UInt64, b Nullable(UInt64)) ENGINE = Memory()</create_query>
+    <fill_query>insert into t_opt_orderby select * from  generateRandom('a UInt64, b Nullable(UInt64)') limit 10000000</fill_query>
path: tests/performance/order_by_single_nullable.xml
user: login: rschu1ze
body: @taiyang-li I guess a performance tests for the low-cardinality optimization will also make sense.
created_at: 2025-03-24T18:28:21Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,14 @@
+<test>
+    <create_query>CREATE TABLE IF NOT EXISTS t_opt_orderby (a UInt64, b Nullable(UInt64)) ENGINE = Memory()</create_query>
+    <fill_query>insert into t_opt_orderby select * from  generateRandom('a UInt64, b Nullable(UInt64)') limit 10000000</fill_query>
path: tests/performance/order_by_single_nullable.xml
user: login: taiyang-li
body: I think it was covered by sort.xml 
![image](https://github.com/user-attachments/assets/5f94fbf5-a562-4709-aff6-7bcb6738bf72)
 
created_at: 2025-03-25T01:49:26Z
author_association: CONTRIBUTOR

PR comments:
user: login: clickhouse-gh[bot]
created_at: 2025-03-18T08:35:52Z
body: Workflow [[PR](https://s3.amazonaws.com/clickhouse-test-reports/json.html?PR=77789&sha=latest&name_0=PR)], commit [f0e65066]
author_association: CONTRIBUTOR
user: login: Algunenano
created_at: 2025-03-18T18:32:50Z
body: `group_by_fixed_keys	5` has slowed down (`+1.465x`) but I don't see how that's possible, and running it locally I see the same perf.
author_association: MEMBER
user: login: taiyang-li
created_at: 2025-03-19T06:29:28Z
body: > `group_by_fixed_keys 5` has slowed down (`+1.465x`) but I don't see how that's possible, and running it locally I see the same perf.

Me either. 
author_association: CONTRIBUTOR
user: login: taiyang-li
created_at: 2025-03-19T08:27:54Z
body: @Algunenano I can't reproduce it in my local environment. Maybe it is just a noise

```
./clickhouse benchmark --concurrency 1  -i 20   --max_threads=1 --query "select a, b from group_by_fk group by a, b format Null"

Before
localhost:9001, queries: 20, QPS: 1.010, RPS: 10096806.800, MiB/s: 77.033, result RPS: 0.000, result MiB/s: 0.000.
0%              0.756 sec.
10%             0.960 sec.
20%             0.969 sec.
30%             0.978 sec.
40%             0.979 sec.
50%             0.983 sec.
60%             0.988 sec.
70%             0.993 sec.
80%             0.999 sec.
90%             1.008 sec.
95%             1.010 sec.
99%             1.343 sec.
99.9%           1.343 sec.
99.99%          1.343 sec.

After
localhost:9001, queries: 20, QPS: 0.995, RPS: 9948095.166, MiB/s: 75.898, result RPS: 0.000, result MiB/s: 0.000.
0%              0.861 sec.
10%             0.974 sec.
20%             0.975 sec.
30%             0.979 sec.
40%             0.986 sec.
50%             0.999 sec.
60%             1.000 sec.
70%             1.015 sec.
80%             1.036 sec.
90%             1.046 sec.
95%             1.049 sec.
99%             1.120 sec.
99.9%           1.120 sec.
99.99%          1.120 sec. 
```
author_association: CONTRIBUTOR