Repository: ClickHouse/ClickHouse
Pull request #77187: Add `arrayLevenshtein*` functions to calculate distance for arrays
Author: login: Felixoid
Labels: name: pr-feature
description: Pull request with new product feature
name: pr-synced-to-cloud
description: The PR is synced to the cloud repo
Created: 2025-03-05T19:19:37Z | Merged: 2025-03-28T17:28:36Z

<!---
A technical comment, you are free to remove or leave it as it is when PR is created
The following categories are used in the next scripts, update them accordingly
utils/changelog/changelog.py
tests/ci/cancel_and_rerun_workflow_lambda/app.py
-->
### Changelog category (leave one):
- New Feature


### Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):
Add functions `arrayLevenshteinDistance`, `arrayLevenshteinDistanceWeighted`, and `arraySimilarity`.

### Documentation entry for user-facing changes

- [ ] Documentation is written (mandatory for new features)

<!---
Directly edit documentation source files in the "docs" folder with the same pull-request as code changes

or

Add a user-readable short description of the changes that should be added to docs.clickhouse.com below.

At a minimum, the following information should be added (but add more as needed).
- Motivation: Why is this function, table engine, etc. useful to ClickHouse users?

- Parameters: If the feature being added takes arguments, options or is influenced by settings, please list them below with a brief explanation.

- Example use: A query or command.
-->



Diff:
diff --git a/ci/jobs/scripts/check_style/aspell-ignore/en/aspell-dict.txt b/ci/jobs/scripts/check_style/aspell-ignore/en/aspell-dict.txt
index 5f921be48d23..328369f23ccb 100644
--- a/ci/jobs/scripts/check_style/aspell-ignore/en/aspell-dict.txt
+++ b/ci/jobs/scripts/check_style/aspell-ignore/en/aspell-dict.txt
@@ -1305,6 +1305,8 @@ arrayJoin
 arrayLast
 arrayLastIndex
 arrayLastOrNull
+arrayLevenshteinDistance
+arrayLevenshteinDistanceWeighted
 arrayMap
 arrayMax
 arrayMin
@@ -1332,6 +1334,7 @@ arrayShiftLeft
 arrayShiftRight
 arrayShingles
 arrayShuffle
+arraySimilarity
 arraySlice
 arraySort
 arraySplit
diff --git a/docs/en/sql-reference/functions/array-functions.md b/docs/en/sql-reference/functions/array-functions.md
index f04fa947492f..ab49e1e71547 100644
--- a/docs/en/sql-reference/functions/array-functions.md
+++ b/docs/en/sql-reference/functions/array-functions.md
@@ -3374,6 +3374,121 @@ Result:
 └─────────────────────────────────────────────────────────────┘
 ```
 
+## arrayLevenshteinDistance {#arraylevenshteindistance}
+
+Calculates Levenshtein distance for two arrays.
+
+**Syntax**
+
+```sql
+arrayLevenshteinDistance(from, to)
+```
+
+**Arguments**
+
+- `from` — first array
+- `to` — second array
+
+**Returned Value**
+
+- Levenshtein distance between the first and the second arrays
+
+**Examples**
+
+Query:
+
+```sql
+SELECT arrayLevenshteinDistance([1, 2, 4], [1, 2, 3])
+```
+
+Result:
+
+```text
+
+┌─arrayLevenshteinDistance([1, 2, 4], [1, 2, 3])─┐
+│                                              1 │
+└────────────────────────────────────────────────┘
+
+```
+
+## arrayLevenshteinDistanceWeighted {#arraylevenshteindistanceweighted}
+
+Calculates Levenshtein distance for two arrays with custom weights for each element. Number of elements for array and its weights should match
+
+**Syntax**
+
+```sql
+arrayLevenshteinDistanceWeighted(from, to, from_weights, to_weights)
+```
+
+**Arguments**
+
+- `from` — first array
+- `to` — second array
+- `from_weights` — weights for the first array
+- `to_weights` — weights for the second array
+
+**Returned Value**
+
+- Levenshtein distance between the first and the second arrays with custom weights for each element
+
+**Examples**
+
+Query:
+
+```sql
+SELECT arrayLevenshteinDistanceWeighted(['A', 'B', 'C'], ['A', 'K', 'L'], [1.0, 2, 3], [3.0, 4, 5])
+```
+
+Result:
+
+```text
+
+┌─arrayLevenshteinDistanceWeighted(['A', 'B', 'C'], ['A', 'K', 'L'], [1.0, 2, 3], [3.0, 4, 5])─┐
+│                                                                                           14 │
+└──────────────────────────────────────────────────────────────────────────────────────────────┘
+
+```
+
+## arraySimilarity {#arraysimilarity}
+
+Calculates arrays' similarity from 0 to 1 based on weighed Levenshtein distance. Accepts the same arguments as `arrayLevenshteinDistanceWeighted` function.
+
+**Syntax**
+
+```sql
+arraySimilarity(from, to, from_weights, to_weights)
+```
+
+**Arguments**
+
+- `from` — first array
+- `to` — second array
+- `from_weights` — weights for the first array
+- `to_weights` — weights for the second array
+
+**Returned Value**
+
+- Similarity of two arrays based on the weighted Levenshtein distance
+
+**Examples**
+
+Query:
+
+```sql
+SELECT arraySimilarity(['A', 'B', 'C'], ['A', 'K', 'L'], [1.0, 2, 3], [3.0, 4, 5])
+```
+
+Result:
+
+```text
+
+┌─arraySimilarity(['A', 'B', 'C'], ['A', 'K', 'L'], [1.0, 2, 3], [3.0, 4, 5])─┐
+│                                                          0.2222222222222222 │
+└─────────────────────────────────────────────────────────────────────────────┘
+
+```
+
 ## Distance functions {#distance-functions}
 
 All supported functions are described in [distance functions documentation](../../sql-reference/functions/distance-functions.md).
diff --git a/src/Common/FunctionDocumentation.cpp b/src/Common/FunctionDocumentation.cpp
index 946dc4eb0208..049bf07bbd81 100644
--- a/src/Common/FunctionDocumentation.cpp
+++ b/src/Common/FunctionDocumentation.cpp
@@ -7,7 +7,7 @@ std::string FunctionDocumentation::argumentsAsString() const
 {
     std::string res;
     for (const auto & [name, desc] : arguments)
-        res += "- " + name + ":" + desc + "\n";
+        res += "- `" + name + "` — " + desc + "\n";
     return res;
 }
 
@@ -20,6 +20,7 @@ std::string FunctionDocumentation::examplesAsString() const
         res += "```sql\n";
         res += query + "\n";
         res += "```\n\n";
+        res += "Result:\n\n";
         res += "```text\n";
         res += result + "\n";
         res += "```\n";
diff --git a/src/Common/NamePrompter.h b/src/Common/NamePrompter.h
index cc72554657f6..e25fc4d8400e 100644
--- a/src/Common/NamePrompter.h
+++ b/src/Common/NamePrompter.h
@@ -41,7 +41,7 @@ class NamePrompter
 
         if (prompt.size() <= name.size() + mistake_factor && prompt.size() + mistake_factor >= name.size())
         {
-            size_t distance = levenshteinDistance(prompt, name);
+            size_t distance = levenshteinDistanceCaseInsensitive(prompt, name);
             if (distance <= mistake_factor)
             {
                 queue.emplace(distance, ind);
diff --git a/src/Common/isLocalAddress.cpp b/src/Common/isLocalAddress.cpp
index 399de9d89a0a..433dbebc37d8 100644
--- a/src/Common/isLocalAddress.cpp
+++ b/src/Common/isLocalAddress.cpp
@@ -132,7 +132,7 @@ size_t getHostNamePrefixDistance(const std::string & local_hostname, const std::
 
 size_t getHostNameLevenshteinDistance(const std::string & local_hostname, const std::string & host)
 {
-    return levenshteinDistance(local_hostname, host);
+    return levenshteinDistanceCaseInsensitive(local_hostname, host);
 }
 
 }
diff --git a/src/Common/levenshteinDistance.cpp b/src/Common/levenshteinDistance.cpp
index 3ab80af94bb6..3325f739a50a 100644
--- a/src/Common/levenshteinDistance.cpp
+++ b/src/Common/levenshteinDistance.cpp
@@ -5,7 +5,7 @@
 namespace DB
 {
 
-size_t levenshteinDistance(const String & lhs, const String & rhs)
+size_t levenshteinDistanceCaseInsensitive(const String & lhs, const String & rhs)
 {
     size_t m = lhs.size();
     size_t n = rhs.size();
diff --git a/src/Common/levenshteinDistance.h b/src/Common/levenshteinDistance.h
index b062f7c1d73b..398e02763eb4 100644
--- a/src/Common/levenshteinDistance.h
+++ b/src/Common/levenshteinDistance.h
@@ -1,12 +1,112 @@
 #pragma once
 
 #include <base/types.h>
+#include <Common/PODArray.h>
+#include <Common/iota.h>
+#include <span>
+
+#include <numeric>
 
 namespace DB
 {
 
+namespace ErrorCodes
+{
+    extern const int SIZES_OF_ARRAYS_DONT_MATCH;
+}
+
 /// How many steps if we want to change lhs to rhs.
 /// Details in https://en.wikipedia.org/wiki/Levenshtein_distance
-size_t levenshteinDistance(const String & lhs, const String & rhs);
+size_t levenshteinDistanceCaseInsensitive(const String & lhs, const String & rhs);
+
+/// The same function as DB::levenshteinDistanceCaseInsensitive, but case sensitive for a more generic approach
+template <class Element = char>
+size_t levenshteinDistance(std::span<const Element> lhs, std::span<const Element> rhs)
+{
+    const size_t m = lhs.size();
+    const size_t n = rhs.size();
+    if (m==0 || n==0)
+    {
+        return m + n;
+    }
+    PODArray<size_t> v0(n + 1);
+
+    iota(v0.data() + 1, n, size_t(1));
+
+    for (size_t j = 1; j <= m; ++j)
+    {
+        v0[0] = j;
+        size_t prev = j - 1;
+        for (size_t i = 1; i <= n; ++i)
+        {
+            size_t old = v0[i];
+            v0[i] = std::min(prev + (lhs[j - 1] != rhs[i - 1]),
+                    std::min(v0[i - 1], v0[i]) + 1);
+            prev = old;
+        }
+    }
+    return v0[n];
+};
+
+/// Levenshtein Distance with weights, is used to calculate custom distance from lhs to rhs
+template <class Element, class Weight>
+Weight levenshteinDistanceWeighted(std::span<const Element> lhs, std::span<const Element> rhs,
+                                   std::span<const Weight> lhs_weights, std::span<const Weight> rhs_weights)
+{
+    if (lhs.size() != lhs_weights.size() || rhs.size() != rhs_weights.size())
+        throw Exception(
+            ErrorCodes::SIZES_OF_ARRAYS_DONT_MATCH,
+            "Sizes of arrays don't match: {} != {} or {} != {}",
+            lhs.size(),
+            lhs_weights.size(),
+            rhs.size(),
+            rhs_weights.size());
+
+    auto sum_span = [](const std::span<const Weight> & span) -> Weight
+    {
+        return std::accumulate(span.begin(), span.end(), Weight{});
+    };
+    if (lhs.empty() || rhs.empty())
+    {
+        return sum_span(lhs_weights) + sum_span(rhs_weights);
+    }
+
+    // Consume minimum memory by allocating sliding vector for min `m`
+    if (lhs.size() > rhs.size())
+    {
+        std::swap(lhs, rhs);
+        std::swap(lhs_weights, rhs_weights);
+    }
+
+    const size_t m = lhs.size();
+    const size_t n = rhs.size();
+
+    PODArray<Weight> row(m + 1);
+
+    row[0] = 0;
+    std::partial_sum(lhs_weights.begin(), lhs_weights.end(), row.begin() + 1);
+
+    for (size_t j = 1; j <= n; ++j)
+    {
+        Weight prev = row[0];
+        row[0] += rhs_weights[j - 1];
+        for (size_t i = 1; i <= m; ++i)
+        {
+            if (lhs[i - 1] == rhs[j - 1])
+            {
+                row[i] = prev;
+                continue;
+            }
+
+            prev = row[i];
+
+            row[i] = std::min({row[i] + rhs_weights[j - 1], // deletion
+                               row[i - 1] + lhs_weights[i - 1], // insertion
+                               row[i - 1] + lhs_weights[i - 1] + rhs_weights[j - 1]}); // substitusion
+        }
+    }
+
+    return row[m];
+};
 
 }
diff --git a/src/Functions/array/arrayLevenshtein.cpp b/src/Functions/array/arrayLevenshtein.cpp
new file mode 100644
index 000000000000..15976a15c1b5
--- /dev/null
+++ b/src/Functions/array/arrayLevenshtein.cpp
@@ -0,0 +1,640 @@
+#include <Columns/ColumnArray.h>
+#include <Columns/ColumnDecimal.h>
+#include <Columns/ColumnFixedString.h>
+#include <Columns/ColumnNullable.h>
+#include <Columns/ColumnString.h>
+#include <Columns/ColumnTuple.h>
+#include <Common/levenshteinDistance.h>
+#include <Common/PODArray.h>
+#include <Common/iota.h>
+#include <DataTypes/DataTypeArray.h>
+#include <DataTypes/DataTypeNullable.h>
+#include <DataTypes/DataTypeTuple.h>
+#include <DataTypes/DataTypesNumber.h>
+#include <Functions/FunctionFactory.h>
+#include <Functions/FunctionHelpers.h>
+#include <IO/WriteHelpers.h>
+
+#include <numeric>
+#include <span>
+
+namespace DB
+{
+
+namespace ErrorCodes
+{
+    extern const int ILLEGAL_TYPE_OF_ARGUMENT;
+    extern const int LOGICAL_ERROR;
+    extern const int SIZES_OF_ARRAYS_DONT_MATCH;
+}
+
+/// arrayLevenshteinDistance([1,2,3,4], [1,3,2,4]) = 2
+/// arrayLevenshteinDistanceWeighted([1,2,3,4], [1,3,2,4]) = 2
+template <typename T>
+class FunctionArrayLevenshtein : public IFunction
+{
+public:
+    static constexpr auto name = T::name;
+    static FunctionPtr create(ContextPtr) { return std::make_shared<FunctionArrayLevenshtein<T>>(); }
+
+    String getName() const override { return name; }
+
+    size_t getNumberOfArguments() const override { return T::arguments; }
+    bool useDefaultImplementationForConstants() const override { return true; }
+
+    bool isSuitableForShortCircuitArgumentsExecution(const DataTypesWithConstInfo & /*arguments*/) const override { return true; }
+
+    DataTypePtr getReturnTypeImpl(const ColumnsWithTypeAndName & arguments) const override
+    {
+        FunctionArgumentDescriptors args_descriptors;
+        args_descriptors = FunctionArgumentDescriptors{
+            {"from", static_cast<FunctionArgumentDescriptor::TypeValidator>(&isArray), nullptr, "Array"},
+            {"to", static_cast<FunctionArgumentDescriptor::TypeValidator>(&isArray), nullptr, "Array"},
+            {"from_weights", static_cast<FunctionArgumentDescriptor::TypeValidator>(&isArray), nullptr, "Array"},
+            {"to_weights", static_cast<FunctionArgumentDescriptor::TypeValidator>(&isArray), nullptr, "Array"},
+        };
+        validateFunctionArguments(*this, arguments, args_descriptors);
+        std::vector<DataTypePtr> nested_types;
+        nested_types.reserve(2);
+        for (size_t index = 2; index < 4; ++index)
+        {
+            const DataTypeArray * array_type = checkAndGetDataType<DataTypeArray>(arguments[index].type.get());
+            const DataTypePtr nested_type = array_type->getNestedType();
+            nested_types.emplace_back(nested_type);
+            if (!(isFloat(nested_type) || isInteger(nested_type)))
+                throw Exception(
+                    ErrorCodes::ILLEGAL_TYPE_OF_ARGUMENT,
+                    "Argument {} of function {} must be a numeric array. Found {} instead.",
+                    toString(index + 1),
+                    getName(),
+                    nested_type->getName());
+        }
+
+        if (nested_types[0]->getName() != nested_types[1]->getName())
+            throw Exception(
+                ErrorCodes::ILLEGAL_TYPE_OF_ARGUMENT,
+                "Arguments 3 and 4 of function {} must be arrays of the same types. Found {} and {} instead.",
+                getName(),
+                nested_types[0]->getName(),
+                nested_types[1]->getName());
+        return std::make_shared<DataTypeFloat64>();
+    }
+
+    ColumnPtr executeImpl(const ColumnsWithTypeAndName & arguments, const DataTypePtr &, size_t input_rows_count) const override
+    {
+        size_t num_arguments = arguments.size();
+
+        Columns holders(num_arguments);
+        std::vector<const ColumnArray *> columns(num_arguments);
+
+        for (size_t i = 0; i < num_arguments; ++i)
+        {
+            holders[i] = arguments[i].column->convertToFullColumnIfConst();
+            if (holders[i]->size() != input_rows_count)
+                throw Exception(
+                    ErrorCodes::LOGICAL_ERROR,
+                    "Function {} has unequal number of rows in columns: "
+                    "expected {}, got {} for column {}",
+                    getName(),
+                    input_rows_count,
+                    holders[i]->size(),
+                    holders[i]->getName());
+            columns[i] = assert_cast<const ColumnArray*>(holders[i].get());
+        }
+        return execute(columns);
+    }
+private:
+    ColumnPtr execute(std::vector<const ColumnArray *>) const
+    {
+        throw Exception(
+            ErrorCodes::LOGICAL_ERROR,
+            "Unknown function {}. "
+            "Supported names: 'arrayLevenshteinDistance', 'arrayLevenshteinDistanceWeighted', 'arraySimilarity'",
+            getName());
+    }
+
+    template <typename N, typename Result>
+    bool levenshteinString(std::vector<const ColumnArray *> columns, Result::Container & res_values) const
+    {
+        const N * from_data = checkAndGetColumn<N>(&columns[0]->getData());
+        const N * to_data = checkAndGetColumn<N>(&columns[1]->getData());
+        if (!from_data || !to_data)
+            return false;
+        if (T::arguments == 4)
+        {
+            if constexpr (std::is_same_v<Result, ColumnFloat64>)
+            {
+                if (!(
+                    levenshteinWeightedString<N, UInt8>(columns, res_values)
+                    || levenshteinWeightedString<N, UInt16>(columns, res_values)
+                    || levenshteinWeightedString<N, UInt32>(columns, res_values)
+                    || levenshteinWeightedString<N, UInt64>(columns, res_values)
+                    || levenshteinWeightedString<N, UInt128>(columns, res_values)
+                    || levenshteinWeightedString<N, UInt256>(columns, res_values)
+                    || levenshteinWeightedString<N, Int8>(columns, res_values)
+                    || levenshteinWeightedString<N, Int16>(columns, res_values)
+                    || levenshteinWeightedString<N, Int32>(columns, res_values)
+                    || levenshteinWeightedString<N, Int64>(columns, res_values)
+                    || levenshteinWeightedString<N, Int128>(columns, res_values)
+                    || levenshteinWeightedString<N, Int256>(columns, res_values)
+                    || levenshteinWeightedString<N, Float32>(columns, res_values)
+                    || levenshteinWeightedString<N, Float64>(columns, res_values)))
+                    throw Exception(
+                        ErrorCodes::LOGICAL_ERROR,
+                        "Unexpected code branch of function {}. No fitting levenshteinWeightedString",
+                        getName());
+                return true;
+            }
+            throw Exception(
+                ErrorCodes::LOGICAL_ERROR,
+                "Unexpected code branch of function {}. Wrong expected result type in levenshteinString",
+                getName());
+        }
+        const ColumnArray::Offsets & from_offsets = columns[0]->getOffsets();
+        ColumnArray::Offset prev_from_offset = 0;
+
+        const ColumnArray::Offsets & to_offsets = columns[1]->getOffsets();
+        ColumnArray::Offset prev_to_offset = 0;
+        const auto extract_array = [](const N * data, size_t prev_offset, size_t count)
+        {
+            std::vector<StringRef> temp;
+            temp.reserve(count);
+            for (size_t j = 0; j < count; ++j) { temp.emplace_back(data->getDataAt(prev_offset + j)); }
+            return temp;
+        };
+
+        for (size_t row = 0; row < columns[0]->size(); row++)
+        {
+            const size_t m = from_offsets[row] - prev_from_offset;
+            const size_t n = to_offsets[row] - prev_to_offset;
+            const std::vector<StringRef> from = extract_array(from_data, prev_from_offset, m);
+            const std::vector<StringRef> to = extract_array(to_data, prev_to_offset, n);
+            prev_from_offset = from_offsets[row];
+            prev_to_offset = to_offsets[row];
+            res_values[row] = levenshteinDistance<StringRef>(from, to);
+        }
+        return true;
+    }
+
+    template <typename N, typename Result>
+    bool levenshteinNumber(std::vector<const ColumnArray *> columns, Result::Container & res_values) const
+    {
+        const ColumnVectorOrDecimal<N> * column_from = checkAndGetColumn<ColumnVectorOrDecimal<N>>(&columns[0]->getData());
+        const ColumnVectorOrDecimal<N> * column_to = checkAndGetColumn<ColumnVectorOrDecimal<N>>(&columns[1]->getData());
+        if (!column_from || !column_to)
+            return false;
+
+        if (T::arguments == 4)
+        {
+            if constexpr (std::is_same_v<Result, ColumnFloat64>)
+            {
+                if (!(
+                    levenshteinWeightedNumber<N, UInt8>(columns, res_values)
+                    || levenshteinWeightedNumber<N, UInt16>(columns, res_values)
+                    || levenshteinWeightedNumber<N, UInt32>(columns, res_values)
+                    || levenshteinWeightedNumber<N, UInt64>(columns, res_values)
+                    || levenshteinWeightedNumber<N, UInt128>(columns, res_values)
+                    || levenshteinWeightedNumber<N, UInt256>(columns, res_values)
+                    || levenshteinWeightedNumber<N, Int8>(columns, res_values)
+                    || levenshteinWeightedNumber<N, Int16>(columns, res_values)
+                    || levenshteinWeightedNumber<N, Int32>(columns, res_values)
+                    || levenshteinWeightedNumber<N, Int64>(columns, res_values)
+                    || levenshteinWeightedNumber<N, Int128>(columns, res_values)
+                    || levenshteinWeightedNumber<N, Int256>(columns, res_values)
+                    || levenshteinWeightedNumber<N, Float32>(columns, res_values)
+                    || levenshteinWeightedNumber<N, Float64>(columns, res_values)))
+                    throw Exception(
+                        ErrorCodes::LOGICAL_ERROR,
+                        "Unexpected code branch of function {}. No fitting levenshteinWeightedNumber",
+                        getName());
+                return true;
+            }
+            throw Exception(
+                ErrorCodes::LOGICAL_ERROR,
+                "Unexpected code branch of function {}. Wrong expected result type in levenshteinNumber",
+                getName());
+        }
+
+        const ColumnArray::Offsets & from_offsets = columns[0]->getOffsets();
+        ColumnArray::Offset prev_from_offset = 0;
+
+        const ColumnArray::Offsets & to_offsets = columns[1]->getOffsets();
+        ColumnArray::Offset prev_to_offset = 0;
+
+        for (size_t row = 0; row < columns[0]->size(); row++)
+        {
+            std::span<const N> from(column_from->getData().begin() + prev_from_offset, from_offsets[row] - prev_from_offset);
+            prev_from_offset = from_offsets[row];
+            std::span<const N> to(column_to->getData().begin() + prev_to_offset, to_offsets[row] - prev_to_offset);
+            prev_to_offset = to_offsets[row];
+
+            res_values[row] = levenshteinDistance<N>(from, to);
+        }
+        return true;
+    }
+
+    template<typename Result>
+    void levenshteinGeneric(std::vector<const ColumnArray *> columns, Result::Container & res_values) const
+    {
+        if (T::arguments == 4)
+        {
+            if constexpr (std::is_same_v<Result, ColumnFloat64>)
+            {
+                if (!(
+                    levenshteinWeightedGeneric<UInt8>(columns, res_values)
+                    || levenshteinWeightedGeneric<UInt16>(columns, res_values)
+                    || levenshteinWeightedGeneric<UInt32>(columns, res_values)
+                    || levenshteinWeightedGeneric<UInt64>(columns, res_values)
+                    || levenshteinWeightedGeneric<UInt128>(columns, res_values)
+                    || levenshteinWeightedGeneric<UInt256>(columns, res_values)
+                    || levenshteinWeightedGeneric<Int8>(columns, res_values)
+                    || levenshteinWeightedGeneric<Int16>(columns, res_values)
+                    || levenshteinWeightedGeneric<Int32>(columns, res_values)
+                    || levenshteinWeightedGeneric<Int64>(columns, res_values)
+                    || levenshteinWeightedGeneric<Int128>(columns, res_values)
+                    || levenshteinWeightedGeneric<Int256>(columns, res_values)
+                    || levenshteinWeightedGeneric<Float32>(columns, res_values)
+                    || levenshteinWeightedGeneric<Float64>(columns, res_values)))
+                    throw Exception(
+                        ErrorCodes::LOGICAL_ERROR,
+                        "Unexpected code branch of function {}. No fitting levenshteinWeightedGeneric",
+                        getName());
+                return;
+            }
+            throw Exception(
+                ErrorCodes::LOGICAL_ERROR,
+                "Unexpected code branch of function {}. Wrong expected result type in levenshteinGeneric",
+                getName());
+        }
+
+        const ColumnArray * column_from = columns[0];
+        const ColumnArray * column_to = columns[1];
+        for (size_t row = 0; row < column_from->size(); row++)
+        {
+            // Effective Levenshtein realization from Common/levenshteinDistance
+            Array from = (*column_from)[row].safeGet<Array>();
+            Array to = (*column_to)[row].safeGet<Array>();
+            res_values[row] = levenshteinDistance<Field>(from, to);
+        }
+    }
+
+    template <typename N, typename W>
+    bool levenshteinWeightedString(std::vector<const ColumnArray *> columns, ColumnFloat64::Container & res_values) const
+    {
+        const N * from_data = checkAndGetColumn<N>(&columns[0]->getData());
+        const N * to_data = checkAndGetColumn<N>(&columns[1]->getData());
+        if (!from_data || !to_data)
+            return false;
+        const ColumnArray::Offsets & from_offsets = columns[0]->getOffsets();
+        ColumnArray::Offset prev_from_offset = 0;
+
+        const ColumnArray::Offsets & to_offsets = columns[1]->getOffsets();
+        ColumnArray::Offset prev_to_offset = 0;
+
+        const ColumnVector<W> * column_from_weights = checkAndGetColumn<ColumnVector<W>>(&columns[2]->getData());
+        const ColumnVector<W> * column_to_weights = checkAndGetColumn<ColumnVector<W>>(&columns[3]->getData());
+        if (!column_from_weights || !column_to_weights)
+            return false;
+
+        const ColumnArray::Offsets & from_weights_offsets = columns[2]->getOffsets();
+        ColumnArray::Offset prev_from_weights_offset = 0;
+
+        const ColumnArray::Offsets & to_weights_offsets = columns[3]->getOffsets();
+        ColumnArray::Offset prev_to_weights_offset = 0;
+
+        const auto extract_array = [](const N * data, size_t prev_offset, size_t count)
+        {
+            std::vector<StringRef> temp;
+            temp.reserve(count);
+            for (size_t j = 0; j < count; ++j) { temp.emplace_back(data->getDataAt(prev_offset + j)); }
+            return temp;
+        };
+
+        for (size_t row = 0; row < columns[0]->size(); row++)
+        {
+            const size_t m = from_offsets[row] - prev_from_offset;
+            const size_t n = to_offsets[row] - prev_to_offset;
+            const std::vector<StringRef> from = extract_array(from_data, prev_from_offset, m);
+            const std::vector<StringRef> to = extract_array(to_data, prev_to_offset, n);
+            prev_from_offset = from_offsets[row];
+            prev_to_offset = to_offsets[row];
+
+            std::span<const W> from_weights(column_from_weights->getData().begin() + prev_from_weights_offset, from_weights_offsets[row] - prev_from_weights_offset);
+            prev_from_weights_offset = from_weights_offsets[row];
+            std::span<const W> to_weights(column_to_weights->getData().begin() + prev_to_weights_offset, to_weights_offsets[row] - prev_to_weights_offset);
+            prev_to_weights_offset = to_weights_offsets[row];
+
+            res_values[row] = static_cast<Float64>(DB::levenshteinDistanceWeighted<StringRef, W>(from, to, from_weights, to_weights));
+        }
+        return true;
+    }
+
+    template <typename N, typename W>
+    bool levenshteinWeightedNumber(std::vector<const ColumnArray *> columns, ColumnFloat64::Container & res_values) const
+    {
+        const ColumnVectorOrDecimal<N> * column_from = checkAndGetColumn<ColumnVectorOrDecimal<N>>(&columns[0]->getData());
+        const ColumnVectorOrDecimal<N> * column_to = checkAndGetColumn<ColumnVectorOrDecimal<N>>(&columns[1]->getData());
+        if (!column_from || !column_to)
+            // just to be on the safe side, it's already checked
+            return false;
+
+        const ColumnArray::Offsets & from_offsets = columns[0]->getOffsets();
+        ColumnArray::Offset prev_from_offset = 0;
+        const ColumnArray::Offsets & to_offsets = columns[1]->getOffsets();
+        ColumnArray::Offset prev_to_offset = 0;
+
+        const ColumnVector<W> * column_from_weights = checkAndGetColumn<ColumnVector<W>>(&columns[2]->getData());
+        const ColumnVector<W> * column_to_weights = checkAndGetColumn<ColumnVector<W>>(&columns[3]->getData());
+        if (!column_from_weights || !column_to_weights)
+            return false;
+
+        const ColumnArray::Offsets & from_weights_offsets = columns[2]->getOffsets();
+        ColumnArray::Offset prev_from_weights_offset = 0;
+        const ColumnArray::Offsets & to_weights_offsets = columns[3]->getOffsets();
+        ColumnArray::Offset prev_to_weights_offset = 0;
+
+        for (size_t row = 0; row < columns[0]->size(); row++)
+        {
+            std::span<const N> from(column_from->getData().begin() + prev_from_offset, from_offsets[row] - prev_from_offset);
+            prev_from_offset = from_offsets[row];
+            std::span<const N> to(column_to->getData().begin() + prev_to_offset, to_offsets[row] - prev_to_offset);
+            prev_to_offset = to_offsets[row];
+
+            std::span<const W> from_weights(column_from_weights->getData().begin() + prev_from_weights_offset, from_weights_offsets[row] - prev_from_weights_offset);
+            prev_from_weights_offset = from_weights_offsets[row];
+            std::span<const W> to_weights(column_to_weights->getData().begin() + prev_to_weights_offset, to_weights_offsets[row] - prev_to_weights_offset);
+            prev_to_weights_offset = to_weights_offsets[row];
+
+            res_values[row] = static_cast<Float64>(DB::levenshteinDistanceWeighted<N, W>(from, to, from_weights, to_weights));
+        }
+        return true;
+    }
+
+    template<typename W>
+    bool levenshteinWeightedGeneric(std::vector<const ColumnArray *> columns, ColumnFloat64::Container & res_values) const
+    {
+        const ColumnArray * column_from = columns[0];
+        const ColumnArray * column_to = columns[1];
+
+        const ColumnVector<W> * column_from_weights = checkAndGetColumn<ColumnVector<W>>(&columns[2]->getData());
+        const ColumnVector<W> * column_to_weights = checkAndGetColumn<ColumnVector<W>>(&columns[3]->getData());
+        if (!column_from_weights || !column_to_weights)
+            return false;
+        const ColumnArray::Offsets & from_weights_offsets = columns[2]->getOffsets();
+        ColumnArray::Offset prev_from_weights_offset = 0;
+
+        const ColumnArray::Offsets & to_weights_offsets = columns[3]->getOffsets();
+        ColumnArray::Offset prev_to_weights_offset = 0;
+
+        for (size_t row = 0; row < column_from->size(); row++)
+        {
+            // Effective Levenshtein realization from Common/levenshteinDistance
+            Array from = (*column_from)[row].safeGet<Array>();
+            Array to = (*column_to)[row].safeGet<Array>();
+
+            std::span<const W> from_weights(column_from_weights->getData().begin() + prev_from_weights_offset, from_weights_offsets[row] - prev_from_weights_offset);
+            prev_from_weights_offset = from_weights_offsets[row];
+            std::span<const W> to_weights(column_to_weights->getData().begin() + prev_to_weights_offset, to_weights_offsets[row] - prev_to_weights_offset);
+            prev_to_weights_offset = to_weights_offsets[row];
+
+            res_values[row] = static_cast<Float64>(DB::levenshteinDistanceWeighted<Field, W>(from, to, from_weights, to_weights));
+        }
+        return true;
+    }
+
+    ColumnPtr levenshteinImpl(std::vector<const ColumnArray *> columns) const
+    {
+        auto res = ColumnUInt32::create();
+        ColumnUInt32::Container & res_values = res->getData();
+        res_values.resize(columns[0]->size());
+        if (levenshteinNumber<UInt8, ColumnUInt32>(columns, res_values) || levenshteinNumber<UInt16, ColumnUInt32>(columns, res_values)
+            || levenshteinNumber<UInt32, ColumnUInt32>(columns, res_values) || levenshteinNumber<UInt64, ColumnUInt32>(columns, res_values)
+            || levenshteinNumber<UInt128, ColumnUInt32>(columns, res_values) || levenshteinNumber<UInt256, ColumnUInt32>(columns, res_values)
+            || levenshteinNumber<Int8, ColumnUInt32>(columns, res_values) || levenshteinNumber<Int16, ColumnUInt32>(columns, res_values)
+            || levenshteinNumber<Int32, ColumnUInt32>(columns, res_values) || levenshteinNumber<Int64, ColumnUInt32>(columns, res_values)
+            || levenshteinNumber<Int128, ColumnUInt32>(columns, res_values) || levenshteinNumber<Int256, ColumnUInt32>(columns, res_values)
+            || levenshteinNumber<Float32, ColumnUInt32>(columns, res_values) || levenshteinNumber<Float64, ColumnUInt32>(columns, res_values)
+            || levenshteinNumber<Decimal32, ColumnUInt32>(columns, res_values) || levenshteinNumber<Decimal64, ColumnUInt32>(columns, res_values)
+            || levenshteinNumber<Decimal128, ColumnUInt32>(columns, res_values) || levenshteinNumber<Decimal256, ColumnUInt32>(columns, res_values)
+            || levenshteinNumber<DateTime64, ColumnUInt32>(columns, res_values)
+            || levenshteinString<ColumnString, ColumnUInt32>(columns, res_values) || levenshteinString<ColumnFixedString, ColumnUInt32>(columns, res_values))
+            return res;
+        levenshteinGeneric<ColumnUInt32>(columns, res_values);
+        return res;
+    }
+
+    ColumnPtr weightedLevenshteinImpl(std::vector<const ColumnArray *> columns) const
+    {
+        for (size_t i = 0; i < 2; i++)
+        {
+            const ColumnArray * hs_column = columns[i];
+            const ColumnArray * weights_column = columns[i+2];
+            for (size_t row = 0; row < hs_column->size(); row++)
+            {
+                Array hs = (*hs_column)[row].safeGet<Array>();
+                Array weights = (*weights_column)[row].safeGet<Array>();
+                if (hs.size() != weights.size())
+                    throw Exception(
+                        ErrorCodes::SIZES_OF_ARRAYS_DONT_MATCH,
+                        "Arguments {} ({}, size {}) and {} ({}, size {}) of function {} must be arrays of the same size",
+                        toString(i + 1),
+                        hs_column->getName(),
+                        hs.size(),
+                        toString(i + 3),
+                        weights_column->getName(),
+                        weights.size(),
+                        getName());
+            }
+        }
+        auto res = ColumnFloat64::create();
+        ColumnFloat64::Container & res_values = res->getData();
+        res_values.resize(columns[0]->size());
+        if (levenshteinNumber<UInt8, ColumnFloat64>(columns, res_values) || levenshteinNumber<UInt16, ColumnFloat64>(columns, res_values)
+            || levenshteinNumber<UInt32, ColumnFloat64>(columns, res_values) || levenshteinNumber<UInt64, ColumnFloat64>(columns, res_values)
+            || levenshteinNumber<UInt128, ColumnFloat64>(columns, res_values) || levenshteinNumber<UInt256, ColumnFloat64>(columns, res_values)
+            || levenshteinNumber<Int8, ColumnFloat64>(columns, res_values) || levenshteinNumber<Int16, ColumnFloat64>(columns, res_values)
+            || levenshteinNumber<Int32, ColumnFloat64>(columns, res_values) || levenshteinNumber<Int64, ColumnFloat64>(columns, res_values)
+            || levenshteinNumber<Int128, ColumnFloat64>(columns, res_values) || levenshteinNumber<Int256, ColumnFloat64>(columns, res_values)
+            || levenshteinNumber<Float32, ColumnFloat64>(columns, res_values) || levenshteinNumber<Float64, ColumnFloat64>(columns, res_values)
+            || levenshteinNumber<Decimal32, ColumnFloat64>(columns, res_values) || levenshteinNumber<Decimal64, ColumnFloat64>(columns, res_values)
+            || levenshteinNumber<Decimal128, ColumnFloat64>(columns, res_values) || levenshteinNumber<Decimal256, ColumnFloat64>(columns, res_values)
+            || levenshteinNumber<DateTime64, ColumnFloat64>(columns, res_values)
+            || levenshteinString<ColumnString, ColumnFloat64>(columns, res_values) || levenshteinString<ColumnFixedString, ColumnFloat64>(columns, res_values))
+            return res;
+        levenshteinGeneric<ColumnFloat64>(columns, res_values);
+        return res;
+    }
+
+    template <typename W>
+    bool similarity(std::vector<const ColumnArray *> columns, ColumnPtr distance, ColumnFloat64::Container & res_values) const
+    {
+        const ColumnVector<W> * column_from_weights = checkAndGetColumn<ColumnVector<W>>(&columns[2]->getData());
+        const ColumnVector<W> * column_to_weights = checkAndGetColumn<ColumnVector<W>>(&columns[3]->getData());
+        if (!column_from_weights || !column_to_weights)
+            return false;
+
+        const ColumnArray::Offsets & from_weights_offsets = columns[2]->getOffsets();
+        ColumnArray::Offset prev_from_weights_offset = 0;
+        const ColumnArray::Offsets & to_weights_offsets = columns[3]->getOffsets();
+        ColumnArray::Offset prev_to_weights_offset = 0;
+
+        for (size_t row = 0; row < distance->size(); row++)
+        {
+            std::span<const W> from_weights(column_from_weights->getData().begin() + prev_from_weights_offset, from_weights_offsets[row] - prev_from_weights_offset);
+            prev_from_weights_offset = from_weights_offsets[row];
+            std::span<const W> to_weights(column_to_weights->getData().begin() + prev_to_weights_offset, to_weights_offsets[row] - prev_to_weights_offset);
+            prev_to_weights_offset = to_weights_offsets[row];
+
+            if (distance->getFloat64(row) == 0)
+            {
+                res_values[row] = 1.0;
+                continue;
+            }
+            W weights_sum = std::accumulate(from_weights.begin(), from_weights.end(), W{}) +
+                            std::accumulate(to_weights.begin(), to_weights.end(), W{});
+            if (weights_sum == 0)
+            {
+                res_values[row] = 1.0;
+                continue;
+            }
+            res_values[row] = 1.0 - (distance->getFloat64(row) / weights_sum);
+        }
+        return true;
+    }
+};
+
+struct SimpleLevenshtein
+{
+    static constexpr auto name{"arrayLevenshteinDistance"};
+    static constexpr size_t arguments = 2;
+};
+
+template <>
+DataTypePtr FunctionArrayLevenshtein<SimpleLevenshtein>::getReturnTypeImpl(const ColumnsWithTypeAndName & arguments) const
+{
+    FunctionArgumentDescriptors args_descriptors;
+    args_descriptors = FunctionArgumentDescriptors{
+        {"from", static_cast<FunctionArgumentDescriptor::TypeValidator>(&isArray), nullptr, "Array"},
+        {"to", static_cast<FunctionArgumentDescriptor::TypeValidator>(&isArray), nullptr, "Array"},
+    };
+    validateFunctionArguments(*this, arguments, args_descriptors);
+    return std::make_shared<DataTypeUInt32>();
+}
+
+
+template <>
+ColumnPtr FunctionArrayLevenshtein<SimpleLevenshtein>::execute(std::vector<const ColumnArray *> columns) const
+{
+    return levenshteinImpl(columns);
+}
+
+struct Weighted
+{
+    static constexpr auto name{"arrayLevenshteinDistanceWeighted"};
+    static constexpr size_t arguments = 4;
+};
+
+template <>
+ColumnPtr FunctionArrayLevenshtein<Weighted>::execute(std::vector<const ColumnArray *> columns) const
+{
+    return weightedLevenshteinImpl(columns);
+}
+
+struct Similarity
+{
+    static constexpr auto name{"arraySimilarity"};
+    static constexpr size_t arguments = 4;
+};
+
+template <>
+ColumnPtr FunctionArrayLevenshtein<Similarity>::execute(std::vector<const ColumnArray *> columns) const
+{
+    ColumnPtr distance = weightedLevenshteinImpl(columns);
+    auto result = ColumnFloat64::create();
+    ColumnFloat64::Container & res_values = result->getData();
+    res_values.resize(distance->size());
+    if (!(
+        similarity<UInt8>(columns, distance, res_values) || similarity<UInt16>(columns, distance, res_values)
+        || similarity<UInt16>(columns, distance, res_values) || similarity<UInt64>(columns, distance, res_values)
+        || similarity<UInt128>(columns, distance, res_values) || similarity<UInt256>(columns, distance, res_values)
+        || similarity<Int8>(columns, distance, res_values) || similarity<Int16>(columns, distance, res_values)
+        || similarity<Int16>(columns, distance, res_values) || similarity<Int64>(columns, distance, res_values)
+        || similarity<Int128>(columns, distance, res_values) || similarity<Int256>(columns, distance, res_values)
+        || similarity<Float32>(columns, distance, res_values) || similarity<Float64>(columns, distance, res_values)))
+        throw Exception(
+                ErrorCodes::LOGICAL_ERROR,
+                "Unexpected code branch of function {}. No matching column types for {} and {}",
+                getName(),
+                columns[2]->getName(),
+                columns[3]->getName());
+    return result;
+}
+
+REGISTER_FUNCTION(ArrayLevenshtein)
+{
+    factory.registerFunction<FunctionArrayLevenshtein<SimpleLevenshtein>>(
+        {.description = R"(
+Calculates Levenshtein distance for two arrays.
+)",
+         .syntax{"arrayLevenshteinDistance(from, to)"},
+         .arguments{{"from", "first array"}, {"to", "second array"}},
+         .returned_value{"Levenshtein distance between the first and the second arrays"},
+         .examples{{{
+             "Query",
+             "SELECT arrayLevenshteinDistance([1, 2, 4], [1, 2, 3])",
+             R"(
+┌─arrayLevenshteinDistance([1, 2, 4], [1, 2, 3])─┐
+│                                              1 │
+└────────────────────────────────────────────────┘
+)",
+         }}},
+         .category{"Arrays"}});
+
+    factory.registerFunction<FunctionArrayLevenshtein<Weighted>>(
+        {.description = R"(
+Calculates Levenshtein distance for two arrays with custom weights for each element. Number of elements for array and its weights should match
+)",
+         .syntax{"arrayLevenshteinDistanceWeighted(from, to, from_weights, to_weights)"},
+         .arguments{
+             {"from", "first array"},
+             {"to", "second array"},
+             {"from_weights", "weights for the first array"},
+             {"to_weights", "weights for the second array"},
+         },
+         .returned_value{"Levenshtein distance between the first and the second arrays with custom weights for each element"},
+         .examples{{{
+            "Query",
+            "SELECT arrayLevenshteinDistanceWeighted(['A', 'B', 'C'], ['A', 'K', 'L'], [1.0, 2, 3], [3.0, 4, 5])",
+            R"(
+┌─arrayLevenshteinDistanceWeighted(['A', 'B', 'C'], ['A', 'K', 'L'], [1.0, 2, 3], [3.0, 4, 5])─┐
+│                                                                                           14 │
+└──────────────────────────────────────────────────────────────────────────────────────────────┘
+)",
+         }}},
+         .category{"Arrays"}});
+
+    factory.registerFunction<FunctionArrayLevenshtein<Similarity>>(
+        {.description = R"(
+Calculates arrays' similarity from 0 to 1 based on weighed Levenshtein distance. Accepts the same arguments as `arrayLevenshteinDistanceWeighted` function.
+)",
+         .syntax{"arraySimilarity(from, to, from_weights, to_weights)"},
+         .arguments{
+             {"from", "first array"},
+             {"to", "second array"},
+             {"from_weights", "weights for the first array"},
+             {"to_weights", "weights for the second array"},
+         },
+         .returned_value{"Similarity of two arrays based on the weighted Levenshtein distance"},
+         .examples{{{
+            "Query",
+            "SELECT arraySimilarity(['A', 'B', 'C'], ['A', 'K', 'L'], [1.0, 2, 3], [3.0, 4, 5])",
+            R"(
+┌─arraySimilarity(['A', 'B', 'C'], ['A', 'K', 'L'], [1.0, 2, 3], [3.0, 4, 5])─┐
+│                                                          0.2222222222222222 │
+└─────────────────────────────────────────────────────────────────────────────┘
+)",
+         }}},
+         .category{"Arrays"}});
+
+}
+}
diff --git a/src/Functions/array/arrayZip.cpp b/src/Functions/array/arrayZip.cpp
index 822d0229d329..0d1a6408dabc 100644
--- a/src/Functions/array/arrayZip.cpp
+++ b/src/Functions/array/arrayZip.cpp
@@ -49,7 +49,7 @@ class FunctionArrayZip : public IFunction
                     "Argument {} of function {} must be array. Found {} instead.",
                     toString(index + 1),
                     getName(),
-                    arguments[0].type->getName());
+                    arguments[index].type->getName());
 
             auto nested_type = array_type->getNestedType();
             if constexpr (allow_unaligned)
diff --git a/tests/queries/0_stateless/03369_function_arrayLevenshtein.reference b/tests/queries/0_stateless/03369_function_arrayLevenshtein.reference
new file mode 100644
index 000000000000..3df86ec2c266
--- /dev/null
+++ b/tests/queries/0_stateless/03369_function_arrayLevenshtein.reference
@@ -0,0 +1,35 @@
+0
+1
+2
+2
+4
+4
+0
+
+1	1	1	1	1	1
+
+2
+2
+6
+14
+0
+3
+0
+10
+14
+4
+4
+
+2	2	2	2	2	2
+
+0.8
+0.8
+0.4
+0.22222
+1
+0
+1
+0
+0.33333
+0.77778
+0.71429
diff --git a/tests/queries/0_stateless/03369_function_arrayLevenshtein.sql b/tests/queries/0_stateless/03369_function_arrayLevenshtein.sql
new file mode 100644
index 000000000000..b32d5e74616b
--- /dev/null
+++ b/tests/queries/0_stateless/03369_function_arrayLevenshtein.sql
@@ -0,0 +1,62 @@
+-- arrayLevenshteinDistance
+CREATE TABLE simple_levenshtein (lhs Array(UInt8), rhs Array(UInt8)) ENGINE MergeTree ORDER BY tuple();
+INSERT INTO simple_levenshtein VALUES
+  ([1, 2, 3, 4], [1, 2, 3, 4]),
+  ([1, 2, 3, 4], [1, 3, 3, 4]),
+  ([1, 2, 3, 4], [1, 3, 2, 4]),
+  ([1, 4], [1, 2, 3, 4]),
+  ([1, 2, 3, 4], []),
+  ([], [1, 3, 2, 4]),
+  ([], []);
+SELECT arrayLevenshteinDistance(lhs, rhs) FROM simple_levenshtein;
+SELECT '';
+
+-- arrayLevenshteinDistance for different types
+SELECT arrayLevenshteinDistance(['1', '2'], ['1']),
+  arrayLevenshteinDistance([toFixedString('1', 16), toFixedString('2', 16)], [toFixedString('1', 16)]),
+  arrayLevenshteinDistance([toUInt16(1)], [toUInt16(2), 1]),
+  arrayLevenshteinDistance([toFloat32(1.1), 2], [toFloat32(1.1)]),
+  arrayLevenshteinDistance([toFloat64(1.1), 2], [toFloat64(1.1)]),
+  arrayLevenshteinDistance([toDate('2025-01-01'), toDate('2025-01-02')], [toDate('2025-01-01')]);
+SELECT '';
+
+-- arrayLevenshteinDistanceWeighted
+CREATE TABLE weighted_levenshtein (lhs Array(String), rhs Array(String), lhs_weights Array(Float64), rhs_weights Array(Float64)) ENGINE MergeTree ORDER BY tuple();
+INSERT INTO weighted_levenshtein VALUES
+  (['A', 'B', 'C'], ['A', 'C'], [1, 2, 3], [1, 3]),
+  (['A', 'C'], ['A', 'B', 'C'], [1, 3], [1, 2, 3]),
+  (['A', 'B'], ['A', 'C'], [1, 2], [3, 4]),
+  (['A', 'B', 'C'], ['A', 'K', 'L'], [1, 2, 3], [3, 4, 5]),
+  ([], [], [], []),
+  (['A', 'B'], [], [1, 2], []),
+  (['A', 'B'], ['A', 'B'], [1, 2], [2, 1]),
+  (['A', 'B'], ['C', 'D'], [1, 2], [3, 4]),
+  (['A', 'B', 'C'], ['C', 'B', 'A'], [1, 2, 3], [4, 5, 6]),
+  (['A', 'B'], ['C', 'A', 'B'], [1, 2], [4, 5, 6]),
+  (['A', 'B', 'C', 'D', 'E', 'F', 'G'], ['A', 'B', 'X', 'D', 'E', 'Y', 'G'], [1, 1, 1, 1, 1, 1, 1], [1, 1, 1, 1, 1, 1, 1]);
+SELECT arrayLevenshteinDistanceWeighted(lhs, rhs, lhs_weights, rhs_weights) FROM weighted_levenshtein;
+SELECT '';
+
+-- arrayLevenshteinDistance for different types
+SELECT arrayLevenshteinDistanceWeighted(['1', '2'], ['1'], [1., 2], [1.]),
+  arrayLevenshteinDistanceWeighted([toFixedString('1', 16), toFixedString('2', 16)], [toFixedString('1', 16)], [1., 2], [1.]),
+  arrayLevenshteinDistanceWeighted([toUInt16(1)], [toUInt16(2), 1], [1.], [2., 1]),
+  arrayLevenshteinDistanceWeighted([toFloat32(1.1), 2], [toFloat32(1.1)], [1., 2], [1.]),
+  arrayLevenshteinDistanceWeighted([toFloat64(1.1), 2], [toFloat64(1.1)], [1., 2], [1.]),
+  arrayLevenshteinDistanceWeighted([toDate('2025-01-01'), toDate('2025-01-02')], [toDate('2025-01-01')], [1., 2], [1.]);
+SELECT '';
+
+-- arraySimilarity
+SELECT round(arraySimilarity(lhs, rhs, lhs_weights, rhs_weights), 5) FROM weighted_levenshtein;
+
+-- errors NUMBER_OF_ARGUMENTS_DOESNT_MATCH
+SELECT arrayLevenshteinDistance(lhs, rhs, lhs_weights, rhs_weights) FROM weighted_levenshtein; -- { serverError NUMBER_OF_ARGUMENTS_DOESNT_MATCH }
+SELECT arrayLevenshteinDistanceWeighted(lhs, rhs) FROM simple_levenshtein; -- { serverError NUMBER_OF_ARGUMENTS_DOESNT_MATCH }
+SELECT round(arraySimilarity(lhs, rhs), 5) FROM simple_levenshtein; -- { serverError NUMBER_OF_ARGUMENTS_DOESNT_MATCH }
+-- errors ILLEGAL_TYPE_OF_ARGUMENT
+SELECT arrayLevenshteinDistance(1, [1]); -- { serverError ILLEGAL_TYPE_OF_ARGUMENT }
+SELECT arrayLevenshteinDistanceWeighted([1], 1, [1, 2], [1]); -- { serverError ILLEGAL_TYPE_OF_ARGUMENT }
+SELECT arraySimilarity([1, 2], 1, [1, 2], [1]); -- { serverError ILLEGAL_TYPE_OF_ARGUMENT }
+--
+SELECT arrayLevenshteinDistanceWeighted([1, 2], [1], [1., 2], [1., 2]); -- { serverError SIZES_OF_ARRAYS_DONT_MATCH }
+SELECT arraySimilarity([1, 2], [1], [1., 2], [1., 2]); -- { serverError SIZES_OF_ARRAYS_DONT_MATCH }


Reviews:
user: login: Felixoid
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-09T13:20:27Z
user: login: Felixoid
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-09T15:51:57Z
user: login: Felixoid
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-09T18:46:54Z
user: login: Felixoid
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-10T12:07:40Z
user: login: Felixoid
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-10T13:51:45Z
user: login: Michicosun
body: About `Levenshtein` function calculation - let's simplify interface to calculate only single row of answer and move it outside of the Function class scope. Also I assume we need to support custom implementations for `numeric` and `string-like` Arrays, because the current implementation with pure `Fields` is not efficient, but probably it is acceptable for other types.

Example about type dispatch https://github.com/ClickHouse/ClickHouse/blob/79c95f1d4a5c3b85fe89395e580ae42cbaaef9b3/src/Functions/seriesDecomposeSTL.cpp#L105
state: CHANGES_REQUESTED
author_association: MEMBER
submitted_at: 2025-03-10T19:09:05Z
user: login: Felixoid
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-10T19:59:39Z
user: login: Felixoid
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-11T14:48:00Z
user: login: Felixoid
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-11T14:48:34Z
user: login: Michicosun
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-24T09:38:17Z
user: login: Michicosun
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-24T11:35:57Z
user: login: Felixoid
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-24T21:26:05Z
user: login: Felixoid
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-24T23:33:45Z
user: login: Felixoid
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-26T06:46:42Z
user: login: Felixoid
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-26T08:15:13Z
user: login: Felixoid
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-26T08:15:34Z
user: login: Michicosun
state: CHANGES_REQUESTED
author_association: MEMBER
submitted_at: 2025-03-27T15:24:28Z
user: login: Felixoid
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-27T15:49:16Z
user: login: Michicosun
state: APPROVED
author_association: MEMBER
submitted_at: 2025-03-27T16:32:19Z
user: login: Felixoid
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-28T11:57:35Z

Review comments:
diff_hunk: @@ -0,0 +1,368 @@
+#include <Columns/ColumnArray.h>
+#include <Columns/ColumnNullable.h>
+#include <Columns/ColumnTuple.h>
+#include <Common/PODArray.h>
+#include <Common/iota.h>
+#include <DataTypes/DataTypeArray.h>
+#include <DataTypes/DataTypeNullable.h>
+#include <DataTypes/DataTypeTuple.h>
+#include <DataTypes/DataTypesNumber.h>
+#include <Functions/FunctionFactory.h>
+#include <Functions/FunctionHelpers.h>
+#include <IO/WriteHelpers.h>
+
+#include <numeric>
+
+namespace DB
+{
+
+namespace ErrorCodes
+{
+extern const int ILLEGAL_TYPE_OF_ARGUMENT;
+extern const int LOGICAL_ERROR;
+extern const int SIZES_OF_ARRAYS_DONT_MATCH;
+}
+
+/// arrayLevenshtein([1,2,3,4], [1,3,2,4]) = 2
+/// arrayLevenshteinWeighted([1,2,3,4], [1,3,2,4]) = 2
+template <typename T>
+class FunctionArrayLevenshtein : public IFunction
+{
+public:
+    static constexpr auto name = T::name;
+    static FunctionPtr create(ContextPtr) { return std::make_shared<FunctionArrayLevenshtein<T>>(); }
+
+    String getName() const override { return name; }
+
+    bool isVariadic() const override { return false; }
+    size_t getNumberOfArguments() const override { return T::arguments; }
+    bool useDefaultImplementationForConstants() const override { return true; }
+
+    bool isSuitableForShortCircuitArgumentsExecution(const DataTypesWithConstInfo & /*arguments*/) const override { return false; }
+
+    DataTypePtr getReturnTypeImpl(const ColumnsWithTypeAndName &) const override
+    {
+        throw Exception(
+            ErrorCodes::LOGICAL_ERROR,
+            "Unknown function {}. "
+            "Supported names: 'arrayLevenshtein', 'arrayLevenshteinWeighted', 'arraySimilarity'",
+            T::name);
+    }
+
+    ColumnPtr executeImpl(
+        const ColumnsWithTypeAndName & arguments, const DataTypePtr & /*result_type*/, size_t input_rows_count
+    ) const override
+    {
+        size_t num_arguments = arguments.size();
+
+        Columns holders(num_arguments);
+        std::vector<const ColumnArray *> columns(num_arguments);
+
+        for (size_t i = 0; i < num_arguments; ++i)
+        {
+            holders[i] = arguments[i].column->convertToFullColumnIfConst();
+            if (holders[i]->size() != input_rows_count)
+                throw Exception(
+                    ErrorCodes::LOGICAL_ERROR,
+                    "Function {} has unequal number of rows in columns: "
+                    "expected {}, got {} for column {}",
+                    getName(),
+                    input_rows_count,
+                    holders[i]->size(),
+                    holders[i]->getName());
+            columns[i] = assert_cast<const ColumnArray*>(holders[i].get());
+        }
+        switch (T::impl)
+        {
+            case 0:
+                return simpleLevenshteinImpl(columns);
+            case 1:
+                return weightedLevenshteinImpl(columns, false);
+            case 2:
+                return weightedLevenshteinImpl(columns, true);
+        }
+    }
+private:
+    DataTypes checkArguments(const ColumnsWithTypeAndName & arguments) const
+    {
+        DataTypes arguments_types;
+        // First two arguments lhs and rhs are always arrays
+        for (size_t index = 0; index < arguments.size(); ++index)
+        {
+            const DataTypeArray * array_type = checkAndGetDataType<DataTypeArray>(arguments[index].type.get());
+
+            if (!array_type)
+                throw Exception(
+                    ErrorCodes::ILLEGAL_TYPE_OF_ARGUMENT,
+                    "Argument {} of function {} must be array. Found {} instead.",
+                    toString(index + 1),
+                    getName(),
+                    arguments[index].type->getName());
+
+            auto nested_type = array_type->getNestedType();
+            arguments_types.emplace_back(nested_type);
+        }
+
+        assert(arguments.size() == T::arguments);
+        return arguments_types;
+    }
+
+    ColumnPtr simpleLevenshteinImpl(std::vector<const ColumnArray *> columns) const
+    {
+        const ColumnArray * column_lhs = columns[0];
+        const ColumnArray * column_rhs = columns[1];
+        auto res = ColumnUInt32::create();
+        ColumnUInt32::Container & res_values = res->getData();
+        res_values.resize(column_lhs->size());
+        for (size_t row = 0; row < column_lhs->size(); row++)
+        {
+            // Effective Levenshtein realization from Common/levenshteinDistance
+            Array lhs = (*column_lhs)[row].safeGet<Array>();
+            Array rhs = (*column_rhs)[row].safeGet<Array>();
+            const size_t m = lhs.size();
+            const size_t n = rhs.size();
+            if (m==0 || n==0)
+            {
+                res_values[row] = static_cast<UInt32>(std::max(m, n));
+                continue;
+            }
+            PODArrayWithStackMemory<size_t, 32> v0(n + 1);
+
+            iota(v0.data() + 1, n, size_t(1));
+
+            for (size_t j = 1; j <= m; ++j)
+            {
+                v0[0] = j;
+                size_t prev = j - 1;
+                for (size_t i = 1; i <= n; ++i)
+                {
+                    size_t old = v0[i];
+                    v0[i] = std::min(prev + (lhs[j - 1] != rhs[i - 1]),
+                                     std::min(v0[i - 1], v0[i]) + 1);
+                    prev = old;
+                }
+            }
+            res_values[row] = static_cast<UInt32>(v0[n]);
+        }
+        return res;
+    }
+
+    ColumnPtr weightedLevenshteinImpl(std::vector<const ColumnArray *> columns, bool similarity) const
+    {
+        for (size_t i = 0; i < 2; i++)
+        {
+            const ColumnArray * hs_column = columns[i];
+            const ColumnArray * weights_column = columns[i+2];
+            for (size_t row = 0; row < hs_column->size(); row++)
+            {
+                Array hs = (*hs_column)[row].safeGet<Array>();
+                Array weights = (*weights_column)[row].safeGet<Array>();
+                if (hs.size() != weights.size())
+                    throw Exception(
+                        ErrorCodes::SIZES_OF_ARRAYS_DONT_MATCH,
+                        "Arguments {} ({}, size {}) and {} ({}, size {}) of function {} must be arrays of the same size.",
+                        toString(i + 1),
+                        hs_column->getName(),
+                        hs.size(),
+                        toString(i + 3),
+                        weights_column->getName(),
+                        weights.size(),
+                        getName());
+            }
+        }
+        auto res = ColumnFloat64::create();
+        ColumnFloat64::Container & res_values = res->getData();
+        res_values.resize(columns[0]->size());
+        for (size_t row = 0; row < columns[0]->size(); row++)
+        {
+            // Levenshtein with sliding vectors and weighted elements
+            // https://www.codeproject.com/Articles/13525/Fast-memory-efficient-Levenshtein-algorithm
+            Array lhs = (*columns[0])[row].safeGet<Array>();
+            Array rhs = (*columns[1])[row].safeGet<Array>();
+
+            Array array_l_weights = (*columns[2])[row].safeGet<Array>();
+            std::vector<Float64> l_weights(array_l_weights.size());
+            for (size_t i = 0; i < array_l_weights.size(); i++)
+                l_weights[i] = array_l_weights[i].safeGet<Float64>();
+            Array array_r_weights = (*columns[3])[row].safeGet<Array>();
+            std::vector<Float64> r_weights(array_r_weights.size());
+            for (size_t j = 0; j < array_r_weights.size(); j++)
+                r_weights[j] = array_r_weights[j].safeGet<Float64>();
path: src/Functions/array/arrayLevenshtein.cpp
user: login: Felixoid
body: I am unsure if it's memory efficient or if it's better to use `safeGet` per element access. Maybe the latest commit is unnecessary.
created_at: 2025-03-09T13:20:27Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,368 @@
+#include <Columns/ColumnArray.h>
+#include <Columns/ColumnNullable.h>
+#include <Columns/ColumnTuple.h>
+#include <Common/PODArray.h>
+#include <Common/iota.h>
+#include <DataTypes/DataTypeArray.h>
+#include <DataTypes/DataTypeNullable.h>
+#include <DataTypes/DataTypeTuple.h>
+#include <DataTypes/DataTypesNumber.h>
+#include <Functions/FunctionFactory.h>
+#include <Functions/FunctionHelpers.h>
+#include <IO/WriteHelpers.h>
+
+#include <numeric>
+
+namespace DB
+{
+
+namespace ErrorCodes
+{
+extern const int ILLEGAL_TYPE_OF_ARGUMENT;
+extern const int LOGICAL_ERROR;
+extern const int SIZES_OF_ARRAYS_DONT_MATCH;
+}
+
+/// arrayLevenshtein([1,2,3,4], [1,3,2,4]) = 2
+/// arrayLevenshteinWeighted([1,2,3,4], [1,3,2,4]) = 2
+template <typename T>
+class FunctionArrayLevenshtein : public IFunction
+{
+public:
+    static constexpr auto name = T::name;
+    static FunctionPtr create(ContextPtr) { return std::make_shared<FunctionArrayLevenshtein<T>>(); }
+
+    String getName() const override { return name; }
+
+    bool isVariadic() const override { return false; }
+    size_t getNumberOfArguments() const override { return T::arguments; }
+    bool useDefaultImplementationForConstants() const override { return true; }
+
+    bool isSuitableForShortCircuitArgumentsExecution(const DataTypesWithConstInfo & /*arguments*/) const override { return false; }
+
+    DataTypePtr getReturnTypeImpl(const ColumnsWithTypeAndName &) const override
+    {
+        throw Exception(
+            ErrorCodes::LOGICAL_ERROR,
+            "Unknown function {}. "
+            "Supported names: 'arrayLevenshtein', 'arrayLevenshteinWeighted', 'arraySimilarity'",
+            T::name);
+    }
+
+    ColumnPtr executeImpl(
+        const ColumnsWithTypeAndName & arguments, const DataTypePtr & /*result_type*/, size_t input_rows_count
+    ) const override
+    {
+        size_t num_arguments = arguments.size();
+
+        Columns holders(num_arguments);
+        std::vector<const ColumnArray *> columns(num_arguments);
+
+        for (size_t i = 0; i < num_arguments; ++i)
+        {
+            holders[i] = arguments[i].column->convertToFullColumnIfConst();
+            if (holders[i]->size() != input_rows_count)
+                throw Exception(
+                    ErrorCodes::LOGICAL_ERROR,
+                    "Function {} has unequal number of rows in columns: "
+                    "expected {}, got {} for column {}",
+                    getName(),
+                    input_rows_count,
+                    holders[i]->size(),
+                    holders[i]->getName());
+            columns[i] = assert_cast<const ColumnArray*>(holders[i].get());
+        }
+        switch (T::impl)
+        {
+            case 0:
+                return simpleLevenshteinImpl(columns);
+            case 1:
+                return weightedLevenshteinImpl(columns, false);
+            case 2:
+                return weightedLevenshteinImpl(columns, true);
+        }
+    }
+private:
+    DataTypes checkArguments(const ColumnsWithTypeAndName & arguments) const
+    {
+        DataTypes arguments_types;
+        // First two arguments lhs and rhs are always arrays
+        for (size_t index = 0; index < arguments.size(); ++index)
+        {
+            const DataTypeArray * array_type = checkAndGetDataType<DataTypeArray>(arguments[index].type.get());
+
+            if (!array_type)
+                throw Exception(
+                    ErrorCodes::ILLEGAL_TYPE_OF_ARGUMENT,
+                    "Argument {} of function {} must be array. Found {} instead.",
+                    toString(index + 1),
+                    getName(),
+                    arguments[index].type->getName());
+
+            auto nested_type = array_type->getNestedType();
+            arguments_types.emplace_back(nested_type);
+        }
+
+        assert(arguments.size() == T::arguments);
+        return arguments_types;
+    }
+
+    ColumnPtr simpleLevenshteinImpl(std::vector<const ColumnArray *> columns) const
+    {
+        const ColumnArray * column_lhs = columns[0];
+        const ColumnArray * column_rhs = columns[1];
+        auto res = ColumnUInt32::create();
+        ColumnUInt32::Container & res_values = res->getData();
+        res_values.resize(column_lhs->size());
+        for (size_t row = 0; row < column_lhs->size(); row++)
+        {
+            // Effective Levenshtein realization from Common/levenshteinDistance
+            Array lhs = (*column_lhs)[row].safeGet<Array>();
+            Array rhs = (*column_rhs)[row].safeGet<Array>();
+            const size_t m = lhs.size();
+            const size_t n = rhs.size();
+            if (m==0 || n==0)
+            {
+                res_values[row] = static_cast<UInt32>(std::max(m, n));
+                continue;
+            }
+            PODArrayWithStackMemory<size_t, 32> v0(n + 1);
+
+            iota(v0.data() + 1, n, size_t(1));
+
+            for (size_t j = 1; j <= m; ++j)
+            {
+                v0[0] = j;
+                size_t prev = j - 1;
+                for (size_t i = 1; i <= n; ++i)
+                {
+                    size_t old = v0[i];
+                    v0[i] = std::min(prev + (lhs[j - 1] != rhs[i - 1]),
+                                     std::min(v0[i - 1], v0[i]) + 1);
+                    prev = old;
+                }
+            }
+            res_values[row] = static_cast<UInt32>(v0[n]);
+        }
+        return res;
+    }
+
+    ColumnPtr weightedLevenshteinImpl(std::vector<const ColumnArray *> columns, bool similarity) const
+    {
+        for (size_t i = 0; i < 2; i++)
+        {
+            const ColumnArray * hs_column = columns[i];
+            const ColumnArray * weights_column = columns[i+2];
+            for (size_t row = 0; row < hs_column->size(); row++)
+            {
+                Array hs = (*hs_column)[row].safeGet<Array>();
+                Array weights = (*weights_column)[row].safeGet<Array>();
+                if (hs.size() != weights.size())
+                    throw Exception(
+                        ErrorCodes::SIZES_OF_ARRAYS_DONT_MATCH,
+                        "Arguments {} ({}, size {}) and {} ({}, size {}) of function {} must be arrays of the same size.",
+                        toString(i + 1),
+                        hs_column->getName(),
+                        hs.size(),
+                        toString(i + 3),
+                        weights_column->getName(),
+                        weights.size(),
+                        getName());
+            }
+        }
+        auto res = ColumnFloat64::create();
+        ColumnFloat64::Container & res_values = res->getData();
+        res_values.resize(columns[0]->size());
+        for (size_t row = 0; row < columns[0]->size(); row++)
+        {
+            // Levenshtein with sliding vectors and weighted elements
+            // https://www.codeproject.com/Articles/13525/Fast-memory-efficient-Levenshtein-algorithm
+            Array lhs = (*columns[0])[row].safeGet<Array>();
+            Array rhs = (*columns[1])[row].safeGet<Array>();
+
+            Array array_l_weights = (*columns[2])[row].safeGet<Array>();
+            std::vector<Float64> l_weights(array_l_weights.size());
+            for (size_t i = 0; i < array_l_weights.size(); i++)
+                l_weights[i] = array_l_weights[i].safeGet<Float64>();
+            Array array_r_weights = (*columns[3])[row].safeGet<Array>();
+            std::vector<Float64> r_weights(array_r_weights.size());
+            for (size_t j = 0; j < array_r_weights.size(); j++)
+                r_weights[j] = array_r_weights[j].safeGet<Float64>();
path: src/Functions/array/arrayLevenshtein.cpp
user: login: Felixoid
body: Found how to make it with lambda, and improved some other things
created_at: 2025-03-09T15:51:57Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,350 @@
+#include <Columns/ColumnArray.h>
+#include <Columns/ColumnNullable.h>
+#include <Columns/ColumnTuple.h>
+#include <Common/PODArray.h>
+#include <Common/iota.h>
+#include <DataTypes/DataTypeArray.h>
+#include <DataTypes/DataTypeNullable.h>
+#include <DataTypes/DataTypeTuple.h>
+#include <DataTypes/DataTypesNumber.h>
+#include <Functions/FunctionFactory.h>
+#include <Functions/FunctionHelpers.h>
+#include <IO/WriteHelpers.h>
+
+#include <numeric>
+
+namespace DB
+{
+
+namespace ErrorCodes
+{
+extern const int ILLEGAL_TYPE_OF_ARGUMENT;
+extern const int LOGICAL_ERROR;
+extern const int NUMBER_OF_ARGUMENTS_DOESNT_MATCH;
+extern const int SIZES_OF_ARRAYS_DONT_MATCH;
+}
+
+/// arrayLevenshtein([1,2,3,4], [1,3,2,4]) = 2
+/// arrayLevenshteinWeighted([1,2,3,4], [1,3,2,4]) = 2
+template <typename T>
+class FunctionArrayLevenshtein : public IFunction
+{
+public:
+    static constexpr auto name = T::name;
+    static FunctionPtr create(ContextPtr) { return std::make_shared<FunctionArrayLevenshtein<T>>(); }
+
+    String getName() const override { return name; }
+
+    bool isVariadic() const override { return false; }
+    size_t getNumberOfArguments() const override { return T::arguments; }
+    bool useDefaultImplementationForConstants() const override { return true; }
+
+    bool isSuitableForShortCircuitArgumentsExecution(const DataTypesWithConstInfo & /*arguments*/) const override { return false; }
+
+    DataTypePtr getReturnTypeImpl(const ColumnsWithTypeAndName & arguments) const override
+    {
+        DataTypes arguments_types;
+        // First two arguments lhs and rhs are always arrays
+        for (size_t index = 0; index < arguments.size(); ++index)
+        {
+            const DataTypeArray * array_type = checkAndGetDataType<DataTypeArray>(arguments[index].type.get());
+
+            if (!array_type)
+                throw Exception(
+                    ErrorCodes::ILLEGAL_TYPE_OF_ARGUMENT,
+                    "Argument {} of function {} must be array. Found {} instead.",
+                    toString(index + 1),
+                    getName(),
+                    arguments[index].type->getName());
+
+            auto nested_type = array_type->getNestedType();
+            arguments_types.emplace_back(nested_type);
+        }
+
+        if (arguments.size() != T::arguments)
+          throw Exception(
+                  ErrorCodes::NUMBER_OF_ARGUMENTS_DOESNT_MATCH,
+                  "Function {}'s arguments number must be {}.",
+                  getName(),
+                  T::arguments);
+
+        switch (T::impl)
+        {
+            case 0:
+                return std::make_shared<DataTypeUInt32>();
+            case 1:
+            case 2:
+                for (size_t index = 2; index < 4; ++index)
+                {
+                  if (!WhichDataType(arguments_types[index]).isFloat64())
+                    throw Exception(
+                        ErrorCodes::ILLEGAL_TYPE_OF_ARGUMENT,
+                        "Argument {} of function {} must be array of Float64. Found {} instead.",
+                        toString(index + 1),
+                        getName(),
+                        arguments_types[index]->getName());
+                }
+                return std::make_shared<DataTypeFloat64>();
+        }
+        throw Exception(
+            ErrorCodes::LOGICAL_ERROR,
+            "Unknown function {}. "
+            "Supported names: 'arrayLevenshtein', 'arrayLevenshteinWeighted', 'arraySimilarity'",
+            T::name);
+    }
+
+    ColumnPtr executeImpl(
+        const ColumnsWithTypeAndName & arguments, const DataTypePtr & /*result_type*/, size_t input_rows_count
+    ) const override
+    {
+        size_t num_arguments = arguments.size();
+
+        Columns holders(num_arguments);
+        std::vector<const ColumnArray *> columns(num_arguments);
+
+        for (size_t i = 0; i < num_arguments; ++i)
+        {
+            holders[i] = arguments[i].column->convertToFullColumnIfConst();
+            if (holders[i]->size() != input_rows_count)
+                throw Exception(
+                    ErrorCodes::LOGICAL_ERROR,
+                    "Function {} has unequal number of rows in columns: "
+                    "expected {}, got {} for column {}",
+                    getName(),
+                    input_rows_count,
+                    holders[i]->size(),
+                    holders[i]->getName());
+            columns[i] = assert_cast<const ColumnArray*>(holders[i].get());
+        }
+        switch (T::impl)
+        {
+            case 0:
+                return simpleLevenshteinImpl(columns);
+            case 1:
+                return weightedLevenshteinImpl(columns, false);
+            case 2:
+                return weightedLevenshteinImpl(columns, true);
+        }
+    }
+private:
+    ColumnPtr simpleLevenshteinImpl(std::vector<const ColumnArray *> columns) const
+    {
+        const ColumnArray * column_lhs = columns[0];
+        const ColumnArray * column_rhs = columns[1];
+        auto res = ColumnUInt32::create();
+        ColumnUInt32::Container & res_values = res->getData();
+        res_values.resize(column_lhs->size());
+        for (size_t row = 0; row < column_lhs->size(); row++)
+        {
+            // Effective Levenshtein realization from Common/levenshteinDistance
+            Array lhs = (*column_lhs)[row].safeGet<Array>();
+            Array rhs = (*column_rhs)[row].safeGet<Array>();
+            const size_t m = lhs.size();
+            const size_t n = rhs.size();
+            if (m==0 || n==0)
+            {
+                res_values[row] = static_cast<UInt32>(std::max(m, n));
+                continue;
+            }
+            PODArrayWithStackMemory<size_t, 32> v0(n + 1);
+
+            iota(v0.data() + 1, n, size_t(1));
+
+            for (size_t j = 1; j <= m; ++j)
+            {
+                v0[0] = j;
+                size_t prev = j - 1;
+                for (size_t i = 1; i <= n; ++i)
+                {
+                    size_t old = v0[i];
+                    v0[i] = std::min(prev + (lhs[j - 1] != rhs[i - 1]),
+                                     std::min(v0[i - 1], v0[i]) + 1);
+                    prev = old;
+                }
+            }
+            res_values[row] = static_cast<UInt32>(v0[n]);
+        }
+        return res;
+    }
+
+    ColumnPtr weightedLevenshteinImpl(std::vector<const ColumnArray *> columns, bool similarity) const
+    {
+        for (size_t i = 0; i < 2; i++)
+        {
+            const ColumnArray * hs_column = columns[i];
+            const ColumnArray * weights_column = columns[i+2];
+            for (size_t row = 0; row < hs_column->size(); row++)
path: src/Functions/array/arrayLevenshtein.cpp
user: login: Felixoid
body: Does it mean, that all rows must be checked in the beginning of the execution? So, the block will start proceeding only after it's ready?

Maybe, the better place for it is inside the main cycle over the rows below?
created_at: 2025-03-09T18:46:53Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,350 @@
+#include <Columns/ColumnArray.h>
+#include <Columns/ColumnNullable.h>
+#include <Columns/ColumnTuple.h>
+#include <Common/PODArray.h>
+#include <Common/iota.h>
+#include <DataTypes/DataTypeArray.h>
+#include <DataTypes/DataTypeNullable.h>
+#include <DataTypes/DataTypeTuple.h>
+#include <DataTypes/DataTypesNumber.h>
+#include <Functions/FunctionFactory.h>
+#include <Functions/FunctionHelpers.h>
+#include <IO/WriteHelpers.h>
+
+#include <numeric>
+
+namespace DB
+{
+
+namespace ErrorCodes
+{
+extern const int ILLEGAL_TYPE_OF_ARGUMENT;
+extern const int LOGICAL_ERROR;
+extern const int NUMBER_OF_ARGUMENTS_DOESNT_MATCH;
+extern const int SIZES_OF_ARRAYS_DONT_MATCH;
+}
+
+/// arrayLevenshtein([1,2,3,4], [1,3,2,4]) = 2
+/// arrayLevenshteinWeighted([1,2,3,4], [1,3,2,4]) = 2
+template <typename T>
+class FunctionArrayLevenshtein : public IFunction
+{
+public:
+    static constexpr auto name = T::name;
+    static FunctionPtr create(ContextPtr) { return std::make_shared<FunctionArrayLevenshtein<T>>(); }
+
+    String getName() const override { return name; }
+
+    bool isVariadic() const override { return false; }
+    size_t getNumberOfArguments() const override { return T::arguments; }
+    bool useDefaultImplementationForConstants() const override { return true; }
+
+    bool isSuitableForShortCircuitArgumentsExecution(const DataTypesWithConstInfo & /*arguments*/) const override { return false; }
+
+    DataTypePtr getReturnTypeImpl(const ColumnsWithTypeAndName & arguments) const override
+    {
+        DataTypes arguments_types;
+        // First two arguments lhs and rhs are always arrays
+        for (size_t index = 0; index < arguments.size(); ++index)
+        {
+            const DataTypeArray * array_type = checkAndGetDataType<DataTypeArray>(arguments[index].type.get());
+
+            if (!array_type)
+                throw Exception(
+                    ErrorCodes::ILLEGAL_TYPE_OF_ARGUMENT,
+                    "Argument {} of function {} must be array. Found {} instead.",
+                    toString(index + 1),
+                    getName(),
+                    arguments[index].type->getName());
+
+            auto nested_type = array_type->getNestedType();
+            arguments_types.emplace_back(nested_type);
+        }
+
+        if (arguments.size() != T::arguments)
+          throw Exception(
+                  ErrorCodes::NUMBER_OF_ARGUMENTS_DOESNT_MATCH,
+                  "Function {}'s arguments number must be {}.",
+                  getName(),
+                  T::arguments);
+
+        switch (T::impl)
+        {
+            case 0:
+                return std::make_shared<DataTypeUInt32>();
+            case 1:
+            case 2:
+                for (size_t index = 2; index < 4; ++index)
+                {
+                  if (!WhichDataType(arguments_types[index]).isFloat64())
+                    throw Exception(
+                        ErrorCodes::ILLEGAL_TYPE_OF_ARGUMENT,
+                        "Argument {} of function {} must be array of Float64. Found {} instead.",
+                        toString(index + 1),
+                        getName(),
+                        arguments_types[index]->getName());
+                }
+                return std::make_shared<DataTypeFloat64>();
+        }
+        throw Exception(
+            ErrorCodes::LOGICAL_ERROR,
+            "Unknown function {}. "
+            "Supported names: 'arrayLevenshtein', 'arrayLevenshteinWeighted', 'arraySimilarity'",
+            T::name);
+    }
+
+    ColumnPtr executeImpl(
+        const ColumnsWithTypeAndName & arguments, const DataTypePtr & /*result_type*/, size_t input_rows_count
+    ) const override
+    {
+        size_t num_arguments = arguments.size();
+
+        Columns holders(num_arguments);
+        std::vector<const ColumnArray *> columns(num_arguments);
+
+        for (size_t i = 0; i < num_arguments; ++i)
+        {
+            holders[i] = arguments[i].column->convertToFullColumnIfConst();
+            if (holders[i]->size() != input_rows_count)
+                throw Exception(
+                    ErrorCodes::LOGICAL_ERROR,
+                    "Function {} has unequal number of rows in columns: "
+                    "expected {}, got {} for column {}",
+                    getName(),
+                    input_rows_count,
+                    holders[i]->size(),
+                    holders[i]->getName());
+            columns[i] = assert_cast<const ColumnArray*>(holders[i].get());
+        }
+        switch (T::impl)
+        {
+            case 0:
+                return simpleLevenshteinImpl(columns);
+            case 1:
+                return weightedLevenshteinImpl(columns, false);
+            case 2:
+                return weightedLevenshteinImpl(columns, true);
+        }
+    }
+private:
+    ColumnPtr simpleLevenshteinImpl(std::vector<const ColumnArray *> columns) const
+    {
+        const ColumnArray * column_lhs = columns[0];
+        const ColumnArray * column_rhs = columns[1];
+        auto res = ColumnUInt32::create();
+        ColumnUInt32::Container & res_values = res->getData();
+        res_values.resize(column_lhs->size());
+        for (size_t row = 0; row < column_lhs->size(); row++)
+        {
+            // Effective Levenshtein realization from Common/levenshteinDistance
+            Array lhs = (*column_lhs)[row].safeGet<Array>();
+            Array rhs = (*column_rhs)[row].safeGet<Array>();
+            const size_t m = lhs.size();
+            const size_t n = rhs.size();
+            if (m==0 || n==0)
+            {
+                res_values[row] = static_cast<UInt32>(std::max(m, n));
+                continue;
+            }
+            PODArrayWithStackMemory<size_t, 32> v0(n + 1);
+
+            iota(v0.data() + 1, n, size_t(1));
+
+            for (size_t j = 1; j <= m; ++j)
+            {
+                v0[0] = j;
+                size_t prev = j - 1;
+                for (size_t i = 1; i <= n; ++i)
+                {
+                    size_t old = v0[i];
+                    v0[i] = std::min(prev + (lhs[j - 1] != rhs[i - 1]),
+                                     std::min(v0[i - 1], v0[i]) + 1);
+                    prev = old;
+                }
+            }
+            res_values[row] = static_cast<UInt32>(v0[n]);
+        }
+        return res;
+    }
+
+    ColumnPtr weightedLevenshteinImpl(std::vector<const ColumnArray *> columns, bool similarity) const
+    {
+        for (size_t i = 0; i < 2; i++)
+        {
+            const ColumnArray * hs_column = columns[i];
+            const ColumnArray * weights_column = columns[i+2];
+            for (size_t row = 0; row < hs_column->size(); row++)
path: src/Functions/array/arrayLevenshtein.cpp
user: login: Felixoid
body: I thought of it tonight, and it looks safe. The whole block should be there already, otherwise `size_t input_rows_count` wouldn't make any sense.
created_at: 2025-03-10T12:07:40Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,349 @@
+#include <Columns/ColumnArray.h>
+#include <Columns/ColumnNullable.h>
+#include <Columns/ColumnTuple.h>
+#include <Common/PODArray.h>
+#include <Common/iota.h>
+#include <DataTypes/DataTypeArray.h>
+#include <DataTypes/DataTypeNullable.h>
+#include <DataTypes/DataTypeTuple.h>
+#include <DataTypes/DataTypesNumber.h>
+#include <Functions/FunctionFactory.h>
+#include <Functions/FunctionHelpers.h>
+#include <IO/WriteHelpers.h>
+
+#include <numeric>
+
+namespace DB
+{
+
+namespace ErrorCodes
+{
+extern const int ILLEGAL_TYPE_OF_ARGUMENT;
+extern const int LOGICAL_ERROR;
+extern const int NUMBER_OF_ARGUMENTS_DOESNT_MATCH;
+extern const int SIZES_OF_ARRAYS_DONT_MATCH;
+}
+
+/// arrayLevenshtein([1,2,3,4], [1,3,2,4]) = 2
+/// arrayLevenshteinWeighted([1,2,3,4], [1,3,2,4]) = 2
+template <typename T>
+class FunctionArrayLevenshtein : public IFunction
+{
+public:
+    static constexpr auto name = T::name;
+    static FunctionPtr create(ContextPtr) { return std::make_shared<FunctionArrayLevenshtein<T>>(); }
+
+    String getName() const override { return name; }
+
+    bool isVariadic() const override { return false; }
+    size_t getNumberOfArguments() const override { return T::arguments; }
+    bool useDefaultImplementationForConstants() const override { return true; }
+
+    bool isSuitableForShortCircuitArgumentsExecution(const DataTypesWithConstInfo & /*arguments*/) const override { return true; }
+
+    DataTypePtr getReturnTypeImpl(const ColumnsWithTypeAndName & arguments) const override
+    {
+        if (arguments.size() != T::arguments)
+          throw Exception(
+                  ErrorCodes::NUMBER_OF_ARGUMENTS_DOESNT_MATCH,
+                  "Function {}'s arguments number must be {}.",
+                  getName(),
+                  T::arguments);
+
+        DataTypes arguments_types;
+        for (size_t index = 0; index < arguments.size(); ++index)
+        {
+            const DataTypeArray * array_type = checkAndGetDataType<DataTypeArray>(arguments[index].type.get());
+
+            if (!array_type)
+                throw Exception(
+                    ErrorCodes::ILLEGAL_TYPE_OF_ARGUMENT,
+                    "Argument {} of function {} must be array. Found {} instead.",
+                    toString(index + 1),
+                    getName(),
+                    arguments[index].type->getName());
+
+            auto nested_type = array_type->getNestedType();
+            arguments_types.emplace_back(nested_type);
+        }
+
+        switch (T::impl)
+        {
+            case 0:
+                return std::make_shared<DataTypeUInt32>();
+            case 1:
+            case 2:
+                for (size_t index = 2; index < 4; ++index)
+                {
+                  if (!WhichDataType(arguments_types[index]).isFloat64())
+                    throw Exception(
+                        ErrorCodes::ILLEGAL_TYPE_OF_ARGUMENT,
+                        "Argument {} of function {} must be array of Float64. Found {} instead.",
+                        toString(index + 1),
+                        getName(),
+                        arguments_types[index]->getName());
+                }
+                return std::make_shared<DataTypeFloat64>();
+        }
+        throw Exception(
+            ErrorCodes::LOGICAL_ERROR,
+            "Unknown function {}. "
+            "Supported names: 'arrayLevenshtein', 'arrayLevenshteinWeighted', 'arraySimilarity'",
+            T::name);
+    }
+
+    ColumnPtr executeImpl(
+        const ColumnsWithTypeAndName & arguments, const DataTypePtr & /*result_type*/, size_t input_rows_count
+    ) const override
+    {
+        size_t num_arguments = arguments.size();
+
+        Columns holders(num_arguments);
+        std::vector<const ColumnArray *> columns(num_arguments);
+
+        for (size_t i = 0; i < num_arguments; ++i)
+        {
+            holders[i] = arguments[i].column->convertToFullColumnIfConst();
+            if (holders[i]->size() != input_rows_count)
+                throw Exception(
+                    ErrorCodes::LOGICAL_ERROR,
+                    "Function {} has unequal number of rows in columns: "
+                    "expected {}, got {} for column {}",
+                    getName(),
+                    input_rows_count,
+                    holders[i]->size(),
+                    holders[i]->getName());
+            columns[i] = assert_cast<const ColumnArray*>(holders[i].get());
+        }
+        switch (T::impl)
+        {
+            case 0:
+                return simpleLevenshteinImpl(columns);
+            case 1:
+                return weightedLevenshteinImpl(columns, false);
+            case 2:
+                return weightedLevenshteinImpl(columns, true);
+        }
+    }
+private:
+    ColumnPtr simpleLevenshteinImpl(std::vector<const ColumnArray *> columns) const
+    {
+        const ColumnArray * column_lhs = columns[0];
+        const ColumnArray * column_rhs = columns[1];
+        auto res = ColumnUInt32::create();
+        ColumnUInt32::Container & res_values = res->getData();
+        res_values.resize(column_lhs->size());
+        for (size_t row = 0; row < column_lhs->size(); row++)
+        {
+            // Effective Levenshtein realization from Common/levenshteinDistance
+            Array lhs = (*column_lhs)[row].safeGet<Array>();
+            Array rhs = (*column_rhs)[row].safeGet<Array>();
+            const size_t m = lhs.size();
+            const size_t n = rhs.size();
+            if (m==0 || n==0)
+            {
+                res_values[row] = static_cast<UInt32>(std::max(m, n));
+                continue;
+            }
+            PODArrayWithStackMemory<size_t, 32> v0(n + 1);
+
+            iota(v0.data() + 1, n, size_t(1));
+
+            for (size_t j = 1; j <= m; ++j)
+            {
+                v0[0] = j;
+                size_t prev = j - 1;
+                for (size_t i = 1; i <= n; ++i)
+                {
+                    size_t old = v0[i];
+                    v0[i] = std::min(prev + (lhs[j - 1] != rhs[i - 1]),
+                                     std::min(v0[i - 1], v0[i]) + 1);
+                    prev = old;
+                }
+            }
+            res_values[row] = static_cast<UInt32>(v0[n]);
+        }
+        return res;
+    }
+
+    ColumnPtr weightedLevenshteinImpl(std::vector<const ColumnArray *> columns, bool similarity) const
+    {
+        for (size_t i = 0; i < 2; i++)
+        {
+            const ColumnArray * hs_column = columns[i];
+            const ColumnArray * weights_column = columns[i+2];
+            for (size_t row = 0; row < hs_column->size(); row++)
+            {
+                Array hs = (*hs_column)[row].safeGet<Array>();
+                Array weights = (*weights_column)[row].safeGet<Array>();
+                if (hs.size() != weights.size())
+                    throw Exception(
+                        ErrorCodes::SIZES_OF_ARRAYS_DONT_MATCH,
+                        "Arguments {} ({}, size {}) and {} ({}, size {}) of function {} must be arrays of the same size.",
+                        toString(i + 1),
+                        hs_column->getName(),
+                        hs.size(),
+                        toString(i + 3),
+                        weights_column->getName(),
+                        weights.size(),
+                        getName());
+            }
+        }
+        auto res = ColumnFloat64::create();
+        ColumnFloat64::Container & res_values = res->getData();
+        res_values.resize(columns[0]->size());
+        auto get_float = [](const Field & element) -> Float64 { return element.safeGet<Float64>(); };
+        auto sum_array = [](const Array & array) -> Float64 {
+            return std::accumulate(array.begin(), array.end(), 0.0, [](Float64 acc, const Field &field){return acc + field.safeGet<Float64>();});
+        };
+        for (size_t row = 0; row < columns[0]->size(); row++)
+        {
+            // Levenshtein with sliding vectors and weighted elements
+            // https://www.codeproject.com/Articles/13525/Fast-memory-efficient-Levenshtein-algorithm
+            Array lhs = (*columns[0])[row].safeGet<Array>();
+            Array rhs = (*columns[1])[row].safeGet<Array>();
+            Array lhs_w = (*columns[2])[row].safeGet<Array>();
+            Array rhs_w = (*columns[3])[row].safeGet<Array>();
+            const size_t m = lhs.size();
+            const size_t n = rhs.size();
+            if (m==0 || n==0)
+            {
+                if (similarity)
+                    res_values[row] = m == n;
+                else
+                    res_values[row] = sum_array(lhs_w) + sum_array(rhs_w);
+                continue;
+            }
+            PODArrayWithStackMemory<Float64, 64> v0(m + 1);
path: src/Functions/array/arrayLevenshtein.cpp
user: login: Felixoid
body: It would be more efficient to do next two things:

- Find the `min(max(lhs.size()), max(rhs.size))`, allocate v0 and v1 once by it
- in the cycle by `row` swap the lhs/rhs, if they don't fit the size

Should be much less memory usage.


I guess, I'll leave it for the next optimization
created_at: 2025-03-10T13:51:45Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,349 @@
+#include <Columns/ColumnArray.h>
+#include <Columns/ColumnNullable.h>
+#include <Columns/ColumnTuple.h>
+#include <Common/PODArray.h>
+#include <Common/iota.h>
+#include <DataTypes/DataTypeArray.h>
+#include <DataTypes/DataTypeNullable.h>
+#include <DataTypes/DataTypeTuple.h>
+#include <DataTypes/DataTypesNumber.h>
+#include <Functions/FunctionFactory.h>
+#include <Functions/FunctionHelpers.h>
+#include <IO/WriteHelpers.h>
+
+#include <numeric>
+
+namespace DB
+{
+
+namespace ErrorCodes
+{
+extern const int ILLEGAL_TYPE_OF_ARGUMENT;
+extern const int LOGICAL_ERROR;
+extern const int NUMBER_OF_ARGUMENTS_DOESNT_MATCH;
+extern const int SIZES_OF_ARRAYS_DONT_MATCH;
+}
+
+/// arrayLevenshtein([1,2,3,4], [1,3,2,4]) = 2
+/// arrayLevenshteinWeighted([1,2,3,4], [1,3,2,4]) = 2
+template <typename T>
+class FunctionArrayLevenshtein : public IFunction
+{
+public:
+    static constexpr auto name = T::name;
+    static FunctionPtr create(ContextPtr) { return std::make_shared<FunctionArrayLevenshtein<T>>(); }
+
+    String getName() const override { return name; }
+
+    bool isVariadic() const override { return false; }
+    size_t getNumberOfArguments() const override { return T::arguments; }
+    bool useDefaultImplementationForConstants() const override { return true; }
+
+    bool isSuitableForShortCircuitArgumentsExecution(const DataTypesWithConstInfo & /*arguments*/) const override { return true; }
+
+    DataTypePtr getReturnTypeImpl(const ColumnsWithTypeAndName & arguments) const override
+    {
+        if (arguments.size() != T::arguments)
+          throw Exception(
+                  ErrorCodes::NUMBER_OF_ARGUMENTS_DOESNT_MATCH,
+                  "Function {}'s arguments number must be {}.",
+                  getName(),
+                  T::arguments);
+
+        DataTypes arguments_types;
+        for (size_t index = 0; index < arguments.size(); ++index)
+        {
+            const DataTypeArray * array_type = checkAndGetDataType<DataTypeArray>(arguments[index].type.get());
+
+            if (!array_type)
+                throw Exception(
+                    ErrorCodes::ILLEGAL_TYPE_OF_ARGUMENT,
+                    "Argument {} of function {} must be array. Found {} instead.",
+                    toString(index + 1),
+                    getName(),
+                    arguments[index].type->getName());
+
+            auto nested_type = array_type->getNestedType();
+            arguments_types.emplace_back(nested_type);
+        }
+
+        switch (T::impl)
+        {
+            case 0:
+                return std::make_shared<DataTypeUInt32>();
+            case 1:
+            case 2:
+                for (size_t index = 2; index < 4; ++index)
+                {
+                  if (!WhichDataType(arguments_types[index]).isFloat64())
+                    throw Exception(
+                        ErrorCodes::ILLEGAL_TYPE_OF_ARGUMENT,
+                        "Argument {} of function {} must be array of Float64. Found {} instead.",
+                        toString(index + 1),
+                        getName(),
+                        arguments_types[index]->getName());
+                }
+                return std::make_shared<DataTypeFloat64>();
+        }
+        throw Exception(
+            ErrorCodes::LOGICAL_ERROR,
+            "Unknown function {}. "
+            "Supported names: 'arrayLevenshtein', 'arrayLevenshteinWeighted', 'arraySimilarity'",
+            T::name);
+    }
+
+    ColumnPtr executeImpl(
+        const ColumnsWithTypeAndName & arguments, const DataTypePtr & /*result_type*/, size_t input_rows_count
+    ) const override
+    {
+        size_t num_arguments = arguments.size();
+
+        Columns holders(num_arguments);
+        std::vector<const ColumnArray *> columns(num_arguments);
+
+        for (size_t i = 0; i < num_arguments; ++i)
+        {
+            holders[i] = arguments[i].column->convertToFullColumnIfConst();
+            if (holders[i]->size() != input_rows_count)
+                throw Exception(
+                    ErrorCodes::LOGICAL_ERROR,
+                    "Function {} has unequal number of rows in columns: "
+                    "expected {}, got {} for column {}",
+                    getName(),
+                    input_rows_count,
+                    holders[i]->size(),
+                    holders[i]->getName());
+            columns[i] = assert_cast<const ColumnArray*>(holders[i].get());
+        }
+        switch (T::impl)
+        {
+            case 0:
+                return simpleLevenshteinImpl(columns);
+            case 1:
+                return weightedLevenshteinImpl(columns, false);
+            case 2:
+                return weightedLevenshteinImpl(columns, true);
+        }
+    }
+private:
+    ColumnPtr simpleLevenshteinImpl(std::vector<const ColumnArray *> columns) const
+    {
+        const ColumnArray * column_lhs = columns[0];
+        const ColumnArray * column_rhs = columns[1];
+        auto res = ColumnUInt32::create();
+        ColumnUInt32::Container & res_values = res->getData();
+        res_values.resize(column_lhs->size());
+        for (size_t row = 0; row < column_lhs->size(); row++)
+        {
+            // Effective Levenshtein realization from Common/levenshteinDistance
+            Array lhs = (*column_lhs)[row].safeGet<Array>();
+            Array rhs = (*column_rhs)[row].safeGet<Array>();
+            const size_t m = lhs.size();
+            const size_t n = rhs.size();
+            if (m==0 || n==0)
+            {
+                res_values[row] = static_cast<UInt32>(std::max(m, n));
+                continue;
+            }
+            PODArrayWithStackMemory<size_t, 32> v0(n + 1);
+
+            iota(v0.data() + 1, n, size_t(1));
+
+            for (size_t j = 1; j <= m; ++j)
+            {
+                v0[0] = j;
+                size_t prev = j - 1;
+                for (size_t i = 1; i <= n; ++i)
+                {
+                    size_t old = v0[i];
+                    v0[i] = std::min(prev + (lhs[j - 1] != rhs[i - 1]),
+                                     std::min(v0[i - 1], v0[i]) + 1);
+                    prev = old;
+                }
+            }
+            res_values[row] = static_cast<UInt32>(v0[n]);
+        }
+        return res;
+    }
+
+    ColumnPtr weightedLevenshteinImpl(std::vector<const ColumnArray *> columns, bool similarity) const
+    {
+        for (size_t i = 0; i < 2; i++)
+        {
+            const ColumnArray * hs_column = columns[i];
+            const ColumnArray * weights_column = columns[i+2];
+            for (size_t row = 0; row < hs_column->size(); row++)
+            {
+                Array hs = (*hs_column)[row].safeGet<Array>();
+                Array weights = (*weights_column)[row].safeGet<Array>();
+                if (hs.size() != weights.size())
+                    throw Exception(
+                        ErrorCodes::SIZES_OF_ARRAYS_DONT_MATCH,
+                        "Arguments {} ({}, size {}) and {} ({}, size {}) of function {} must be arrays of the same size.",
+                        toString(i + 1),
+                        hs_column->getName(),
+                        hs.size(),
+                        toString(i + 3),
+                        weights_column->getName(),
+                        weights.size(),
+                        getName());
+            }
+        }
+        auto res = ColumnFloat64::create();
+        ColumnFloat64::Container & res_values = res->getData();
+        res_values.resize(columns[0]->size());
+        auto get_float = [](const Field & element) -> Float64 { return element.safeGet<Float64>(); };
+        auto sum_array = [](const Array & array) -> Float64 {
+            return std::accumulate(array.begin(), array.end(), 0.0, [](Float64 acc, const Field &field){return acc + field.safeGet<Float64>();});
+        };
+        for (size_t row = 0; row < columns[0]->size(); row++)
+        {
+            // Levenshtein with sliding vectors and weighted elements
+            // https://www.codeproject.com/Articles/13525/Fast-memory-efficient-Levenshtein-algorithm
path: src/Functions/array/arrayLevenshtein.cpp
user: login: Michicosun
body: this is https://en.wikipedia.org/wiki/Wagner%E2%80%93Fischer_algorithm algorithm

created_at: 2025-03-10T14:49:13Z
author_association: MEMBER
diff_hunk: @@ -3372,6 +3372,121 @@ Result:
 └─────────────────────────────────────────────────────────────┘
 ```
 
+## arrayLevenshtein {#arraylevenshtein}
+
+Calculates Levenshtein distance for two arrays.
+
+**Syntax**
+
+```sql
+arrayLevenshtein(lhs, rhs)
+```
+
+**Arguments**
+
+- `lhs` — left-hand side array
+- `rhs` — right-hand side array
path: docs/en/sql-reference/functions/array-functions.md
user: login: Michicosun
body: let's use different names, for example `from` and `to`, to emphasize that the distance of transition of the array `from` to `to` will be calculated
created_at: 2025-03-10T14:51:10Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,349 @@
+#include <Columns/ColumnArray.h>
+#include <Columns/ColumnNullable.h>
+#include <Columns/ColumnTuple.h>
+#include <Common/PODArray.h>
+#include <Common/iota.h>
+#include <DataTypes/DataTypeArray.h>
+#include <DataTypes/DataTypeNullable.h>
+#include <DataTypes/DataTypeTuple.h>
+#include <DataTypes/DataTypesNumber.h>
+#include <Functions/FunctionFactory.h>
+#include <Functions/FunctionHelpers.h>
+#include <IO/WriteHelpers.h>
+
+#include <numeric>
+
+namespace DB
+{
+
+namespace ErrorCodes
+{
+extern const int ILLEGAL_TYPE_OF_ARGUMENT;
+extern const int LOGICAL_ERROR;
+extern const int NUMBER_OF_ARGUMENTS_DOESNT_MATCH;
+extern const int SIZES_OF_ARRAYS_DONT_MATCH;
path: src/Functions/array/arrayLevenshtein.cpp
user: login: Michicosun
body: ```suggestion
    extern const int ILLEGAL_TYPE_OF_ARGUMENT;
    extern const int LOGICAL_ERROR;
    extern const int NUMBER_OF_ARGUMENTS_DOESNT_MATCH;
    extern const int SIZES_OF_ARRAYS_DONT_MATCH;
```
created_at: 2025-03-10T14:52:09Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,349 @@
+#include <Columns/ColumnArray.h>
+#include <Columns/ColumnNullable.h>
+#include <Columns/ColumnTuple.h>
+#include <Common/PODArray.h>
+#include <Common/iota.h>
+#include <DataTypes/DataTypeArray.h>
+#include <DataTypes/DataTypeNullable.h>
+#include <DataTypes/DataTypeTuple.h>
+#include <DataTypes/DataTypesNumber.h>
+#include <Functions/FunctionFactory.h>
+#include <Functions/FunctionHelpers.h>
+#include <IO/WriteHelpers.h>
+
+#include <numeric>
+
+namespace DB
+{
+
+namespace ErrorCodes
+{
+extern const int ILLEGAL_TYPE_OF_ARGUMENT;
+extern const int LOGICAL_ERROR;
+extern const int NUMBER_OF_ARGUMENTS_DOESNT_MATCH;
+extern const int SIZES_OF_ARRAYS_DONT_MATCH;
+}
+
+/// arrayLevenshtein([1,2,3,4], [1,3,2,4]) = 2
+/// arrayLevenshteinWeighted([1,2,3,4], [1,3,2,4]) = 2
+template <typename T>
+class FunctionArrayLevenshtein : public IFunction
+{
+public:
+    static constexpr auto name = T::name;
+    static FunctionPtr create(ContextPtr) { return std::make_shared<FunctionArrayLevenshtein<T>>(); }
+
+    String getName() const override { return name; }
+
+    bool isVariadic() const override { return false; }
path: src/Functions/array/arrayLevenshtein.cpp
user: login: Michicosun
body: Can be removed, because it is the default implementation from `IFunction`
created_at: 2025-03-10T15:54:08Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,349 @@
+#include <Columns/ColumnArray.h>
+#include <Columns/ColumnNullable.h>
+#include <Columns/ColumnTuple.h>
+#include <Common/PODArray.h>
+#include <Common/iota.h>
+#include <DataTypes/DataTypeArray.h>
+#include <DataTypes/DataTypeNullable.h>
+#include <DataTypes/DataTypeTuple.h>
+#include <DataTypes/DataTypesNumber.h>
+#include <Functions/FunctionFactory.h>
+#include <Functions/FunctionHelpers.h>
+#include <IO/WriteHelpers.h>
+
+#include <numeric>
+
+namespace DB
+{
+
+namespace ErrorCodes
+{
+extern const int ILLEGAL_TYPE_OF_ARGUMENT;
+extern const int LOGICAL_ERROR;
+extern const int NUMBER_OF_ARGUMENTS_DOESNT_MATCH;
+extern const int SIZES_OF_ARRAYS_DONT_MATCH;
+}
+
+/// arrayLevenshtein([1,2,3,4], [1,3,2,4]) = 2
+/// arrayLevenshteinWeighted([1,2,3,4], [1,3,2,4]) = 2
+template <typename T>
+class FunctionArrayLevenshtein : public IFunction
+{
+public:
+    static constexpr auto name = T::name;
+    static FunctionPtr create(ContextPtr) { return std::make_shared<FunctionArrayLevenshtein<T>>(); }
+
+    String getName() const override { return name; }
+
+    bool isVariadic() const override { return false; }
+    size_t getNumberOfArguments() const override { return T::arguments; }
+    bool useDefaultImplementationForConstants() const override { return true; }
+
+    bool isSuitableForShortCircuitArgumentsExecution(const DataTypesWithConstInfo & /*arguments*/) const override { return true; }
+
+    DataTypePtr getReturnTypeImpl(const ColumnsWithTypeAndName & arguments) const override
+    {
+        if (arguments.size() != T::arguments)
path: src/Functions/array/arrayLevenshtein.cpp
user: login: Michicosun
body: It is better to use `validateFunctionArguments` with `FunctionArgumentDescriptors` for validation.

For example check `seriesDecomposeSTL.cpp`
created_at: 2025-03-10T16:07:08Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,349 @@
+#include <Columns/ColumnArray.h>
+#include <Columns/ColumnNullable.h>
+#include <Columns/ColumnTuple.h>
+#include <Common/PODArray.h>
+#include <Common/iota.h>
+#include <DataTypes/DataTypeArray.h>
+#include <DataTypes/DataTypeNullable.h>
+#include <DataTypes/DataTypeTuple.h>
+#include <DataTypes/DataTypesNumber.h>
+#include <Functions/FunctionFactory.h>
+#include <Functions/FunctionHelpers.h>
+#include <IO/WriteHelpers.h>
+
+#include <numeric>
+
+namespace DB
+{
+
+namespace ErrorCodes
+{
+extern const int ILLEGAL_TYPE_OF_ARGUMENT;
+extern const int LOGICAL_ERROR;
+extern const int NUMBER_OF_ARGUMENTS_DOESNT_MATCH;
+extern const int SIZES_OF_ARRAYS_DONT_MATCH;
+}
+
+/// arrayLevenshtein([1,2,3,4], [1,3,2,4]) = 2
+/// arrayLevenshteinWeighted([1,2,3,4], [1,3,2,4]) = 2
+template <typename T>
+class FunctionArrayLevenshtein : public IFunction
+{
+public:
+    static constexpr auto name = T::name;
+    static FunctionPtr create(ContextPtr) { return std::make_shared<FunctionArrayLevenshtein<T>>(); }
+
+    String getName() const override { return name; }
+
+    bool isVariadic() const override { return false; }
+    size_t getNumberOfArguments() const override { return T::arguments; }
+    bool useDefaultImplementationForConstants() const override { return true; }
+
+    bool isSuitableForShortCircuitArgumentsExecution(const DataTypesWithConstInfo & /*arguments*/) const override { return true; }
+
+    DataTypePtr getReturnTypeImpl(const ColumnsWithTypeAndName & arguments) const override
+    {
+        if (arguments.size() != T::arguments)
+          throw Exception(
+                  ErrorCodes::NUMBER_OF_ARGUMENTS_DOESNT_MATCH,
+                  "Function {}'s arguments number must be {}.",
+                  getName(),
+                  T::arguments);
+
+        DataTypes arguments_types;
+        for (size_t index = 0; index < arguments.size(); ++index)
+        {
+            const DataTypeArray * array_type = checkAndGetDataType<DataTypeArray>(arguments[index].type.get());
+
+            if (!array_type)
+                throw Exception(
+                    ErrorCodes::ILLEGAL_TYPE_OF_ARGUMENT,
+                    "Argument {} of function {} must be array. Found {} instead.",
+                    toString(index + 1),
+                    getName(),
+                    arguments[index].type->getName());
+
+            auto nested_type = array_type->getNestedType();
+            arguments_types.emplace_back(nested_type);
+        }
+
+        switch (T::impl)
+        {
+            case 0:
+                return std::make_shared<DataTypeUInt32>();
+            case 1:
+            case 2:
+                for (size_t index = 2; index < 4; ++index)
+                {
+                  if (!WhichDataType(arguments_types[index]).isFloat64())
+                    throw Exception(
+                        ErrorCodes::ILLEGAL_TYPE_OF_ARGUMENT,
+                        "Argument {} of function {} must be array of Float64. Found {} instead.",
+                        toString(index + 1),
+                        getName(),
+                        arguments_types[index]->getName());
+                }
+                return std::make_shared<DataTypeFloat64>();
+        }
+        throw Exception(
+            ErrorCodes::LOGICAL_ERROR,
+            "Unknown function {}. "
+            "Supported names: 'arrayLevenshtein', 'arrayLevenshteinWeighted', 'arraySimilarity'",
+            T::name);
+    }
+
+    ColumnPtr executeImpl(
+        const ColumnsWithTypeAndName & arguments, const DataTypePtr & /*result_type*/, size_t input_rows_count
+    ) const override
path: src/Functions/array/arrayLevenshtein.cpp
user: login: Michicosun
body: ```suggestion
    ColumnPtr executeImpl(const ColumnsWithTypeAndName & arguments, const DataTypePtr & /*result_type*/, size_t input_rows_count) const override
```
created_at: 2025-03-10T16:08:13Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,349 @@
+#include <Columns/ColumnArray.h>
+#include <Columns/ColumnNullable.h>
+#include <Columns/ColumnTuple.h>
+#include <Common/PODArray.h>
+#include <Common/iota.h>
+#include <DataTypes/DataTypeArray.h>
+#include <DataTypes/DataTypeNullable.h>
+#include <DataTypes/DataTypeTuple.h>
+#include <DataTypes/DataTypesNumber.h>
+#include <Functions/FunctionFactory.h>
+#include <Functions/FunctionHelpers.h>
+#include <IO/WriteHelpers.h>
+
+#include <numeric>
+
+namespace DB
+{
+
+namespace ErrorCodes
+{
+extern const int ILLEGAL_TYPE_OF_ARGUMENT;
+extern const int LOGICAL_ERROR;
+extern const int NUMBER_OF_ARGUMENTS_DOESNT_MATCH;
+extern const int SIZES_OF_ARRAYS_DONT_MATCH;
+}
+
+/// arrayLevenshtein([1,2,3,4], [1,3,2,4]) = 2
+/// arrayLevenshteinWeighted([1,2,3,4], [1,3,2,4]) = 2
+template <typename T>
+class FunctionArrayLevenshtein : public IFunction
+{
+public:
+    static constexpr auto name = T::name;
+    static FunctionPtr create(ContextPtr) { return std::make_shared<FunctionArrayLevenshtein<T>>(); }
+
+    String getName() const override { return name; }
+
+    bool isVariadic() const override { return false; }
+    size_t getNumberOfArguments() const override { return T::arguments; }
+    bool useDefaultImplementationForConstants() const override { return true; }
+
+    bool isSuitableForShortCircuitArgumentsExecution(const DataTypesWithConstInfo & /*arguments*/) const override { return true; }
+
+    DataTypePtr getReturnTypeImpl(const ColumnsWithTypeAndName & arguments) const override
+    {
+        if (arguments.size() != T::arguments)
+          throw Exception(
+                  ErrorCodes::NUMBER_OF_ARGUMENTS_DOESNT_MATCH,
+                  "Function {}'s arguments number must be {}.",
+                  getName(),
+                  T::arguments);
+
+        DataTypes arguments_types;
+        for (size_t index = 0; index < arguments.size(); ++index)
+        {
+            const DataTypeArray * array_type = checkAndGetDataType<DataTypeArray>(arguments[index].type.get());
+
+            if (!array_type)
+                throw Exception(
+                    ErrorCodes::ILLEGAL_TYPE_OF_ARGUMENT,
+                    "Argument {} of function {} must be array. Found {} instead.",
+                    toString(index + 1),
+                    getName(),
+                    arguments[index].type->getName());
+
+            auto nested_type = array_type->getNestedType();
+            arguments_types.emplace_back(nested_type);
+        }
+
+        switch (T::impl)
+        {
+            case 0:
+                return std::make_shared<DataTypeUInt32>();
+            case 1:
+            case 2:
+                for (size_t index = 2; index < 4; ++index)
+                {
+                  if (!WhichDataType(arguments_types[index]).isFloat64())
+                    throw Exception(
+                        ErrorCodes::ILLEGAL_TYPE_OF_ARGUMENT,
+                        "Argument {} of function {} must be array of Float64. Found {} instead.",
+                        toString(index + 1),
+                        getName(),
+                        arguments_types[index]->getName());
+                }
+                return std::make_shared<DataTypeFloat64>();
+        }
+        throw Exception(
+            ErrorCodes::LOGICAL_ERROR,
+            "Unknown function {}. "
+            "Supported names: 'arrayLevenshtein', 'arrayLevenshteinWeighted', 'arraySimilarity'",
+            T::name);
+    }
+
+    ColumnPtr executeImpl(
+        const ColumnsWithTypeAndName & arguments, const DataTypePtr & /*result_type*/, size_t input_rows_count
+    ) const override
+    {
+        size_t num_arguments = arguments.size();
+
+        Columns holders(num_arguments);
+        std::vector<const ColumnArray *> columns(num_arguments);
+
+        for (size_t i = 0; i < num_arguments; ++i)
+        {
+            holders[i] = arguments[i].column->convertToFullColumnIfConst();
+            if (holders[i]->size() != input_rows_count)
+                throw Exception(
+                    ErrorCodes::LOGICAL_ERROR,
+                    "Function {} has unequal number of rows in columns: "
+                    "expected {}, got {} for column {}",
+                    getName(),
+                    input_rows_count,
+                    holders[i]->size(),
+                    holders[i]->getName());
+            columns[i] = assert_cast<const ColumnArray*>(holders[i].get());
+        }
+        switch (T::impl)
+        {
+            case 0:
+                return simpleLevenshteinImpl(columns);
+            case 1:
+                return weightedLevenshteinImpl(columns, false);
+            case 2:
+                return weightedLevenshteinImpl(columns, true);
+        }
+    }
+private:
+    ColumnPtr simpleLevenshteinImpl(std::vector<const ColumnArray *> columns) const
+    {
+        const ColumnArray * column_lhs = columns[0];
+        const ColumnArray * column_rhs = columns[1];
+        auto res = ColumnUInt32::create();
+        ColumnUInt32::Container & res_values = res->getData();
+        res_values.resize(column_lhs->size());
+        for (size_t row = 0; row < column_lhs->size(); row++)
+        {
+            // Effective Levenshtein realization from Common/levenshteinDistance
+            Array lhs = (*column_lhs)[row].safeGet<Array>();
+            Array rhs = (*column_rhs)[row].safeGet<Array>();
+            const size_t m = lhs.size();
+            const size_t n = rhs.size();
+            if (m==0 || n==0)
+            {
+                res_values[row] = static_cast<UInt32>(std::max(m, n));
+                continue;
+            }
+            PODArrayWithStackMemory<size_t, 32> v0(n + 1);
+
+            iota(v0.data() + 1, n, size_t(1));
+
+            for (size_t j = 1; j <= m; ++j)
+            {
+                v0[0] = j;
+                size_t prev = j - 1;
+                for (size_t i = 1; i <= n; ++i)
+                {
+                    size_t old = v0[i];
+                    v0[i] = std::min(prev + (lhs[j - 1] != rhs[i - 1]),
+                                     std::min(v0[i - 1], v0[i]) + 1);
+                    prev = old;
+                }
+            }
+            res_values[row] = static_cast<UInt32>(v0[n]);
+        }
+        return res;
+    }
+
+    ColumnPtr weightedLevenshteinImpl(std::vector<const ColumnArray *> columns, bool similarity) const
path: src/Functions/array/arrayLevenshtein.cpp
user: login: Michicosun
body: `similarity` looks like an out of this function's scope feature, maybe it will be simplier to calculate `weightedLevenshtein` and then map it uniformly to weighted sum linear space?
created_at: 2025-03-10T17:28:19Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,349 @@
+#include <Columns/ColumnArray.h>
+#include <Columns/ColumnNullable.h>
+#include <Columns/ColumnTuple.h>
+#include <Common/PODArray.h>
+#include <Common/iota.h>
+#include <DataTypes/DataTypeArray.h>
+#include <DataTypes/DataTypeNullable.h>
+#include <DataTypes/DataTypeTuple.h>
+#include <DataTypes/DataTypesNumber.h>
+#include <Functions/FunctionFactory.h>
+#include <Functions/FunctionHelpers.h>
+#include <IO/WriteHelpers.h>
+
+#include <numeric>
+
+namespace DB
+{
+
+namespace ErrorCodes
+{
+extern const int ILLEGAL_TYPE_OF_ARGUMENT;
+extern const int LOGICAL_ERROR;
+extern const int NUMBER_OF_ARGUMENTS_DOESNT_MATCH;
+extern const int SIZES_OF_ARRAYS_DONT_MATCH;
+}
+
+/// arrayLevenshtein([1,2,3,4], [1,3,2,4]) = 2
+/// arrayLevenshteinWeighted([1,2,3,4], [1,3,2,4]) = 2
+template <typename T>
+class FunctionArrayLevenshtein : public IFunction
path: src/Functions/array/arrayLevenshtein.cpp
user: login: Michicosun
body: why not implement it as 3 classes instead?

this kind of switches looks weird:
```
        switch (T::impl)
        {
            case 0:
                return simpleLevenshteinImpl(columns);
            case 1:
                return weightedLevenshteinImpl(columns, false);
            case 2:
                return weightedLevenshteinImpl(columns, true);
        }
```
created_at: 2025-03-10T18:39:33Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,349 @@
+#include <Columns/ColumnArray.h>
+#include <Columns/ColumnNullable.h>
+#include <Columns/ColumnTuple.h>
+#include <Common/PODArray.h>
+#include <Common/iota.h>
+#include <DataTypes/DataTypeArray.h>
+#include <DataTypes/DataTypeNullable.h>
+#include <DataTypes/DataTypeTuple.h>
+#include <DataTypes/DataTypesNumber.h>
+#include <Functions/FunctionFactory.h>
+#include <Functions/FunctionHelpers.h>
+#include <IO/WriteHelpers.h>
+
+#include <numeric>
+
+namespace DB
+{
+
+namespace ErrorCodes
+{
+extern const int ILLEGAL_TYPE_OF_ARGUMENT;
+extern const int LOGICAL_ERROR;
+extern const int NUMBER_OF_ARGUMENTS_DOESNT_MATCH;
+extern const int SIZES_OF_ARRAYS_DONT_MATCH;
+}
+
+/// arrayLevenshtein([1,2,3,4], [1,3,2,4]) = 2
+/// arrayLevenshteinWeighted([1,2,3,4], [1,3,2,4]) = 2
+template <typename T>
+class FunctionArrayLevenshtein : public IFunction
path: src/Functions/array/arrayLevenshtein.cpp
user: login: Felixoid
body: `weightedLevenshteinImpl` and `similarity` are almost identical except for a slight difference.

I didn't want to repeat shared code with `handler`.
created_at: 2025-03-10T19:59:39Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,349 @@
+#include <Columns/ColumnArray.h>
+#include <Columns/ColumnNullable.h>
+#include <Columns/ColumnTuple.h>
+#include <Common/PODArray.h>
+#include <Common/iota.h>
+#include <DataTypes/DataTypeArray.h>
+#include <DataTypes/DataTypeNullable.h>
+#include <DataTypes/DataTypeTuple.h>
+#include <DataTypes/DataTypesNumber.h>
+#include <Functions/FunctionFactory.h>
+#include <Functions/FunctionHelpers.h>
+#include <IO/WriteHelpers.h>
+
+#include <numeric>
+
+namespace DB
+{
+
+namespace ErrorCodes
+{
+extern const int ILLEGAL_TYPE_OF_ARGUMENT;
+extern const int LOGICAL_ERROR;
+extern const int NUMBER_OF_ARGUMENTS_DOESNT_MATCH;
+extern const int SIZES_OF_ARRAYS_DONT_MATCH;
+}
+
+/// arrayLevenshtein([1,2,3,4], [1,3,2,4]) = 2
+/// arrayLevenshteinWeighted([1,2,3,4], [1,3,2,4]) = 2
+template <typename T>
+class FunctionArrayLevenshtein : public IFunction
+{
+public:
+    static constexpr auto name = T::name;
+    static FunctionPtr create(ContextPtr) { return std::make_shared<FunctionArrayLevenshtein<T>>(); }
+
+    String getName() const override { return name; }
+
+    bool isVariadic() const override { return false; }
+    size_t getNumberOfArguments() const override { return T::arguments; }
+    bool useDefaultImplementationForConstants() const override { return true; }
+
+    bool isSuitableForShortCircuitArgumentsExecution(const DataTypesWithConstInfo & /*arguments*/) const override { return true; }
+
+    DataTypePtr getReturnTypeImpl(const ColumnsWithTypeAndName & arguments) const override
+    {
+        if (arguments.size() != T::arguments)
+          throw Exception(
+                  ErrorCodes::NUMBER_OF_ARGUMENTS_DOESNT_MATCH,
+                  "Function {}'s arguments number must be {}.",
+                  getName(),
+                  T::arguments);
+
+        DataTypes arguments_types;
+        for (size_t index = 0; index < arguments.size(); ++index)
+        {
+            const DataTypeArray * array_type = checkAndGetDataType<DataTypeArray>(arguments[index].type.get());
+
+            if (!array_type)
+                throw Exception(
+                    ErrorCodes::ILLEGAL_TYPE_OF_ARGUMENT,
+                    "Argument {} of function {} must be array. Found {} instead.",
+                    toString(index + 1),
+                    getName(),
+                    arguments[index].type->getName());
+
+            auto nested_type = array_type->getNestedType();
+            arguments_types.emplace_back(nested_type);
+        }
+
+        switch (T::impl)
+        {
+            case 0:
+                return std::make_shared<DataTypeUInt32>();
+            case 1:
+            case 2:
+                for (size_t index = 2; index < 4; ++index)
+                {
+                  if (!WhichDataType(arguments_types[index]).isFloat64())
+                    throw Exception(
+                        ErrorCodes::ILLEGAL_TYPE_OF_ARGUMENT,
+                        "Argument {} of function {} must be array of Float64. Found {} instead.",
+                        toString(index + 1),
+                        getName(),
+                        arguments_types[index]->getName());
+                }
+                return std::make_shared<DataTypeFloat64>();
+        }
+        throw Exception(
+            ErrorCodes::LOGICAL_ERROR,
+            "Unknown function {}. "
+            "Supported names: 'arrayLevenshtein', 'arrayLevenshteinWeighted', 'arraySimilarity'",
+            T::name);
+    }
+
+    ColumnPtr executeImpl(
+        const ColumnsWithTypeAndName & arguments, const DataTypePtr & /*result_type*/, size_t input_rows_count
+    ) const override
path: src/Functions/array/arrayLevenshtein.cpp
user: login: Felixoid
body: What is general rule of thumb for the line length?
created_at: 2025-03-11T14:48:00Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,349 @@
+#include <Columns/ColumnArray.h>
+#include <Columns/ColumnNullable.h>
+#include <Columns/ColumnTuple.h>
+#include <Common/PODArray.h>
+#include <Common/iota.h>
+#include <DataTypes/DataTypeArray.h>
+#include <DataTypes/DataTypeNullable.h>
+#include <DataTypes/DataTypeTuple.h>
+#include <DataTypes/DataTypesNumber.h>
+#include <Functions/FunctionFactory.h>
+#include <Functions/FunctionHelpers.h>
+#include <IO/WriteHelpers.h>
+
+#include <numeric>
+
+namespace DB
+{
+
+namespace ErrorCodes
+{
+extern const int ILLEGAL_TYPE_OF_ARGUMENT;
+extern const int LOGICAL_ERROR;
+extern const int NUMBER_OF_ARGUMENTS_DOESNT_MATCH;
+extern const int SIZES_OF_ARRAYS_DONT_MATCH;
+}
+
+/// arrayLevenshtein([1,2,3,4], [1,3,2,4]) = 2
+/// arrayLevenshteinWeighted([1,2,3,4], [1,3,2,4]) = 2
+template <typename T>
+class FunctionArrayLevenshtein : public IFunction
path: src/Functions/array/arrayLevenshtein.cpp
user: login: Felixoid
body: Moved to `execute`
created_at: 2025-03-11T14:48:34Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,349 @@
+#include <Columns/ColumnArray.h>
+#include <Columns/ColumnNullable.h>
+#include <Columns/ColumnTuple.h>
+#include <Common/PODArray.h>
+#include <Common/iota.h>
+#include <DataTypes/DataTypeArray.h>
+#include <DataTypes/DataTypeNullable.h>
+#include <DataTypes/DataTypeTuple.h>
+#include <DataTypes/DataTypesNumber.h>
+#include <Functions/FunctionFactory.h>
+#include <Functions/FunctionHelpers.h>
+#include <IO/WriteHelpers.h>
+
+#include <numeric>
+
+namespace DB
+{
+
+namespace ErrorCodes
+{
+extern const int ILLEGAL_TYPE_OF_ARGUMENT;
+extern const int LOGICAL_ERROR;
+extern const int NUMBER_OF_ARGUMENTS_DOESNT_MATCH;
+extern const int SIZES_OF_ARRAYS_DONT_MATCH;
+}
+
+/// arrayLevenshtein([1,2,3,4], [1,3,2,4]) = 2
+/// arrayLevenshteinWeighted([1,2,3,4], [1,3,2,4]) = 2
+template <typename T>
+class FunctionArrayLevenshtein : public IFunction
+{
+public:
+    static constexpr auto name = T::name;
+    static FunctionPtr create(ContextPtr) { return std::make_shared<FunctionArrayLevenshtein<T>>(); }
+
+    String getName() const override { return name; }
+
+    bool isVariadic() const override { return false; }
+    size_t getNumberOfArguments() const override { return T::arguments; }
+    bool useDefaultImplementationForConstants() const override { return true; }
+
+    bool isSuitableForShortCircuitArgumentsExecution(const DataTypesWithConstInfo & /*arguments*/) const override { return true; }
+
+    DataTypePtr getReturnTypeImpl(const ColumnsWithTypeAndName & arguments) const override
+    {
+        if (arguments.size() != T::arguments)
+          throw Exception(
+                  ErrorCodes::NUMBER_OF_ARGUMENTS_DOESNT_MATCH,
+                  "Function {}'s arguments number must be {}.",
+                  getName(),
+                  T::arguments);
+
+        DataTypes arguments_types;
+        for (size_t index = 0; index < arguments.size(); ++index)
+        {
+            const DataTypeArray * array_type = checkAndGetDataType<DataTypeArray>(arguments[index].type.get());
+
+            if (!array_type)
+                throw Exception(
+                    ErrorCodes::ILLEGAL_TYPE_OF_ARGUMENT,
+                    "Argument {} of function {} must be array. Found {} instead.",
+                    toString(index + 1),
+                    getName(),
+                    arguments[index].type->getName());
+
+            auto nested_type = array_type->getNestedType();
+            arguments_types.emplace_back(nested_type);
+        }
+
+        switch (T::impl)
+        {
+            case 0:
+                return std::make_shared<DataTypeUInt32>();
+            case 1:
+            case 2:
+                for (size_t index = 2; index < 4; ++index)
+                {
+                  if (!WhichDataType(arguments_types[index]).isFloat64())
+                    throw Exception(
+                        ErrorCodes::ILLEGAL_TYPE_OF_ARGUMENT,
+                        "Argument {} of function {} must be array of Float64. Found {} instead.",
+                        toString(index + 1),
+                        getName(),
+                        arguments_types[index]->getName());
+                }
+                return std::make_shared<DataTypeFloat64>();
+        }
+        throw Exception(
+            ErrorCodes::LOGICAL_ERROR,
+            "Unknown function {}. "
+            "Supported names: 'arrayLevenshtein', 'arrayLevenshteinWeighted', 'arraySimilarity'",
+            T::name);
+    }
+
+    ColumnPtr executeImpl(
+        const ColumnsWithTypeAndName & arguments, const DataTypePtr & /*result_type*/, size_t input_rows_count
+    ) const override
path: src/Functions/array/arrayLevenshtein.cpp
user: login: Michicosun
body: As far as I know 140 symbols.
created_at: 2025-03-24T09:38:17Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,626 @@
+#include <Columns/ColumnArray.h>
+#include <Columns/ColumnDecimal.h>
+#include <Columns/ColumnFixedString.h>
+#include <Columns/ColumnNullable.h>
+#include <Columns/ColumnString.h>
+#include <Columns/ColumnTuple.h>
+#include <Common/PODArray.h>
+#include <Common/iota.h>
+#include <DataTypes/DataTypeArray.h>
+#include <DataTypes/DataTypeNullable.h>
+#include <DataTypes/DataTypeTuple.h>
+#include <DataTypes/DataTypesNumber.h>
+#include <Functions/FunctionFactory.h>
+#include <Functions/FunctionHelpers.h>
+#include <IO/WriteHelpers.h>
+
+#include <numeric>
+
+namespace DB
+{
+
+namespace ErrorCodes
+{
+    extern const int ILLEGAL_TYPE_OF_ARGUMENT;
+    extern const int LOGICAL_ERROR;
+    extern const int SIZES_OF_ARRAYS_DONT_MATCH;
+}
+
+/// arrayLevenshtein([1,2,3,4], [1,3,2,4]) = 2
+/// arrayLevenshteinWeighted([1,2,3,4], [1,3,2,4]) = 2
+template <typename T>
+class FunctionArrayLevenshtein : public IFunction
+{
+public:
+    static constexpr auto name = T::name;
+    static FunctionPtr create(ContextPtr) { return std::make_shared<FunctionArrayLevenshtein<T>>(); }
+
+    String getName() const override { return name; }
+
+    size_t getNumberOfArguments() const override { return T::arguments; }
+    bool useDefaultImplementationForConstants() const override { return true; }
+
+    bool isSuitableForShortCircuitArgumentsExecution(const DataTypesWithConstInfo & /*arguments*/) const override { return true; }
+
+    DataTypePtr getReturnTypeImpl(const ColumnsWithTypeAndName & arguments) const override
+    {
+        FunctionArgumentDescriptors args_descriptors;
+        args_descriptors = FunctionArgumentDescriptors{
+            {"from", static_cast<FunctionArgumentDescriptor::TypeValidator>(&isArray), nullptr, "Array"},
+            {"to", static_cast<FunctionArgumentDescriptor::TypeValidator>(&isArray), nullptr, "Array"},
+            {"from_weights", static_cast<FunctionArgumentDescriptor::TypeValidator>(&isArray), nullptr, "Array"},
+            {"to_weights", static_cast<FunctionArgumentDescriptor::TypeValidator>(&isArray), nullptr, "Array"},
+        };
+        validateFunctionArguments(*this, arguments, args_descriptors);
+        std::vector<DataTypePtr> nested_types;
+        nested_types.reserve(2);
+        for (size_t index = 2; index < 4; ++index)
+        {
+            const DataTypeArray * array_type = checkAndGetDataType<DataTypeArray>(arguments[index].type.get());
+            const DataTypePtr nested_type = array_type->getNestedType();
+            nested_types.emplace_back(nested_type);
+            // TODO: extend to work with different types?
+            if (!WhichDataType(nested_type).isFloat64())
+                throw Exception(
+                    ErrorCodes::ILLEGAL_TYPE_OF_ARGUMENT,
+                    "Argument {} of function {} must be array of Float64. Found {} instead.",
+                    toString(index + 1),
+                    getName(),
+                    nested_type->getName());
+        }
+
+        if (nested_types[0]->getName() != nested_types[1]->getName())
+            throw Exception(
+                ErrorCodes::ILLEGAL_TYPE_OF_ARGUMENT,
+                "Arguments 3 and 4 of function {} must be arrays of the same types. Found {} and {} instead.",
+                getName(),
+                nested_types[0]->getName(),
+                nested_types[1]->getName());
+        return std::make_shared<DataTypeFloat64>();
+    }
+
+    ColumnPtr executeImpl(const ColumnsWithTypeAndName & arguments, const DataTypePtr &, size_t input_rows_count) const override
+    {
+        size_t num_arguments = arguments.size();
+
+        Columns holders(num_arguments);
+        std::vector<const ColumnArray *> columns(num_arguments);
+
+        for (size_t i = 0; i < num_arguments; ++i)
+        {
+            holders[i] = arguments[i].column->convertToFullColumnIfConst();
+            if (holders[i]->size() != input_rows_count)
+                throw Exception(
+                    ErrorCodes::LOGICAL_ERROR,
+                    "Function {} has unequal number of rows in columns: "
+                    "expected {}, got {} for column {}",
+                    getName(),
+                    input_rows_count,
+                    holders[i]->size(),
+                    holders[i]->getName());
+            columns[i] = assert_cast<const ColumnArray*>(holders[i].get());
+        }
+        return execute(columns);
+    }
+private:
+    ColumnPtr execute(std::vector<const ColumnArray *>) const
+    {
+        throw Exception(
+            ErrorCodes::LOGICAL_ERROR,
+            "Unknown function {}. "
+            "Supported names: 'arrayLevenshtein', 'arrayLevenshteinWeighted', 'arraySimilarity'",
+            T::name);
+    }
+
+    template<typename N>
+    UInt32 levenshteinDistance(const std::vector<N> from, const std::vector<N> to) const
path: src/Functions/array/arrayLevenshtein.cpp
user: login: Michicosun
body: Maybe we can move `levenshteinDistance` and `levenshteinDistanceWeighted` to `/Common` and make them completely generic wdyt?

For example this kind of signature:

```cpp
template <class Underlying>
size_t levenshteinDistance(std::span<Underlying> from, std::span<Underlying> to);
```

btw it seems that `levenshteinDistanceWeighted` can be also implemented via single array using the same technique, saving the diagonal element (v0[j]) as old


created_at: 2025-03-24T10:27:53Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,626 @@
+#include <Columns/ColumnArray.h>
+#include <Columns/ColumnDecimal.h>
+#include <Columns/ColumnFixedString.h>
+#include <Columns/ColumnNullable.h>
+#include <Columns/ColumnString.h>
+#include <Columns/ColumnTuple.h>
+#include <Common/PODArray.h>
+#include <Common/iota.h>
+#include <DataTypes/DataTypeArray.h>
+#include <DataTypes/DataTypeNullable.h>
+#include <DataTypes/DataTypeTuple.h>
+#include <DataTypes/DataTypesNumber.h>
+#include <Functions/FunctionFactory.h>
+#include <Functions/FunctionHelpers.h>
+#include <IO/WriteHelpers.h>
+
+#include <numeric>
+
+namespace DB
+{
+
+namespace ErrorCodes
+{
+    extern const int ILLEGAL_TYPE_OF_ARGUMENT;
+    extern const int LOGICAL_ERROR;
+    extern const int SIZES_OF_ARRAYS_DONT_MATCH;
+}
+
+/// arrayLevenshtein([1,2,3,4], [1,3,2,4]) = 2
+/// arrayLevenshteinWeighted([1,2,3,4], [1,3,2,4]) = 2
+template <typename T>
+class FunctionArrayLevenshtein : public IFunction
+{
+public:
+    static constexpr auto name = T::name;
+    static FunctionPtr create(ContextPtr) { return std::make_shared<FunctionArrayLevenshtein<T>>(); }
+
+    String getName() const override { return name; }
+
+    size_t getNumberOfArguments() const override { return T::arguments; }
+    bool useDefaultImplementationForConstants() const override { return true; }
+
+    bool isSuitableForShortCircuitArgumentsExecution(const DataTypesWithConstInfo & /*arguments*/) const override { return true; }
+
+    DataTypePtr getReturnTypeImpl(const ColumnsWithTypeAndName & arguments) const override
+    {
+        FunctionArgumentDescriptors args_descriptors;
+        args_descriptors = FunctionArgumentDescriptors{
+            {"from", static_cast<FunctionArgumentDescriptor::TypeValidator>(&isArray), nullptr, "Array"},
+            {"to", static_cast<FunctionArgumentDescriptor::TypeValidator>(&isArray), nullptr, "Array"},
+            {"from_weights", static_cast<FunctionArgumentDescriptor::TypeValidator>(&isArray), nullptr, "Array"},
+            {"to_weights", static_cast<FunctionArgumentDescriptor::TypeValidator>(&isArray), nullptr, "Array"},
+        };
+        validateFunctionArguments(*this, arguments, args_descriptors);
+        std::vector<DataTypePtr> nested_types;
+        nested_types.reserve(2);
+        for (size_t index = 2; index < 4; ++index)
+        {
+            const DataTypeArray * array_type = checkAndGetDataType<DataTypeArray>(arguments[index].type.get());
+            const DataTypePtr nested_type = array_type->getNestedType();
+            nested_types.emplace_back(nested_type);
+            // TODO: extend to work with different types?
+            if (!WhichDataType(nested_type).isFloat64())
+                throw Exception(
+                    ErrorCodes::ILLEGAL_TYPE_OF_ARGUMENT,
+                    "Argument {} of function {} must be array of Float64. Found {} instead.",
+                    toString(index + 1),
+                    getName(),
+                    nested_type->getName());
+        }
+
+        if (nested_types[0]->getName() != nested_types[1]->getName())
+            throw Exception(
+                ErrorCodes::ILLEGAL_TYPE_OF_ARGUMENT,
+                "Arguments 3 and 4 of function {} must be arrays of the same types. Found {} and {} instead.",
+                getName(),
+                nested_types[0]->getName(),
+                nested_types[1]->getName());
+        return std::make_shared<DataTypeFloat64>();
+    }
+
+    ColumnPtr executeImpl(const ColumnsWithTypeAndName & arguments, const DataTypePtr &, size_t input_rows_count) const override
+    {
+        size_t num_arguments = arguments.size();
+
+        Columns holders(num_arguments);
+        std::vector<const ColumnArray *> columns(num_arguments);
+
+        for (size_t i = 0; i < num_arguments; ++i)
+        {
+            holders[i] = arguments[i].column->convertToFullColumnIfConst();
+            if (holders[i]->size() != input_rows_count)
+                throw Exception(
+                    ErrorCodes::LOGICAL_ERROR,
+                    "Function {} has unequal number of rows in columns: "
+                    "expected {}, got {} for column {}",
+                    getName(),
+                    input_rows_count,
+                    holders[i]->size(),
+                    holders[i]->getName());
+            columns[i] = assert_cast<const ColumnArray*>(holders[i].get());
+        }
+        return execute(columns);
+    }
+private:
+    ColumnPtr execute(std::vector<const ColumnArray *>) const
+    {
+        throw Exception(
+            ErrorCodes::LOGICAL_ERROR,
+            "Unknown function {}. "
+            "Supported names: 'arrayLevenshtein', 'arrayLevenshteinWeighted', 'arraySimilarity'",
+            T::name);
+    }
+
+    template<typename N>
+    UInt32 levenshteinDistance(const std::vector<N> from, const std::vector<N> to) const
+    {
+        const size_t m = from.size();
+        const size_t n = to.size();
+        if (m==0 || n==0)
+        {
+            return static_cast<UInt32>(m + n);
+        }
+        PODArrayWithStackMemory<size_t, 32> v0(n + 1);
+
+        iota(v0.data() + 1, n, size_t(1));
+
+        for (size_t j = 1; j <= m; ++j)
+        {
+            v0[0] = j;
+            size_t prev = j - 1;
+            for (size_t i = 1; i <= n; ++i)
+            {
+                size_t old = v0[i];
+                v0[i] = std::min(prev + (from[j - 1] != to[i - 1]),
+                        std::min(v0[i - 1], v0[i]) + 1);
+                prev = old;
+            }
+        }
+        return static_cast<UInt32>(v0[n]);
+    }
+
+    template <typename N>
+    bool levenshteinString(std::vector<const ColumnArray *> columns, ColumnUInt32::Container & res_values) const
+    {
+        const N * from_data = checkAndGetColumn<N>(&columns[0]->getData());
+        const N * to_data = checkAndGetColumn<N>(&columns[1]->getData());
+        if (!from_data || !to_data)
+            return false;
+        const ColumnArray::Offsets & from_offsets = columns[0]->getOffsets();
+        ColumnArray::Offset prev_from_offset = 0;
+
+        const ColumnArray::Offsets & to_offsets = columns[1]->getOffsets();
+        ColumnArray::Offset prev_to_offset = 0;
+
+        for (size_t row = 0; row < columns[0]->size(); row++)
+        {
+            const size_t m = from_offsets[row] - prev_from_offset;
+            const size_t n = to_offsets[row] - prev_to_offset;
+            const std::vector<StringRef> & from = [&]()
+            {
+                std::vector<StringRef> temp;
+                temp.reserve(m);
+                for (size_t j = 0; j < m; ++j) { temp.emplace_back(from_data->getDataAt(prev_from_offset + j)); }
+                return temp;
+            }();
+            const std::vector<StringRef> & to = [&]()
+            {
+                std::vector<StringRef> temp;
+                temp.reserve(m);
+                for (size_t j = 0; j < n; ++j) { temp.emplace_back(to_data->getDataAt(prev_to_offset + j)); }
+                return temp;
+            }();
path: src/Functions/array/arrayLevenshtein.cpp
user: login: Michicosun
body: ```suggestion
            const auto extract_array = [](const N * data, size_t prev_offset, size_t count)
            {
                std::vector<StringRef> temp;
                temp.reserve(count);
                for (size_t j = 0; j < count; ++j) { temp.emplace_back(data->getDataAt(prev_offset + j)); }
                return temp;
            };
            
            std::vector<StringRef> from = extract_array(from_data, prev_from_offset, m);
            std::vector<StringRef> to = extract_array(to_data, prev_to_offset, n);
```
created_at: 2025-03-24T10:33:15Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,626 @@
+#include <Columns/ColumnArray.h>
+#include <Columns/ColumnDecimal.h>
+#include <Columns/ColumnFixedString.h>
+#include <Columns/ColumnNullable.h>
+#include <Columns/ColumnString.h>
+#include <Columns/ColumnTuple.h>
+#include <Common/PODArray.h>
+#include <Common/iota.h>
+#include <DataTypes/DataTypeArray.h>
+#include <DataTypes/DataTypeNullable.h>
+#include <DataTypes/DataTypeTuple.h>
+#include <DataTypes/DataTypesNumber.h>
+#include <Functions/FunctionFactory.h>
+#include <Functions/FunctionHelpers.h>
+#include <IO/WriteHelpers.h>
+
+#include <numeric>
+
+namespace DB
+{
+
+namespace ErrorCodes
+{
+    extern const int ILLEGAL_TYPE_OF_ARGUMENT;
+    extern const int LOGICAL_ERROR;
+    extern const int SIZES_OF_ARRAYS_DONT_MATCH;
+}
+
+/// arrayLevenshtein([1,2,3,4], [1,3,2,4]) = 2
+/// arrayLevenshteinWeighted([1,2,3,4], [1,3,2,4]) = 2
+template <typename T>
+class FunctionArrayLevenshtein : public IFunction
+{
+public:
+    static constexpr auto name = T::name;
+    static FunctionPtr create(ContextPtr) { return std::make_shared<FunctionArrayLevenshtein<T>>(); }
+
+    String getName() const override { return name; }
+
+    size_t getNumberOfArguments() const override { return T::arguments; }
+    bool useDefaultImplementationForConstants() const override { return true; }
+
+    bool isSuitableForShortCircuitArgumentsExecution(const DataTypesWithConstInfo & /*arguments*/) const override { return true; }
+
+    DataTypePtr getReturnTypeImpl(const ColumnsWithTypeAndName & arguments) const override
+    {
+        FunctionArgumentDescriptors args_descriptors;
+        args_descriptors = FunctionArgumentDescriptors{
+            {"from", static_cast<FunctionArgumentDescriptor::TypeValidator>(&isArray), nullptr, "Array"},
+            {"to", static_cast<FunctionArgumentDescriptor::TypeValidator>(&isArray), nullptr, "Array"},
+            {"from_weights", static_cast<FunctionArgumentDescriptor::TypeValidator>(&isArray), nullptr, "Array"},
+            {"to_weights", static_cast<FunctionArgumentDescriptor::TypeValidator>(&isArray), nullptr, "Array"},
+        };
+        validateFunctionArguments(*this, arguments, args_descriptors);
+        std::vector<DataTypePtr> nested_types;
+        nested_types.reserve(2);
+        for (size_t index = 2; index < 4; ++index)
+        {
+            const DataTypeArray * array_type = checkAndGetDataType<DataTypeArray>(arguments[index].type.get());
+            const DataTypePtr nested_type = array_type->getNestedType();
+            nested_types.emplace_back(nested_type);
+            // TODO: extend to work with different types?
+            if (!WhichDataType(nested_type).isFloat64())
+                throw Exception(
+                    ErrorCodes::ILLEGAL_TYPE_OF_ARGUMENT,
+                    "Argument {} of function {} must be array of Float64. Found {} instead.",
+                    toString(index + 1),
+                    getName(),
+                    nested_type->getName());
+        }
+
+        if (nested_types[0]->getName() != nested_types[1]->getName())
+            throw Exception(
+                ErrorCodes::ILLEGAL_TYPE_OF_ARGUMENT,
+                "Arguments 3 and 4 of function {} must be arrays of the same types. Found {} and {} instead.",
+                getName(),
+                nested_types[0]->getName(),
+                nested_types[1]->getName());
+        return std::make_shared<DataTypeFloat64>();
+    }
+
+    ColumnPtr executeImpl(const ColumnsWithTypeAndName & arguments, const DataTypePtr &, size_t input_rows_count) const override
+    {
+        size_t num_arguments = arguments.size();
+
+        Columns holders(num_arguments);
+        std::vector<const ColumnArray *> columns(num_arguments);
+
+        for (size_t i = 0; i < num_arguments; ++i)
+        {
+            holders[i] = arguments[i].column->convertToFullColumnIfConst();
+            if (holders[i]->size() != input_rows_count)
+                throw Exception(
+                    ErrorCodes::LOGICAL_ERROR,
+                    "Function {} has unequal number of rows in columns: "
+                    "expected {}, got {} for column {}",
+                    getName(),
+                    input_rows_count,
+                    holders[i]->size(),
+                    holders[i]->getName());
+            columns[i] = assert_cast<const ColumnArray*>(holders[i].get());
+        }
+        return execute(columns);
+    }
+private:
+    ColumnPtr execute(std::vector<const ColumnArray *>) const
+    {
+        throw Exception(
+            ErrorCodes::LOGICAL_ERROR,
+            "Unknown function {}. "
+            "Supported names: 'arrayLevenshtein', 'arrayLevenshteinWeighted', 'arraySimilarity'",
+            T::name);
+    }
+
+    template<typename N>
+    UInt32 levenshteinDistance(const std::vector<N> from, const std::vector<N> to) const
+    {
+        const size_t m = from.size();
+        const size_t n = to.size();
+        if (m==0 || n==0)
+        {
+            return static_cast<UInt32>(m + n);
+        }
+        PODArrayWithStackMemory<size_t, 32> v0(n + 1);
+
+        iota(v0.data() + 1, n, size_t(1));
+
+        for (size_t j = 1; j <= m; ++j)
+        {
+            v0[0] = j;
+            size_t prev = j - 1;
+            for (size_t i = 1; i <= n; ++i)
+            {
+                size_t old = v0[i];
+                v0[i] = std::min(prev + (from[j - 1] != to[i - 1]),
+                        std::min(v0[i - 1], v0[i]) + 1);
+                prev = old;
+            }
+        }
+        return static_cast<UInt32>(v0[n]);
+    }
+
+    template <typename N>
+    bool levenshteinString(std::vector<const ColumnArray *> columns, ColumnUInt32::Container & res_values) const
+    {
+        const N * from_data = checkAndGetColumn<N>(&columns[0]->getData());
+        const N * to_data = checkAndGetColumn<N>(&columns[1]->getData());
+        if (!from_data || !to_data)
+            return false;
+        const ColumnArray::Offsets & from_offsets = columns[0]->getOffsets();
+        ColumnArray::Offset prev_from_offset = 0;
+
+        const ColumnArray::Offsets & to_offsets = columns[1]->getOffsets();
+        ColumnArray::Offset prev_to_offset = 0;
+
+        for (size_t row = 0; row < columns[0]->size(); row++)
+        {
+            const size_t m = from_offsets[row] - prev_from_offset;
+            const size_t n = to_offsets[row] - prev_to_offset;
+            const std::vector<StringRef> & from = [&]()
+            {
+                std::vector<StringRef> temp;
+                temp.reserve(m);
+                for (size_t j = 0; j < m; ++j) { temp.emplace_back(from_data->getDataAt(prev_from_offset + j)); }
+                return temp;
+            }();
+            const std::vector<StringRef> & to = [&]()
+            {
+                std::vector<StringRef> temp;
+                temp.reserve(m);
+                for (size_t j = 0; j < n; ++j) { temp.emplace_back(to_data->getDataAt(prev_to_offset + j)); }
+                return temp;
+            }();
+            prev_from_offset = from_offsets[row];
+            prev_to_offset = to_offsets[row];
+            res_values[row] = levenshteinDistance<StringRef>(from, to);
+        }
+        return true;
+    }
+
+    template <typename N>
+    bool levenshteinNumber(std::vector<const ColumnArray *> columns, ColumnUInt32::Container & res_values) const
+    {
+        const ColumnVectorOrDecimal<N> * column_from = checkAndGetColumn<ColumnVectorOrDecimal<N>>(&columns[0]->getData());
+        const ColumnVectorOrDecimal<N> * column_to = checkAndGetColumn<ColumnVectorOrDecimal<N>>(&columns[1]->getData());
+        if (!column_from || !column_to)
+            return false;
+        const PaddedPODArray<N> & vec_from = column_from->getData();
+        const ColumnArray::Offsets & from_offsets = columns[0]->getOffsets();
+        ColumnArray::Offset prev_from_offset = 0;
+
+        const PaddedPODArray<N> & vec_to = column_to->getData();
+        const ColumnArray::Offsets & to_offsets = columns[1]->getOffsets();
+        ColumnArray::Offset prev_to_offset = 0;
+
+        for (size_t row = 0; row < columns[0]->size(); row++)
+        {
+            const std::vector<N> from(vec_from.begin() + prev_from_offset, vec_from.begin() + from_offsets[row]);
+            prev_from_offset = from_offsets[row];
+            const std::vector<N> to(vec_to.begin() + prev_to_offset, vec_to.begin() + to_offsets[row]);
+            prev_to_offset = to_offsets[row];
path: src/Functions/array/arrayLevenshtein.cpp
user: login: Michicosun
body: here and in other places if you use `std::span` in `levenshteinDistance` you can even do not copy values, just use view over ColumnArray's data, like this:

```cpp
std::span<N> from(vec_from.begin() + prev_from_offset, vec_from.begin() + from_offsets[row])
``` 
created_at: 2025-03-24T10:36:51Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,626 @@
+#include <Columns/ColumnArray.h>
+#include <Columns/ColumnDecimal.h>
+#include <Columns/ColumnFixedString.h>
+#include <Columns/ColumnNullable.h>
+#include <Columns/ColumnString.h>
+#include <Columns/ColumnTuple.h>
+#include <Common/PODArray.h>
+#include <Common/iota.h>
+#include <DataTypes/DataTypeArray.h>
+#include <DataTypes/DataTypeNullable.h>
+#include <DataTypes/DataTypeTuple.h>
+#include <DataTypes/DataTypesNumber.h>
+#include <Functions/FunctionFactory.h>
+#include <Functions/FunctionHelpers.h>
+#include <IO/WriteHelpers.h>
+
+#include <numeric>
+
+namespace DB
+{
+
+namespace ErrorCodes
+{
+    extern const int ILLEGAL_TYPE_OF_ARGUMENT;
+    extern const int LOGICAL_ERROR;
+    extern const int SIZES_OF_ARRAYS_DONT_MATCH;
+}
+
+/// arrayLevenshtein([1,2,3,4], [1,3,2,4]) = 2
+/// arrayLevenshteinWeighted([1,2,3,4], [1,3,2,4]) = 2
+template <typename T>
+class FunctionArrayLevenshtein : public IFunction
+{
+public:
+    static constexpr auto name = T::name;
+    static FunctionPtr create(ContextPtr) { return std::make_shared<FunctionArrayLevenshtein<T>>(); }
+
+    String getName() const override { return name; }
+
+    size_t getNumberOfArguments() const override { return T::arguments; }
+    bool useDefaultImplementationForConstants() const override { return true; }
+
+    bool isSuitableForShortCircuitArgumentsExecution(const DataTypesWithConstInfo & /*arguments*/) const override { return true; }
+
+    DataTypePtr getReturnTypeImpl(const ColumnsWithTypeAndName & arguments) const override
+    {
+        FunctionArgumentDescriptors args_descriptors;
+        args_descriptors = FunctionArgumentDescriptors{
+            {"from", static_cast<FunctionArgumentDescriptor::TypeValidator>(&isArray), nullptr, "Array"},
+            {"to", static_cast<FunctionArgumentDescriptor::TypeValidator>(&isArray), nullptr, "Array"},
+            {"from_weights", static_cast<FunctionArgumentDescriptor::TypeValidator>(&isArray), nullptr, "Array"},
+            {"to_weights", static_cast<FunctionArgumentDescriptor::TypeValidator>(&isArray), nullptr, "Array"},
+        };
+        validateFunctionArguments(*this, arguments, args_descriptors);
+        std::vector<DataTypePtr> nested_types;
+        nested_types.reserve(2);
+        for (size_t index = 2; index < 4; ++index)
+        {
+            const DataTypeArray * array_type = checkAndGetDataType<DataTypeArray>(arguments[index].type.get());
+            const DataTypePtr nested_type = array_type->getNestedType();
+            nested_types.emplace_back(nested_type);
+            // TODO: extend to work with different types?
+            if (!WhichDataType(nested_type).isFloat64())
+                throw Exception(
+                    ErrorCodes::ILLEGAL_TYPE_OF_ARGUMENT,
+                    "Argument {} of function {} must be array of Float64. Found {} instead.",
+                    toString(index + 1),
+                    getName(),
+                    nested_type->getName());
+        }
+
+        if (nested_types[0]->getName() != nested_types[1]->getName())
+            throw Exception(
+                ErrorCodes::ILLEGAL_TYPE_OF_ARGUMENT,
+                "Arguments 3 and 4 of function {} must be arrays of the same types. Found {} and {} instead.",
+                getName(),
+                nested_types[0]->getName(),
+                nested_types[1]->getName());
+        return std::make_shared<DataTypeFloat64>();
+    }
+
+    ColumnPtr executeImpl(const ColumnsWithTypeAndName & arguments, const DataTypePtr &, size_t input_rows_count) const override
+    {
+        size_t num_arguments = arguments.size();
+
+        Columns holders(num_arguments);
+        std::vector<const ColumnArray *> columns(num_arguments);
+
+        for (size_t i = 0; i < num_arguments; ++i)
+        {
+            holders[i] = arguments[i].column->convertToFullColumnIfConst();
+            if (holders[i]->size() != input_rows_count)
+                throw Exception(
+                    ErrorCodes::LOGICAL_ERROR,
+                    "Function {} has unequal number of rows in columns: "
+                    "expected {}, got {} for column {}",
+                    getName(),
+                    input_rows_count,
+                    holders[i]->size(),
+                    holders[i]->getName());
+            columns[i] = assert_cast<const ColumnArray*>(holders[i].get());
+        }
+        return execute(columns);
+    }
+private:
+    ColumnPtr execute(std::vector<const ColumnArray *>) const
+    {
+        throw Exception(
+            ErrorCodes::LOGICAL_ERROR,
+            "Unknown function {}. "
+            "Supported names: 'arrayLevenshtein', 'arrayLevenshteinWeighted', 'arraySimilarity'",
+            T::name);
+    }
+
+    template<typename N>
+    UInt32 levenshteinDistance(const std::vector<N> from, const std::vector<N> to) const
+    {
+        const size_t m = from.size();
+        const size_t n = to.size();
+        if (m==0 || n==0)
+        {
+            return static_cast<UInt32>(m + n);
+        }
+        PODArrayWithStackMemory<size_t, 32> v0(n + 1);
+
+        iota(v0.data() + 1, n, size_t(1));
+
+        for (size_t j = 1; j <= m; ++j)
+        {
+            v0[0] = j;
+            size_t prev = j - 1;
+            for (size_t i = 1; i <= n; ++i)
+            {
+                size_t old = v0[i];
+                v0[i] = std::min(prev + (from[j - 1] != to[i - 1]),
+                        std::min(v0[i - 1], v0[i]) + 1);
+                prev = old;
+            }
+        }
+        return static_cast<UInt32>(v0[n]);
+    }
+
+    template <typename N>
+    bool levenshteinString(std::vector<const ColumnArray *> columns, ColumnUInt32::Container & res_values) const
+    {
+        const N * from_data = checkAndGetColumn<N>(&columns[0]->getData());
+        const N * to_data = checkAndGetColumn<N>(&columns[1]->getData());
+        if (!from_data || !to_data)
+            return false;
+        const ColumnArray::Offsets & from_offsets = columns[0]->getOffsets();
+        ColumnArray::Offset prev_from_offset = 0;
+
+        const ColumnArray::Offsets & to_offsets = columns[1]->getOffsets();
+        ColumnArray::Offset prev_to_offset = 0;
+
+        for (size_t row = 0; row < columns[0]->size(); row++)
+        {
+            const size_t m = from_offsets[row] - prev_from_offset;
+            const size_t n = to_offsets[row] - prev_to_offset;
+            const std::vector<StringRef> & from = [&]()
+            {
+                std::vector<StringRef> temp;
+                temp.reserve(m);
+                for (size_t j = 0; j < m; ++j) { temp.emplace_back(from_data->getDataAt(prev_from_offset + j)); }
+                return temp;
+            }();
+            const std::vector<StringRef> & to = [&]()
+            {
+                std::vector<StringRef> temp;
+                temp.reserve(m);
+                for (size_t j = 0; j < n; ++j) { temp.emplace_back(to_data->getDataAt(prev_to_offset + j)); }
+                return temp;
+            }();
+            prev_from_offset = from_offsets[row];
+            prev_to_offset = to_offsets[row];
+            res_values[row] = levenshteinDistance<StringRef>(from, to);
+        }
+        return true;
+    }
+
+    template <typename N>
+    bool levenshteinNumber(std::vector<const ColumnArray *> columns, ColumnUInt32::Container & res_values) const
+    {
+        const ColumnVectorOrDecimal<N> * column_from = checkAndGetColumn<ColumnVectorOrDecimal<N>>(&columns[0]->getData());
+        const ColumnVectorOrDecimal<N> * column_to = checkAndGetColumn<ColumnVectorOrDecimal<N>>(&columns[1]->getData());
+        if (!column_from || !column_to)
+            return false;
+        const PaddedPODArray<N> & vec_from = column_from->getData();
+        const ColumnArray::Offsets & from_offsets = columns[0]->getOffsets();
+        ColumnArray::Offset prev_from_offset = 0;
+
+        const PaddedPODArray<N> & vec_to = column_to->getData();
+        const ColumnArray::Offsets & to_offsets = columns[1]->getOffsets();
+        ColumnArray::Offset prev_to_offset = 0;
+
+        for (size_t row = 0; row < columns[0]->size(); row++)
+        {
+            const std::vector<N> from(vec_from.begin() + prev_from_offset, vec_from.begin() + from_offsets[row]);
+            prev_from_offset = from_offsets[row];
+            const std::vector<N> to(vec_to.begin() + prev_to_offset, vec_to.begin() + to_offsets[row]);
+            prev_to_offset = to_offsets[row];
+
+            res_values[row] = levenshteinDistance<N>(from, to);
+        }
+        return true;
+    }
+
+    void levenshteinGeneric(std::vector<const ColumnArray *> columns, ColumnUInt32::Container & res_values) const
+    {
+        const ColumnArray * column_from = columns[0];
+        const ColumnArray * column_to = columns[1];
+        for (size_t row = 0; row < column_from->size(); row++)
+        {
+            // Effective Levenshtein realization from Common/levenshteinDistance
+            Array from = (*column_from)[row].safeGet<Array>();
+            Array to = (*column_to)[row].safeGet<Array>();
+            const std::vector<Field> from_vec(from.begin(), from.end());
+            const std::vector<Field> to_vec(to.begin(), to.end());
path: src/Functions/array/arrayLevenshtein.cpp
user: login: Michicosun
body: `Array` is already a vector according to the definition, why do you need to copy it here?
created_at: 2025-03-24T10:39:30Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,626 @@
+#include <Columns/ColumnArray.h>
+#include <Columns/ColumnDecimal.h>
+#include <Columns/ColumnFixedString.h>
+#include <Columns/ColumnNullable.h>
+#include <Columns/ColumnString.h>
+#include <Columns/ColumnTuple.h>
+#include <Common/PODArray.h>
+#include <Common/iota.h>
+#include <DataTypes/DataTypeArray.h>
+#include <DataTypes/DataTypeNullable.h>
+#include <DataTypes/DataTypeTuple.h>
+#include <DataTypes/DataTypesNumber.h>
+#include <Functions/FunctionFactory.h>
+#include <Functions/FunctionHelpers.h>
+#include <IO/WriteHelpers.h>
+
+#include <numeric>
+
+namespace DB
+{
+
+namespace ErrorCodes
+{
+    extern const int ILLEGAL_TYPE_OF_ARGUMENT;
+    extern const int LOGICAL_ERROR;
+    extern const int SIZES_OF_ARRAYS_DONT_MATCH;
+}
+
+/// arrayLevenshtein([1,2,3,4], [1,3,2,4]) = 2
+/// arrayLevenshteinWeighted([1,2,3,4], [1,3,2,4]) = 2
+template <typename T>
+class FunctionArrayLevenshtein : public IFunction
+{
+public:
+    static constexpr auto name = T::name;
+    static FunctionPtr create(ContextPtr) { return std::make_shared<FunctionArrayLevenshtein<T>>(); }
+
+    String getName() const override { return name; }
+
+    size_t getNumberOfArguments() const override { return T::arguments; }
+    bool useDefaultImplementationForConstants() const override { return true; }
+
+    bool isSuitableForShortCircuitArgumentsExecution(const DataTypesWithConstInfo & /*arguments*/) const override { return true; }
+
+    DataTypePtr getReturnTypeImpl(const ColumnsWithTypeAndName & arguments) const override
+    {
+        FunctionArgumentDescriptors args_descriptors;
+        args_descriptors = FunctionArgumentDescriptors{
+            {"from", static_cast<FunctionArgumentDescriptor::TypeValidator>(&isArray), nullptr, "Array"},
+            {"to", static_cast<FunctionArgumentDescriptor::TypeValidator>(&isArray), nullptr, "Array"},
+            {"from_weights", static_cast<FunctionArgumentDescriptor::TypeValidator>(&isArray), nullptr, "Array"},
+            {"to_weights", static_cast<FunctionArgumentDescriptor::TypeValidator>(&isArray), nullptr, "Array"},
+        };
+        validateFunctionArguments(*this, arguments, args_descriptors);
+        std::vector<DataTypePtr> nested_types;
+        nested_types.reserve(2);
+        for (size_t index = 2; index < 4; ++index)
+        {
+            const DataTypeArray * array_type = checkAndGetDataType<DataTypeArray>(arguments[index].type.get());
+            const DataTypePtr nested_type = array_type->getNestedType();
+            nested_types.emplace_back(nested_type);
+            // TODO: extend to work with different types?
+            if (!WhichDataType(nested_type).isFloat64())
+                throw Exception(
+                    ErrorCodes::ILLEGAL_TYPE_OF_ARGUMENT,
+                    "Argument {} of function {} must be array of Float64. Found {} instead.",
+                    toString(index + 1),
+                    getName(),
+                    nested_type->getName());
+        }
+
+        if (nested_types[0]->getName() != nested_types[1]->getName())
+            throw Exception(
+                ErrorCodes::ILLEGAL_TYPE_OF_ARGUMENT,
+                "Arguments 3 and 4 of function {} must be arrays of the same types. Found {} and {} instead.",
+                getName(),
+                nested_types[0]->getName(),
+                nested_types[1]->getName());
+        return std::make_shared<DataTypeFloat64>();
+    }
+
+    ColumnPtr executeImpl(const ColumnsWithTypeAndName & arguments, const DataTypePtr &, size_t input_rows_count) const override
+    {
+        size_t num_arguments = arguments.size();
+
+        Columns holders(num_arguments);
+        std::vector<const ColumnArray *> columns(num_arguments);
+
+        for (size_t i = 0; i < num_arguments; ++i)
+        {
+            holders[i] = arguments[i].column->convertToFullColumnIfConst();
+            if (holders[i]->size() != input_rows_count)
+                throw Exception(
+                    ErrorCodes::LOGICAL_ERROR,
+                    "Function {} has unequal number of rows in columns: "
+                    "expected {}, got {} for column {}",
+                    getName(),
+                    input_rows_count,
+                    holders[i]->size(),
+                    holders[i]->getName());
+            columns[i] = assert_cast<const ColumnArray*>(holders[i].get());
+        }
+        return execute(columns);
+    }
+private:
+    ColumnPtr execute(std::vector<const ColumnArray *>) const
+    {
+        throw Exception(
+            ErrorCodes::LOGICAL_ERROR,
+            "Unknown function {}. "
+            "Supported names: 'arrayLevenshtein', 'arrayLevenshteinWeighted', 'arraySimilarity'",
+            T::name);
+    }
path: src/Functions/array/arrayLevenshtein.cpp
user: login: Michicosun
body: Just don't implement it here, the compiler will check it automatically for each class specialization.

created_at: 2025-03-24T10:58:41Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,626 @@
+#include <Columns/ColumnArray.h>
+#include <Columns/ColumnDecimal.h>
+#include <Columns/ColumnFixedString.h>
+#include <Columns/ColumnNullable.h>
+#include <Columns/ColumnString.h>
+#include <Columns/ColumnTuple.h>
+#include <Common/PODArray.h>
+#include <Common/iota.h>
+#include <DataTypes/DataTypeArray.h>
+#include <DataTypes/DataTypeNullable.h>
+#include <DataTypes/DataTypeTuple.h>
+#include <DataTypes/DataTypesNumber.h>
+#include <Functions/FunctionFactory.h>
+#include <Functions/FunctionHelpers.h>
+#include <IO/WriteHelpers.h>
+
+#include <numeric>
+
+namespace DB
+{
+
+namespace ErrorCodes
+{
+    extern const int ILLEGAL_TYPE_OF_ARGUMENT;
+    extern const int LOGICAL_ERROR;
+    extern const int SIZES_OF_ARRAYS_DONT_MATCH;
+}
+
+/// arrayLevenshtein([1,2,3,4], [1,3,2,4]) = 2
+/// arrayLevenshteinWeighted([1,2,3,4], [1,3,2,4]) = 2
+template <typename T>
+class FunctionArrayLevenshtein : public IFunction
+{
+public:
+    static constexpr auto name = T::name;
+    static FunctionPtr create(ContextPtr) { return std::make_shared<FunctionArrayLevenshtein<T>>(); }
+
+    String getName() const override { return name; }
+
+    size_t getNumberOfArguments() const override { return T::arguments; }
+    bool useDefaultImplementationForConstants() const override { return true; }
+
+    bool isSuitableForShortCircuitArgumentsExecution(const DataTypesWithConstInfo & /*arguments*/) const override { return true; }
+
+    DataTypePtr getReturnTypeImpl(const ColumnsWithTypeAndName & arguments) const override
+    {
+        FunctionArgumentDescriptors args_descriptors;
+        args_descriptors = FunctionArgumentDescriptors{
+            {"from", static_cast<FunctionArgumentDescriptor::TypeValidator>(&isArray), nullptr, "Array"},
+            {"to", static_cast<FunctionArgumentDescriptor::TypeValidator>(&isArray), nullptr, "Array"},
+            {"from_weights", static_cast<FunctionArgumentDescriptor::TypeValidator>(&isArray), nullptr, "Array"},
+            {"to_weights", static_cast<FunctionArgumentDescriptor::TypeValidator>(&isArray), nullptr, "Array"},
+        };
+        validateFunctionArguments(*this, arguments, args_descriptors);
+        std::vector<DataTypePtr> nested_types;
+        nested_types.reserve(2);
+        for (size_t index = 2; index < 4; ++index)
+        {
+            const DataTypeArray * array_type = checkAndGetDataType<DataTypeArray>(arguments[index].type.get());
+            const DataTypePtr nested_type = array_type->getNestedType();
+            nested_types.emplace_back(nested_type);
+            // TODO: extend to work with different types?
+            if (!WhichDataType(nested_type).isFloat64())
+                throw Exception(
+                    ErrorCodes::ILLEGAL_TYPE_OF_ARGUMENT,
+                    "Argument {} of function {} must be array of Float64. Found {} instead.",
+                    toString(index + 1),
+                    getName(),
+                    nested_type->getName());
+        }
+
+        if (nested_types[0]->getName() != nested_types[1]->getName())
+            throw Exception(
+                ErrorCodes::ILLEGAL_TYPE_OF_ARGUMENT,
+                "Arguments 3 and 4 of function {} must be arrays of the same types. Found {} and {} instead.",
+                getName(),
+                nested_types[0]->getName(),
+                nested_types[1]->getName());
+        return std::make_shared<DataTypeFloat64>();
+    }
+
+    ColumnPtr executeImpl(const ColumnsWithTypeAndName & arguments, const DataTypePtr &, size_t input_rows_count) const override
+    {
+        size_t num_arguments = arguments.size();
+
+        Columns holders(num_arguments);
+        std::vector<const ColumnArray *> columns(num_arguments);
+
+        for (size_t i = 0; i < num_arguments; ++i)
+        {
+            holders[i] = arguments[i].column->convertToFullColumnIfConst();
+            if (holders[i]->size() != input_rows_count)
+                throw Exception(
+                    ErrorCodes::LOGICAL_ERROR,
+                    "Function {} has unequal number of rows in columns: "
+                    "expected {}, got {} for column {}",
+                    getName(),
+                    input_rows_count,
+                    holders[i]->size(),
+                    holders[i]->getName());
+            columns[i] = assert_cast<const ColumnArray*>(holders[i].get());
+        }
+        return execute(columns);
+    }
+private:
+    ColumnPtr execute(std::vector<const ColumnArray *>) const
+    {
+        throw Exception(
+            ErrorCodes::LOGICAL_ERROR,
+            "Unknown function {}. "
+            "Supported names: 'arrayLevenshtein', 'arrayLevenshteinWeighted', 'arraySimilarity'",
+            T::name);
+    }
+
+    template<typename N>
+    UInt32 levenshteinDistance(const std::vector<N> from, const std::vector<N> to) const
+    {
+        const size_t m = from.size();
+        const size_t n = to.size();
+        if (m==0 || n==0)
+        {
+            return static_cast<UInt32>(m + n);
+        }
+        PODArrayWithStackMemory<size_t, 32> v0(n + 1);
+
+        iota(v0.data() + 1, n, size_t(1));
+
+        for (size_t j = 1; j <= m; ++j)
+        {
+            v0[0] = j;
+            size_t prev = j - 1;
+            for (size_t i = 1; i <= n; ++i)
+            {
+                size_t old = v0[i];
+                v0[i] = std::min(prev + (from[j - 1] != to[i - 1]),
+                        std::min(v0[i - 1], v0[i]) + 1);
+                prev = old;
+            }
+        }
+        return static_cast<UInt32>(v0[n]);
+    }
+
+    template <typename N>
+    bool levenshteinString(std::vector<const ColumnArray *> columns, ColumnUInt32::Container & res_values) const
+    {
+        const N * from_data = checkAndGetColumn<N>(&columns[0]->getData());
+        const N * to_data = checkAndGetColumn<N>(&columns[1]->getData());
+        if (!from_data || !to_data)
+            return false;
+        const ColumnArray::Offsets & from_offsets = columns[0]->getOffsets();
+        ColumnArray::Offset prev_from_offset = 0;
+
+        const ColumnArray::Offsets & to_offsets = columns[1]->getOffsets();
+        ColumnArray::Offset prev_to_offset = 0;
+
+        for (size_t row = 0; row < columns[0]->size(); row++)
+        {
+            const size_t m = from_offsets[row] - prev_from_offset;
+            const size_t n = to_offsets[row] - prev_to_offset;
+            const std::vector<StringRef> & from = [&]()
+            {
+                std::vector<StringRef> temp;
+                temp.reserve(m);
+                for (size_t j = 0; j < m; ++j) { temp.emplace_back(from_data->getDataAt(prev_from_offset + j)); }
+                return temp;
+            }();
+            const std::vector<StringRef> & to = [&]()
+            {
+                std::vector<StringRef> temp;
+                temp.reserve(m);
+                for (size_t j = 0; j < n; ++j) { temp.emplace_back(to_data->getDataAt(prev_to_offset + j)); }
+                return temp;
+            }();
+            prev_from_offset = from_offsets[row];
+            prev_to_offset = to_offsets[row];
+            res_values[row] = levenshteinDistance<StringRef>(from, to);
+        }
+        return true;
+    }
+
+    template <typename N>
+    bool levenshteinNumber(std::vector<const ColumnArray *> columns, ColumnUInt32::Container & res_values) const
+    {
+        const ColumnVectorOrDecimal<N> * column_from = checkAndGetColumn<ColumnVectorOrDecimal<N>>(&columns[0]->getData());
+        const ColumnVectorOrDecimal<N> * column_to = checkAndGetColumn<ColumnVectorOrDecimal<N>>(&columns[1]->getData());
+        if (!column_from || !column_to)
+            return false;
+        const PaddedPODArray<N> & vec_from = column_from->getData();
+        const ColumnArray::Offsets & from_offsets = columns[0]->getOffsets();
+        ColumnArray::Offset prev_from_offset = 0;
+
+        const PaddedPODArray<N> & vec_to = column_to->getData();
+        const ColumnArray::Offsets & to_offsets = columns[1]->getOffsets();
+        ColumnArray::Offset prev_to_offset = 0;
+
+        for (size_t row = 0; row < columns[0]->size(); row++)
+        {
+            const std::vector<N> from(vec_from.begin() + prev_from_offset, vec_from.begin() + from_offsets[row]);
+            prev_from_offset = from_offsets[row];
+            const std::vector<N> to(vec_to.begin() + prev_to_offset, vec_to.begin() + to_offsets[row]);
+            prev_to_offset = to_offsets[row];
+
+            res_values[row] = levenshteinDistance<N>(from, to);
+        }
+        return true;
+    }
+
+    void levenshteinGeneric(std::vector<const ColumnArray *> columns, ColumnUInt32::Container & res_values) const
+    {
+        const ColumnArray * column_from = columns[0];
+        const ColumnArray * column_to = columns[1];
+        for (size_t row = 0; row < column_from->size(); row++)
+        {
+            // Effective Levenshtein realization from Common/levenshteinDistance
+            Array from = (*column_from)[row].safeGet<Array>();
+            Array to = (*column_to)[row].safeGet<Array>();
+            const std::vector<Field> from_vec(from.begin(), from.end());
+            const std::vector<Field> to_vec(to.begin(), to.end());
+            res_values[row] = levenshteinDistance<Field>(from_vec, to_vec);
+        }
+    }
+
+    template<typename N>
+    Float64 levenshteinDistanceWeighted(const std::vector<N> from, const std::vector<N> to,
+                                        const std::vector<Float64> from_weights, const std::vector<Float64> to_weights) const
+    {
+        auto sum_vec = [](const std::vector<Float64> & vector) -> Float64
+        {
+            return std::accumulate(vector.begin(), vector.end(), .0);
+        };
+        const size_t m = std::min(from.size(), to.size());
+        const size_t n = std::max(from.size(), to.size());
+        if (m==0 || n==0)
+        {
+            return sum_vec(from_weights) + sum_vec(to_weights);
+        }
+        // Consume minimum memory by allocating sliding vectors for min `m`
+        auto & lhs = (from.size() <= to.size()) ? from : to;
+        const std::vector<Float64> & lhs_w = (from.size() <= to.size()) ? from_weights : to_weights;
+        auto & rhs = (from.size() <= to.size()) ? to : from;
+        const std::vector<Float64> & rhs_w = (from.size() <= to.size()) ? to_weights : from_weights;
+
+        PODArrayWithStackMemory<Float64, 64> v0(m + 1);
+        PODArrayWithStackMemory<Float64, 64> v1(m + 1);
+
+        v0[0] = 0;
+        std::partial_sum(lhs_w.begin(), lhs_w.end(), v0.begin() + 1);
+
+        for (size_t i = 0; i < n; ++i)
+        {
+            v1[0] = v0[0] + rhs_w[i];
+            for (size_t j = 0; j < m; ++j)
+            {
+                if (lhs[j] == rhs[i])
+                {
+                    v1[j + 1] = v0[j];
+                    continue;
+                }
+
+                v1[j+1] = std::min({v0[j + 1] + rhs_w[i],          // deletion
+                                    v1[j] + lhs_w[j],              // insertion
+                                    v0[j] + lhs_w[j] + rhs_w[i]}); // substitusion
+            }
+            std::swap(v0, v1);
+        }
+        return v0[m];
+    }
+
+    template <typename N>
+    bool levenshteinWeightedString(std::vector<const ColumnArray *> columns, ColumnFloat64::Container & res_values) const
+    {
+        const N * from_data = checkAndGetColumn<N>(&columns[0]->getData());
+        const N * to_data = checkAndGetColumn<N>(&columns[1]->getData());
+        if (!from_data || !to_data)
+            return false;
+        const ColumnArray::Offsets & from_offsets = columns[0]->getOffsets();
+        ColumnArray::Offset prev_from_offset = 0;
+
+        const ColumnArray::Offsets & to_offsets = columns[1]->getOffsets();
+        ColumnArray::Offset prev_to_offset = 0;
+
+        const ColumnVector<Float64> * column_from_weights = checkAndGetColumn<ColumnVector<Float64>>(&columns[2]->getData());
+        const ColumnVector<Float64> * column_to_weights = checkAndGetColumn<ColumnVector<Float64>>(&columns[3]->getData());
+        if (!column_from_weights || !column_to_weights)
+            throw Exception(
+                ErrorCodes::LOGICAL_ERROR,
+                "Function {} wrong type of weight columns",
+                getName());
+        const PaddedPODArray<Float64> & vec_from_weights = column_from_weights->getData();
+        const ColumnArray::Offsets & from_weights_offsets = columns[2]->getOffsets();
+        ColumnArray::Offset prev_from_weights_offset = 0;
+
+        const PaddedPODArray<Float64> & vec_to_weights = column_to_weights->getData();
+        const ColumnArray::Offsets & to_weights_offsets = columns[3]->getOffsets();
+        ColumnArray::Offset prev_to_weights_offset = 0;
+
+        for (size_t row = 0; row < columns[0]->size(); row++)
+        {
+            const size_t m = from_offsets[row] - prev_from_offset;
+            const size_t n = to_offsets[row] - prev_to_offset;
+            const std::vector<StringRef> & from = [&]()
+            {
+                std::vector<StringRef> temp;
+                temp.reserve(m);
+                for (size_t j = 0; j < m; ++j) { temp.emplace_back(from_data->getDataAt(prev_from_offset + j)); }
+                return temp;
+            }();
+            const std::vector<StringRef> & to = [&]()
+            {
+                std::vector<StringRef> temp;
+                temp.reserve(m);
+                for (size_t j = 0; j < n; ++j) { temp.emplace_back(to_data->getDataAt(prev_to_offset + j)); }
+                return temp;
+            }();
+            prev_from_offset = from_offsets[row];
+            prev_to_offset = to_offsets[row];
+
+            const std::vector<Float64> from_weights(vec_from_weights.begin() + prev_from_weights_offset, vec_from_weights.begin() + from_weights_offsets[row]);
+            prev_from_weights_offset = from_weights_offsets[row];
+            const std::vector<Float64> to_weights(vec_to_weights.begin() + prev_to_weights_offset, vec_to_weights.begin() + to_weights_offsets[row]);
+            prev_to_weights_offset = to_weights_offsets[row];
+
+            res_values[row] = levenshteinDistanceWeighted<StringRef>(from, to, from_weights, to_weights);
+        }
+        return true;
+    }
+
+    template <typename N>
+    bool levenshteinWeightedNumber(std::vector<const ColumnArray *> columns, ColumnFloat64::Container & res_values) const
+    {
+        const ColumnVectorOrDecimal<N> * column_from = checkAndGetColumn<ColumnVectorOrDecimal<N>>(&columns[0]->getData());
+        const ColumnVectorOrDecimal<N> * column_to = checkAndGetColumn<ColumnVectorOrDecimal<N>>(&columns[1]->getData());
+        if (!column_from || !column_to)
+            return false;
+        const PaddedPODArray<N> & vec_from = column_from->getData();
+        const ColumnArray::Offsets & from_offsets = columns[0]->getOffsets();
+        ColumnArray::Offset prev_from_offset = 0;
+
+        const PaddedPODArray<N> & vec_to = column_to->getData();
+        const ColumnArray::Offsets & to_offsets = columns[1]->getOffsets();
+        ColumnArray::Offset prev_to_offset = 0;
+
+        const ColumnVector<Float64> * column_from_weights = checkAndGetColumn<ColumnVector<Float64>>(&columns[2]->getData());
+        const ColumnVector<Float64> * column_to_weights = checkAndGetColumn<ColumnVector<Float64>>(&columns[3]->getData());
+        if (!column_from_weights || !column_to_weights)
+            throw Exception(
+                ErrorCodes::LOGICAL_ERROR,
+                "Function {} wrong type of weight columns",
+                getName());
+        const PaddedPODArray<Float64> & vec_from_weights = column_from_weights->getData();
+        const ColumnArray::Offsets & from_weights_offsets = columns[2]->getOffsets();
+        ColumnArray::Offset prev_from_weights_offset = 0;
+
+        const PaddedPODArray<Float64> & vec_to_weights = column_to_weights->getData();
+        const ColumnArray::Offsets & to_weights_offsets = columns[3]->getOffsets();
+        ColumnArray::Offset prev_to_weights_offset = 0;
+
+        for (size_t row = 0; row < columns[0]->size(); row++)
+        {
+            const std::vector<N> from(vec_from.begin() + prev_from_offset, vec_from.begin() + from_offsets[row]);
+            prev_from_offset = from_offsets[row];
+            const std::vector<N> to(vec_to.begin() + prev_to_offset, vec_to.begin() + to_offsets[row]);
+            prev_to_offset = to_offsets[row];
+
+            const std::vector<Float64> from_weights(vec_from_weights.begin() + prev_from_weights_offset, vec_from_weights.begin() + from_weights_offsets[row]);
+            prev_from_weights_offset = from_weights_offsets[row];
+            const std::vector<Float64> to_weights(vec_to_weights.begin() + prev_to_weights_offset, vec_to_weights.begin() + to_weights_offsets[row]);
+            prev_to_weights_offset = to_weights_offsets[row];
+
+            res_values[row] = levenshteinDistanceWeighted<N>(from, to, from_weights, to_weights);
+        }
+        return true;
+    }
+
+    void levenshteinWeightedGeneric(std::vector<const ColumnArray *> columns, ColumnFloat64::Container & res_values) const
+    {
+        const ColumnArray * column_from = columns[0];
+        const ColumnArray * column_to = columns[1];
+
+        const ColumnVector<Float64> * column_from_weights = checkAndGetColumn<ColumnVector<Float64>>(&columns[2]->getData());
+        const ColumnVector<Float64> * column_to_weights = checkAndGetColumn<ColumnVector<Float64>>(&columns[3]->getData());
+        if (!column_from_weights || !column_to_weights)
+            throw Exception(
+                ErrorCodes::LOGICAL_ERROR,
+                "Function {} wrong type of weight columns",
+                getName());
+        const PaddedPODArray<Float64> & vec_from_weights = column_from_weights->getData();
+        const ColumnArray::Offsets & from_weights_offsets = columns[2]->getOffsets();
+        ColumnArray::Offset prev_from_weights_offset = 0;
+
+        const PaddedPODArray<Float64> & vec_to_weights = column_to_weights->getData();
+        const ColumnArray::Offsets & to_weights_offsets = columns[3]->getOffsets();
+        ColumnArray::Offset prev_to_weights_offset = 0;
+
+        for (size_t row = 0; row < column_from->size(); row++)
+        {
+            // Effective Levenshtein realization from Common/levenshteinDistance
+            Array from = (*column_from)[row].safeGet<Array>();
+            Array to = (*column_to)[row].safeGet<Array>();
+            const std::vector<Field> from_vec(from.begin(), from.end());
+            const std::vector<Field> to_vec(to.begin(), to.end());
+
+            const std::vector<Float64> from_weights(vec_from_weights.begin() + prev_from_weights_offset, vec_from_weights.begin() + from_weights_offsets[row]);
+            prev_from_weights_offset = from_weights_offsets[row];
+            const std::vector<Float64> to_weights(vec_to_weights.begin() + prev_to_weights_offset, vec_to_weights.begin() + to_weights_offsets[row]);
+            prev_to_weights_offset = to_weights_offsets[row];
+
+            res_values[row] = levenshteinDistanceWeighted<Field>(from_vec, to_vec, from_weights, to_weights);
+        }
+    }
+
+    MutableColumnPtr weightedLevenshteinImpl(std::vector<const ColumnArray *> columns) const
+    {
+        for (size_t i = 0; i < 2; i++)
+        {
+            const ColumnArray * hs_column = columns[i];
+            const ColumnArray * weights_column = columns[i+2];
+            for (size_t row = 0; row < hs_column->size(); row++)
+            {
+                Array hs = (*hs_column)[row].safeGet<Array>();
+                Array weights = (*weights_column)[row].safeGet<Array>();
+                if (hs.size() != weights.size())
+                    throw Exception(
+                        ErrorCodes::SIZES_OF_ARRAYS_DONT_MATCH,
+                        "Arguments {} ({}, size {}) and {} ({}, size {}) of function {} must be arrays of the same size",
+                        toString(i + 1),
+                        hs_column->getName(),
+                        hs.size(),
+                        toString(i + 3),
+                        weights_column->getName(),
+                        weights.size(),
+                        getName());
+            }
+        }
+        auto res = ColumnFloat64::create();
+        ColumnFloat64::Container & res_values = res->getData();
+        res_values.resize(columns[0]->size());
+        if (levenshteinWeightedNumber<UInt8>(columns, res_values) || levenshteinWeightedNumber<UInt16>(columns, res_values)
+            || levenshteinWeightedNumber<UInt32>(columns, res_values) || levenshteinWeightedNumber<UInt64>(columns, res_values)
+            || levenshteinWeightedNumber<UInt128>(columns, res_values) || levenshteinWeightedNumber<UInt256>(columns, res_values)
+            || levenshteinWeightedNumber<Int8>(columns, res_values) || levenshteinWeightedNumber<Int16>(columns, res_values)
+            || levenshteinWeightedNumber<Int32>(columns, res_values) || levenshteinWeightedNumber<Int64>(columns, res_values)
+            || levenshteinWeightedNumber<Int128>(columns, res_values) || levenshteinWeightedNumber<Int256>(columns, res_values)
+            || levenshteinWeightedNumber<Float32>(columns, res_values) || levenshteinWeightedNumber<Float64>(columns, res_values)
+            || levenshteinWeightedNumber<Decimal32>(columns, res_values) || levenshteinWeightedNumber<Decimal64>(columns, res_values)
+            || levenshteinWeightedNumber<Decimal128>(columns, res_values) || levenshteinWeightedNumber<Decimal256>(columns, res_values)
+            || levenshteinWeightedNumber<DateTime64>(columns, res_values)
+            || levenshteinWeightedString<ColumnString>(columns, res_values) || levenshteinWeightedString<ColumnFixedString>(columns, res_values))
+            return res;
+        levenshteinWeightedGeneric(columns, res_values);
+        return res;
+    }
+};
+
+struct SimpleLevenshtein
+{
+    static constexpr auto name{"arrayLevenshtein"};
+    static constexpr size_t arguments = 2;
+};
+
+template <>
+DataTypePtr FunctionArrayLevenshtein<SimpleLevenshtein>::getReturnTypeImpl(const ColumnsWithTypeAndName & arguments) const
+{
+    FunctionArgumentDescriptors args_descriptors;
+    args_descriptors = FunctionArgumentDescriptors{
+        {"from", static_cast<FunctionArgumentDescriptor::TypeValidator>(&isArray), nullptr, "Array"},
+        {"to", static_cast<FunctionArgumentDescriptor::TypeValidator>(&isArray), nullptr, "Array"},
+    };
+    validateFunctionArguments(*this, arguments, args_descriptors);
+    return std::make_shared<DataTypeUInt32>();
+}
+
+
+template <>
+ColumnPtr FunctionArrayLevenshtein<SimpleLevenshtein>::execute(std::vector<const ColumnArray *> columns) const
+{
+    auto res = ColumnUInt32::create();
+    ColumnUInt32::Container & res_values = res->getData();
+    res_values.resize(columns[0]->size());
+    if (levenshteinNumber<UInt8>(columns, res_values) || levenshteinNumber<UInt16>(columns, res_values)
+        || levenshteinNumber<UInt32>(columns, res_values) || levenshteinNumber<UInt64>(columns, res_values)
+        || levenshteinNumber<UInt128>(columns, res_values) || levenshteinNumber<UInt256>(columns, res_values)
+        || levenshteinNumber<Int8>(columns, res_values) || levenshteinNumber<Int16>(columns, res_values)
+        || levenshteinNumber<Int32>(columns, res_values) || levenshteinNumber<Int64>(columns, res_values)
+        || levenshteinNumber<Int128>(columns, res_values) || levenshteinNumber<Int256>(columns, res_values)
+        || levenshteinNumber<Float32>(columns, res_values) || levenshteinNumber<Float64>(columns, res_values)
+        || levenshteinNumber<Decimal32>(columns, res_values) || levenshteinNumber<Decimal64>(columns, res_values)
+        || levenshteinNumber<Decimal128>(columns, res_values) || levenshteinNumber<Decimal256>(columns, res_values)
+        || levenshteinNumber<DateTime64>(columns, res_values)
+        || levenshteinString<ColumnString>(columns, res_values) || levenshteinString<ColumnFixedString>(columns, res_values))
+        return res;
+    levenshteinGeneric(columns, res_values);
+    return res;
+}
+
+struct Weighted
+{
+    static constexpr auto name{"arrayLevenshteinWeighted"};
+    static constexpr size_t arguments = 4;
+};
+
+template <>
+ColumnPtr FunctionArrayLevenshtein<Weighted>::execute(std::vector<const ColumnArray *> columns) const
+{
+    return weightedLevenshteinImpl(columns);
+}
+
+struct Similarity
+{
+    static constexpr auto name{"arraySimilarity"};
+    static constexpr size_t arguments = 4;
+};
+
+template <>
+ColumnPtr FunctionArrayLevenshtein<Similarity>::execute(std::vector<const ColumnArray *> columns) const
+{
+     MutableColumnPtr distance = weightedLevenshteinImpl(columns);
+     auto result = ColumnFloat64::create();
+     ColumnFloat64::Container & res_values = result->getData();
+     res_values.resize(distance->size());
+     const ColumnVector<Float64> * column_from_weights = checkAndGetColumn<ColumnVector<Float64>>(&columns[2]->getData());
+     const ColumnVector<Float64> * column_to_weights = checkAndGetColumn<ColumnVector<Float64>>(&columns[3]->getData());
+     if (!column_from_weights || !column_to_weights)
+         throw Exception(
+                 ErrorCodes::LOGICAL_ERROR,
+                 "Function {} wrong type of weight columns",
+                 getName());
path: src/Functions/array/arrayLevenshtein.cpp
user: login: Michicosun
body: you don't need this check it is already performed in `weightedLevenshteinImpl `
created_at: 2025-03-24T11:05:24Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,626 @@
+#include <Columns/ColumnArray.h>
+#include <Columns/ColumnDecimal.h>
+#include <Columns/ColumnFixedString.h>
+#include <Columns/ColumnNullable.h>
+#include <Columns/ColumnString.h>
+#include <Columns/ColumnTuple.h>
+#include <Common/PODArray.h>
+#include <Common/iota.h>
+#include <DataTypes/DataTypeArray.h>
+#include <DataTypes/DataTypeNullable.h>
+#include <DataTypes/DataTypeTuple.h>
+#include <DataTypes/DataTypesNumber.h>
+#include <Functions/FunctionFactory.h>
+#include <Functions/FunctionHelpers.h>
+#include <IO/WriteHelpers.h>
+
+#include <numeric>
+
+namespace DB
+{
+
+namespace ErrorCodes
+{
+    extern const int ILLEGAL_TYPE_OF_ARGUMENT;
+    extern const int LOGICAL_ERROR;
+    extern const int SIZES_OF_ARRAYS_DONT_MATCH;
+}
+
+/// arrayLevenshtein([1,2,3,4], [1,3,2,4]) = 2
+/// arrayLevenshteinWeighted([1,2,3,4], [1,3,2,4]) = 2
+template <typename T>
+class FunctionArrayLevenshtein : public IFunction
+{
+public:
+    static constexpr auto name = T::name;
+    static FunctionPtr create(ContextPtr) { return std::make_shared<FunctionArrayLevenshtein<T>>(); }
+
+    String getName() const override { return name; }
+
+    size_t getNumberOfArguments() const override { return T::arguments; }
+    bool useDefaultImplementationForConstants() const override { return true; }
+
+    bool isSuitableForShortCircuitArgumentsExecution(const DataTypesWithConstInfo & /*arguments*/) const override { return true; }
+
+    DataTypePtr getReturnTypeImpl(const ColumnsWithTypeAndName & arguments) const override
+    {
+        FunctionArgumentDescriptors args_descriptors;
+        args_descriptors = FunctionArgumentDescriptors{
+            {"from", static_cast<FunctionArgumentDescriptor::TypeValidator>(&isArray), nullptr, "Array"},
+            {"to", static_cast<FunctionArgumentDescriptor::TypeValidator>(&isArray), nullptr, "Array"},
+            {"from_weights", static_cast<FunctionArgumentDescriptor::TypeValidator>(&isArray), nullptr, "Array"},
+            {"to_weights", static_cast<FunctionArgumentDescriptor::TypeValidator>(&isArray), nullptr, "Array"},
+        };
+        validateFunctionArguments(*this, arguments, args_descriptors);
+        std::vector<DataTypePtr> nested_types;
+        nested_types.reserve(2);
+        for (size_t index = 2; index < 4; ++index)
+        {
+            const DataTypeArray * array_type = checkAndGetDataType<DataTypeArray>(arguments[index].type.get());
+            const DataTypePtr nested_type = array_type->getNestedType();
+            nested_types.emplace_back(nested_type);
+            // TODO: extend to work with different types?
+            if (!WhichDataType(nested_type).isFloat64())
+                throw Exception(
+                    ErrorCodes::ILLEGAL_TYPE_OF_ARGUMENT,
+                    "Argument {} of function {} must be array of Float64. Found {} instead.",
+                    toString(index + 1),
+                    getName(),
+                    nested_type->getName());
+        }
+
+        if (nested_types[0]->getName() != nested_types[1]->getName())
+            throw Exception(
+                ErrorCodes::ILLEGAL_TYPE_OF_ARGUMENT,
+                "Arguments 3 and 4 of function {} must be arrays of the same types. Found {} and {} instead.",
+                getName(),
+                nested_types[0]->getName(),
+                nested_types[1]->getName());
+        return std::make_shared<DataTypeFloat64>();
+    }
+
+    ColumnPtr executeImpl(const ColumnsWithTypeAndName & arguments, const DataTypePtr &, size_t input_rows_count) const override
+    {
+        size_t num_arguments = arguments.size();
+
+        Columns holders(num_arguments);
+        std::vector<const ColumnArray *> columns(num_arguments);
+
+        for (size_t i = 0; i < num_arguments; ++i)
+        {
+            holders[i] = arguments[i].column->convertToFullColumnIfConst();
+            if (holders[i]->size() != input_rows_count)
+                throw Exception(
+                    ErrorCodes::LOGICAL_ERROR,
+                    "Function {} has unequal number of rows in columns: "
+                    "expected {}, got {} for column {}",
+                    getName(),
+                    input_rows_count,
+                    holders[i]->size(),
+                    holders[i]->getName());
+            columns[i] = assert_cast<const ColumnArray*>(holders[i].get());
+        }
+        return execute(columns);
+    }
+private:
+    ColumnPtr execute(std::vector<const ColumnArray *>) const
+    {
+        throw Exception(
+            ErrorCodes::LOGICAL_ERROR,
+            "Unknown function {}. "
+            "Supported names: 'arrayLevenshtein', 'arrayLevenshteinWeighted', 'arraySimilarity'",
+            T::name);
+    }
path: src/Functions/array/arrayLevenshtein.cpp
user: login: Felixoid
body: No, it complains about the missing method:

```
src/Functions/array/arrayLevenshtein.cpp:102:16: error: use of undeclared identifier 'execute'
  102 |         return execute(columns);
      |                ^
src/Functions/array/arrayLevenshtein.cpp:573:56: error: out-of-line definition of 'execute' does not match any declaration in 'DB::FunctionArrayLevenshtein<DB::SimpleLevenshtein>'
  573 | ColumnPtr FunctionArrayLevenshtein<SimpleLevenshtein>::execute(std::vector<const ColumnArray *> columns) const
      |                                                        ^~~~~~~
```
created_at: 2025-03-24T21:26:05Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,626 @@
+#include <Columns/ColumnArray.h>
+#include <Columns/ColumnDecimal.h>
+#include <Columns/ColumnFixedString.h>
+#include <Columns/ColumnNullable.h>
+#include <Columns/ColumnString.h>
+#include <Columns/ColumnTuple.h>
+#include <Common/PODArray.h>
+#include <Common/iota.h>
+#include <DataTypes/DataTypeArray.h>
+#include <DataTypes/DataTypeNullable.h>
+#include <DataTypes/DataTypeTuple.h>
+#include <DataTypes/DataTypesNumber.h>
+#include <Functions/FunctionFactory.h>
+#include <Functions/FunctionHelpers.h>
+#include <IO/WriteHelpers.h>
+
+#include <numeric>
+
+namespace DB
+{
+
+namespace ErrorCodes
+{
+    extern const int ILLEGAL_TYPE_OF_ARGUMENT;
+    extern const int LOGICAL_ERROR;
+    extern const int SIZES_OF_ARRAYS_DONT_MATCH;
+}
+
+/// arrayLevenshtein([1,2,3,4], [1,3,2,4]) = 2
+/// arrayLevenshteinWeighted([1,2,3,4], [1,3,2,4]) = 2
+template <typename T>
+class FunctionArrayLevenshtein : public IFunction
+{
+public:
+    static constexpr auto name = T::name;
+    static FunctionPtr create(ContextPtr) { return std::make_shared<FunctionArrayLevenshtein<T>>(); }
+
+    String getName() const override { return name; }
+
+    size_t getNumberOfArguments() const override { return T::arguments; }
+    bool useDefaultImplementationForConstants() const override { return true; }
+
+    bool isSuitableForShortCircuitArgumentsExecution(const DataTypesWithConstInfo & /*arguments*/) const override { return true; }
+
+    DataTypePtr getReturnTypeImpl(const ColumnsWithTypeAndName & arguments) const override
+    {
+        FunctionArgumentDescriptors args_descriptors;
+        args_descriptors = FunctionArgumentDescriptors{
+            {"from", static_cast<FunctionArgumentDescriptor::TypeValidator>(&isArray), nullptr, "Array"},
+            {"to", static_cast<FunctionArgumentDescriptor::TypeValidator>(&isArray), nullptr, "Array"},
+            {"from_weights", static_cast<FunctionArgumentDescriptor::TypeValidator>(&isArray), nullptr, "Array"},
+            {"to_weights", static_cast<FunctionArgumentDescriptor::TypeValidator>(&isArray), nullptr, "Array"},
+        };
+        validateFunctionArguments(*this, arguments, args_descriptors);
+        std::vector<DataTypePtr> nested_types;
+        nested_types.reserve(2);
+        for (size_t index = 2; index < 4; ++index)
+        {
+            const DataTypeArray * array_type = checkAndGetDataType<DataTypeArray>(arguments[index].type.get());
+            const DataTypePtr nested_type = array_type->getNestedType();
+            nested_types.emplace_back(nested_type);
+            // TODO: extend to work with different types?
+            if (!WhichDataType(nested_type).isFloat64())
+                throw Exception(
+                    ErrorCodes::ILLEGAL_TYPE_OF_ARGUMENT,
+                    "Argument {} of function {} must be array of Float64. Found {} instead.",
+                    toString(index + 1),
+                    getName(),
+                    nested_type->getName());
+        }
+
+        if (nested_types[0]->getName() != nested_types[1]->getName())
+            throw Exception(
+                ErrorCodes::ILLEGAL_TYPE_OF_ARGUMENT,
+                "Arguments 3 and 4 of function {} must be arrays of the same types. Found {} and {} instead.",
+                getName(),
+                nested_types[0]->getName(),
+                nested_types[1]->getName());
+        return std::make_shared<DataTypeFloat64>();
+    }
+
+    ColumnPtr executeImpl(const ColumnsWithTypeAndName & arguments, const DataTypePtr &, size_t input_rows_count) const override
+    {
+        size_t num_arguments = arguments.size();
+
+        Columns holders(num_arguments);
+        std::vector<const ColumnArray *> columns(num_arguments);
+
+        for (size_t i = 0; i < num_arguments; ++i)
+        {
+            holders[i] = arguments[i].column->convertToFullColumnIfConst();
+            if (holders[i]->size() != input_rows_count)
+                throw Exception(
+                    ErrorCodes::LOGICAL_ERROR,
+                    "Function {} has unequal number of rows in columns: "
+                    "expected {}, got {} for column {}",
+                    getName(),
+                    input_rows_count,
+                    holders[i]->size(),
+                    holders[i]->getName());
+            columns[i] = assert_cast<const ColumnArray*>(holders[i].get());
+        }
+        return execute(columns);
+    }
+private:
+    ColumnPtr execute(std::vector<const ColumnArray *>) const
+    {
+        throw Exception(
+            ErrorCodes::LOGICAL_ERROR,
+            "Unknown function {}. "
+            "Supported names: 'arrayLevenshtein', 'arrayLevenshteinWeighted', 'arraySimilarity'",
+            T::name);
+    }
+
+    template<typename N>
+    UInt32 levenshteinDistance(const std::vector<N> from, const std::vector<N> to) const
path: src/Functions/array/arrayLevenshtein.cpp
user: login: Felixoid
body: I finally nailed how to do it with a single vector, I will rework it and try to move the functions to the `src/Common/levenshteinDistance.*`
created_at: 2025-03-24T23:33:44Z
author_association: MEMBER
diff_hunk: @@ -3372,6 +3372,121 @@ Result:
 └─────────────────────────────────────────────────────────────┘
 ```
 
+## arrayLevenshtein {#arraylevenshtein}
+
+Calculates Levenshtein distance for two arrays.
+
+**Syntax**
+
+```sql
+arrayLevenshtein(lhs, rhs)
+```
+
+**Arguments**
+
+- `lhs` — left-hand side array
+- `rhs` — right-hand side array
path: docs/en/sql-reference/functions/array-functions.md
user: login: Felixoid
body: I agree, we can use them in the documentation. Although, in `size_t levenshteinDistance(const String & lhs, const String & rhs);` I'd leave it as `*hs` for consistency
created_at: 2025-03-26T06:46:42Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,626 @@
+#include <Columns/ColumnArray.h>
+#include <Columns/ColumnDecimal.h>
+#include <Columns/ColumnFixedString.h>
+#include <Columns/ColumnNullable.h>
+#include <Columns/ColumnString.h>
+#include <Columns/ColumnTuple.h>
+#include <Common/PODArray.h>
+#include <Common/iota.h>
+#include <DataTypes/DataTypeArray.h>
+#include <DataTypes/DataTypeNullable.h>
+#include <DataTypes/DataTypeTuple.h>
+#include <DataTypes/DataTypesNumber.h>
+#include <Functions/FunctionFactory.h>
+#include <Functions/FunctionHelpers.h>
+#include <IO/WriteHelpers.h>
+
+#include <numeric>
+
+namespace DB
+{
+
+namespace ErrorCodes
+{
+    extern const int ILLEGAL_TYPE_OF_ARGUMENT;
+    extern const int LOGICAL_ERROR;
+    extern const int SIZES_OF_ARRAYS_DONT_MATCH;
+}
+
+/// arrayLevenshtein([1,2,3,4], [1,3,2,4]) = 2
+/// arrayLevenshteinWeighted([1,2,3,4], [1,3,2,4]) = 2
+template <typename T>
+class FunctionArrayLevenshtein : public IFunction
+{
+public:
+    static constexpr auto name = T::name;
+    static FunctionPtr create(ContextPtr) { return std::make_shared<FunctionArrayLevenshtein<T>>(); }
+
+    String getName() const override { return name; }
+
+    size_t getNumberOfArguments() const override { return T::arguments; }
+    bool useDefaultImplementationForConstants() const override { return true; }
+
+    bool isSuitableForShortCircuitArgumentsExecution(const DataTypesWithConstInfo & /*arguments*/) const override { return true; }
+
+    DataTypePtr getReturnTypeImpl(const ColumnsWithTypeAndName & arguments) const override
+    {
+        FunctionArgumentDescriptors args_descriptors;
+        args_descriptors = FunctionArgumentDescriptors{
+            {"from", static_cast<FunctionArgumentDescriptor::TypeValidator>(&isArray), nullptr, "Array"},
+            {"to", static_cast<FunctionArgumentDescriptor::TypeValidator>(&isArray), nullptr, "Array"},
+            {"from_weights", static_cast<FunctionArgumentDescriptor::TypeValidator>(&isArray), nullptr, "Array"},
+            {"to_weights", static_cast<FunctionArgumentDescriptor::TypeValidator>(&isArray), nullptr, "Array"},
+        };
+        validateFunctionArguments(*this, arguments, args_descriptors);
+        std::vector<DataTypePtr> nested_types;
+        nested_types.reserve(2);
+        for (size_t index = 2; index < 4; ++index)
+        {
+            const DataTypeArray * array_type = checkAndGetDataType<DataTypeArray>(arguments[index].type.get());
+            const DataTypePtr nested_type = array_type->getNestedType();
+            nested_types.emplace_back(nested_type);
+            // TODO: extend to work with different types?
+            if (!WhichDataType(nested_type).isFloat64())
+                throw Exception(
+                    ErrorCodes::ILLEGAL_TYPE_OF_ARGUMENT,
+                    "Argument {} of function {} must be array of Float64. Found {} instead.",
+                    toString(index + 1),
+                    getName(),
+                    nested_type->getName());
+        }
+
+        if (nested_types[0]->getName() != nested_types[1]->getName())
+            throw Exception(
+                ErrorCodes::ILLEGAL_TYPE_OF_ARGUMENT,
+                "Arguments 3 and 4 of function {} must be arrays of the same types. Found {} and {} instead.",
+                getName(),
+                nested_types[0]->getName(),
+                nested_types[1]->getName());
+        return std::make_shared<DataTypeFloat64>();
+    }
+
+    ColumnPtr executeImpl(const ColumnsWithTypeAndName & arguments, const DataTypePtr &, size_t input_rows_count) const override
+    {
+        size_t num_arguments = arguments.size();
+
+        Columns holders(num_arguments);
+        std::vector<const ColumnArray *> columns(num_arguments);
+
+        for (size_t i = 0; i < num_arguments; ++i)
+        {
+            holders[i] = arguments[i].column->convertToFullColumnIfConst();
+            if (holders[i]->size() != input_rows_count)
+                throw Exception(
+                    ErrorCodes::LOGICAL_ERROR,
+                    "Function {} has unequal number of rows in columns: "
+                    "expected {}, got {} for column {}",
+                    getName(),
+                    input_rows_count,
+                    holders[i]->size(),
+                    holders[i]->getName());
+            columns[i] = assert_cast<const ColumnArray*>(holders[i].get());
+        }
+        return execute(columns);
+    }
+private:
+    ColumnPtr execute(std::vector<const ColumnArray *>) const
+    {
+        throw Exception(
+            ErrorCodes::LOGICAL_ERROR,
+            "Unknown function {}. "
+            "Supported names: 'arrayLevenshtein', 'arrayLevenshteinWeighted', 'arraySimilarity'",
+            T::name);
+    }
+
+    template<typename N>
+    UInt32 levenshteinDistance(const std::vector<N> from, const std::vector<N> to) const
+    {
+        const size_t m = from.size();
+        const size_t n = to.size();
+        if (m==0 || n==0)
+        {
+            return static_cast<UInt32>(m + n);
+        }
+        PODArrayWithStackMemory<size_t, 32> v0(n + 1);
+
+        iota(v0.data() + 1, n, size_t(1));
+
+        for (size_t j = 1; j <= m; ++j)
+        {
+            v0[0] = j;
+            size_t prev = j - 1;
+            for (size_t i = 1; i <= n; ++i)
+            {
+                size_t old = v0[i];
+                v0[i] = std::min(prev + (from[j - 1] != to[i - 1]),
+                        std::min(v0[i - 1], v0[i]) + 1);
+                prev = old;
+            }
+        }
+        return static_cast<UInt32>(v0[n]);
+    }
+
+    template <typename N>
+    bool levenshteinString(std::vector<const ColumnArray *> columns, ColumnUInt32::Container & res_values) const
+    {
+        const N * from_data = checkAndGetColumn<N>(&columns[0]->getData());
+        const N * to_data = checkAndGetColumn<N>(&columns[1]->getData());
+        if (!from_data || !to_data)
+            return false;
+        const ColumnArray::Offsets & from_offsets = columns[0]->getOffsets();
+        ColumnArray::Offset prev_from_offset = 0;
+
+        const ColumnArray::Offsets & to_offsets = columns[1]->getOffsets();
+        ColumnArray::Offset prev_to_offset = 0;
+
+        for (size_t row = 0; row < columns[0]->size(); row++)
+        {
+            const size_t m = from_offsets[row] - prev_from_offset;
+            const size_t n = to_offsets[row] - prev_to_offset;
+            const std::vector<StringRef> & from = [&]()
+            {
+                std::vector<StringRef> temp;
+                temp.reserve(m);
+                for (size_t j = 0; j < m; ++j) { temp.emplace_back(from_data->getDataAt(prev_from_offset + j)); }
+                return temp;
+            }();
+            const std::vector<StringRef> & to = [&]()
+            {
+                std::vector<StringRef> temp;
+                temp.reserve(m);
+                for (size_t j = 0; j < n; ++j) { temp.emplace_back(to_data->getDataAt(prev_to_offset + j)); }
+                return temp;
+            }();
path: src/Functions/array/arrayLevenshtein.cpp
user: login: Felixoid
body: Thanks, moved it above the cycle
created_at: 2025-03-26T08:15:12Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,626 @@
+#include <Columns/ColumnArray.h>
+#include <Columns/ColumnDecimal.h>
+#include <Columns/ColumnFixedString.h>
+#include <Columns/ColumnNullable.h>
+#include <Columns/ColumnString.h>
+#include <Columns/ColumnTuple.h>
+#include <Common/PODArray.h>
+#include <Common/iota.h>
+#include <DataTypes/DataTypeArray.h>
+#include <DataTypes/DataTypeNullable.h>
+#include <DataTypes/DataTypeTuple.h>
+#include <DataTypes/DataTypesNumber.h>
+#include <Functions/FunctionFactory.h>
+#include <Functions/FunctionHelpers.h>
+#include <IO/WriteHelpers.h>
+
+#include <numeric>
+
+namespace DB
+{
+
+namespace ErrorCodes
+{
+    extern const int ILLEGAL_TYPE_OF_ARGUMENT;
+    extern const int LOGICAL_ERROR;
+    extern const int SIZES_OF_ARRAYS_DONT_MATCH;
+}
+
+/// arrayLevenshtein([1,2,3,4], [1,3,2,4]) = 2
+/// arrayLevenshteinWeighted([1,2,3,4], [1,3,2,4]) = 2
+template <typename T>
+class FunctionArrayLevenshtein : public IFunction
+{
+public:
+    static constexpr auto name = T::name;
+    static FunctionPtr create(ContextPtr) { return std::make_shared<FunctionArrayLevenshtein<T>>(); }
+
+    String getName() const override { return name; }
+
+    size_t getNumberOfArguments() const override { return T::arguments; }
+    bool useDefaultImplementationForConstants() const override { return true; }
+
+    bool isSuitableForShortCircuitArgumentsExecution(const DataTypesWithConstInfo & /*arguments*/) const override { return true; }
+
+    DataTypePtr getReturnTypeImpl(const ColumnsWithTypeAndName & arguments) const override
+    {
+        FunctionArgumentDescriptors args_descriptors;
+        args_descriptors = FunctionArgumentDescriptors{
+            {"from", static_cast<FunctionArgumentDescriptor::TypeValidator>(&isArray), nullptr, "Array"},
+            {"to", static_cast<FunctionArgumentDescriptor::TypeValidator>(&isArray), nullptr, "Array"},
+            {"from_weights", static_cast<FunctionArgumentDescriptor::TypeValidator>(&isArray), nullptr, "Array"},
+            {"to_weights", static_cast<FunctionArgumentDescriptor::TypeValidator>(&isArray), nullptr, "Array"},
+        };
+        validateFunctionArguments(*this, arguments, args_descriptors);
+        std::vector<DataTypePtr> nested_types;
+        nested_types.reserve(2);
+        for (size_t index = 2; index < 4; ++index)
+        {
+            const DataTypeArray * array_type = checkAndGetDataType<DataTypeArray>(arguments[index].type.get());
+            const DataTypePtr nested_type = array_type->getNestedType();
+            nested_types.emplace_back(nested_type);
+            // TODO: extend to work with different types?
+            if (!WhichDataType(nested_type).isFloat64())
+                throw Exception(
+                    ErrorCodes::ILLEGAL_TYPE_OF_ARGUMENT,
+                    "Argument {} of function {} must be array of Float64. Found {} instead.",
+                    toString(index + 1),
+                    getName(),
+                    nested_type->getName());
+        }
+
+        if (nested_types[0]->getName() != nested_types[1]->getName())
+            throw Exception(
+                ErrorCodes::ILLEGAL_TYPE_OF_ARGUMENT,
+                "Arguments 3 and 4 of function {} must be arrays of the same types. Found {} and {} instead.",
+                getName(),
+                nested_types[0]->getName(),
+                nested_types[1]->getName());
+        return std::make_shared<DataTypeFloat64>();
+    }
+
+    ColumnPtr executeImpl(const ColumnsWithTypeAndName & arguments, const DataTypePtr &, size_t input_rows_count) const override
+    {
+        size_t num_arguments = arguments.size();
+
+        Columns holders(num_arguments);
+        std::vector<const ColumnArray *> columns(num_arguments);
+
+        for (size_t i = 0; i < num_arguments; ++i)
+        {
+            holders[i] = arguments[i].column->convertToFullColumnIfConst();
+            if (holders[i]->size() != input_rows_count)
+                throw Exception(
+                    ErrorCodes::LOGICAL_ERROR,
+                    "Function {} has unequal number of rows in columns: "
+                    "expected {}, got {} for column {}",
+                    getName(),
+                    input_rows_count,
+                    holders[i]->size(),
+                    holders[i]->getName());
+            columns[i] = assert_cast<const ColumnArray*>(holders[i].get());
+        }
+        return execute(columns);
+    }
+private:
+    ColumnPtr execute(std::vector<const ColumnArray *>) const
+    {
+        throw Exception(
+            ErrorCodes::LOGICAL_ERROR,
+            "Unknown function {}. "
+            "Supported names: 'arrayLevenshtein', 'arrayLevenshteinWeighted', 'arraySimilarity'",
+            T::name);
+    }
+
+    template<typename N>
+    UInt32 levenshteinDistance(const std::vector<N> from, const std::vector<N> to) const
+    {
+        const size_t m = from.size();
+        const size_t n = to.size();
+        if (m==0 || n==0)
+        {
+            return static_cast<UInt32>(m + n);
+        }
+        PODArrayWithStackMemory<size_t, 32> v0(n + 1);
+
+        iota(v0.data() + 1, n, size_t(1));
+
+        for (size_t j = 1; j <= m; ++j)
+        {
+            v0[0] = j;
+            size_t prev = j - 1;
+            for (size_t i = 1; i <= n; ++i)
+            {
+                size_t old = v0[i];
+                v0[i] = std::min(prev + (from[j - 1] != to[i - 1]),
+                        std::min(v0[i - 1], v0[i]) + 1);
+                prev = old;
+            }
+        }
+        return static_cast<UInt32>(v0[n]);
+    }
+
+    template <typename N>
+    bool levenshteinString(std::vector<const ColumnArray *> columns, ColumnUInt32::Container & res_values) const
+    {
+        const N * from_data = checkAndGetColumn<N>(&columns[0]->getData());
+        const N * to_data = checkAndGetColumn<N>(&columns[1]->getData());
+        if (!from_data || !to_data)
+            return false;
+        const ColumnArray::Offsets & from_offsets = columns[0]->getOffsets();
+        ColumnArray::Offset prev_from_offset = 0;
+
+        const ColumnArray::Offsets & to_offsets = columns[1]->getOffsets();
+        ColumnArray::Offset prev_to_offset = 0;
+
+        for (size_t row = 0; row < columns[0]->size(); row++)
+        {
+            const size_t m = from_offsets[row] - prev_from_offset;
+            const size_t n = to_offsets[row] - prev_to_offset;
+            const std::vector<StringRef> & from = [&]()
+            {
+                std::vector<StringRef> temp;
+                temp.reserve(m);
+                for (size_t j = 0; j < m; ++j) { temp.emplace_back(from_data->getDataAt(prev_from_offset + j)); }
+                return temp;
+            }();
+            const std::vector<StringRef> & to = [&]()
+            {
+                std::vector<StringRef> temp;
+                temp.reserve(m);
+                for (size_t j = 0; j < n; ++j) { temp.emplace_back(to_data->getDataAt(prev_to_offset + j)); }
+                return temp;
+            }();
+            prev_from_offset = from_offsets[row];
+            prev_to_offset = to_offsets[row];
+            res_values[row] = levenshteinDistance<StringRef>(from, to);
+        }
+        return true;
+    }
+
+    template <typename N>
+    bool levenshteinNumber(std::vector<const ColumnArray *> columns, ColumnUInt32::Container & res_values) const
+    {
+        const ColumnVectorOrDecimal<N> * column_from = checkAndGetColumn<ColumnVectorOrDecimal<N>>(&columns[0]->getData());
+        const ColumnVectorOrDecimal<N> * column_to = checkAndGetColumn<ColumnVectorOrDecimal<N>>(&columns[1]->getData());
+        if (!column_from || !column_to)
+            return false;
+        const PaddedPODArray<N> & vec_from = column_from->getData();
+        const ColumnArray::Offsets & from_offsets = columns[0]->getOffsets();
+        ColumnArray::Offset prev_from_offset = 0;
+
+        const PaddedPODArray<N> & vec_to = column_to->getData();
+        const ColumnArray::Offsets & to_offsets = columns[1]->getOffsets();
+        ColumnArray::Offset prev_to_offset = 0;
+
+        for (size_t row = 0; row < columns[0]->size(); row++)
+        {
+            const std::vector<N> from(vec_from.begin() + prev_from_offset, vec_from.begin() + from_offsets[row]);
+            prev_from_offset = from_offsets[row];
+            const std::vector<N> to(vec_to.begin() + prev_to_offset, vec_to.begin() + to_offsets[row]);
+            prev_to_offset = to_offsets[row];
+
+            res_values[row] = levenshteinDistance<N>(from, to);
+        }
+        return true;
+    }
+
+    void levenshteinGeneric(std::vector<const ColumnArray *> columns, ColumnUInt32::Container & res_values) const
+    {
+        const ColumnArray * column_from = columns[0];
+        const ColumnArray * column_to = columns[1];
+        for (size_t row = 0; row < column_from->size(); row++)
+        {
+            // Effective Levenshtein realization from Common/levenshteinDistance
+            Array from = (*column_from)[row].safeGet<Array>();
+            Array to = (*column_to)[row].safeGet<Array>();
+            const std::vector<Field> from_vec(from.begin(), from.end());
+            const std::vector<Field> to_vec(to.begin(), to.end());
path: src/Functions/array/arrayLevenshtein.cpp
user: login: Felixoid
body: Простите, был напуган
created_at: 2025-03-26T08:15:33Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,648 @@
+#include <Columns/ColumnArray.h>
+#include <Columns/ColumnDecimal.h>
+#include <Columns/ColumnFixedString.h>
+#include <Columns/ColumnNullable.h>
+#include <Columns/ColumnString.h>
+#include <Columns/ColumnTuple.h>
+#include <Common/levenshteinDistance.h>
+#include <Common/PODArray.h>
+#include <Common/iota.h>
+#include <DataTypes/DataTypeArray.h>
+#include <DataTypes/DataTypeNullable.h>
+#include <DataTypes/DataTypeTuple.h>
+#include <DataTypes/DataTypesNumber.h>
+#include <Functions/FunctionFactory.h>
+#include <Functions/FunctionHelpers.h>
+#include <IO/WriteHelpers.h>
+
+#include <numeric>
+#include <span>
+
+namespace DB
+{
+
+namespace ErrorCodes
+{
+    extern const int ILLEGAL_TYPE_OF_ARGUMENT;
+    extern const int LOGICAL_ERROR;
+    extern const int SIZES_OF_ARRAYS_DONT_MATCH;
+}
+
+/// arrayLevenshteinDistance([1,2,3,4], [1,3,2,4]) = 2
+/// arrayLevenshteinDistanceWeighted([1,2,3,4], [1,3,2,4]) = 2
+template <typename T>
+class FunctionArrayLevenshtein : public IFunction
+{
+public:
+    static constexpr auto name = T::name;
+    static FunctionPtr create(ContextPtr) { return std::make_shared<FunctionArrayLevenshtein<T>>(); }
+
+    String getName() const override { return name; }
+
+    size_t getNumberOfArguments() const override { return T::arguments; }
+    bool useDefaultImplementationForConstants() const override { return true; }
+
+    bool isSuitableForShortCircuitArgumentsExecution(const DataTypesWithConstInfo & /*arguments*/) const override { return true; }
+
+    DataTypePtr getReturnTypeImpl(const ColumnsWithTypeAndName & arguments) const override
+    {
+        FunctionArgumentDescriptors args_descriptors;
+        args_descriptors = FunctionArgumentDescriptors{
+            {"from", static_cast<FunctionArgumentDescriptor::TypeValidator>(&isArray), nullptr, "Array"},
+            {"to", static_cast<FunctionArgumentDescriptor::TypeValidator>(&isArray), nullptr, "Array"},
+            {"from_weights", static_cast<FunctionArgumentDescriptor::TypeValidator>(&isArray), nullptr, "Array"},
+            {"to_weights", static_cast<FunctionArgumentDescriptor::TypeValidator>(&isArray), nullptr, "Array"},
+        };
+        validateFunctionArguments(*this, arguments, args_descriptors);
+        std::vector<DataTypePtr> nested_types;
+        nested_types.reserve(2);
+        for (size_t index = 2; index < 4; ++index)
+        {
+            const DataTypeArray * array_type = checkAndGetDataType<DataTypeArray>(arguments[index].type.get());
+            const DataTypePtr nested_type = array_type->getNestedType();
+            nested_types.emplace_back(nested_type);
+            if (!(isFloat(nested_type) || isInteger(nested_type)))
+                throw Exception(
+                    ErrorCodes::ILLEGAL_TYPE_OF_ARGUMENT,
+                    "Argument {} of function {} must be array of Float64. Found {} instead.",
path: src/Functions/array/arrayLevenshtein.cpp
user: login: Michicosun
body: ```suggestion
                    "Argument {} of function {} must be a numeric array. Found {} instead.",
```
created_at: 2025-03-27T14:14:26Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,648 @@
+#include <Columns/ColumnArray.h>
+#include <Columns/ColumnDecimal.h>
+#include <Columns/ColumnFixedString.h>
+#include <Columns/ColumnNullable.h>
+#include <Columns/ColumnString.h>
+#include <Columns/ColumnTuple.h>
+#include <Common/levenshteinDistance.h>
+#include <Common/PODArray.h>
+#include <Common/iota.h>
+#include <DataTypes/DataTypeArray.h>
+#include <DataTypes/DataTypeNullable.h>
+#include <DataTypes/DataTypeTuple.h>
+#include <DataTypes/DataTypesNumber.h>
+#include <Functions/FunctionFactory.h>
+#include <Functions/FunctionHelpers.h>
+#include <IO/WriteHelpers.h>
+
+#include <numeric>
+#include <span>
+
+namespace DB
+{
+
+namespace ErrorCodes
+{
+    extern const int ILLEGAL_TYPE_OF_ARGUMENT;
+    extern const int LOGICAL_ERROR;
+    extern const int SIZES_OF_ARRAYS_DONT_MATCH;
+}
+
+/// arrayLevenshteinDistance([1,2,3,4], [1,3,2,4]) = 2
+/// arrayLevenshteinDistanceWeighted([1,2,3,4], [1,3,2,4]) = 2
+template <typename T>
+class FunctionArrayLevenshtein : public IFunction
+{
+public:
+    static constexpr auto name = T::name;
+    static FunctionPtr create(ContextPtr) { return std::make_shared<FunctionArrayLevenshtein<T>>(); }
+
+    String getName() const override { return name; }
+
+    size_t getNumberOfArguments() const override { return T::arguments; }
+    bool useDefaultImplementationForConstants() const override { return true; }
+
+    bool isSuitableForShortCircuitArgumentsExecution(const DataTypesWithConstInfo & /*arguments*/) const override { return true; }
+
+    DataTypePtr getReturnTypeImpl(const ColumnsWithTypeAndName & arguments) const override
+    {
+        FunctionArgumentDescriptors args_descriptors;
+        args_descriptors = FunctionArgumentDescriptors{
+            {"from", static_cast<FunctionArgumentDescriptor::TypeValidator>(&isArray), nullptr, "Array"},
+            {"to", static_cast<FunctionArgumentDescriptor::TypeValidator>(&isArray), nullptr, "Array"},
+            {"from_weights", static_cast<FunctionArgumentDescriptor::TypeValidator>(&isArray), nullptr, "Array"},
+            {"to_weights", static_cast<FunctionArgumentDescriptor::TypeValidator>(&isArray), nullptr, "Array"},
+        };
+        validateFunctionArguments(*this, arguments, args_descriptors);
+        std::vector<DataTypePtr> nested_types;
+        nested_types.reserve(2);
+        for (size_t index = 2; index < 4; ++index)
+        {
+            const DataTypeArray * array_type = checkAndGetDataType<DataTypeArray>(arguments[index].type.get());
+            const DataTypePtr nested_type = array_type->getNestedType();
+            nested_types.emplace_back(nested_type);
+            if (!(isFloat(nested_type) || isInteger(nested_type)))
+                throw Exception(
+                    ErrorCodes::ILLEGAL_TYPE_OF_ARGUMENT,
+                    "Argument {} of function {} must be array of Float64. Found {} instead.",
+                    toString(index + 1),
+                    getName(),
+                    nested_type->getName());
+        }
+
+        if (nested_types[0]->getName() != nested_types[1]->getName())
+            throw Exception(
+                ErrorCodes::ILLEGAL_TYPE_OF_ARGUMENT,
+                "Arguments 3 and 4 of function {} must be arrays of the same types. Found {} and {} instead.",
+                getName(),
+                nested_types[0]->getName(),
+                nested_types[1]->getName());
+        return std::make_shared<DataTypeFloat64>();
+    }
+
+    ColumnPtr executeImpl(const ColumnsWithTypeAndName & arguments, const DataTypePtr &, size_t input_rows_count) const override
+    {
+        size_t num_arguments = arguments.size();
+
+        Columns holders(num_arguments);
+        std::vector<const ColumnArray *> columns(num_arguments);
+
+        for (size_t i = 0; i < num_arguments; ++i)
+        {
+            holders[i] = arguments[i].column->convertToFullColumnIfConst();
+            if (holders[i]->size() != input_rows_count)
+                throw Exception(
+                    ErrorCodes::LOGICAL_ERROR,
+                    "Function {} has unequal number of rows in columns: "
+                    "expected {}, got {} for column {}",
+                    getName(),
+                    input_rows_count,
+                    holders[i]->size(),
+                    holders[i]->getName());
+            columns[i] = assert_cast<const ColumnArray*>(holders[i].get());
+        }
+        return execute(columns);
+    }
+private:
+    ColumnPtr execute(std::vector<const ColumnArray *>) const
+    {
+        throw Exception(
+            ErrorCodes::LOGICAL_ERROR,
+            "Unknown function {}. "
+            "Supported names: 'arrayLevenshteinDistance', 'arrayLevenshteinDistanceWeighted', 'arraySimilarity'",
+            getName());
+    }
+
+    template <typename N, typename Result>
+    bool levenshteinString(std::vector<const ColumnArray *> columns, Result::Container & res_values) const
+    {
+        const N * from_data = checkAndGetColumn<N>(&columns[0]->getData());
+        const N * to_data = checkAndGetColumn<N>(&columns[1]->getData());
+        if (!from_data || !to_data)
+            return false;
+        if (T::arguments == 4)
+        {
+            if constexpr (std::is_same_v<Result, ColumnFloat64>)
+            {
+                if (!(
+                    levenshteinWeightedString<N, UInt8>(columns, res_values)
+                    || levenshteinWeightedString<N, UInt16>(columns, res_values)
+                    || levenshteinWeightedString<N, UInt32>(columns, res_values)
+                    || levenshteinWeightedString<N, UInt64>(columns, res_values)
+                    || levenshteinWeightedString<N, UInt128>(columns, res_values)
+                    || levenshteinWeightedString<N, UInt256>(columns, res_values)
+                    || levenshteinWeightedString<N, Int8>(columns, res_values)
+                    || levenshteinWeightedString<N, Int16>(columns, res_values)
+                    || levenshteinWeightedString<N, Int32>(columns, res_values)
+                    || levenshteinWeightedString<N, Int64>(columns, res_values)
+                    || levenshteinWeightedString<N, Int128>(columns, res_values)
+                    || levenshteinWeightedString<N, Int256>(columns, res_values)
+                    || levenshteinWeightedString<N, Float32>(columns, res_values)
+                    || levenshteinWeightedString<N, Float64>(columns, res_values)))
+                    throw Exception(
+                        ErrorCodes::LOGICAL_ERROR,
+                        "Unexpected code branch of function {}. No fitting levenshteinWeightedString",
+                        getName());
+                return true;
+            }
+            throw Exception(
+                ErrorCodes::LOGICAL_ERROR,
+                "Unexpected code branch of function {}. Wrong expected result type in levenshteinString",
+                getName());
+        }
+        const ColumnArray::Offsets & from_offsets = columns[0]->getOffsets();
+        ColumnArray::Offset prev_from_offset = 0;
+
+        const ColumnArray::Offsets & to_offsets = columns[1]->getOffsets();
+        ColumnArray::Offset prev_to_offset = 0;
+        const auto extract_array = [](const N * data, size_t prev_offset, size_t count)
+        {
+            std::vector<StringRef> temp;
+            temp.reserve(count);
+            for (size_t j = 0; j < count; ++j) { temp.emplace_back(data->getDataAt(prev_offset + j)); }
+            return temp;
+        };
+
+        for (size_t row = 0; row < columns[0]->size(); row++)
+        {
+            const size_t m = from_offsets[row] - prev_from_offset;
+            const size_t n = to_offsets[row] - prev_to_offset;
+            const std::vector<StringRef> & from_vec = extract_array(from_data, prev_from_offset, m);
+            const std::vector<StringRef> & to_vec = extract_array(to_data, prev_to_offset, n);
+            std::span<const StringRef> from(from_vec.begin(), m);
+            std::span<const StringRef> to(to_vec.begin(), n);
+            prev_from_offset = from_offsets[row];
+            prev_to_offset = to_offsets[row];
+            res_values[row] = levenshteinDistance<StringRef>(from, to);
+        }
+        return true;
+    }
+
+    template <typename N, typename Result>
+    bool levenshteinNumber(std::vector<const ColumnArray *> columns, Result::Container & res_values) const
+    {
+        const ColumnVectorOrDecimal<N> * column_from = checkAndGetColumn<ColumnVectorOrDecimal<N>>(&columns[0]->getData());
+        const ColumnVectorOrDecimal<N> * column_to = checkAndGetColumn<ColumnVectorOrDecimal<N>>(&columns[1]->getData());
+        if (!column_from || !column_to)
+            return false;
+
+        if (T::arguments == 4)
+        {
+            if constexpr (std::is_same_v<Result, ColumnFloat64>)
+            {
+                if (!(
+                    levenshteinWeightedNumber<N, UInt8>(columns, res_values)
+                    || levenshteinWeightedNumber<N, UInt16>(columns, res_values)
+                    || levenshteinWeightedNumber<N, UInt32>(columns, res_values)
+                    || levenshteinWeightedNumber<N, UInt64>(columns, res_values)
+                    || levenshteinWeightedNumber<N, UInt128>(columns, res_values)
+                    || levenshteinWeightedNumber<N, UInt256>(columns, res_values)
+                    || levenshteinWeightedNumber<N, Int8>(columns, res_values)
+                    || levenshteinWeightedNumber<N, Int16>(columns, res_values)
+                    || levenshteinWeightedNumber<N, Int32>(columns, res_values)
+                    || levenshteinWeightedNumber<N, Int64>(columns, res_values)
+                    || levenshteinWeightedNumber<N, Int128>(columns, res_values)
+                    || levenshteinWeightedNumber<N, Int256>(columns, res_values)
+                    || levenshteinWeightedNumber<N, Float32>(columns, res_values)
+                    || levenshteinWeightedNumber<N, Float64>(columns, res_values)))
+                    throw Exception(
+                        ErrorCodes::LOGICAL_ERROR,
+                        "Unexpected code branch of function {}. No fitting levenshteinWeightedNumber",
+                        getName());
+                return true;
+            }
+            throw Exception(
+                ErrorCodes::LOGICAL_ERROR,
+                "Unexpected code branch of function {}. Wrong expected result type in levenshteinNumber",
+                getName());
+        }
+
+        const ColumnArray::Offsets & from_offsets = columns[0]->getOffsets();
+        ColumnArray::Offset prev_from_offset = 0;
+
+        const ColumnArray::Offsets & to_offsets = columns[1]->getOffsets();
+        ColumnArray::Offset prev_to_offset = 0;
+
+        for (size_t row = 0; row < columns[0]->size(); row++)
+        {
+            std::span<const N> from(column_from->getData().begin() + prev_from_offset, from_offsets[row] - prev_from_offset);
+            prev_from_offset = from_offsets[row];
+            std::span<const N> to(column_to->getData().begin() + prev_to_offset, to_offsets[row] - prev_to_offset);
+            prev_to_offset = to_offsets[row];
+
+            res_values[row] = levenshteinDistance<N>(from, to);
+        }
+        return true;
+    }
+
+    template<typename Result>
+    void levenshteinGeneric(std::vector<const ColumnArray *> columns, Result::Container & res_values) const
+    {
+        if (T::arguments == 4)
+        {
+            if constexpr (std::is_same_v<Result, ColumnFloat64>)
+            {
+                if (!(
+                    levenshteinWeightedGeneric<UInt8>(columns, res_values)
+                    || levenshteinWeightedGeneric<UInt16>(columns, res_values)
+                    || levenshteinWeightedGeneric<UInt32>(columns, res_values)
+                    || levenshteinWeightedGeneric<UInt64>(columns, res_values)
+                    || levenshteinWeightedGeneric<UInt128>(columns, res_values)
+                    || levenshteinWeightedGeneric<UInt256>(columns, res_values)
+                    || levenshteinWeightedGeneric<Int8>(columns, res_values)
+                    || levenshteinWeightedGeneric<Int16>(columns, res_values)
+                    || levenshteinWeightedGeneric<Int32>(columns, res_values)
+                    || levenshteinWeightedGeneric<Int64>(columns, res_values)
+                    || levenshteinWeightedGeneric<Int128>(columns, res_values)
+                    || levenshteinWeightedGeneric<Int256>(columns, res_values)
+                    || levenshteinWeightedGeneric<Float32>(columns, res_values)
+                    || levenshteinWeightedGeneric<Float64>(columns, res_values)))
+                    throw Exception(
+                        ErrorCodes::LOGICAL_ERROR,
+                        "Unexpected code branch of function {}. No fitting levenshteinWeightedGeneric",
+                        getName());
+                return;
+            }
+            throw Exception(
+                ErrorCodes::LOGICAL_ERROR,
+                "Unexpected code branch of function {}. Wrong expected result type in levenshteinGeneric",
+                getName());
+        }
+
+        const ColumnArray * column_from = columns[0];
+        const ColumnArray * column_to = columns[1];
+        for (size_t row = 0; row < column_from->size(); row++)
+        {
+            // Effective Levenshtein realization from Common/levenshteinDistance
+            Array from = (*column_from)[row].safeGet<Array>();
+            Array to = (*column_to)[row].safeGet<Array>();
+            std::span<const Field> from_vec(from.begin(), from.end());
+            std::span<const Field> to_vec(to.begin(), to.end());
+            res_values[row] = levenshteinDistance<Field>(from_vec, to_vec);
+        }
+    }
+
+    template <typename N, typename W>
+    bool levenshteinWeightedString(std::vector<const ColumnArray *> columns, ColumnFloat64::Container & res_values) const
+    {
+        const N * from_data = checkAndGetColumn<N>(&columns[0]->getData());
+        const N * to_data = checkAndGetColumn<N>(&columns[1]->getData());
+        if (!from_data || !to_data)
+            return false;
+        const ColumnArray::Offsets & from_offsets = columns[0]->getOffsets();
+        ColumnArray::Offset prev_from_offset = 0;
+
+        const ColumnArray::Offsets & to_offsets = columns[1]->getOffsets();
+        ColumnArray::Offset prev_to_offset = 0;
+
+        const ColumnVector<W> * column_from_weights = checkAndGetColumn<ColumnVector<W>>(&columns[2]->getData());
+        const ColumnVector<W> * column_to_weights = checkAndGetColumn<ColumnVector<W>>(&columns[3]->getData());
+        if (!column_from_weights || !column_to_weights)
+            return false;
+
+        const ColumnArray::Offsets & from_weights_offsets = columns[2]->getOffsets();
+        ColumnArray::Offset prev_from_weights_offset = 0;
+
+        const ColumnArray::Offsets & to_weights_offsets = columns[3]->getOffsets();
+        ColumnArray::Offset prev_to_weights_offset = 0;
+
+        const auto extract_array = [](const N * data, size_t prev_offset, size_t count)
+        {
+            std::vector<StringRef> temp;
+            temp.reserve(count);
+            for (size_t j = 0; j < count; ++j) { temp.emplace_back(data->getDataAt(prev_offset + j)); }
+            return temp;
+        };
+
+        for (size_t row = 0; row < columns[0]->size(); row++)
+        {
+            const size_t m = from_offsets[row] - prev_from_offset;
+            const size_t n = to_offsets[row] - prev_to_offset;
+            const std::vector<StringRef> & from_vec = extract_array(from_data, prev_from_offset, m);
+            const std::vector<StringRef> & to_vec = extract_array(to_data, prev_to_offset, n);
+            std::span<const StringRef> from(from_vec.begin(), m);
+            std::span<const StringRef> to(to_vec.begin(), n);
+            prev_from_offset = from_offsets[row];
+            prev_to_offset = to_offsets[row];
+
+            std::span<const W> from_weights(column_from_weights->getData().begin() + prev_from_weights_offset, from_weights_offsets[row] - prev_from_weights_offset);
+            prev_from_weights_offset = from_weights_offsets[row];
+            std::span<const W> to_weights(column_to_weights->getData().begin() + prev_to_weights_offset, to_weights_offsets[row] - prev_to_weights_offset);
+            prev_to_weights_offset = to_weights_offsets[row];
+
+            res_values[row] = static_cast<Float64>(DB::levenshteinDistanceWeighted<StringRef, W>(from, to, from_weights, to_weights));
+        }
+        return true;
+    }
+
+    template <typename N, typename W>
+    bool levenshteinWeightedNumber(std::vector<const ColumnArray *> columns, ColumnFloat64::Container & res_values) const
+    {
+        const ColumnVectorOrDecimal<N> * column_from = checkAndGetColumn<ColumnVectorOrDecimal<N>>(&columns[0]->getData());
+        const ColumnVectorOrDecimal<N> * column_to = checkAndGetColumn<ColumnVectorOrDecimal<N>>(&columns[1]->getData());
+        if (!column_from || !column_to)
+            // just to be on the safe side, it's already checked
+            return false;
+
+        const ColumnArray::Offsets & from_offsets = columns[0]->getOffsets();
+        ColumnArray::Offset prev_from_offset = 0;
+        const ColumnArray::Offsets & to_offsets = columns[1]->getOffsets();
+        ColumnArray::Offset prev_to_offset = 0;
+
+        const ColumnVector<W> * column_from_weights = checkAndGetColumn<ColumnVector<W>>(&columns[2]->getData());
+        const ColumnVector<W> * column_to_weights = checkAndGetColumn<ColumnVector<W>>(&columns[3]->getData());
+        if (!column_from_weights || !column_to_weights)
+            return false;
+
+        const ColumnArray::Offsets & from_weights_offsets = columns[2]->getOffsets();
+        ColumnArray::Offset prev_from_weights_offset = 0;
+        const ColumnArray::Offsets & to_weights_offsets = columns[3]->getOffsets();
+        ColumnArray::Offset prev_to_weights_offset = 0;
+
+        for (size_t row = 0; row < columns[0]->size(); row++)
+        {
+            std::span<const N> from(column_from->getData().begin() + prev_from_offset, from_offsets[row] - prev_from_offset);
+            prev_from_offset = from_offsets[row];
+            std::span<const N> to(column_to->getData().begin() + prev_to_offset, to_offsets[row] - prev_to_offset);
+            prev_to_offset = to_offsets[row];
+
+            std::span<const W> from_weights(column_from_weights->getData().begin() + prev_from_weights_offset, from_weights_offsets[row] - prev_from_weights_offset);
+            prev_from_weights_offset = from_weights_offsets[row];
+            std::span<const W> to_weights(column_to_weights->getData().begin() + prev_to_weights_offset, to_weights_offsets[row] - prev_to_weights_offset);
+            prev_to_weights_offset = to_weights_offsets[row];
+
+            res_values[row] = static_cast<Float64>(DB::levenshteinDistanceWeighted<N, W>(from, to, from_weights, to_weights));
+        }
+        return true;
+    }
+
+    template<typename W>
+    bool levenshteinWeightedGeneric(std::vector<const ColumnArray *> columns, ColumnFloat64::Container & res_values) const
+    {
+        const ColumnArray * column_from = columns[0];
+        const ColumnArray * column_to = columns[1];
+
+        const ColumnVector<W> * column_from_weights = checkAndGetColumn<ColumnVector<W>>(&columns[2]->getData());
+        const ColumnVector<W> * column_to_weights = checkAndGetColumn<ColumnVector<W>>(&columns[3]->getData());
+        if (!column_from_weights || !column_to_weights)
+            return false;
+        const ColumnArray::Offsets & from_weights_offsets = columns[2]->getOffsets();
+        ColumnArray::Offset prev_from_weights_offset = 0;
+
+        const ColumnArray::Offsets & to_weights_offsets = columns[3]->getOffsets();
+        ColumnArray::Offset prev_to_weights_offset = 0;
+
+        for (size_t row = 0; row < column_from->size(); row++)
+        {
+            // Effective Levenshtein realization from Common/levenshteinDistance
+            Array from = (*column_from)[row].safeGet<Array>();
+            Array to = (*column_to)[row].safeGet<Array>();
+            std::span<const Field> from_vec(from.begin(), from.end());
+            std::span<const Field> to_vec(to.begin(), to.end());
path: src/Functions/array/arrayLevenshtein.cpp
user: login: Michicosun
body: you do not need this explicit construction, because you can pass `from` and `to` directly into `levenshteinDistanceWeighted` they will be implicitly casted into span.
created_at: 2025-03-27T14:22:40Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,648 @@
+#include <Columns/ColumnArray.h>
+#include <Columns/ColumnDecimal.h>
+#include <Columns/ColumnFixedString.h>
+#include <Columns/ColumnNullable.h>
+#include <Columns/ColumnString.h>
+#include <Columns/ColumnTuple.h>
+#include <Common/levenshteinDistance.h>
+#include <Common/PODArray.h>
+#include <Common/iota.h>
+#include <DataTypes/DataTypeArray.h>
+#include <DataTypes/DataTypeNullable.h>
+#include <DataTypes/DataTypeTuple.h>
+#include <DataTypes/DataTypesNumber.h>
+#include <Functions/FunctionFactory.h>
+#include <Functions/FunctionHelpers.h>
+#include <IO/WriteHelpers.h>
+
+#include <numeric>
+#include <span>
+
+namespace DB
+{
+
+namespace ErrorCodes
+{
+    extern const int ILLEGAL_TYPE_OF_ARGUMENT;
+    extern const int LOGICAL_ERROR;
+    extern const int SIZES_OF_ARRAYS_DONT_MATCH;
+}
+
+/// arrayLevenshteinDistance([1,2,3,4], [1,3,2,4]) = 2
+/// arrayLevenshteinDistanceWeighted([1,2,3,4], [1,3,2,4]) = 2
+template <typename T>
+class FunctionArrayLevenshtein : public IFunction
+{
+public:
+    static constexpr auto name = T::name;
+    static FunctionPtr create(ContextPtr) { return std::make_shared<FunctionArrayLevenshtein<T>>(); }
+
+    String getName() const override { return name; }
+
+    size_t getNumberOfArguments() const override { return T::arguments; }
+    bool useDefaultImplementationForConstants() const override { return true; }
+
+    bool isSuitableForShortCircuitArgumentsExecution(const DataTypesWithConstInfo & /*arguments*/) const override { return true; }
+
+    DataTypePtr getReturnTypeImpl(const ColumnsWithTypeAndName & arguments) const override
+    {
+        FunctionArgumentDescriptors args_descriptors;
+        args_descriptors = FunctionArgumentDescriptors{
+            {"from", static_cast<FunctionArgumentDescriptor::TypeValidator>(&isArray), nullptr, "Array"},
+            {"to", static_cast<FunctionArgumentDescriptor::TypeValidator>(&isArray), nullptr, "Array"},
+            {"from_weights", static_cast<FunctionArgumentDescriptor::TypeValidator>(&isArray), nullptr, "Array"},
+            {"to_weights", static_cast<FunctionArgumentDescriptor::TypeValidator>(&isArray), nullptr, "Array"},
+        };
+        validateFunctionArguments(*this, arguments, args_descriptors);
+        std::vector<DataTypePtr> nested_types;
+        nested_types.reserve(2);
+        for (size_t index = 2; index < 4; ++index)
+        {
+            const DataTypeArray * array_type = checkAndGetDataType<DataTypeArray>(arguments[index].type.get());
+            const DataTypePtr nested_type = array_type->getNestedType();
+            nested_types.emplace_back(nested_type);
+            if (!(isFloat(nested_type) || isInteger(nested_type)))
+                throw Exception(
+                    ErrorCodes::ILLEGAL_TYPE_OF_ARGUMENT,
+                    "Argument {} of function {} must be array of Float64. Found {} instead.",
+                    toString(index + 1),
+                    getName(),
+                    nested_type->getName());
+        }
+
+        if (nested_types[0]->getName() != nested_types[1]->getName())
+            throw Exception(
+                ErrorCodes::ILLEGAL_TYPE_OF_ARGUMENT,
+                "Arguments 3 and 4 of function {} must be arrays of the same types. Found {} and {} instead.",
+                getName(),
+                nested_types[0]->getName(),
+                nested_types[1]->getName());
+        return std::make_shared<DataTypeFloat64>();
+    }
+
+    ColumnPtr executeImpl(const ColumnsWithTypeAndName & arguments, const DataTypePtr &, size_t input_rows_count) const override
+    {
+        size_t num_arguments = arguments.size();
+
+        Columns holders(num_arguments);
+        std::vector<const ColumnArray *> columns(num_arguments);
+
+        for (size_t i = 0; i < num_arguments; ++i)
+        {
+            holders[i] = arguments[i].column->convertToFullColumnIfConst();
+            if (holders[i]->size() != input_rows_count)
+                throw Exception(
+                    ErrorCodes::LOGICAL_ERROR,
+                    "Function {} has unequal number of rows in columns: "
+                    "expected {}, got {} for column {}",
+                    getName(),
+                    input_rows_count,
+                    holders[i]->size(),
+                    holders[i]->getName());
+            columns[i] = assert_cast<const ColumnArray*>(holders[i].get());
+        }
+        return execute(columns);
+    }
+private:
+    ColumnPtr execute(std::vector<const ColumnArray *>) const
+    {
+        throw Exception(
+            ErrorCodes::LOGICAL_ERROR,
+            "Unknown function {}. "
+            "Supported names: 'arrayLevenshteinDistance', 'arrayLevenshteinDistanceWeighted', 'arraySimilarity'",
+            getName());
+    }
+
+    template <typename N, typename Result>
+    bool levenshteinString(std::vector<const ColumnArray *> columns, Result::Container & res_values) const
+    {
+        const N * from_data = checkAndGetColumn<N>(&columns[0]->getData());
+        const N * to_data = checkAndGetColumn<N>(&columns[1]->getData());
+        if (!from_data || !to_data)
+            return false;
+        if (T::arguments == 4)
+        {
+            if constexpr (std::is_same_v<Result, ColumnFloat64>)
+            {
+                if (!(
+                    levenshteinWeightedString<N, UInt8>(columns, res_values)
+                    || levenshteinWeightedString<N, UInt16>(columns, res_values)
+                    || levenshteinWeightedString<N, UInt32>(columns, res_values)
+                    || levenshteinWeightedString<N, UInt64>(columns, res_values)
+                    || levenshteinWeightedString<N, UInt128>(columns, res_values)
+                    || levenshteinWeightedString<N, UInt256>(columns, res_values)
+                    || levenshteinWeightedString<N, Int8>(columns, res_values)
+                    || levenshteinWeightedString<N, Int16>(columns, res_values)
+                    || levenshteinWeightedString<N, Int32>(columns, res_values)
+                    || levenshteinWeightedString<N, Int64>(columns, res_values)
+                    || levenshteinWeightedString<N, Int128>(columns, res_values)
+                    || levenshteinWeightedString<N, Int256>(columns, res_values)
+                    || levenshteinWeightedString<N, Float32>(columns, res_values)
+                    || levenshteinWeightedString<N, Float64>(columns, res_values)))
+                    throw Exception(
+                        ErrorCodes::LOGICAL_ERROR,
+                        "Unexpected code branch of function {}. No fitting levenshteinWeightedString",
+                        getName());
+                return true;
+            }
+            throw Exception(
+                ErrorCodes::LOGICAL_ERROR,
+                "Unexpected code branch of function {}. Wrong expected result type in levenshteinString",
+                getName());
+        }
+        const ColumnArray::Offsets & from_offsets = columns[0]->getOffsets();
+        ColumnArray::Offset prev_from_offset = 0;
+
+        const ColumnArray::Offsets & to_offsets = columns[1]->getOffsets();
+        ColumnArray::Offset prev_to_offset = 0;
+        const auto extract_array = [](const N * data, size_t prev_offset, size_t count)
+        {
+            std::vector<StringRef> temp;
+            temp.reserve(count);
+            for (size_t j = 0; j < count; ++j) { temp.emplace_back(data->getDataAt(prev_offset + j)); }
+            return temp;
+        };
+
+        for (size_t row = 0; row < columns[0]->size(); row++)
+        {
+            const size_t m = from_offsets[row] - prev_from_offset;
+            const size_t n = to_offsets[row] - prev_to_offset;
+            const std::vector<StringRef> & from_vec = extract_array(from_data, prev_from_offset, m);
+            const std::vector<StringRef> & to_vec = extract_array(to_data, prev_to_offset, n);
+            std::span<const StringRef> from(from_vec.begin(), m);
+            std::span<const StringRef> to(to_vec.begin(), n);
+            prev_from_offset = from_offsets[row];
+            prev_to_offset = to_offsets[row];
+            res_values[row] = levenshteinDistance<StringRef>(from, to);
+        }
+        return true;
+    }
+
+    template <typename N, typename Result>
+    bool levenshteinNumber(std::vector<const ColumnArray *> columns, Result::Container & res_values) const
+    {
+        const ColumnVectorOrDecimal<N> * column_from = checkAndGetColumn<ColumnVectorOrDecimal<N>>(&columns[0]->getData());
+        const ColumnVectorOrDecimal<N> * column_to = checkAndGetColumn<ColumnVectorOrDecimal<N>>(&columns[1]->getData());
+        if (!column_from || !column_to)
+            return false;
+
+        if (T::arguments == 4)
+        {
+            if constexpr (std::is_same_v<Result, ColumnFloat64>)
+            {
+                if (!(
+                    levenshteinWeightedNumber<N, UInt8>(columns, res_values)
+                    || levenshteinWeightedNumber<N, UInt16>(columns, res_values)
+                    || levenshteinWeightedNumber<N, UInt32>(columns, res_values)
+                    || levenshteinWeightedNumber<N, UInt64>(columns, res_values)
+                    || levenshteinWeightedNumber<N, UInt128>(columns, res_values)
+                    || levenshteinWeightedNumber<N, UInt256>(columns, res_values)
+                    || levenshteinWeightedNumber<N, Int8>(columns, res_values)
+                    || levenshteinWeightedNumber<N, Int16>(columns, res_values)
+                    || levenshteinWeightedNumber<N, Int32>(columns, res_values)
+                    || levenshteinWeightedNumber<N, Int64>(columns, res_values)
+                    || levenshteinWeightedNumber<N, Int128>(columns, res_values)
+                    || levenshteinWeightedNumber<N, Int256>(columns, res_values)
+                    || levenshteinWeightedNumber<N, Float32>(columns, res_values)
+                    || levenshteinWeightedNumber<N, Float64>(columns, res_values)))
+                    throw Exception(
+                        ErrorCodes::LOGICAL_ERROR,
+                        "Unexpected code branch of function {}. No fitting levenshteinWeightedNumber",
+                        getName());
+                return true;
+            }
+            throw Exception(
+                ErrorCodes::LOGICAL_ERROR,
+                "Unexpected code branch of function {}. Wrong expected result type in levenshteinNumber",
+                getName());
+        }
+
+        const ColumnArray::Offsets & from_offsets = columns[0]->getOffsets();
+        ColumnArray::Offset prev_from_offset = 0;
+
+        const ColumnArray::Offsets & to_offsets = columns[1]->getOffsets();
+        ColumnArray::Offset prev_to_offset = 0;
+
+        for (size_t row = 0; row < columns[0]->size(); row++)
+        {
+            std::span<const N> from(column_from->getData().begin() + prev_from_offset, from_offsets[row] - prev_from_offset);
+            prev_from_offset = from_offsets[row];
+            std::span<const N> to(column_to->getData().begin() + prev_to_offset, to_offsets[row] - prev_to_offset);
+            prev_to_offset = to_offsets[row];
+
+            res_values[row] = levenshteinDistance<N>(from, to);
+        }
+        return true;
+    }
+
+    template<typename Result>
+    void levenshteinGeneric(std::vector<const ColumnArray *> columns, Result::Container & res_values) const
+    {
+        if (T::arguments == 4)
+        {
+            if constexpr (std::is_same_v<Result, ColumnFloat64>)
+            {
+                if (!(
+                    levenshteinWeightedGeneric<UInt8>(columns, res_values)
+                    || levenshteinWeightedGeneric<UInt16>(columns, res_values)
+                    || levenshteinWeightedGeneric<UInt32>(columns, res_values)
+                    || levenshteinWeightedGeneric<UInt64>(columns, res_values)
+                    || levenshteinWeightedGeneric<UInt128>(columns, res_values)
+                    || levenshteinWeightedGeneric<UInt256>(columns, res_values)
+                    || levenshteinWeightedGeneric<Int8>(columns, res_values)
+                    || levenshteinWeightedGeneric<Int16>(columns, res_values)
+                    || levenshteinWeightedGeneric<Int32>(columns, res_values)
+                    || levenshteinWeightedGeneric<Int64>(columns, res_values)
+                    || levenshteinWeightedGeneric<Int128>(columns, res_values)
+                    || levenshteinWeightedGeneric<Int256>(columns, res_values)
+                    || levenshteinWeightedGeneric<Float32>(columns, res_values)
+                    || levenshteinWeightedGeneric<Float64>(columns, res_values)))
+                    throw Exception(
+                        ErrorCodes::LOGICAL_ERROR,
+                        "Unexpected code branch of function {}. No fitting levenshteinWeightedGeneric",
+                        getName());
+                return;
+            }
+            throw Exception(
+                ErrorCodes::LOGICAL_ERROR,
+                "Unexpected code branch of function {}. Wrong expected result type in levenshteinGeneric",
+                getName());
+        }
+
+        const ColumnArray * column_from = columns[0];
+        const ColumnArray * column_to = columns[1];
+        for (size_t row = 0; row < column_from->size(); row++)
+        {
+            // Effective Levenshtein realization from Common/levenshteinDistance
+            Array from = (*column_from)[row].safeGet<Array>();
+            Array to = (*column_to)[row].safeGet<Array>();
+            std::span<const Field> from_vec(from.begin(), from.end());
+            std::span<const Field> to_vec(to.begin(), to.end());
+            res_values[row] = levenshteinDistance<Field>(from_vec, to_vec);
+        }
+    }
+
+    template <typename N, typename W>
+    bool levenshteinWeightedString(std::vector<const ColumnArray *> columns, ColumnFloat64::Container & res_values) const
+    {
+        const N * from_data = checkAndGetColumn<N>(&columns[0]->getData());
+        const N * to_data = checkAndGetColumn<N>(&columns[1]->getData());
+        if (!from_data || !to_data)
+            return false;
+        const ColumnArray::Offsets & from_offsets = columns[0]->getOffsets();
+        ColumnArray::Offset prev_from_offset = 0;
+
+        const ColumnArray::Offsets & to_offsets = columns[1]->getOffsets();
+        ColumnArray::Offset prev_to_offset = 0;
+
+        const ColumnVector<W> * column_from_weights = checkAndGetColumn<ColumnVector<W>>(&columns[2]->getData());
+        const ColumnVector<W> * column_to_weights = checkAndGetColumn<ColumnVector<W>>(&columns[3]->getData());
+        if (!column_from_weights || !column_to_weights)
+            return false;
+
+        const ColumnArray::Offsets & from_weights_offsets = columns[2]->getOffsets();
+        ColumnArray::Offset prev_from_weights_offset = 0;
+
+        const ColumnArray::Offsets & to_weights_offsets = columns[3]->getOffsets();
+        ColumnArray::Offset prev_to_weights_offset = 0;
+
+        const auto extract_array = [](const N * data, size_t prev_offset, size_t count)
+        {
+            std::vector<StringRef> temp;
+            temp.reserve(count);
+            for (size_t j = 0; j < count; ++j) { temp.emplace_back(data->getDataAt(prev_offset + j)); }
+            return temp;
+        };
+
+        for (size_t row = 0; row < columns[0]->size(); row++)
+        {
+            const size_t m = from_offsets[row] - prev_from_offset;
+            const size_t n = to_offsets[row] - prev_to_offset;
+            const std::vector<StringRef> & from_vec = extract_array(from_data, prev_from_offset, m);
+            const std::vector<StringRef> & to_vec = extract_array(to_data, prev_to_offset, n);
+            std::span<const StringRef> from(from_vec.begin(), m);
+            std::span<const StringRef> to(to_vec.begin(), n);
path: src/Functions/array/arrayLevenshtein.cpp
user: login: Michicosun
body: also
created_at: 2025-03-27T14:24:52Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,648 @@
+#include <Columns/ColumnArray.h>
+#include <Columns/ColumnDecimal.h>
+#include <Columns/ColumnFixedString.h>
+#include <Columns/ColumnNullable.h>
+#include <Columns/ColumnString.h>
+#include <Columns/ColumnTuple.h>
+#include <Common/levenshteinDistance.h>
+#include <Common/PODArray.h>
+#include <Common/iota.h>
+#include <DataTypes/DataTypeArray.h>
+#include <DataTypes/DataTypeNullable.h>
+#include <DataTypes/DataTypeTuple.h>
+#include <DataTypes/DataTypesNumber.h>
+#include <Functions/FunctionFactory.h>
+#include <Functions/FunctionHelpers.h>
+#include <IO/WriteHelpers.h>
+
+#include <numeric>
+#include <span>
+
+namespace DB
+{
+
+namespace ErrorCodes
+{
+    extern const int ILLEGAL_TYPE_OF_ARGUMENT;
+    extern const int LOGICAL_ERROR;
+    extern const int SIZES_OF_ARRAYS_DONT_MATCH;
+}
+
+/// arrayLevenshteinDistance([1,2,3,4], [1,3,2,4]) = 2
+/// arrayLevenshteinDistanceWeighted([1,2,3,4], [1,3,2,4]) = 2
+template <typename T>
+class FunctionArrayLevenshtein : public IFunction
+{
+public:
+    static constexpr auto name = T::name;
+    static FunctionPtr create(ContextPtr) { return std::make_shared<FunctionArrayLevenshtein<T>>(); }
+
+    String getName() const override { return name; }
+
+    size_t getNumberOfArguments() const override { return T::arguments; }
+    bool useDefaultImplementationForConstants() const override { return true; }
+
+    bool isSuitableForShortCircuitArgumentsExecution(const DataTypesWithConstInfo & /*arguments*/) const override { return true; }
+
+    DataTypePtr getReturnTypeImpl(const ColumnsWithTypeAndName & arguments) const override
+    {
+        FunctionArgumentDescriptors args_descriptors;
+        args_descriptors = FunctionArgumentDescriptors{
+            {"from", static_cast<FunctionArgumentDescriptor::TypeValidator>(&isArray), nullptr, "Array"},
+            {"to", static_cast<FunctionArgumentDescriptor::TypeValidator>(&isArray), nullptr, "Array"},
+            {"from_weights", static_cast<FunctionArgumentDescriptor::TypeValidator>(&isArray), nullptr, "Array"},
+            {"to_weights", static_cast<FunctionArgumentDescriptor::TypeValidator>(&isArray), nullptr, "Array"},
+        };
+        validateFunctionArguments(*this, arguments, args_descriptors);
+        std::vector<DataTypePtr> nested_types;
+        nested_types.reserve(2);
+        for (size_t index = 2; index < 4; ++index)
+        {
+            const DataTypeArray * array_type = checkAndGetDataType<DataTypeArray>(arguments[index].type.get());
+            const DataTypePtr nested_type = array_type->getNestedType();
+            nested_types.emplace_back(nested_type);
+            if (!(isFloat(nested_type) || isInteger(nested_type)))
+                throw Exception(
+                    ErrorCodes::ILLEGAL_TYPE_OF_ARGUMENT,
+                    "Argument {} of function {} must be array of Float64. Found {} instead.",
+                    toString(index + 1),
+                    getName(),
+                    nested_type->getName());
+        }
+
+        if (nested_types[0]->getName() != nested_types[1]->getName())
+            throw Exception(
+                ErrorCodes::ILLEGAL_TYPE_OF_ARGUMENT,
+                "Arguments 3 and 4 of function {} must be arrays of the same types. Found {} and {} instead.",
+                getName(),
+                nested_types[0]->getName(),
+                nested_types[1]->getName());
+        return std::make_shared<DataTypeFloat64>();
+    }
+
+    ColumnPtr executeImpl(const ColumnsWithTypeAndName & arguments, const DataTypePtr &, size_t input_rows_count) const override
+    {
+        size_t num_arguments = arguments.size();
+
+        Columns holders(num_arguments);
+        std::vector<const ColumnArray *> columns(num_arguments);
+
+        for (size_t i = 0; i < num_arguments; ++i)
+        {
+            holders[i] = arguments[i].column->convertToFullColumnIfConst();
+            if (holders[i]->size() != input_rows_count)
+                throw Exception(
+                    ErrorCodes::LOGICAL_ERROR,
+                    "Function {} has unequal number of rows in columns: "
+                    "expected {}, got {} for column {}",
+                    getName(),
+                    input_rows_count,
+                    holders[i]->size(),
+                    holders[i]->getName());
+            columns[i] = assert_cast<const ColumnArray*>(holders[i].get());
+        }
+        return execute(columns);
+    }
+private:
+    ColumnPtr execute(std::vector<const ColumnArray *>) const
+    {
+        throw Exception(
+            ErrorCodes::LOGICAL_ERROR,
+            "Unknown function {}. "
+            "Supported names: 'arrayLevenshteinDistance', 'arrayLevenshteinDistanceWeighted', 'arraySimilarity'",
+            getName());
+    }
+
+    template <typename N, typename Result>
+    bool levenshteinString(std::vector<const ColumnArray *> columns, Result::Container & res_values) const
+    {
+        const N * from_data = checkAndGetColumn<N>(&columns[0]->getData());
+        const N * to_data = checkAndGetColumn<N>(&columns[1]->getData());
+        if (!from_data || !to_data)
+            return false;
+        if (T::arguments == 4)
+        {
+            if constexpr (std::is_same_v<Result, ColumnFloat64>)
+            {
+                if (!(
+                    levenshteinWeightedString<N, UInt8>(columns, res_values)
+                    || levenshteinWeightedString<N, UInt16>(columns, res_values)
+                    || levenshteinWeightedString<N, UInt32>(columns, res_values)
+                    || levenshteinWeightedString<N, UInt64>(columns, res_values)
+                    || levenshteinWeightedString<N, UInt128>(columns, res_values)
+                    || levenshteinWeightedString<N, UInt256>(columns, res_values)
+                    || levenshteinWeightedString<N, Int8>(columns, res_values)
+                    || levenshteinWeightedString<N, Int16>(columns, res_values)
+                    || levenshteinWeightedString<N, Int32>(columns, res_values)
+                    || levenshteinWeightedString<N, Int64>(columns, res_values)
+                    || levenshteinWeightedString<N, Int128>(columns, res_values)
+                    || levenshteinWeightedString<N, Int256>(columns, res_values)
+                    || levenshteinWeightedString<N, Float32>(columns, res_values)
+                    || levenshteinWeightedString<N, Float64>(columns, res_values)))
+                    throw Exception(
+                        ErrorCodes::LOGICAL_ERROR,
+                        "Unexpected code branch of function {}. No fitting levenshteinWeightedString",
+                        getName());
+                return true;
+            }
+            throw Exception(
+                ErrorCodes::LOGICAL_ERROR,
+                "Unexpected code branch of function {}. Wrong expected result type in levenshteinString",
+                getName());
+        }
+        const ColumnArray::Offsets & from_offsets = columns[0]->getOffsets();
+        ColumnArray::Offset prev_from_offset = 0;
+
+        const ColumnArray::Offsets & to_offsets = columns[1]->getOffsets();
+        ColumnArray::Offset prev_to_offset = 0;
+        const auto extract_array = [](const N * data, size_t prev_offset, size_t count)
+        {
+            std::vector<StringRef> temp;
+            temp.reserve(count);
+            for (size_t j = 0; j < count; ++j) { temp.emplace_back(data->getDataAt(prev_offset + j)); }
+            return temp;
+        };
+
+        for (size_t row = 0; row < columns[0]->size(); row++)
+        {
+            const size_t m = from_offsets[row] - prev_from_offset;
+            const size_t n = to_offsets[row] - prev_to_offset;
+            const std::vector<StringRef> & from_vec = extract_array(from_data, prev_from_offset, m);
+            const std::vector<StringRef> & to_vec = extract_array(to_data, prev_to_offset, n);
+            std::span<const StringRef> from(from_vec.begin(), m);
+            std::span<const StringRef> to(to_vec.begin(), n);
+            prev_from_offset = from_offsets[row];
+            prev_to_offset = to_offsets[row];
+            res_values[row] = levenshteinDistance<StringRef>(from, to);
+        }
+        return true;
+    }
+
+    template <typename N, typename Result>
+    bool levenshteinNumber(std::vector<const ColumnArray *> columns, Result::Container & res_values) const
+    {
+        const ColumnVectorOrDecimal<N> * column_from = checkAndGetColumn<ColumnVectorOrDecimal<N>>(&columns[0]->getData());
+        const ColumnVectorOrDecimal<N> * column_to = checkAndGetColumn<ColumnVectorOrDecimal<N>>(&columns[1]->getData());
+        if (!column_from || !column_to)
+            return false;
+
+        if (T::arguments == 4)
+        {
+            if constexpr (std::is_same_v<Result, ColumnFloat64>)
+            {
+                if (!(
+                    levenshteinWeightedNumber<N, UInt8>(columns, res_values)
+                    || levenshteinWeightedNumber<N, UInt16>(columns, res_values)
+                    || levenshteinWeightedNumber<N, UInt32>(columns, res_values)
+                    || levenshteinWeightedNumber<N, UInt64>(columns, res_values)
+                    || levenshteinWeightedNumber<N, UInt128>(columns, res_values)
+                    || levenshteinWeightedNumber<N, UInt256>(columns, res_values)
+                    || levenshteinWeightedNumber<N, Int8>(columns, res_values)
+                    || levenshteinWeightedNumber<N, Int16>(columns, res_values)
+                    || levenshteinWeightedNumber<N, Int32>(columns, res_values)
+                    || levenshteinWeightedNumber<N, Int64>(columns, res_values)
+                    || levenshteinWeightedNumber<N, Int128>(columns, res_values)
+                    || levenshteinWeightedNumber<N, Int256>(columns, res_values)
+                    || levenshteinWeightedNumber<N, Float32>(columns, res_values)
+                    || levenshteinWeightedNumber<N, Float64>(columns, res_values)))
+                    throw Exception(
+                        ErrorCodes::LOGICAL_ERROR,
+                        "Unexpected code branch of function {}. No fitting levenshteinWeightedNumber",
+                        getName());
+                return true;
+            }
+            throw Exception(
+                ErrorCodes::LOGICAL_ERROR,
+                "Unexpected code branch of function {}. Wrong expected result type in levenshteinNumber",
+                getName());
+        }
+
+        const ColumnArray::Offsets & from_offsets = columns[0]->getOffsets();
+        ColumnArray::Offset prev_from_offset = 0;
+
+        const ColumnArray::Offsets & to_offsets = columns[1]->getOffsets();
+        ColumnArray::Offset prev_to_offset = 0;
+
+        for (size_t row = 0; row < columns[0]->size(); row++)
+        {
+            std::span<const N> from(column_from->getData().begin() + prev_from_offset, from_offsets[row] - prev_from_offset);
+            prev_from_offset = from_offsets[row];
+            std::span<const N> to(column_to->getData().begin() + prev_to_offset, to_offsets[row] - prev_to_offset);
+            prev_to_offset = to_offsets[row];
+
+            res_values[row] = levenshteinDistance<N>(from, to);
+        }
+        return true;
+    }
+
+    template<typename Result>
+    void levenshteinGeneric(std::vector<const ColumnArray *> columns, Result::Container & res_values) const
+    {
+        if (T::arguments == 4)
+        {
+            if constexpr (std::is_same_v<Result, ColumnFloat64>)
+            {
+                if (!(
+                    levenshteinWeightedGeneric<UInt8>(columns, res_values)
+                    || levenshteinWeightedGeneric<UInt16>(columns, res_values)
+                    || levenshteinWeightedGeneric<UInt32>(columns, res_values)
+                    || levenshteinWeightedGeneric<UInt64>(columns, res_values)
+                    || levenshteinWeightedGeneric<UInt128>(columns, res_values)
+                    || levenshteinWeightedGeneric<UInt256>(columns, res_values)
+                    || levenshteinWeightedGeneric<Int8>(columns, res_values)
+                    || levenshteinWeightedGeneric<Int16>(columns, res_values)
+                    || levenshteinWeightedGeneric<Int32>(columns, res_values)
+                    || levenshteinWeightedGeneric<Int64>(columns, res_values)
+                    || levenshteinWeightedGeneric<Int128>(columns, res_values)
+                    || levenshteinWeightedGeneric<Int256>(columns, res_values)
+                    || levenshteinWeightedGeneric<Float32>(columns, res_values)
+                    || levenshteinWeightedGeneric<Float64>(columns, res_values)))
+                    throw Exception(
+                        ErrorCodes::LOGICAL_ERROR,
+                        "Unexpected code branch of function {}. No fitting levenshteinWeightedGeneric",
+                        getName());
+                return;
+            }
+            throw Exception(
+                ErrorCodes::LOGICAL_ERROR,
+                "Unexpected code branch of function {}. Wrong expected result type in levenshteinGeneric",
+                getName());
+        }
+
+        const ColumnArray * column_from = columns[0];
+        const ColumnArray * column_to = columns[1];
+        for (size_t row = 0; row < column_from->size(); row++)
+        {
+            // Effective Levenshtein realization from Common/levenshteinDistance
+            Array from = (*column_from)[row].safeGet<Array>();
+            Array to = (*column_to)[row].safeGet<Array>();
+            std::span<const Field> from_vec(from.begin(), from.end());
+            std::span<const Field> to_vec(to.begin(), to.end());
path: src/Functions/array/arrayLevenshtein.cpp
user: login: Michicosun
body: also
created_at: 2025-03-27T14:35:41Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,648 @@
+#include <Columns/ColumnArray.h>
+#include <Columns/ColumnDecimal.h>
+#include <Columns/ColumnFixedString.h>
+#include <Columns/ColumnNullable.h>
+#include <Columns/ColumnString.h>
+#include <Columns/ColumnTuple.h>
+#include <Common/levenshteinDistance.h>
+#include <Common/PODArray.h>
+#include <Common/iota.h>
+#include <DataTypes/DataTypeArray.h>
+#include <DataTypes/DataTypeNullable.h>
+#include <DataTypes/DataTypeTuple.h>
+#include <DataTypes/DataTypesNumber.h>
+#include <Functions/FunctionFactory.h>
+#include <Functions/FunctionHelpers.h>
+#include <IO/WriteHelpers.h>
+
+#include <numeric>
+#include <span>
+
+namespace DB
+{
+
+namespace ErrorCodes
+{
+    extern const int ILLEGAL_TYPE_OF_ARGUMENT;
+    extern const int LOGICAL_ERROR;
+    extern const int SIZES_OF_ARRAYS_DONT_MATCH;
+}
+
+/// arrayLevenshteinDistance([1,2,3,4], [1,3,2,4]) = 2
+/// arrayLevenshteinDistanceWeighted([1,2,3,4], [1,3,2,4]) = 2
+template <typename T>
+class FunctionArrayLevenshtein : public IFunction
+{
+public:
+    static constexpr auto name = T::name;
+    static FunctionPtr create(ContextPtr) { return std::make_shared<FunctionArrayLevenshtein<T>>(); }
+
+    String getName() const override { return name; }
+
+    size_t getNumberOfArguments() const override { return T::arguments; }
+    bool useDefaultImplementationForConstants() const override { return true; }
+
+    bool isSuitableForShortCircuitArgumentsExecution(const DataTypesWithConstInfo & /*arguments*/) const override { return true; }
+
+    DataTypePtr getReturnTypeImpl(const ColumnsWithTypeAndName & arguments) const override
+    {
+        FunctionArgumentDescriptors args_descriptors;
+        args_descriptors = FunctionArgumentDescriptors{
+            {"from", static_cast<FunctionArgumentDescriptor::TypeValidator>(&isArray), nullptr, "Array"},
+            {"to", static_cast<FunctionArgumentDescriptor::TypeValidator>(&isArray), nullptr, "Array"},
+            {"from_weights", static_cast<FunctionArgumentDescriptor::TypeValidator>(&isArray), nullptr, "Array"},
+            {"to_weights", static_cast<FunctionArgumentDescriptor::TypeValidator>(&isArray), nullptr, "Array"},
+        };
+        validateFunctionArguments(*this, arguments, args_descriptors);
+        std::vector<DataTypePtr> nested_types;
+        nested_types.reserve(2);
+        for (size_t index = 2; index < 4; ++index)
+        {
+            const DataTypeArray * array_type = checkAndGetDataType<DataTypeArray>(arguments[index].type.get());
+            const DataTypePtr nested_type = array_type->getNestedType();
+            nested_types.emplace_back(nested_type);
+            if (!(isFloat(nested_type) || isInteger(nested_type)))
+                throw Exception(
+                    ErrorCodes::ILLEGAL_TYPE_OF_ARGUMENT,
+                    "Argument {} of function {} must be array of Float64. Found {} instead.",
+                    toString(index + 1),
+                    getName(),
+                    nested_type->getName());
+        }
+
+        if (nested_types[0]->getName() != nested_types[1]->getName())
+            throw Exception(
+                ErrorCodes::ILLEGAL_TYPE_OF_ARGUMENT,
+                "Arguments 3 and 4 of function {} must be arrays of the same types. Found {} and {} instead.",
+                getName(),
+                nested_types[0]->getName(),
+                nested_types[1]->getName());
+        return std::make_shared<DataTypeFloat64>();
+    }
+
+    ColumnPtr executeImpl(const ColumnsWithTypeAndName & arguments, const DataTypePtr &, size_t input_rows_count) const override
+    {
+        size_t num_arguments = arguments.size();
+
+        Columns holders(num_arguments);
+        std::vector<const ColumnArray *> columns(num_arguments);
+
+        for (size_t i = 0; i < num_arguments; ++i)
+        {
+            holders[i] = arguments[i].column->convertToFullColumnIfConst();
+            if (holders[i]->size() != input_rows_count)
+                throw Exception(
+                    ErrorCodes::LOGICAL_ERROR,
+                    "Function {} has unequal number of rows in columns: "
+                    "expected {}, got {} for column {}",
+                    getName(),
+                    input_rows_count,
+                    holders[i]->size(),
+                    holders[i]->getName());
+            columns[i] = assert_cast<const ColumnArray*>(holders[i].get());
+        }
+        return execute(columns);
+    }
+private:
+    ColumnPtr execute(std::vector<const ColumnArray *>) const
+    {
+        throw Exception(
+            ErrorCodes::LOGICAL_ERROR,
+            "Unknown function {}. "
+            "Supported names: 'arrayLevenshteinDistance', 'arrayLevenshteinDistanceWeighted', 'arraySimilarity'",
+            getName());
+    }
+
+    template <typename N, typename Result>
+    bool levenshteinString(std::vector<const ColumnArray *> columns, Result::Container & res_values) const
+    {
+        const N * from_data = checkAndGetColumn<N>(&columns[0]->getData());
+        const N * to_data = checkAndGetColumn<N>(&columns[1]->getData());
+        if (!from_data || !to_data)
+            return false;
+        if (T::arguments == 4)
+        {
+            if constexpr (std::is_same_v<Result, ColumnFloat64>)
+            {
+                if (!(
+                    levenshteinWeightedString<N, UInt8>(columns, res_values)
+                    || levenshteinWeightedString<N, UInt16>(columns, res_values)
+                    || levenshteinWeightedString<N, UInt32>(columns, res_values)
+                    || levenshteinWeightedString<N, UInt64>(columns, res_values)
+                    || levenshteinWeightedString<N, UInt128>(columns, res_values)
+                    || levenshteinWeightedString<N, UInt256>(columns, res_values)
+                    || levenshteinWeightedString<N, Int8>(columns, res_values)
+                    || levenshteinWeightedString<N, Int16>(columns, res_values)
+                    || levenshteinWeightedString<N, Int32>(columns, res_values)
+                    || levenshteinWeightedString<N, Int64>(columns, res_values)
+                    || levenshteinWeightedString<N, Int128>(columns, res_values)
+                    || levenshteinWeightedString<N, Int256>(columns, res_values)
+                    || levenshteinWeightedString<N, Float32>(columns, res_values)
+                    || levenshteinWeightedString<N, Float64>(columns, res_values)))
+                    throw Exception(
+                        ErrorCodes::LOGICAL_ERROR,
+                        "Unexpected code branch of function {}. No fitting levenshteinWeightedString",
+                        getName());
+                return true;
+            }
+            throw Exception(
+                ErrorCodes::LOGICAL_ERROR,
+                "Unexpected code branch of function {}. Wrong expected result type in levenshteinString",
+                getName());
+        }
+        const ColumnArray::Offsets & from_offsets = columns[0]->getOffsets();
+        ColumnArray::Offset prev_from_offset = 0;
+
+        const ColumnArray::Offsets & to_offsets = columns[1]->getOffsets();
+        ColumnArray::Offset prev_to_offset = 0;
+        const auto extract_array = [](const N * data, size_t prev_offset, size_t count)
+        {
+            std::vector<StringRef> temp;
+            temp.reserve(count);
+            for (size_t j = 0; j < count; ++j) { temp.emplace_back(data->getDataAt(prev_offset + j)); }
+            return temp;
+        };
+
+        for (size_t row = 0; row < columns[0]->size(); row++)
+        {
+            const size_t m = from_offsets[row] - prev_from_offset;
+            const size_t n = to_offsets[row] - prev_to_offset;
+            const std::vector<StringRef> & from_vec = extract_array(from_data, prev_from_offset, m);
+            const std::vector<StringRef> & to_vec = extract_array(to_data, prev_to_offset, n);
+            std::span<const StringRef> from(from_vec.begin(), m);
+            std::span<const StringRef> to(to_vec.begin(), n);
path: src/Functions/array/arrayLevenshtein.cpp
user: login: Michicosun
body: also
created_at: 2025-03-27T14:37:04Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,648 @@
+#include <Columns/ColumnArray.h>
+#include <Columns/ColumnDecimal.h>
+#include <Columns/ColumnFixedString.h>
+#include <Columns/ColumnNullable.h>
+#include <Columns/ColumnString.h>
+#include <Columns/ColumnTuple.h>
+#include <Common/levenshteinDistance.h>
+#include <Common/PODArray.h>
+#include <Common/iota.h>
+#include <DataTypes/DataTypeArray.h>
+#include <DataTypes/DataTypeNullable.h>
+#include <DataTypes/DataTypeTuple.h>
+#include <DataTypes/DataTypesNumber.h>
+#include <Functions/FunctionFactory.h>
+#include <Functions/FunctionHelpers.h>
+#include <IO/WriteHelpers.h>
+
+#include <numeric>
+#include <span>
+
+namespace DB
+{
+
+namespace ErrorCodes
+{
+    extern const int ILLEGAL_TYPE_OF_ARGUMENT;
+    extern const int LOGICAL_ERROR;
+    extern const int SIZES_OF_ARRAYS_DONT_MATCH;
+}
+
+/// arrayLevenshteinDistance([1,2,3,4], [1,3,2,4]) = 2
+/// arrayLevenshteinDistanceWeighted([1,2,3,4], [1,3,2,4]) = 2
+template <typename T>
+class FunctionArrayLevenshtein : public IFunction
+{
+public:
+    static constexpr auto name = T::name;
+    static FunctionPtr create(ContextPtr) { return std::make_shared<FunctionArrayLevenshtein<T>>(); }
+
+    String getName() const override { return name; }
+
+    size_t getNumberOfArguments() const override { return T::arguments; }
+    bool useDefaultImplementationForConstants() const override { return true; }
+
+    bool isSuitableForShortCircuitArgumentsExecution(const DataTypesWithConstInfo & /*arguments*/) const override { return true; }
+
+    DataTypePtr getReturnTypeImpl(const ColumnsWithTypeAndName & arguments) const override
+    {
+        FunctionArgumentDescriptors args_descriptors;
+        args_descriptors = FunctionArgumentDescriptors{
+            {"from", static_cast<FunctionArgumentDescriptor::TypeValidator>(&isArray), nullptr, "Array"},
+            {"to", static_cast<FunctionArgumentDescriptor::TypeValidator>(&isArray), nullptr, "Array"},
+            {"from_weights", static_cast<FunctionArgumentDescriptor::TypeValidator>(&isArray), nullptr, "Array"},
+            {"to_weights", static_cast<FunctionArgumentDescriptor::TypeValidator>(&isArray), nullptr, "Array"},
+        };
+        validateFunctionArguments(*this, arguments, args_descriptors);
+        std::vector<DataTypePtr> nested_types;
+        nested_types.reserve(2);
+        for (size_t index = 2; index < 4; ++index)
+        {
+            const DataTypeArray * array_type = checkAndGetDataType<DataTypeArray>(arguments[index].type.get());
+            const DataTypePtr nested_type = array_type->getNestedType();
+            nested_types.emplace_back(nested_type);
+            if (!(isFloat(nested_type) || isInteger(nested_type)))
+                throw Exception(
+                    ErrorCodes::ILLEGAL_TYPE_OF_ARGUMENT,
+                    "Argument {} of function {} must be array of Float64. Found {} instead.",
+                    toString(index + 1),
+                    getName(),
+                    nested_type->getName());
+        }
+
+        if (nested_types[0]->getName() != nested_types[1]->getName())
+            throw Exception(
+                ErrorCodes::ILLEGAL_TYPE_OF_ARGUMENT,
+                "Arguments 3 and 4 of function {} must be arrays of the same types. Found {} and {} instead.",
+                getName(),
+                nested_types[0]->getName(),
+                nested_types[1]->getName());
+        return std::make_shared<DataTypeFloat64>();
+    }
+
+    ColumnPtr executeImpl(const ColumnsWithTypeAndName & arguments, const DataTypePtr &, size_t input_rows_count) const override
+    {
+        size_t num_arguments = arguments.size();
+
+        Columns holders(num_arguments);
+        std::vector<const ColumnArray *> columns(num_arguments);
+
+        for (size_t i = 0; i < num_arguments; ++i)
+        {
+            holders[i] = arguments[i].column->convertToFullColumnIfConst();
+            if (holders[i]->size() != input_rows_count)
+                throw Exception(
+                    ErrorCodes::LOGICAL_ERROR,
+                    "Function {} has unequal number of rows in columns: "
+                    "expected {}, got {} for column {}",
+                    getName(),
+                    input_rows_count,
+                    holders[i]->size(),
+                    holders[i]->getName());
+            columns[i] = assert_cast<const ColumnArray*>(holders[i].get());
+        }
+        return execute(columns);
+    }
+private:
+    ColumnPtr execute(std::vector<const ColumnArray *>) const
+    {
+        throw Exception(
+            ErrorCodes::LOGICAL_ERROR,
+            "Unknown function {}. "
+            "Supported names: 'arrayLevenshteinDistance', 'arrayLevenshteinDistanceWeighted', 'arraySimilarity'",
+            getName());
+    }
+
+    template <typename N, typename Result>
+    bool levenshteinString(std::vector<const ColumnArray *> columns, Result::Container & res_values) const
+    {
+        const N * from_data = checkAndGetColumn<N>(&columns[0]->getData());
+        const N * to_data = checkAndGetColumn<N>(&columns[1]->getData());
+        if (!from_data || !to_data)
+            return false;
+        if (T::arguments == 4)
+        {
+            if constexpr (std::is_same_v<Result, ColumnFloat64>)
+            {
+                if (!(
+                    levenshteinWeightedString<N, UInt8>(columns, res_values)
+                    || levenshteinWeightedString<N, UInt16>(columns, res_values)
+                    || levenshteinWeightedString<N, UInt32>(columns, res_values)
+                    || levenshteinWeightedString<N, UInt64>(columns, res_values)
+                    || levenshteinWeightedString<N, UInt128>(columns, res_values)
+                    || levenshteinWeightedString<N, UInt256>(columns, res_values)
+                    || levenshteinWeightedString<N, Int8>(columns, res_values)
+                    || levenshteinWeightedString<N, Int16>(columns, res_values)
+                    || levenshteinWeightedString<N, Int32>(columns, res_values)
+                    || levenshteinWeightedString<N, Int64>(columns, res_values)
+                    || levenshteinWeightedString<N, Int128>(columns, res_values)
+                    || levenshteinWeightedString<N, Int256>(columns, res_values)
+                    || levenshteinWeightedString<N, Float32>(columns, res_values)
+                    || levenshteinWeightedString<N, Float64>(columns, res_values)))
+                    throw Exception(
+                        ErrorCodes::LOGICAL_ERROR,
+                        "Unexpected code branch of function {}. No fitting levenshteinWeightedString",
+                        getName());
+                return true;
+            }
+            throw Exception(
+                ErrorCodes::LOGICAL_ERROR,
+                "Unexpected code branch of function {}. Wrong expected result type in levenshteinString",
+                getName());
+        }
+        const ColumnArray::Offsets & from_offsets = columns[0]->getOffsets();
+        ColumnArray::Offset prev_from_offset = 0;
+
+        const ColumnArray::Offsets & to_offsets = columns[1]->getOffsets();
+        ColumnArray::Offset prev_to_offset = 0;
+        const auto extract_array = [](const N * data, size_t prev_offset, size_t count)
+        {
+            std::vector<StringRef> temp;
+            temp.reserve(count);
+            for (size_t j = 0; j < count; ++j) { temp.emplace_back(data->getDataAt(prev_offset + j)); }
+            return temp;
+        };
+
+        for (size_t row = 0; row < columns[0]->size(); row++)
+        {
+            const size_t m = from_offsets[row] - prev_from_offset;
+            const size_t n = to_offsets[row] - prev_to_offset;
+            const std::vector<StringRef> & from_vec = extract_array(from_data, prev_from_offset, m);
+            const std::vector<StringRef> & to_vec = extract_array(to_data, prev_to_offset, n);
+            std::span<const StringRef> from(from_vec.begin(), m);
+            std::span<const StringRef> to(to_vec.begin(), n);
+            prev_from_offset = from_offsets[row];
+            prev_to_offset = to_offsets[row];
+            res_values[row] = levenshteinDistance<StringRef>(from, to);
+        }
+        return true;
+    }
+
+    template <typename N, typename Result>
+    bool levenshteinNumber(std::vector<const ColumnArray *> columns, Result::Container & res_values) const
+    {
+        const ColumnVectorOrDecimal<N> * column_from = checkAndGetColumn<ColumnVectorOrDecimal<N>>(&columns[0]->getData());
+        const ColumnVectorOrDecimal<N> * column_to = checkAndGetColumn<ColumnVectorOrDecimal<N>>(&columns[1]->getData());
+        if (!column_from || !column_to)
+            return false;
+
+        if (T::arguments == 4)
+        {
+            if constexpr (std::is_same_v<Result, ColumnFloat64>)
+            {
+                if (!(
+                    levenshteinWeightedNumber<N, UInt8>(columns, res_values)
+                    || levenshteinWeightedNumber<N, UInt16>(columns, res_values)
+                    || levenshteinWeightedNumber<N, UInt32>(columns, res_values)
+                    || levenshteinWeightedNumber<N, UInt64>(columns, res_values)
+                    || levenshteinWeightedNumber<N, UInt128>(columns, res_values)
+                    || levenshteinWeightedNumber<N, UInt256>(columns, res_values)
+                    || levenshteinWeightedNumber<N, Int8>(columns, res_values)
+                    || levenshteinWeightedNumber<N, Int16>(columns, res_values)
+                    || levenshteinWeightedNumber<N, Int32>(columns, res_values)
+                    || levenshteinWeightedNumber<N, Int64>(columns, res_values)
+                    || levenshteinWeightedNumber<N, Int128>(columns, res_values)
+                    || levenshteinWeightedNumber<N, Int256>(columns, res_values)
+                    || levenshteinWeightedNumber<N, Float32>(columns, res_values)
+                    || levenshteinWeightedNumber<N, Float64>(columns, res_values)))
+                    throw Exception(
+                        ErrorCodes::LOGICAL_ERROR,
+                        "Unexpected code branch of function {}. No fitting levenshteinWeightedNumber",
+                        getName());
+                return true;
+            }
+            throw Exception(
+                ErrorCodes::LOGICAL_ERROR,
+                "Unexpected code branch of function {}. Wrong expected result type in levenshteinNumber",
+                getName());
+        }
+
+        const ColumnArray::Offsets & from_offsets = columns[0]->getOffsets();
+        ColumnArray::Offset prev_from_offset = 0;
+
+        const ColumnArray::Offsets & to_offsets = columns[1]->getOffsets();
+        ColumnArray::Offset prev_to_offset = 0;
+
+        for (size_t row = 0; row < columns[0]->size(); row++)
+        {
+            std::span<const N> from(column_from->getData().begin() + prev_from_offset, from_offsets[row] - prev_from_offset);
+            prev_from_offset = from_offsets[row];
+            std::span<const N> to(column_to->getData().begin() + prev_to_offset, to_offsets[row] - prev_to_offset);
+            prev_to_offset = to_offsets[row];
+
+            res_values[row] = levenshteinDistance<N>(from, to);
+        }
+        return true;
+    }
+
+    template<typename Result>
+    void levenshteinGeneric(std::vector<const ColumnArray *> columns, Result::Container & res_values) const
+    {
+        if (T::arguments == 4)
+        {
+            if constexpr (std::is_same_v<Result, ColumnFloat64>)
+            {
+                if (!(
+                    levenshteinWeightedGeneric<UInt8>(columns, res_values)
+                    || levenshteinWeightedGeneric<UInt16>(columns, res_values)
+                    || levenshteinWeightedGeneric<UInt32>(columns, res_values)
+                    || levenshteinWeightedGeneric<UInt64>(columns, res_values)
+                    || levenshteinWeightedGeneric<UInt128>(columns, res_values)
+                    || levenshteinWeightedGeneric<UInt256>(columns, res_values)
+                    || levenshteinWeightedGeneric<Int8>(columns, res_values)
+                    || levenshteinWeightedGeneric<Int16>(columns, res_values)
+                    || levenshteinWeightedGeneric<Int32>(columns, res_values)
+                    || levenshteinWeightedGeneric<Int64>(columns, res_values)
+                    || levenshteinWeightedGeneric<Int128>(columns, res_values)
+                    || levenshteinWeightedGeneric<Int256>(columns, res_values)
+                    || levenshteinWeightedGeneric<Float32>(columns, res_values)
+                    || levenshteinWeightedGeneric<Float64>(columns, res_values)))
+                    throw Exception(
+                        ErrorCodes::LOGICAL_ERROR,
+                        "Unexpected code branch of function {}. No fitting levenshteinWeightedGeneric",
+                        getName());
+                return;
+            }
+            throw Exception(
+                ErrorCodes::LOGICAL_ERROR,
+                "Unexpected code branch of function {}. Wrong expected result type in levenshteinGeneric",
+                getName());
+        }
+
+        const ColumnArray * column_from = columns[0];
+        const ColumnArray * column_to = columns[1];
+        for (size_t row = 0; row < column_from->size(); row++)
+        {
+            // Effective Levenshtein realization from Common/levenshteinDistance
+            Array from = (*column_from)[row].safeGet<Array>();
+            Array to = (*column_to)[row].safeGet<Array>();
+            std::span<const Field> from_vec(from.begin(), from.end());
+            std::span<const Field> to_vec(to.begin(), to.end());
+            res_values[row] = levenshteinDistance<Field>(from_vec, to_vec);
+        }
+    }
+
+    template <typename N, typename W>
+    bool levenshteinWeightedString(std::vector<const ColumnArray *> columns, ColumnFloat64::Container & res_values) const
+    {
+        const N * from_data = checkAndGetColumn<N>(&columns[0]->getData());
+        const N * to_data = checkAndGetColumn<N>(&columns[1]->getData());
+        if (!from_data || !to_data)
+            return false;
+        const ColumnArray::Offsets & from_offsets = columns[0]->getOffsets();
+        ColumnArray::Offset prev_from_offset = 0;
+
+        const ColumnArray::Offsets & to_offsets = columns[1]->getOffsets();
+        ColumnArray::Offset prev_to_offset = 0;
+
+        const ColumnVector<W> * column_from_weights = checkAndGetColumn<ColumnVector<W>>(&columns[2]->getData());
+        const ColumnVector<W> * column_to_weights = checkAndGetColumn<ColumnVector<W>>(&columns[3]->getData());
+        if (!column_from_weights || !column_to_weights)
+            return false;
+
+        const ColumnArray::Offsets & from_weights_offsets = columns[2]->getOffsets();
+        ColumnArray::Offset prev_from_weights_offset = 0;
+
+        const ColumnArray::Offsets & to_weights_offsets = columns[3]->getOffsets();
+        ColumnArray::Offset prev_to_weights_offset = 0;
+
+        const auto extract_array = [](const N * data, size_t prev_offset, size_t count)
+        {
+            std::vector<StringRef> temp;
+            temp.reserve(count);
+            for (size_t j = 0; j < count; ++j) { temp.emplace_back(data->getDataAt(prev_offset + j)); }
+            return temp;
+        };
+
+        for (size_t row = 0; row < columns[0]->size(); row++)
+        {
+            const size_t m = from_offsets[row] - prev_from_offset;
+            const size_t n = to_offsets[row] - prev_to_offset;
+            const std::vector<StringRef> & from_vec = extract_array(from_data, prev_from_offset, m);
+            const std::vector<StringRef> & to_vec = extract_array(to_data, prev_to_offset, n);
path: src/Functions/array/arrayLevenshtein.cpp
user: login: Michicosun
body: ```suggestion
            const std::vector<StringRef> from_vec = extract_array(from_data, prev_from_offset, m);
            const std::vector<StringRef> to_vec = extract_array(to_data, prev_to_offset, n);
```
created_at: 2025-03-27T15:11:09Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,648 @@
+#include <Columns/ColumnArray.h>
+#include <Columns/ColumnDecimal.h>
+#include <Columns/ColumnFixedString.h>
+#include <Columns/ColumnNullable.h>
+#include <Columns/ColumnString.h>
+#include <Columns/ColumnTuple.h>
+#include <Common/levenshteinDistance.h>
+#include <Common/PODArray.h>
+#include <Common/iota.h>
+#include <DataTypes/DataTypeArray.h>
+#include <DataTypes/DataTypeNullable.h>
+#include <DataTypes/DataTypeTuple.h>
+#include <DataTypes/DataTypesNumber.h>
+#include <Functions/FunctionFactory.h>
+#include <Functions/FunctionHelpers.h>
+#include <IO/WriteHelpers.h>
+
+#include <numeric>
+#include <span>
+
+namespace DB
+{
+
+namespace ErrorCodes
+{
+    extern const int ILLEGAL_TYPE_OF_ARGUMENT;
+    extern const int LOGICAL_ERROR;
+    extern const int SIZES_OF_ARRAYS_DONT_MATCH;
+}
+
+/// arrayLevenshteinDistance([1,2,3,4], [1,3,2,4]) = 2
+/// arrayLevenshteinDistanceWeighted([1,2,3,4], [1,3,2,4]) = 2
+template <typename T>
+class FunctionArrayLevenshtein : public IFunction
+{
+public:
+    static constexpr auto name = T::name;
+    static FunctionPtr create(ContextPtr) { return std::make_shared<FunctionArrayLevenshtein<T>>(); }
+
+    String getName() const override { return name; }
+
+    size_t getNumberOfArguments() const override { return T::arguments; }
+    bool useDefaultImplementationForConstants() const override { return true; }
+
+    bool isSuitableForShortCircuitArgumentsExecution(const DataTypesWithConstInfo & /*arguments*/) const override { return true; }
+
+    DataTypePtr getReturnTypeImpl(const ColumnsWithTypeAndName & arguments) const override
+    {
+        FunctionArgumentDescriptors args_descriptors;
+        args_descriptors = FunctionArgumentDescriptors{
+            {"from", static_cast<FunctionArgumentDescriptor::TypeValidator>(&isArray), nullptr, "Array"},
+            {"to", static_cast<FunctionArgumentDescriptor::TypeValidator>(&isArray), nullptr, "Array"},
+            {"from_weights", static_cast<FunctionArgumentDescriptor::TypeValidator>(&isArray), nullptr, "Array"},
+            {"to_weights", static_cast<FunctionArgumentDescriptor::TypeValidator>(&isArray), nullptr, "Array"},
+        };
+        validateFunctionArguments(*this, arguments, args_descriptors);
+        std::vector<DataTypePtr> nested_types;
+        nested_types.reserve(2);
+        for (size_t index = 2; index < 4; ++index)
+        {
+            const DataTypeArray * array_type = checkAndGetDataType<DataTypeArray>(arguments[index].type.get());
+            const DataTypePtr nested_type = array_type->getNestedType();
+            nested_types.emplace_back(nested_type);
+            if (!(isFloat(nested_type) || isInteger(nested_type)))
+                throw Exception(
+                    ErrorCodes::ILLEGAL_TYPE_OF_ARGUMENT,
+                    "Argument {} of function {} must be array of Float64. Found {} instead.",
+                    toString(index + 1),
+                    getName(),
+                    nested_type->getName());
+        }
+
+        if (nested_types[0]->getName() != nested_types[1]->getName())
+            throw Exception(
+                ErrorCodes::ILLEGAL_TYPE_OF_ARGUMENT,
+                "Arguments 3 and 4 of function {} must be arrays of the same types. Found {} and {} instead.",
+                getName(),
+                nested_types[0]->getName(),
+                nested_types[1]->getName());
+        return std::make_shared<DataTypeFloat64>();
+    }
+
+    ColumnPtr executeImpl(const ColumnsWithTypeAndName & arguments, const DataTypePtr &, size_t input_rows_count) const override
+    {
+        size_t num_arguments = arguments.size();
+
+        Columns holders(num_arguments);
+        std::vector<const ColumnArray *> columns(num_arguments);
+
+        for (size_t i = 0; i < num_arguments; ++i)
+        {
+            holders[i] = arguments[i].column->convertToFullColumnIfConst();
+            if (holders[i]->size() != input_rows_count)
+                throw Exception(
+                    ErrorCodes::LOGICAL_ERROR,
+                    "Function {} has unequal number of rows in columns: "
+                    "expected {}, got {} for column {}",
+                    getName(),
+                    input_rows_count,
+                    holders[i]->size(),
+                    holders[i]->getName());
+            columns[i] = assert_cast<const ColumnArray*>(holders[i].get());
+        }
+        return execute(columns);
+    }
+private:
+    ColumnPtr execute(std::vector<const ColumnArray *>) const
+    {
+        throw Exception(
+            ErrorCodes::LOGICAL_ERROR,
+            "Unknown function {}. "
+            "Supported names: 'arrayLevenshteinDistance', 'arrayLevenshteinDistanceWeighted', 'arraySimilarity'",
+            getName());
+    }
+
+    template <typename N, typename Result>
+    bool levenshteinString(std::vector<const ColumnArray *> columns, Result::Container & res_values) const
+    {
+        const N * from_data = checkAndGetColumn<N>(&columns[0]->getData());
+        const N * to_data = checkAndGetColumn<N>(&columns[1]->getData());
+        if (!from_data || !to_data)
+            return false;
+        if (T::arguments == 4)
+        {
+            if constexpr (std::is_same_v<Result, ColumnFloat64>)
+            {
+                if (!(
+                    levenshteinWeightedString<N, UInt8>(columns, res_values)
+                    || levenshteinWeightedString<N, UInt16>(columns, res_values)
+                    || levenshteinWeightedString<N, UInt32>(columns, res_values)
+                    || levenshteinWeightedString<N, UInt64>(columns, res_values)
+                    || levenshteinWeightedString<N, UInt128>(columns, res_values)
+                    || levenshteinWeightedString<N, UInt256>(columns, res_values)
+                    || levenshteinWeightedString<N, Int8>(columns, res_values)
+                    || levenshteinWeightedString<N, Int16>(columns, res_values)
+                    || levenshteinWeightedString<N, Int32>(columns, res_values)
+                    || levenshteinWeightedString<N, Int64>(columns, res_values)
+                    || levenshteinWeightedString<N, Int128>(columns, res_values)
+                    || levenshteinWeightedString<N, Int256>(columns, res_values)
+                    || levenshteinWeightedString<N, Float32>(columns, res_values)
+                    || levenshteinWeightedString<N, Float64>(columns, res_values)))
+                    throw Exception(
+                        ErrorCodes::LOGICAL_ERROR,
+                        "Unexpected code branch of function {}. No fitting levenshteinWeightedString",
+                        getName());
+                return true;
+            }
+            throw Exception(
+                ErrorCodes::LOGICAL_ERROR,
+                "Unexpected code branch of function {}. Wrong expected result type in levenshteinString",
+                getName());
+        }
+        const ColumnArray::Offsets & from_offsets = columns[0]->getOffsets();
+        ColumnArray::Offset prev_from_offset = 0;
+
+        const ColumnArray::Offsets & to_offsets = columns[1]->getOffsets();
+        ColumnArray::Offset prev_to_offset = 0;
+        const auto extract_array = [](const N * data, size_t prev_offset, size_t count)
+        {
+            std::vector<StringRef> temp;
+            temp.reserve(count);
+            for (size_t j = 0; j < count; ++j) { temp.emplace_back(data->getDataAt(prev_offset + j)); }
+            return temp;
+        };
+
+        for (size_t row = 0; row < columns[0]->size(); row++)
+        {
+            const size_t m = from_offsets[row] - prev_from_offset;
+            const size_t n = to_offsets[row] - prev_to_offset;
+            const std::vector<StringRef> & from_vec = extract_array(from_data, prev_from_offset, m);
+            const std::vector<StringRef> & to_vec = extract_array(to_data, prev_to_offset, n);
path: src/Functions/array/arrayLevenshtein.cpp
user: login: Michicosun
body: ```suggestion
            const std::vector<StringRef> from_vec = extract_array(from_data, prev_from_offset, m);
            const std::vector<StringRef> to_vec = extract_array(to_data, prev_to_offset, n);
```
created_at: 2025-03-27T15:13:36Z
author_association: MEMBER
diff_hunk: @@ -1,12 +1,112 @@
 #pragma once
 
 #include <base/types.h>
+#include <Common/PODArray.h>
+#include <Common/iota.h>
+#include <span>
+
+#include <numeric>
 
 namespace DB
 {
 
+namespace ErrorCodes
+{
+    extern const int SIZES_OF_ARRAYS_DONT_MATCH;
+}
+
 /// How many steps if we want to change lhs to rhs.
 /// Details in https://en.wikipedia.org/wiki/Levenshtein_distance
 size_t levenshteinDistance(const String & lhs, const String & rhs);
path: src/Common/levenshteinDistance.h
user: login: Michicosun
body: Let's remove the old variant and the cpp file, because `levenshteinDistance<char>` will work for strings.
created_at: 2025-03-27T15:23:05Z
author_association: MEMBER
diff_hunk: @@ -1,12 +1,112 @@
 #pragma once
 
 #include <base/types.h>
+#include <Common/PODArray.h>
+#include <Common/iota.h>
+#include <span>
+
+#include <numeric>
 
 namespace DB
 {
 
+namespace ErrorCodes
+{
+    extern const int SIZES_OF_ARRAYS_DONT_MATCH;
+}
+
 /// How many steps if we want to change lhs to rhs.
 /// Details in https://en.wikipedia.org/wiki/Levenshtein_distance
 size_t levenshteinDistance(const String & lhs, const String & rhs);
 
+/// The same function as DB::levenshteinDistance, but for a more generic approach
+template <class Underlying>
path: src/Common/levenshteinDistance.h
user: login: Michicosun
body: ```suggestion
template <class Symbol = char>
```



created_at: 2025-03-27T15:24:01Z
author_association: MEMBER
diff_hunk: @@ -1,12 +1,112 @@
 #pragma once
 
 #include <base/types.h>
+#include <Common/PODArray.h>
+#include <Common/iota.h>
+#include <span>
+
+#include <numeric>
 
 namespace DB
 {
 
+namespace ErrorCodes
+{
+    extern const int SIZES_OF_ARRAYS_DONT_MATCH;
+}
+
 /// How many steps if we want to change lhs to rhs.
 /// Details in https://en.wikipedia.org/wiki/Levenshtein_distance
 size_t levenshteinDistance(const String & lhs, const String & rhs);
 
+/// The same function as DB::levenshteinDistance, but for a more generic approach
+template <class Underlying>
path: src/Common/levenshteinDistance.h
user: login: Felixoid
body: I'll use `Element` here and below
created_at: 2025-03-27T15:49:16Z
author_association: MEMBER
diff_hunk: @@ -1,12 +1,112 @@
 #pragma once
 
 #include <base/types.h>
+#include <Common/PODArray.h>
+#include <Common/iota.h>
+#include <span>
+
+#include <numeric>
 
 namespace DB
 {
 
+namespace ErrorCodes
+{
+    extern const int SIZES_OF_ARRAYS_DONT_MATCH;
+}
+
 /// How many steps if we want to change lhs to rhs.
 /// Details in https://en.wikipedia.org/wiki/Levenshtein_distance
 size_t levenshteinDistance(const String & lhs, const String & rhs);
path: src/Common/levenshteinDistance.h
user: login: Felixoid
body: Unfortunately, it didn't work out https://s3.amazonaws.com/clickhouse-test-reports/json.html?PR=77187&sha=2eff9d005643d1c0570ffbde6388455782c1664c&name_0=PR&name_1=Fast+test&name_2=Tests

The old function is case insensitive, and I don't think it makes sense to make another template for it.

I'll bring it back and rename to `levenshteinDistanceCaseInsensitive`
created_at: 2025-03-28T11:57:35Z
author_association: MEMBER

PR comments:
user: login: clickhouse-gh[bot]
created_at: 2025-03-05T19:20:04Z
body: Workflow [[PR](https://s3.amazonaws.com/clickhouse-test-reports/json.html?PR=77187&sha=latest&name_0=PR)], commit [7a0b310e]
author_association: CONTRIBUTOR
user: login: Felixoid
created_at: 2025-03-26T08:20:16Z
body: I am out of ideas on reducing the boilerplate between levenshtein / weighted, especially between different weighted methods. There are a lot of repeating operations, but I don't see how to handle them.
author_association: MEMBER
user: login: Felixoid
created_at: 2025-03-27T11:24:05Z
body: [test_storage_rabbitmq/test_failed_connection.py::test_rabbitmq_restore_failed_connection_without_losses_2](https://github.com/ClickHouse/ClickHouse/issues/71049#issuecomment-2754975789) is broken.

Docs are in progress to fix in https://github.com/ClickHouse/ClickHouse/pull/78356

Markin `Mergeable Check` green manually
author_association: MEMBER
user: login: clickgapai
created_at: 2026-04-03T18:44:37Z
body: Hi @Felixoid @Michicosun — while reviewing this PR I found the following:

- **Bug (P0):** levenshteinDistanceWeighted fails to update prev on match, producing wrong distance for consecutive matching elements — https://github.com/ClickHouse/ClickHouse/issues/101725

Happy to discuss — close anything that's wrong or already addressed.
author_association: CONTRIBUTOR