Repository: TileDB-Inc/TileDB
Pull request #5506: fix: Overflow-detection correctness issues in double-delta and bit-width-reduction filters
Author: login: rroelke
Created: 2025-04-29T01:55:26Z | Merged: 2025-05-08T16:49:49Z

[SC-65154](https://app.shortcut.com/tiledb-inc/story/65154/overflow-buffer-on-read) surfaced an issue in a filter pipeline where data would be written, but not read back properly.  Investigating identified that this could happen in the event of arithmetic overflow.  Both the double delta and bit width reduction filters have errors in their overflow detection.

The issue reporter's filter pipeline fed double delta into bit width reduction.  Since double delta produces essentially binary blobs as output containing sign bits, this unsurprisingly produces very wide bit widths.  And apparently arithmetic overflows.  This caused the bit width reduction to write down some incorrect bits, which un-filtered into invalid input to the double delta filter.

The fix of course is to correct the overflow detection.  The C++ standard library does not provide routines to do this, so we must write our own and assert their correctness.

~~That brings in the more significant part of this pull request, which is the introduction of Rust into core library builds.  The Rust standard library *does* provide overflow detection routines.  We can write our own in C++ (which can be in-lined), and then write unit tests which compare their output against code written in Rust.~~

~~As such this is a light step towards using Rust inside of the core library - this pull request only introduces small functions in test code, which we hope will enable us to divorce the build configuration and CI details from the novelty of doing something interesting in Rust.~~

EDIT: the Rust bits are all reverted so we can merge the bug fix (which is awaited upstream and will be a good candidate for 2.28) from the Rust component (which is not desired for 2.28)

---
TYPE: BUG
DESC: Fix overflow detection errors in double delta and bit width reduction filters



Diff:
diff --git a/.github/workflows/unit-test-runs.yml b/.github/workflows/unit-test-runs.yml
index 66520b8ed6c..74ed2ef5490 100644
--- a/.github/workflows/unit-test-runs.yml
+++ b/.github/workflows/unit-test-runs.yml
@@ -1,6 +1,7 @@
 name: unit-test-standalone
 on:
   workflow_call:
+  workflow_dispatch:
 
 env:
   VCPKG_BINARY_SOURCES: 'clear;x-gha,readwrite'
diff --git a/test/CMakeLists.txt b/test/CMakeLists.txt
index 575928d198c..4ad14ab1d70 100644
--- a/test/CMakeLists.txt
+++ b/test/CMakeLists.txt
@@ -41,6 +41,7 @@ set(TILEDB_TEST_SUPPORT_SOURCES
   support/src/helpers-dimension.h
   support/src/vfs_helpers.cc
   support/src/serialization_wrappers.cc
+  support/rapidcheck/show_datatype.cc
   )
 
 set(TILEDB_UNIT_TEST_SOURCES
diff --git a/test/support/rapidcheck/datatype.h b/test/support/rapidcheck/datatype.h
new file mode 100644
index 00000000000..48065ef56a8
--- /dev/null
+++ b/test/support/rapidcheck/datatype.h
@@ -0,0 +1,109 @@
+/**
+ * @file test/support/rapidcheck/datatype.h
+ *
+ * @section LICENSE
+ *
+ * The MIT License
+ *
+ * @copyright Copyright (c) 2025 TileDB, Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to deal
+ * in the Software without restriction, including without limitation the rights
+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+ * copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+ * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+ * THE SOFTWARE.
+ *
+ * @section DESCRIPTION
+ *
+ * This file defines rapidcheck generators for datatypes.
+ */
+
+#ifndef TILEDB_RAPIDCHECK_DATATYPE_H
+#define TILEDB_RAPIDCHECK_DATATYPE_H
+
+namespace rc {
+using namespace tiledb::sm;
+
+template <>
+struct Arbitrary<Datatype> {
+  static Gen<Datatype> arbitrary() {
+    return rc::gen::element(
+        Datatype::INT32,
+        Datatype::INT64,
+        Datatype::FLOAT32,
+        Datatype::FLOAT64,
+        Datatype::CHAR,
+        Datatype::INT8,
+        Datatype::UINT8,
+        Datatype::INT16,
+        Datatype::UINT16,
+        Datatype::UINT32,
+        Datatype::UINT64,
+        Datatype::STRING_ASCII,
+        Datatype::STRING_UTF8,
+        Datatype::STRING_UTF16,
+        Datatype::STRING_UTF32,
+        Datatype::STRING_UCS2,
+        Datatype::STRING_UCS4,
+        Datatype::ANY,
+        Datatype::DATETIME_YEAR,
+        Datatype::DATETIME_MONTH,
+        Datatype::DATETIME_WEEK,
+        Datatype::DATETIME_DAY,
+        Datatype::DATETIME_HR,
+        Datatype::DATETIME_MIN,
+        Datatype::DATETIME_SEC,
+        Datatype::DATETIME_MS,
+        Datatype::DATETIME_US,
+        Datatype::DATETIME_NS,
+        Datatype::DATETIME_PS,
+        Datatype::DATETIME_FS,
+        Datatype::DATETIME_AS,
+        Datatype::TIME_HR,
+        Datatype::TIME_MIN,
+        Datatype::TIME_SEC,
+        Datatype::TIME_MS,
+        Datatype::TIME_US,
+        Datatype::TIME_NS,
+        Datatype::TIME_PS,
+        Datatype::TIME_FS,
+        Datatype::TIME_AS,
+        Datatype::BLOB,
+        Datatype::BOOL,
+        Datatype::GEOM_WKB,
+        Datatype::GEOM_WKT);
+  };
+};
+
+[[maybe_unused]] static Gen<std::vector<uint8_t>> make_input_bytes(
+    Datatype input_type) {
+  auto gen_elt = rc::gen::container<std::vector<uint8_t>>(
+      datatype_size(input_type), gen::arbitrary<uint8_t>());
+  auto gen_container =
+      gen::nonEmpty(gen::container<std::vector<std::vector<uint8_t>>>(gen_elt));
+  return gen::map(
+      gen_container, [input_type](std::vector<std::vector<uint8_t>> elts) {
+        std::vector<uint8_t> flat;
+        flat.reserve(elts.size() * datatype_size(input_type));
+        for (const std::vector<uint8_t>& elt : elts) {
+          flat.insert(flat.end(), elt.begin(), elt.end());
+        }
+        return flat;
+      });
+}
+
+}  // namespace rc
+
+#endif
diff --git a/test/support/rapidcheck/show.h b/test/support/rapidcheck/show.h
index de742caf512..1b561601d60 100644
--- a/test/support/rapidcheck/show.h
+++ b/test/support/rapidcheck/show.h
@@ -39,10 +39,12 @@
 
 #include <ostream>
 
+#include "tiledb/sm/enums/datatype.h"
+
 namespace tiledb {
 namespace sm {
 class ASTNode;
-}
+}  // namespace sm
 }  // namespace tiledb
 
 // forward declarations of `showValue` overloads
@@ -57,6 +59,13 @@ namespace rc::detail {
 template <stdx::is_optional_v T>
 void showValue(const T& value, std::ostream& os);
 
+/**
+ * Specializes `show` for Datatype.
+ *
+ * Requires adding `show_datatype.cc` to source list.
+ */
+void showValue(const tiledb::sm::Datatype& value, std::ostream& os);
+
 /**
  * Specializes `show` for query ASTNode
  */
diff --git a/test/support/rapidcheck/show_datatype.cc b/test/support/rapidcheck/show_datatype.cc
new file mode 100644
index 00000000000..04ea2bb2ad8
--- /dev/null
+++ b/test/support/rapidcheck/show_datatype.cc
@@ -0,0 +1,46 @@
+/**
+ * @file test/support/rapidcheck/show_datatype.cc
+ *
+ * @section LICENSE
+ *
+ * The MIT License
+ *
+ * @copyright Copyright (c) 2022-2025 TileDB, Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to deal
+ * in the Software without restriction, including without limitation the rights
+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+ * copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+ * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+ * THE SOFTWARE.
+ *
+ * @section DESCRIPTION
+ *
+ * This file provides forward declarations of `rc::detail::showValue`
+ * overloads, which seemingly must be included prior to the rapidcheck
+ * header files.
+ */
+
+#include <test/support/rapidcheck/show.h>
+
+#include "tiledb/sm/enums/query_condition_op.h"
+#include "tiledb/sm/query/ast/query_ast.h"
+
+namespace rc::detail {
+
+void showValue(const tiledb::sm::Datatype& value, std::ostream& os) {
+  os << tiledb::sm::datatype_str(value);
+}
+
+}  // namespace rc::detail
diff --git a/test/support/tdb_catch.h b/test/support/tdb_catch.h
index bdb0f1956f7..1423f0c746b 100644
--- a/test/support/tdb_catch.h
+++ b/test/support/tdb_catch.h
@@ -57,6 +57,19 @@
 #include <catch2/reporters/catch_reporter_event_listener.hpp>
 #include <catch2/reporters/catch_reporter_registrars.hpp>
 
+namespace Catch {
+template <typename T>
+struct StringMaker<std::optional<T>> {
+  static std::string convert(std::optional<T> const& value) {
+    if (value.has_value()) {
+      return "Some(" + StringMaker<T>::convert(value.value()) + ")";
+    } else {
+      return "None";
+    }
+  }
+};
+}  // namespace Catch
+
 /*
  * Clean up preprocessor definitions
  */
diff --git a/tiledb/common/arithmetic.h b/tiledb/common/arithmetic.h
new file mode 100644
index 00000000000..08ced92e703
--- /dev/null
+++ b/tiledb/common/arithmetic.h
@@ -0,0 +1,254 @@
+/**
+ * @file   arithmetic.h
+ *
+ * @section LICENSE
+ *
+ * The MIT License
+ *
+ * @copyright Copyright (c) 2025 TileDB, Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to deal
+ * in the Software without restriction, including without limitation the rights
+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+ * copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+ * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+ * THE SOFTWARE.
+ *
+ * @section DESCRIPTION
+ *
+ * This file defines common non-primitive arithmetic functions.
+ */
+#ifndef TILEDB_ARITHMETIC_H
+#define TILEDB_ARITHMETIC_H
+
+#include "tiledb/common/unreachable.h"
+
+#include <cstdint>
+#include <limits>
+#include <optional>
+
+namespace tiledb::common {
+
+/**
+ * Provides the member typedef `type` which is the 64-bit integral type
+ * with the same signed-ness as the integral type `T`,
+ * if `T` itself is narrower than 64 bits.
+ */
+template <typename T>
+struct extended_integral_type;
+
+template <>
+struct extended_integral_type<int8_t> {
+  using type = int64_t;
+};
+
+template <>
+struct extended_integral_type<int16_t> {
+  using type = int64_t;
+};
+
+template <>
+struct extended_integral_type<int32_t> {
+  using type = int64_t;
+};
+
+template <>
+struct extended_integral_type<uint8_t> {
+  using type = uint64_t;
+};
+
+template <>
+struct extended_integral_type<uint16_t> {
+  using type = uint64_t;
+};
+
+template <>
+struct extended_integral_type<uint32_t> {
+  using type = uint64_t;
+};
+
+template <typename T>
+using extended_integral_type_t = typename extended_integral_type<T>::type;
+
+/**
+ * Provides the member typedef `type` which is the 64-bit integral type
+ * with the same signed-ness as the integral type `T`.
+ */
+template <typename T>
+struct integral_64 {
+  using type = extended_integral_type_t<T>;
+};
+
+template <>
+struct integral_64<uint64_t> {
+  using type = uint64_t;
+};
+
+template <>
+struct integral_64<int64_t> {
+  using type = int64_t;
+};
+
+/**
+ * Provides functions for performing common arithmetic operations
+ * checking overflow.
+ */
+template <typename T>
+struct checked_arithmetic {
+  /**
+   * @return `Some(a + b)` if it can be represented as type `T`, `None`
+   * otherwise.
+   */
+  static std::optional<T> add(T a, T b) {
+    const extended_integral_type_t<T> a64 = a;
+    const extended_integral_type_t<T> b64 = b;
+    const extended_integral_type_t<T> c64 = a64 + b64;
+
+    if (std::numeric_limits<T>::lowest() <= c64 &&
+        c64 <= std::numeric_limits<T>::max()) {
+      return static_cast<T>(c64);
+    } else {
+      return std::nullopt;
+    }
+  }
+
+  /**
+   * @return `Some(a - b)` if it can be represented as type `T`, `None`
+   * otherwise.
+   */
+  static std::optional<T> sub(T a, T b) {
+    const extended_integral_type_t<T> a64 = a;
+    const extended_integral_type_t<T> b64 = b;
+    const extended_integral_type_t<T> c64 = a64 - b64;
+
+    if (std::numeric_limits<T>::lowest() <= c64 &&
+        c64 <= std::numeric_limits<T>::max()) {
+      return static_cast<T>(c64);
+    } else {
+      return std::nullopt;
+    }
+  }
+};
+
+template <>
+struct checked_arithmetic<uint64_t> {
+  static std::optional<uint64_t> add(uint64_t a, uint64_t b) {
+    if (a <= std::numeric_limits<uint64_t>::max() - b) {
+      return a + b;
+    } else {
+      return std::nullopt;
+    }
+  }
+
+  static std::optional<uint64_t> sub(uint64_t a, uint64_t b) {
+    if (a < b) {
+      return std::nullopt;
+    } else {
+      return a - b;
+    }
+  }
+
+  /**
+   * @return `a - b` if it can be represented as an `int64_t` without undefined
+   * behavior, `std::nullopt` otherwise
+   */
+  static std::optional<int64_t> sub_signed(uint64_t a, uint64_t b) {
+    if (a >= b) {
+      // since `b >= 0` due to unsigned-ness this will not underflow
+      // but we do have to check if it can be represented as `int64_t`
+      if (a - b > static_cast<uint64_t>(std::numeric_limits<int64_t>::max())) {
+        return std::nullopt;
+      } else {
+        return a - b;
+      }
+    }
+
+    // NB: a traditional and "obvious" way of detecting overflow
+    // is using the signed-ness of the subtraction result compared
+    // against the inputs.  This is actually undefined behavior
+    // and could also possibly be optimized out by a compiler
+    // which would make us sad.
+    //
+    // We will instead rely on `a - b == -(b - a)` which decays to the
+    // above case, and we can check for the only possible
+    // result which we don't have to negate.
+    const auto negated = sub_signed(b, a);
+    if (!negated.has_value()) {
+      // edge case: the delta is exactly the non-negatable twos complement value
+      if (b - a ==
+          static_cast<uint64_t>(std::numeric_limits<int64_t>::max()) + 1) {
+        return std::numeric_limits<int64_t>::min();
+      } else {
+        return std::nullopt;
+      }
+    } else {
+      return -negated.value();
+    }
+  }
+};
+
+template <>
+struct checked_arithmetic<int64_t> {
+  static std::optional<int64_t> add(int64_t a, int64_t b) {
+    if (b >= 0) {
+      if (a <= std::numeric_limits<int64_t>::max() - b) {
+        return a + b;
+      } else {
+        return std::nullopt;
+      }
+    } else {
+      if (std::numeric_limits<int64_t>::min() - b <= a) {
+        return a + b;
+      } else {
+        return std::nullopt;
+      }
+    }
+  }
+
+  static std::optional<int64_t> sub(int64_t a, int64_t b) {
+    if (a < b) {
+      // negate so we can apply logic with generality
+      const auto negated = sub(b, a);
+      if (!negated.has_value()) {
+        if (a - b == std::numeric_limits<int64_t>::min()) {
+          return std::numeric_limits<int64_t>::min();
+        } else {
+          return std::nullopt;
+        }
+      } else if (negated.value() == std::numeric_limits<int64_t>::min()) {
+        return std::nullopt;
+      } else {
+        return -negated.value();
+      }
+    } else {
+      // overflow condition `a - b > i64::MAX`
+      // is equivalent to `a > i64::MAX + b`
+      if (b >= 0) {
+        return a - b;
+      } else if (a > std::numeric_limits<int64_t>::max() + b) {
+        return std::nullopt;
+      } else {
+        return a - b;
+      }
+    }
+  }
+
+  static std::optional<int64_t> sub_signed(int64_t a, int64_t b) {
+    return sub(a, b);
+  }
+};
+
+}  // namespace tiledb::common
+
+#endif
diff --git a/tiledb/common/test/CMakeLists.txt b/tiledb/common/test/CMakeLists.txt
index 034cd452920..45a69b8c6fe 100644
--- a/tiledb/common/test/CMakeLists.txt
+++ b/tiledb/common/test/CMakeLists.txt
@@ -26,6 +26,11 @@
 
 include(unit_test)
 
+commence(unit_test arithmetic)
+    this_target_sources(main.cc unit_arithmetic.cc)
+    this_target_link_libraries(rapidcheck)
+conclude(unit_test)
+
 commence(unit_test experimental)
     this_target_object_libraries(baseline)
     this_target_sources(
diff --git a/tiledb/common/test/unit_arithmetic.cc b/tiledb/common/test/unit_arithmetic.cc
new file mode 100644
index 00000000000..2e46d979a93
--- /dev/null
+++ b/tiledb/common/test/unit_arithmetic.cc
@@ -0,0 +1,274 @@
+/**
+ * @file unit_arithmetic.cc
+ *
+ * @section LICENSE
+ *
+ * The MIT License
+ *
+ * @copyright Copyright (c) 2025 TileDB, Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to deal
+ * in the Software without restriction, including without limitation the rights
+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+ * copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+ * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+ * THE SOFTWARE.
+ *
+ * @section DESCRIPTION
+ *
+ * This file tests arithmetic functions defined in `arithmetic.h`.
+ */
+
+#include "tiledb/common/arithmetic.h"
+
+#include <test/support/assert_helpers.h>
+#include <test/support/tdb_catch.h>
+#include <test/support/tdb_rapidcheck.h>
+
+using namespace tiledb::common;
+
+TEST_CASE(
+    "Arithmetic checked_arithmetic uint32_t add", "[arithmetic][rapidcheck]") {
+  SECTION("Example") {
+    CHECK(checked_arithmetic<uint32_t>::add(0, 0) == 0);
+  }
+
+  SECTION("Boundary") {
+    const uint32_t max = std::numeric_limits<uint32_t>::max();
+    CHECK(checked_arithmetic<uint32_t>::add(0, max) == max);
+    CHECK(checked_arithmetic<uint32_t>::add(max, 0) == max);
+    CHECK(checked_arithmetic<uint32_t>::add(10, max - 10) == max);
+    CHECK(checked_arithmetic<uint32_t>::add(max - 10, 10) == max);
+  }
+
+  SECTION("Overflow") {
+    const uint32_t max = std::numeric_limits<uint32_t>::max();
+    CHECK(checked_arithmetic<uint32_t>::add(1, max) == std::nullopt);
+    CHECK(checked_arithmetic<uint32_t>::add(max, 1) == std::nullopt);
+    CHECK(checked_arithmetic<uint32_t>::add(10, max - 9) == std::nullopt);
+    CHECK(checked_arithmetic<uint32_t>::add(max - 9, 10) == std::nullopt);
+  }
+}
+
+TEST_CASE("Arithmetic checked_arithmetic int32_t add", "[arithmetic]") {
+  SECTION("Example") {
+    CHECK(checked_arithmetic<int32_t>::add(0, 0) == 0);
+    CHECK(checked_arithmetic<int32_t>::add(0, 1) == 1);
+    CHECK(checked_arithmetic<int32_t>::add(-1, 0) == -1);
+  }
+
+  const int32_t max = std::numeric_limits<int32_t>::max();
+  const int32_t min = std::numeric_limits<int32_t>::min();
+
+  SECTION("Boundary") {
+    CHECK(checked_arithmetic<int32_t>::add(max, min) == -1);
+    CHECK(checked_arithmetic<int32_t>::add(max, 0) == max);
+    CHECK(checked_arithmetic<int32_t>::add(0, max) == max);
+    CHECK(checked_arithmetic<int32_t>::add(min, 0) == min);
+    CHECK(checked_arithmetic<int32_t>::add(0, min) == min);
+    CHECK(checked_arithmetic<int32_t>::add(max - 10, 10) == max);
+    CHECK(checked_arithmetic<int32_t>::add(10, max - 10) == max);
+    CHECK(checked_arithmetic<int32_t>::add(min + 10, -10) == min);
+    CHECK(checked_arithmetic<int32_t>::add(-10, min + 10) == min);
+  }
+
+  SECTION("Overflow") {
+    CHECK(checked_arithmetic<int32_t>::add(max, 1) == std::nullopt);
+    CHECK(checked_arithmetic<int32_t>::add(1, max) == std::nullopt);
+    CHECK(checked_arithmetic<int32_t>::add(min, -1) == std::nullopt);
+    CHECK(checked_arithmetic<int32_t>::add(-1, min) == std::nullopt);
+    CHECK(checked_arithmetic<int32_t>::add(max - 9, 10) == std::nullopt);
+    CHECK(checked_arithmetic<int32_t>::add(10, max - 9) == std::nullopt);
+    CHECK(checked_arithmetic<int32_t>::add(min + 9, -10) == std::nullopt);
+    CHECK(checked_arithmetic<int32_t>::add(-10, min + 9) == std::nullopt);
+  }
+}
+
+TEST_CASE("Arithmetic checked_arithmetic uint64_t add", "[arithmetic]") {
+  SECTION("Example") {
+    CHECK(checked_arithmetic<uint64_t>::add(0, 0) == 0);
+  }
+
+  SECTION("Boundary") {
+    const uint64_t max = std::numeric_limits<uint64_t>::max();
+    CHECK(checked_arithmetic<uint64_t>::add(0, max) == max);
+    CHECK(checked_arithmetic<uint64_t>::add(max, 0) == max);
+    CHECK(checked_arithmetic<uint64_t>::add(10, max - 10) == max);
+    CHECK(checked_arithmetic<uint64_t>::add(max - 10, 10) == max);
+  }
+
+  SECTION("Overflow") {
+    const uint64_t max = std::numeric_limits<uint64_t>::max();
+    CHECK(checked_arithmetic<uint64_t>::add(1, max) == std::nullopt);
+    CHECK(checked_arithmetic<uint64_t>::add(max, 1) == std::nullopt);
+    CHECK(checked_arithmetic<uint64_t>::add(10, max - 9) == std::nullopt);
+    CHECK(checked_arithmetic<uint64_t>::add(max - 9, 10) == std::nullopt);
+  }
+}
+
+TEST_CASE("Arithmetic checked_arithmetic int64_t add", "[arithmetic]") {
+  SECTION("Example") {
+    CHECK(checked_arithmetic<int32_t>::add(0, 0) == 0);
+    CHECK(checked_arithmetic<int32_t>::add(0, 1) == 1);
+    CHECK(checked_arithmetic<int32_t>::add(-1, 0) == -1);
+  }
+
+  const int32_t max = std::numeric_limits<int32_t>::max();
+  const int32_t min = std::numeric_limits<int32_t>::min();
+
+  SECTION("Boundary") {
+    CHECK(checked_arithmetic<int32_t>::add(max, min) == -1);
+    CHECK(checked_arithmetic<int32_t>::add(max, 0) == max);
+    CHECK(checked_arithmetic<int32_t>::add(0, max) == max);
+    CHECK(checked_arithmetic<int32_t>::add(min, 0) == min);
+    CHECK(checked_arithmetic<int32_t>::add(0, min) == min);
+    CHECK(checked_arithmetic<int32_t>::add(max - 10, 10) == max);
+    CHECK(checked_arithmetic<int32_t>::add(10, max - 10) == max);
+    CHECK(checked_arithmetic<int32_t>::add(min + 10, -10) == min);
+    CHECK(checked_arithmetic<int32_t>::add(-10, min + 10) == min);
+  }
+
+  SECTION("Overflow") {
+    CHECK(checked_arithmetic<int32_t>::add(max, 1) == std::nullopt);
+    CHECK(checked_arithmetic<int32_t>::add(1, max) == std::nullopt);
+    CHECK(checked_arithmetic<int32_t>::add(min, -1) == std::nullopt);
+    CHECK(checked_arithmetic<int32_t>::add(-1, min) == std::nullopt);
+    CHECK(checked_arithmetic<int32_t>::add(max - 9, 10) == std::nullopt);
+    CHECK(checked_arithmetic<int32_t>::add(10, max - 9) == std::nullopt);
+    CHECK(checked_arithmetic<int32_t>::add(min + 9, -10) == std::nullopt);
+    CHECK(checked_arithmetic<int32_t>::add(-10, min + 9) == std::nullopt);
+  }
+}
+
+TEST_CASE("Arithmetic checked_arithmetic uint32_t sub", "[arithmetic]") {
+  rc::prop("a < b", [](uint32_t a) {
+    const uint32_t max = std::numeric_limits<uint32_t>::max();
+    RC_PRE(a != max);
+    const uint32_t b = *rc::gen::inRange<uint32_t>(a + 1, max);
+    const std::optional<uint32_t> c = checked_arithmetic<uint32_t>::sub(a, b);
+    RC_ASSERT(c == std::nullopt);
+  });
+
+  rc::prop("a >= b", [](int32_t a) {
+    const uint32_t b = *rc::gen::inRange<uint32_t>(0, a);
+    const std::optional<uint32_t> c = checked_arithmetic<uint32_t>::sub(a, b);
+    RC_ASSERT(c == a - b);
+  });
+}
+
+TEST_CASE("Arithmetic checked_arithmetic int32_t sub", "[arithmetic]") {
+  const int32_t max = std::numeric_limits<int32_t>::max();
+  const int32_t min = std::numeric_limits<int32_t>::min();
+
+  SECTION("Example") {
+    CHECK(checked_arithmetic<int32_t>::sub(max, 0) == max);
+    CHECK(checked_arithmetic<int32_t>::sub(0, max) == min + 1);
+    CHECK(checked_arithmetic<int32_t>::sub(0, min + 1) == max);
+  }
+
+  SECTION("Overflow") {
+    CHECK(checked_arithmetic<int32_t>::sub(max, -1) == std::nullopt);
+    CHECK(checked_arithmetic<int32_t>::sub(1, -max) == std::nullopt);
+    CHECK(checked_arithmetic<int32_t>::sub(min, 1) == std::nullopt);
+    CHECK(checked_arithmetic<int32_t>::sub(0, min) == std::nullopt);
+  }
+}
+
+TEST_CASE("Arithmetic checked_arithmetic uint64_t sub", "[arithmetic]") {
+  rc::prop("a < b", [](uint64_t a) {
+    const uint64_t max = std::numeric_limits<uint64_t>::max();
+    RC_PRE(a != max);
+    const uint64_t b = *rc::gen::inRange<uint64_t>(a + 1, max);
+    const std::optional<uint64_t> c = checked_arithmetic<uint64_t>::sub(a, b);
+    RC_ASSERT(c == std::nullopt);
+  });
+
+  rc::prop("a >= b", [](int32_t a) {
+    const uint64_t b = *rc::gen::inRange<uint64_t>(0, a);
+    const std::optional<uint64_t> c = checked_arithmetic<uint64_t>::sub(a, b);
+    RC_ASSERT(c == a - b);
+  });
+}
+
+TEST_CASE("Arithmetic checked_arithmetic uint64_t sub_signed", "[arithmetic]") {
+  SECTION("Example") {
+    CHECK(checked_arithmetic<uint64_t>::sub_signed(0, 0) == 0);
+    CHECK(checked_arithmetic<uint64_t>::sub_signed(0, 1) == -1);
+  }
+
+  SECTION("Boundary") {
+    CHECK(
+        checked_arithmetic<uint64_t>::sub_signed(0, 0x7FFFFFFFFFFFFFFF) ==
+        0x8000000000000001);
+    CHECK(
+        checked_arithmetic<uint64_t>::sub_signed(0, 0x8000000000000000) ==
+        0x8000000000000000);
+    CHECK(
+        checked_arithmetic<uint64_t>::sub_signed(
+            0xFFFFFFFFFFFFFFFF, 0x8000000000000000) == 0x7FFFFFFFFFFFFFFF);
+  }
+
+  SECTION("Overflow") {
+    CHECK(
+        checked_arithmetic<uint64_t>::sub_signed(0, 0x8000000000000001) ==
+        std::nullopt);
+    CHECK(
+        checked_arithmetic<uint64_t>::sub_signed(0xFFFFFFFFFFFFFFFF, 0) ==
+        std::nullopt);
+    CHECK(
+        checked_arithmetic<uint64_t>::sub_signed(
+            0xFFFFFFFFFFFFFFFF, 0x7FFFFFFFFFFFFFFE) == std::nullopt);
+    CHECK(
+        checked_arithmetic<uint64_t>::sub_signed(
+            0xFFFFFFFFFFFFFFFF, 0x7FFFFFFFFFFFFFFF) == std::nullopt);
+  }
+}
+
+TEST_CASE("Arithmetic checked_arithmetic int64_t sub", "[arithmetic]") {
+  SECTION("Example") {
+    CHECK(checked_arithmetic<int64_t>::sub(0, 0) == 0);
+    CHECK(checked_arithmetic<int64_t>::sub(0, 1) == -1);
+    CHECK(checked_arithmetic<int64_t>::sub(-1, 0) == -1);
+  }
+
+  SECTION("Boundary") {
+    CHECK(
+        checked_arithmetic<int64_t>::sub(0, 0x7FFFFFFFFFFFFFFF) ==
+        0x8000000000000001);
+    CHECK(
+        checked_arithmetic<int64_t>::sub(0, 0x8000000000000001) ==
+        0x7FFFFFFFFFFFFFFF);
+    CHECK(
+        checked_arithmetic<int64_t>::sub(-1, 0x7FFFFFFFFFFFFFFE) ==
+        0x8000000000000001);
+    CHECK(
+        checked_arithmetic<int64_t>::sub(-1, 0x7FFFFFFFFFFFFFFF) ==
+        0x8000000000000000);
+    CHECK(
+        checked_arithmetic<int64_t>::sub(-1, 0x8000000000000000) ==
+        0x7FFFFFFFFFFFFFFF);
+    CHECK(
+        checked_arithmetic<int64_t>::sub(0x7FFFFFFFFFFFFFFF, 0) ==
+        0x7FFFFFFFFFFFFFFF);
+  }
+
+  SECTION("Overflow") {
+    CHECK(
+        checked_arithmetic<int64_t>::sub(0, 0x8000000000000000) ==
+        std::nullopt);
+    CHECK(
+        checked_arithmetic<int64_t>::sub(0x7FFFFFFFFFFFFFFF, -1) ==
+        std::nullopt);
+  }
+}
diff --git a/tiledb/sm/compressors/dd_compressor.cc b/tiledb/sm/compressors/dd_compressor.cc
index e46b6c4657a..686280b992a 100644
--- a/tiledb/sm/compressors/dd_compressor.cc
+++ b/tiledb/sm/compressors/dd_compressor.cc
@@ -31,6 +31,7 @@
  */
 
 #include "tiledb/sm/compressors/dd_compressor.h"
+#include "tiledb/common/arithmetic.h"
 #include "tiledb/common/logger.h"
 #include "tiledb/sm/buffer/buffer.h"
 #include "tiledb/sm/enums/datatype.h"
@@ -64,7 +65,7 @@ void DoubleDelta::compress(
     case Datatype::BLOB:
     case Datatype::GEOM_WKB:
     case Datatype::GEOM_WKT:
-      return DoubleDelta::compress<std::byte>(input_buffer, output_buffer);
+      return DoubleDelta::compress<uint8_t>(input_buffer, output_buffer);
     case Datatype::INT8:
       return DoubleDelta::compress<int8_t>(input_buffer, output_buffer);
     case Datatype::BOOL:
@@ -83,7 +84,7 @@ void DoubleDelta::compress(
     case Datatype::UINT64:
       return DoubleDelta::compress<uint64_t>(input_buffer, output_buffer);
     case Datatype::CHAR:
-      return DoubleDelta::compress<char>(input_buffer, output_buffer);
+      return DoubleDelta::compress<int8_t>(input_buffer, output_buffer);
     case Datatype::DATETIME_YEAR:
     case Datatype::DATETIME_MONTH:
     case Datatype::DATETIME_WEEK:
@@ -134,7 +135,7 @@ void DoubleDelta::decompress(
     case Datatype::BLOB:
     case Datatype::GEOM_WKB:
     case Datatype::GEOM_WKT:
-      return DoubleDelta::decompress<std::byte>(input_buffer, output_buffer);
+      return DoubleDelta::decompress<uint8_t>(input_buffer, output_buffer);
     case Datatype::INT8:
       return DoubleDelta::decompress<int8_t>(input_buffer, output_buffer);
     case Datatype::BOOL:
@@ -153,7 +154,7 @@ void DoubleDelta::decompress(
     case Datatype::UINT64:
       return DoubleDelta::decompress<uint64_t>(input_buffer, output_buffer);
     case Datatype::CHAR:
-      return DoubleDelta::decompress<char>(input_buffer, output_buffer);
+      return DoubleDelta::decompress<int8_t>(input_buffer, output_buffer);
     case Datatype::DATETIME_YEAR:
     case Datatype::DATETIME_MONTH:
     case Datatype::DATETIME_WEEK:
@@ -215,8 +216,7 @@ void DoubleDelta::compress(ConstBuffer* input_buffer, Buffer* output_buffer) {
 
   // Calculate bitsize (ignoring the sign bit)
   auto in = (T*)input_buffer->data();
-  unsigned int bitsize;
-  compute_bitsize(in, num, &bitsize);
+  const uint32_t bitsize = compute_bitsize<T>(in, num);
   assert(bitsize <= std::numeric_limits<uint8_t>::max());
   auto bitsize_c = static_cast<uint8_t>(bitsize);
 
@@ -257,35 +257,53 @@ void DoubleDelta::compress(ConstBuffer* input_buffer, Buffer* output_buffer) {
 }
 
 template <class T>
-void DoubleDelta::compute_bitsize(T* in, uint64_t num, unsigned int* bitsize) {
+uint32_t DoubleDelta::compute_bitsize(const T* in, uint64_t num) {
   // Find maximum absolute double delta
-  *bitsize = 0;
-  int64_t max = 0;
   if (num <= 2) {
-    *bitsize = 0;
-    return;
-  }
-  int64_t prev_delta = int64_t(in[1]) - int64_t(in[0]);
-  char delta_out_of_bounds = 0;
-  for (uint64_t i = 2; i < num; ++i) {
-    int64_t cur_delta = int64_t(in[i]) - int64_t(in[i - 1]);
-    int64_t dd = cur_delta - prev_delta;
-    delta_out_of_bounds |= (char)(cur_delta < 0 && prev_delta > 0 && dd > 0);
-    delta_out_of_bounds |= (char)(cur_delta > 0 && prev_delta < 0 && dd < 0);
-    max = std::max(std::abs(dd), max);
-    prev_delta = cur_delta;
+    return 0;
   }
-  // Handle error
-  if (delta_out_of_bounds) {
-    throw DoubleDeltaException(
-        "Cannot compress with DoubleDelta; Some negative double delta is out "
-        "of bounds");
+
+  using DeltaType = common::integral_64<T>::type;
+
+  uint64_t max_delta = 0;
+  std::optional<int64_t> prev_delta;
+  for (uint64_t i = 1; i < num; ++i) {
+    const DeltaType cur = (DeltaType)(in[i]);
+    const DeltaType prev = (DeltaType)(in[i - 1]);
+
+    const std::optional<int64_t> cur_delta =
+        common::checked_arithmetic<DeltaType>::sub_signed(cur, prev);
+    if (!cur_delta.has_value()) {
+      throw DoubleDeltaException(
+          "Cannot compress with DoubleDelta: delta exceeds range of int64_t");
+    }
+
+    if (prev_delta.has_value()) {
+      const std::optional<int64_t> dd =
+          common::checked_arithmetic<int64_t>::sub(
+              cur_delta.value(), prev_delta.value());
+      if (!dd.has_value()) {
+        throw DoubleDeltaException(
+            "Cannot compress with DoubleDelta; Some negative double delta is "
+            "out of bounds");
+      }
+      max_delta =
+          std::max(static_cast<uint64_t>(std::abs(dd.value())), max_delta);
+    } else {
+      max_delta = std::abs(cur_delta.value());
+    }
+
+    prev_delta = cur_delta.value();
   }
+
   // Calculate bitsize of the maximum absolute double delta
+  uint32_t bitsize = 0;
   do {
-    ++(*bitsize);
-    max >>= 1;
-  } while (max);
+    ++bitsize;
+    max_delta >>= 1;
+  } while (max_delta);
+
+  return bitsize;
 }
 
 template <class T>
@@ -297,7 +315,7 @@ void DoubleDelta::decompress(
   uint64_t value_size = sizeof(T);
   throw_if_not_ok(input_buffer->read(&bitsize_c, sizeof(uint8_t)));
   throw_if_not_ok(input_buffer->read(&num, sizeof(uint64_t)));
-  auto bitsize = static_cast<unsigned int>(bitsize_c);
+  auto bitsize = static_cast<uint32_t>(bitsize_c);
   auto out = (T*)output_buffer->cur_data();
 
   // Trivial case - no compression
@@ -428,8 +446,6 @@ void DoubleDelta::write_double_delta(
 
 // Explicit template instantiations
 
-template void DoubleDelta::compress<char>(
-    ConstBuffer* input_buffer, Buffer* output_buffer);
 template void DoubleDelta::compress<int8_t>(
     ConstBuffer* input_buffer, Buffer* output_buffer);
 template void DoubleDelta::compress<uint8_t>(
diff --git a/tiledb/sm/compressors/dd_compressor.h b/tiledb/sm/compressors/dd_compressor.h
index ca8ade9b9b7..18b28ab3120 100644
--- a/tiledb/sm/compressors/dd_compressor.h
+++ b/tiledb/sm/compressors/dd_compressor.h
@@ -134,10 +134,10 @@ class DoubleDelta {
    * @tparam The datatype of the values.
    * @param in The input buffer.
    * @param num The number of values in the buffer.
-   * @param bitsize The bitsize of the double deltas to be retrieved.
+   * @return The bitsize of the double deltas to be retrieved.
    */
   template <class T>
-  static void compute_bitsize(T* in, uint64_t num, unsigned int* bitsize);
+  static uint32_t compute_bitsize(const T* in, uint64_t num);
 
   /**
    * Decompression function.
diff --git a/tiledb/sm/filter/bit_width_reduction_filter.cc b/tiledb/sm/filter/bit_width_reduction_filter.cc
index 8699c7698db..b2ceb60c922 100644
--- a/tiledb/sm/filter/bit_width_reduction_filter.cc
+++ b/tiledb/sm/filter/bit_width_reduction_filter.cc
@@ -31,6 +31,7 @@
  */
 
 #include "tiledb/sm/filter/bit_width_reduction_filter.h"
+#include "tiledb/common/arithmetic.h"
 #include "tiledb/common/logger.h"
 #include "tiledb/sm/enums/datatype.h"
 #include "tiledb/sm/enums/filter_option.h"
@@ -416,13 +417,19 @@ uint8_t BitWidthReductionFilter::compute_bits_required(
   buffer->set_offset(orig_offset);
 
   // Check for overflow
-  T range = window_max - window_min;
-  if (range == std::numeric_limits<T>::max())
+  std::optional<T> range = checked_arithmetic<T>::sub(window_max, window_min);
+  if (!range.has_value()) {
     return sizeof(T) * 8;
+  }
+
+  std::optional<T> range_offset = checked_arithmetic<T>::add(range.value(), 1);
+  if (!range_offset.has_value()) {
+    return sizeof(T) * 8;
+  }
 
   // Compute the number of bits required to store the max (normalized) window
   // value, rounding to the nearest C integer type width.
-  uint8_t bits = bits_required(range + 1);
+  uint8_t bits = bits_required(range_offset.value());
   if (bits <= 8)
     bits = 8;
   else if (bits <= 16)
diff --git a/tiledb/sm/filter/test/CMakeLists.txt b/tiledb/sm/filter/test/CMakeLists.txt
index 3bd3c12392c..786dc4afc51 100644
--- a/tiledb/sm/filter/test/CMakeLists.txt
+++ b/tiledb/sm/filter/test/CMakeLists.txt
@@ -51,12 +51,16 @@ commence(unit_test run_filter_pipeline)
         unit_bit_width_reduction_pipeline.cc
         unit_bitshuffle_pipeline.cc
         unit_byteshuffle_pipeline.cc
+        unit_double_delta_pipeline.cc
         unit_encryption_pipeline.cc
         unit_positive_delta_pipeline.cc
         unit_run_filter_pipeline.cc
         unit_webp_pipeline.cc
         unit_xor_pipeline.cc
+        unit_65154.cc
+        ${CMAKE_SOURCE_DIR}/test/support/rapidcheck/show_datatype.cc
     )
+    this_target_link_libraries(rapidcheck)
 conclude(unit_test)
 
 commence(unit_test float_scale_input_val)
diff --git a/tiledb/sm/filter/test/filter_test_support.cc b/tiledb/sm/filter/test/filter_test_support.cc
index 907940fd013..4a44596a3b5 100644
--- a/tiledb/sm/filter/test/filter_test_support.cc
+++ b/tiledb/sm/filter/test/filter_test_support.cc
@@ -114,7 +114,7 @@ void check_run_pipeline_roundtrip(
     ThreadPool& tp,
     shared_ptr<WriterTile> tile,
     std::optional<shared_ptr<WriterTile>>& offsets_tile,
-    FilterPipeline& pipeline,
+    const FilterPipeline& pipeline,
     const TileDataGenerator* test_data,
     shared_ptr<MemoryTracker> memory_tracker) {
   // Run the pipeline forward.
@@ -132,17 +132,17 @@ void check_run_pipeline_roundtrip(
       tile->filtered_buffer(), memory_tracker);
   ChunkData chunk_data;
   unfiltered_tile.load_chunk_data(chunk_data);
-  CHECK(pipeline
-            .run_reverse(
-                &dummy_stats,
-                &unfiltered_tile,
-                nullptr,
-                chunk_data,
-                0,
-                chunk_data.filtered_chunks_.size(),
-                tp.concurrency_level(),
-                config)
-            .ok());
+  REQUIRE(pipeline
+              .run_reverse(
+                  &dummy_stats,
+                  &unfiltered_tile,
+                  nullptr,
+                  chunk_data,
+                  0,
+                  chunk_data.filtered_chunks_.size(),
+                  tp.concurrency_level(),
+                  config)
+              .ok());
 
   // Check the original data is reverted.
   test_data->check_tile_data(unfiltered_tile);
diff --git a/tiledb/sm/filter/test/filter_test_support.h b/tiledb/sm/filter/test/filter_test_support.h
index e1a93081d95..5a4ed2abf6f 100644
--- a/tiledb/sm/filter/test/filter_test_support.h
+++ b/tiledb/sm/filter/test/filter_test_support.h
@@ -127,7 +127,7 @@ void check_run_pipeline_roundtrip(
     ThreadPool& tp,
     shared_ptr<WriterTile> tile,
     std::optional<shared_ptr<WriterTile>>& offsets_tile,
-    FilterPipeline& pipeline,
+    const FilterPipeline& pipeline,
     const TileDataGenerator* test_data,
     shared_ptr<MemoryTracker> memory_tracker);
 
diff --git a/tiledb/sm/filter/test/tile_data_generator.h b/tiledb/sm/filter/test/tile_data_generator.h
index 284b868c9f4..7be5820b214 100644
--- a/tiledb/sm/filter/test/tile_data_generator.h
+++ b/tiledb/sm/filter/test/tile_data_generator.h
@@ -33,6 +33,7 @@
 #ifndef TILEDB_INPUT_TILE_TEST_DATA_H
 #define TILEDB_INPUT_TILE_TEST_DATA_H
 
+#include <test/support/assert_helpers.h>
 #include <test/support/tdb_catch.h>
 #include <algorithm>
 #include <numeric>
@@ -106,6 +107,74 @@ class TileDataGenerator {
   virtual uint64_t original_tile_size() const = 0;
 };
 
+/**
+ * Tile data generator which simply provides and checks data
+ * stored in a `std::vector<uint8_t>`.
+ */
+template <typename Asserter>
+class VecDataGenerator : public TileDataGenerator {
+  Datatype datatype_;
+  std::span<const uint8_t> bytes_;
+
+  static shared_ptr<WriterTile> to_writer_tile(
+      shared_ptr<MemoryTracker> tracker,
+      Datatype input_type,
+      std::span<const uint8_t> bytes) {
+    auto wt = make_shared<WriterTile>(
+        HERE(),
+        constants::format_version,
+        input_type,
+        datatype_size(input_type),  // TODO: fix for cell val num
+        bytes.size(),
+        tracker);
+    wt->write(&bytes[0], 0, bytes.size());
+    return wt;
+  }
+
+ public:
+  VecDataGenerator(Datatype dt, std::span<const uint8_t> bytes)
+      : datatype_(dt)
+      , bytes_(bytes) {
+  }
+
+  uint64_t cell_size() const override {
+    return datatype_size(datatype_);
+  }
+
+  void check_tile_data(const Tile& tile) const override {
+    ASSERTER(tile.size() == bytes_.size());
+
+    // compare inversion in chunks (so that there's a bit more context on each
+    // line in the event of a failure)
+    constexpr uint64_t chunk_size = 128;
+    for (uint64_t i = 0; i < bytes_.size(); i += chunk_size) {
+      const uint64_t offset = i;
+      const uint64_t n = std::min(chunk_size, bytes_.size() - offset);
+
+      std::vector<uint8_t> chunk_in(
+          bytes_.begin() + offset, bytes_.begin() + offset + n);
+      std::vector<uint8_t> chunk_out(n);
+      tile.read(&chunk_out[0], offset, n);
+
+      ASSERTER(chunk_in == chunk_out);
+    }
+  }
+
+  Datatype datatype() const override {
+    return datatype_;
+  }
+
+  std::tuple<shared_ptr<WriterTile>, std::optional<shared_ptr<WriterTile>>>
+  create_writer_tiles(shared_ptr<MemoryTracker> memory_tracker) const override {
+    return std::make_pair(
+        to_writer_tile(memory_tracker, datatype(), bytes_), std::nullopt);
+  }
+
+  uint64_t original_tile_size() const override {
+    return bytes_.size();
+  }
+};
+
 /**
  * This is a simple tile data generator that simply contains incremental values
  * to store int he tile.
diff --git a/tiledb/sm/filter/test/unit_65154.cc b/tiledb/sm/filter/test/unit_65154.cc
new file mode 100644
index 00000000000..01307da9d17
--- /dev/null
+++ b/tiledb/sm/filter/test/unit_65154.cc
@@ -0,0 +1,112 @@
+/**
+ * @file unit_65154.cc
+ *
+ * @section LICENSE
+ *
+ * The MIT License
+ *
+ * @copyright Copyright (c) 2025 TileDB, Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to deal
+ * in the Software without restriction, including without limitation the rights
+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+ * copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+ * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+ * THE SOFTWARE.
+ *
+ * @section DESCRIPTION
+ *
+ * This set of unit tests checks running the filter pipeline with the
+ * filter pipeline used in SC-65154:
+ * `[DoubleDeltaFilter(reinterp_dtype=None),
+ * BitWidthReductionFilter(window=256), ZstdFilter(level=9)]`.
+ */
+
+#include <test/support/assert_helpers.h>
+#include <test/support/tdb_catch.h>
+#include <test/support/tdb_rapidcheck.h>
+
+#include "filter_test_support.h"
+#include "test/support/rapidcheck/datatype.h"
+#include "test/support/src/mem_helpers.h"
+#include "tiledb/sm/compressors/dd_compressor.h"
+#include "tiledb/sm/filter/bit_width_reduction_filter.h"
+#include "tiledb/sm/filter/compression_filter.h"
+#include "tiledb/sm/tile/tile.h"
+
+using namespace tiledb::common;
+using namespace tiledb::sm;
+
+TEST_CASE("Filter: Round trip SC-65154", "[filter][rapidcheck]") {
+  tiledb::sm::Config config;
+
+  ThreadPool thread_pool(4);
+  auto tracker = tiledb::test::create_test_memory_tracker();
+
+  auto doit = [&]<typename Asserter>(
+                  Datatype input_type, const std::vector<uint8_t>& data) {
+    VecDataGenerator<Asserter> tile_gen(input_type, data);
+
+    auto [input_tile, offsets_tile] = tile_gen.create_writer_tiles(tracker);
+
+    FilterPipeline pipeline;
+    pipeline.add_filter(CompressionFilter(
+        Compressor::DOUBLE_DELTA, 0, input_type, Datatype::ANY));
+    pipeline.add_filter(BitWidthReductionFilter(input_type));
+    pipeline.add_filter(CompressionFilter(Compressor::ZSTD, 9, input_type));
+
+    check_run_pipeline_roundtrip(
+        config,
+        thread_pool,
+        input_tile,
+        offsets_tile,
+        pipeline,
+        &tile_gen,
+        tracker);
+  };
+
+  SECTION("Shrinking") {
+    const std::vector<uint8_t> bytes = {
+        0,
+        0,
+        128,
+        64,
+        128,
+        255,
+        127,
+        0,
+        0,
+        0,
+        0,
+        0,
+    };
+
+    doit.operator()<tiledb::test::AsserterCatch>(Datatype::INT32, bytes);
+  }
+
+  SECTION("Rapidcheck int32") {
+    rc::prop("Filter: Round trip SC-65154 int32", [doit]() {
+      const auto bytes = *rc::make_input_bytes(Datatype::INT32);
+      doit.operator()<tiledb::test::AsserterRapidcheck>(Datatype::INT32, bytes);
+    });
+  }
+
+  SECTION("Rapidcheck uint32") {
+    rc::prop("Filter: Round trip SC-65154 uint32", [doit]() {
+      const auto bytes = *rc::make_input_bytes(Datatype::UINT32);
+      doit.operator()<tiledb::test::AsserterRapidcheck>(
+          Datatype::UINT32, bytes);
+    });
+  }
+}
diff --git a/tiledb/sm/filter/test/unit_bit_width_reduction_pipeline.cc b/tiledb/sm/filter/test/unit_bit_width_reduction_pipeline.cc
index 5dd2d9d93f6..4f2cc8e124e 100644
--- a/tiledb/sm/filter/test/unit_bit_width_reduction_pipeline.cc
+++ b/tiledb/sm/filter/test/unit_bit_width_reduction_pipeline.cc
@@ -32,12 +32,20 @@
  *
  */
 
+#include "tiledb/sm/filter/bit_width_reduction_filter.h"
+#include "tiledb/sm/filter/test/filter_test_support.h"
+#include "tiledb/sm/filter/test/tile_data_generator.h"
+#include "tiledb/sm/tile/tile.h"
+
+#include "test/support/rapidcheck/datatype.h"
+
+#include <test/support/assert_helpers.h>
 #include <test/support/src/mem_helpers.h>
 #include <test/support/tdb_catch.h>
+#include <test/support/tdb_rapidcheck.h>
 #include <random>
-#include "../bit_width_reduction_filter.h"
-#include "filter_test_support.h"
 
+using namespace tiledb::common;
 using namespace tiledb::sm;
 
 TEST_CASE("Filter: Test bit width reduction", "[filter][bit-width-reduction]") {
@@ -484,3 +492,50 @@ TEST_CASE(
 
   WhiteboxWriterTile::set_max_tile_chunk_size(constants::max_tile_chunk_size);
 }
+
+TEST_CASE("Filter: Round trip BitWidthReduction", "[filter][rapidcheck]") {
+  tiledb::sm::Config config;
+
+  ThreadPool thread_pool(4);
+  auto tracker = tiledb::test::create_test_memory_tracker();
+
+  auto doit = [&]<typename Asserter>(
+                  Datatype input_type, const std::vector<uint8_t>& data) {
+    VecDataGenerator<Asserter> tile_gen(input_type, data);
+
+    auto [input_tile, offsets_tile] = tile_gen.create_writer_tiles(tracker);
+
+    FilterPipeline pipeline;
+    pipeline.add_filter(BitWidthReductionFilter(input_type));
+
+    check_run_pipeline_roundtrip(
+        config,
+        thread_pool,
+        input_tile,
+        offsets_tile,
+        pipeline,
+        &tile_gen,
+        tracker);
+  };
+
+  SECTION("Example") {
+    const std::vector<uint8_t> data = {0, 0, 0, 0, 0, 0, 0, 1};
+    doit.operator()<tiledb::test::AsserterCatch>(Datatype::UINT64, data);
+  }
+
+  SECTION("Shrinking", "Examples found by rapidcheck") {
+    SECTION("Overflow") {
+      const std::vector<uint8_t> bytes = {0, 128, 127, 127};
+      doit.operator()<tiledb::test::AsserterCatch>(Datatype::INT16, bytes);
+    }
+  }
+
+  SECTION("Rapidcheck") {
+    rc::prop("Filter: Round trip BitWidthReduction", [doit]() {
+      const auto datatype = *rc::gen::arbitrary<Datatype>();
+      const auto bytes = *rc::make_input_bytes(datatype);
+
+      doit.operator()<tiledb::test::AsserterRapidcheck>(datatype, bytes);
+    });
+  }
+}
diff --git a/tiledb/sm/filter/test/unit_double_delta_pipeline.cc b/tiledb/sm/filter/test/unit_double_delta_pipeline.cc
new file mode 100644
index 00000000000..4ed1f110fd2
--- /dev/null
+++ b/tiledb/sm/filter/test/unit_double_delta_pipeline.cc
@@ -0,0 +1,272 @@
+/**
+ * @file unit_double_delta_pipeline.cc
+ *
+ * @section LICENSE
+ *
+ * The MIT License
+ *
+ * @copyright Copyright (c) 2025 TileDB, Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to deal
+ * in the Software without restriction, including without limitation the rights
+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+ * copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+ * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+ * THE SOFTWARE.
+ *
+ * @section DESCRIPTION
+ *
+ * This set of unit tests checks running the filter pipeline with the double
+ * delta filter.
+ */
+
+#include <test/support/assert_helpers.h>
+#include <test/support/tdb_catch.h>
+#include <test/support/tdb_rapidcheck.h>
+
+#include "filter_test_support.h"
+#include "test/support/rapidcheck/datatype.h"
+#include "test/support/src/mem_helpers.h"
+#include "tile_data_generator.h"
+#include "tiledb/common/arithmetic.h"
+#include "tiledb/sm/compressors/dd_compressor.h"
+#include "tiledb/sm/filter/compression_filter.h"
+#include "tiledb/sm/tile/tile.h"
+#include "tiledb/type/datatype_queries.h"
+
+using namespace tiledb::common;
+using namespace tiledb::sm;
+
+namespace rc {
+static Gen<Datatype> make_reinterpret_datatype(Datatype input_type) {
+  auto gen_reinterpret_datatype = gen::suchThat(
+      gen::arbitrary<Datatype>(), [input_type](auto reinterpret_type) {
+        // see filter_buffer.cc
+        switch (reinterpret_type) {
+          case Datatype::FLOAT32:
+          case Datatype::FLOAT64:
+          case Datatype::CHAR:
+          case Datatype::STRING_ASCII:
+          case Datatype::STRING_UTF8:
+          case Datatype::STRING_UTF16:
+          case Datatype::STRING_UTF32:
+          case Datatype::STRING_UCS2:
+          case Datatype::STRING_UCS4:
+            return false;
+          default:
+            return datatype_size(input_type) %
+                       datatype_size(reinterpret_type) ==
+                   0;
+        }
+      });
+  if (datatype_is_real(input_type)) {
+    return gen::suchThat(gen_reinterpret_datatype, [](auto reinterpret_type) {
+      return reinterpret_type != Datatype::ANY;
+    });
+  } else {
+    return gen_reinterpret_datatype;
+  }
+}
+
+}  // namespace rc
+
+/**
+ * @return true if we should expect `bytes` to overflow when interpreted
+ * using `filter_type`
+ */
+static bool expect_overflow(
+    Datatype filter_type, std::span<const uint8_t> data) {
+  if (datatype_size(filter_type) % sizeof(uint64_t) != 0) {
+    return false;
+  }
+
+  std::vector<int64_t> deltae;
+  if (tiledb::type::has_signed_value_type(filter_type)) {
+    std::span<const int64_t> values(
+        reinterpret_cast<const int64_t*>(&data[0]),
+        data.size() / sizeof(int64_t));
+    if (values.size() <= 2) {
+      return false;
+    }
+    for (uint64_t i = 1; i < values.size(); i++) {
+      std::optional<int64_t> delta =
+          checked_arithmetic<int64_t>::sub(values[i], values[i - 1]);
+      if (delta.has_value()) {
+        deltae.push_back(delta.value());
+      } else {
+        return true;
+      }
+    }
+  } else {
+    std::span<const uint64_t> values(
+        reinterpret_cast<const uint64_t*>(&data[0]),
+        data.size() / sizeof(int64_t));
+    if (values.size() <= 2) {
+      return false;
+    }
+    for (uint64_t i = 1; i < values.size(); i++) {
+      std::optional<int64_t> delta =
+          checked_arithmetic<int64_t>::sub_signed(values[i], values[i - 1]);
+      if (delta.has_value()) {
+        deltae.push_back(delta.value());
+      } else {
+        return true;
+      }
+    }
+  }
+  for (uint64_t i = 1; i < deltae.size(); i++) {
+    if (!checked_arithmetic<int64_t>::sub(deltae[i], deltae[i - 1])
+             .has_value()) {
+      return true;
+    }
+  }
+  return false;
+}
+
+TEST_CASE("Filter: Round trip Compressor DoubleDelta", "[filter][rapidcheck]") {
+  tiledb::sm::Config config;
+
+  ThreadPool thread_pool(4);
+  auto tracker = tiledb::test::create_test_memory_tracker();
+
+  auto doit = [&]<typename Asserter>(
+                  Datatype input_type,
+                  Datatype reinterpret_datatype,
+                  const std::vector<uint8_t>& data) {
+    VecDataGenerator<Asserter> tile_gen(input_type, data);
+
+    auto [input_tile, offsets_tile] = tile_gen.create_writer_tiles(tracker);
+
+    FilterPipeline pipeline;
+    pipeline.add_filter(CompressionFilter(
+        Compressor::DOUBLE_DELTA, 0, input_type, reinterpret_datatype));
+
+    bool overflowed = false;
+    try {
+      check_run_pipeline_roundtrip(
+          config,
+          thread_pool,
+          input_tile,
+          offsets_tile,
+          pipeline,
+          &tile_gen,
+          tracker);
+    } catch (const StatusException& e) {
+      const std::string what(e.what());
+      if (what.find("Cannot compress with DoubleDelta: delta exceeds range "
+                    "of int64_t") != std::string::npos) {
+        overflowed = true;
+      } else if (
+          what.find("Some negative double delta is out of bounds") !=
+          std::string::npos) {
+        overflowed = true;
+      } else {
+        throw;
+      }
+    }
+    const Datatype interpret_type =
+        (reinterpret_datatype == Datatype::ANY ? input_type :
+                                                 reinterpret_datatype);
+    ASSERTER(overflowed == expect_overflow(interpret_type, data));
+  };
+
+  SECTION("Example") {
+    const std::vector<uint8_t> data = {0, 0, 0, 0, 0, 0, 0, 1};
+    doit.operator()<tiledb::test::AsserterCatch>(
+        Datatype::UINT64, Datatype::UINT64, data);
+  }
+
+  SECTION("Shrinking 1") {
+    // overflow scenario
+    const std::vector<uint8_t> data = {0, 0, 0, 0,  0, 0, 0, 0, 0, 0, 0, 0,
+                                       0, 0, 0, 64, 0, 0, 0, 0, 0, 0, 0, 0};
+    doit.operator()<tiledb::test::AsserterCatch>(
+        Datatype::UINT64, Datatype::UINT64, data);
+  }
+  SECTION("Shrinking 2") {
+    const std::vector<uint8_t> data = {1, 0, 1};
+    doit.operator()<tiledb::test::AsserterCatch>(
+        Datatype::UINT8, Datatype::ANY, data);
+  }
+  SECTION("Shrinking 3") {
+    const std::vector<uint8_t> data = {0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+                                       0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 128};
+    doit.operator()<tiledb::test::AsserterCatch>(
+        Datatype::INT64, Datatype::ANY, data);
+  }
+  SECTION("Shrinking 4") {
+    const std::vector<uint8_t> data = {0, 0, 0, 0, 0, 0, 0, 35, 0, 0, 0, 0,
+                                       0, 0, 0, 0, 0, 0, 0, 0,  0, 0, 0, 93};
+    doit.operator()<tiledb::test::AsserterCatch>(
+        Datatype::INT64, Datatype::INT64, data);
+  }
+  SECTION("Shrinking 5") {
+    const std::vector<uint8_t> data = {
+        2, 0, 2, 1, 2, 3, 0, 2, 1, 3, 1, 0, 3, 0, 1, 2};
+    doit.operator()<tiledb::test::AsserterCatch>(
+        Datatype::DATETIME_DAY, Datatype::UINT64, data);
+  }
+  SECTION("Shrinking 6") {
+    const std::vector<uint8_t> data = {0, 1, 0, 0, 1, 1, 1, 0};
+    doit.operator()<tiledb::test::AsserterCatch>(
+        Datatype::DATETIME_MONTH, Datatype::UINT64, data);
+  }
+  SECTION("Shrinking 7") {
+    const std::vector<uint8_t> data = {
+        0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 128};
+    doit.operator()<tiledb::test::AsserterCatch>(
+        Datatype::DATETIME_MONTH, Datatype::UINT64, data);
+  }
+
+  SECTION("Rapidcheck all") {
+    rc::prop("Filter: Round trip Compressor Double Delta", [doit]() {
+      const auto datatype = *rc::gen::arbitrary<Datatype>();
+      const auto reinterpret = *rc::make_reinterpret_datatype(datatype);
+      const auto bytes = *rc::make_input_bytes(datatype);
+
+      doit.operator()<tiledb::test::AsserterRapidcheck>(
+          datatype, reinterpret, bytes);
+    });
+  }
+
+  SECTION("Rapidcheck non-overflowing") {
+    rc::prop(
+        "Filter: Round Trip Compressor Double Delta non-overflowing", [doit]() {
+          const auto [datatype, reinterpret] = *rc::gen::suchThat(
+              rc::gen::mapcat(
+                  rc::gen::arbitrary<Datatype>(),
+                  [](Datatype input) {
+                    return rc::gen::pair(
+                        rc::gen::just(input),
+                        rc::make_reinterpret_datatype(input));
+                  }),
+              [](std::pair<Datatype, Datatype> types) {
+                if (datatype_size(types.first) < sizeof(int64_t)) {
+                  return true;
+                } else if (
+                    types.second != Datatype::ANY &&
+                    datatype_size(types.second) < sizeof(int64_t)) {
+                  return true;
+                } else {
+                  return false;
+                }
+              });
+
+          const auto bytes = *rc::make_input_bytes(datatype);
+
+          doit.operator()<tiledb::test::AsserterRapidcheck>(
+              datatype, reinterpret, bytes);
+        });
+  }
+}
diff --git a/tiledb/sm/filter/test/unit_run_filter_pipeline.cc b/tiledb/sm/filter/test/unit_run_filter_pipeline.cc
index b543a3b08a9..7d3abb439d2 100644
--- a/tiledb/sm/filter/test/unit_run_filter_pipeline.cc
+++ b/tiledb/sm/filter/test/unit_run_filter_pipeline.cc
@@ -844,41 +844,15 @@ TEST_CASE("Filter: Test random pipeline", "[filter][random]") {
       // Pos-delta would (correctly) return error after e.g. compression.
       []() { return tdb_new(PositiveDeltaFilter, Datatype::UINT64); }};
 
-  // Create tile data generator.
-  IncrementTileDataGenerator<uint64_t, Datatype::UINT64> tile_data_generator(
-      100);
-  for (int i = 0; i < 100; i++) {
+  auto doit = [&](const FilterPipeline& pipeline) {
+    // Create tile data generator.
+    IncrementTileDataGenerator<uint64_t, Datatype::UINT64> tile_data_generator(
+        100);
+
     // Create fresh input tiles.
     auto&& [tile, offsets_tile] =
         tile_data_generator.create_writer_tiles(tracker);
 
-    // Construct a random pipeline
-    FilterPipeline pipeline;
-    const unsigned max_num_filters = 6;
-    std::random_device rd;
-    auto pipeline_seed = rd();
-    std::mt19937 gen(pipeline_seed);
-    std::uniform_int_distribution<> rng_num_filters(0, max_num_filters),
-        rng_bool(0, 1), rng_constructors(0, (int)(constructors.size() - 1)),
-        rng_constructors_first(0, (int)(constructors_first.size() - 1));
-
-    INFO("Random pipeline seed: " << pipeline_seed);
-
-    auto num_filters = (unsigned)rng_num_filters(gen);
-    for (unsigned j = 0; j < num_filters; j++) {
-      if (j == 0 && rng_bool(gen) == 1) {
-        auto idx = (unsigned)rng_constructors_first(gen);
-        tiledb::sm::Filter* filter = constructors_first[idx]();
-        pipeline.add_filter(*filter);
-        tdb_delete(filter);
-      } else {
-        auto idx = (unsigned)rng_constructors(gen);
-        tiledb::sm::Filter* filter = constructors[idx]();
-        pipeline.add_filter(*filter);
-        tdb_delete(filter);
-      }
-    }
-
     // Check the pipelines run forward and backward without error and return the
     // input data.
     // Run the pipeline tests.
@@ -890,6 +864,50 @@ TEST_CASE("Filter: Test random pipeline", "[filter][random]") {
         pipeline,
         &tile_data_generator,
         tracker);
+  };
+
+  SECTION("Random pipeline") {
+    for (int i = 0; i < 100; i++) {
+      // Construct a random pipeline
+      FilterPipeline pipeline;
+      const unsigned max_num_filters = 6;
+      std::random_device rd;
+      auto pipeline_seed = rd();
+      std::mt19937 gen(pipeline_seed);
+      std::uniform_int_distribution<> rng_num_filters(0, max_num_filters),
+          rng_bool(0, 1), rng_constructors(0, (int)(constructors.size() - 1)),
+          rng_constructors_first(0, (int)(constructors_first.size() - 1));
+
+      INFO("Random pipeline seed: " << pipeline_seed);
+
+      auto num_filters = (unsigned)rng_num_filters(gen);
+      for (unsigned j = 0; j < num_filters; j++) {
+        if (j == 0 && rng_bool(gen) == 1) {
+          auto idx = (unsigned)rng_constructors_first(gen);
+          tiledb::sm::Filter* filter = constructors_first[idx]();
+          pipeline.add_filter(*filter);
+          tdb_delete(filter);
+        } else {
+          auto idx = (unsigned)rng_constructors(gen);
+          tiledb::sm::Filter* filter = constructors[idx]();
+          pipeline.add_filter(*filter);
+          tdb_delete(filter);
+        }
+      }
+
+      doit(pipeline);
+    }
+  }
+
+  SECTION("Example") {
+    FilterPipeline pipeline;
+    pipeline.add_filter(Add1IncludingMetadataFilter(Datatype::UINT64));
+    pipeline.add_filter(Add1IncludingMetadataFilter(Datatype::UINT64));
+    pipeline.add_filter(
+        CompressionFilter(tiledb::sm::Compressor::BZIP2, -1, Datatype::UINT64));
+    pipeline.add_filter(BitWidthReductionFilter(Datatype::UINT64));
+
+    doit(pipeline);
   }
 }
 
diff --git a/tiledb/type/datatype_queries.h b/tiledb/type/datatype_queries.h
new file mode 100644
index 00000000000..cc4db946c3f
--- /dev/null
+++ b/tiledb/type/datatype_queries.h
@@ -0,0 +1,56 @@
+/**
+ * @file tiledb/type/datatype_queries.h
+ *
+ * @section LICENSE
+ *
+ * The MIT License
+ *
+ * @copyright Copyright (c) 2025 TileDB, Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to deal
+ * in the Software without restriction, including without limitation the rights
+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+ * copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+ * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+ * THE SOFTWARE.
+ *
+ * @section DESCRIPTION
+ *
+ * This file provides functions which answer queries about Datatype
+ */
+
+#ifndef TILEDB_DATATYPE_QUERIES_H
+#define TILEDB_DATATYPE_QUERIES_H
+
+#include "tiledb/type/apply_with_type.h"
+#include "tiledb/type/datatype_traits.h"
+
+namespace tiledb::type {
+
+using tiledb::sm::Datatype;
+
+/**
+ * @return whether the `value_type` for a given `Datatype` is signed
+ */
+static bool has_signed_value_type(Datatype dt) {
+  return apply_with_type(
+      [](auto value_type) -> bool {
+        return std::is_signed_v<decltype(value_type)>;
+      },
+      dt);
+}
+
+}  // namespace tiledb::type
+
+#endif
diff --git a/tiledb/type/datatype_traits.h b/tiledb/type/datatype_traits.h
index e38c5883a25..83e3305b0e8 100644
--- a/tiledb/type/datatype_traits.h
+++ b/tiledb/type/datatype_traits.h
@@ -36,10 +36,10 @@
 
 #include "tiledb/sm/enums/datatype.h"
 
-using tiledb::sm::Datatype;
-
 namespace tiledb::type {
 
+using tiledb::sm::Datatype;
+
 /**
  * `datatype_traits`
  *


Reviews:
user: login: davisp
body: Generally speaking I think this all looks good. I made a number of suggestions throughout, none of which I'm particularly tied to so feel more than free to push back on anything specific.

I do think we should probably keep all Rust and related C++ in a specific directory (whether it be `tiledb/oxidize` or `test/support/oxidize` with a slight preference for the former because of obvious hopes and motivations). For the time being we'll likely want to ensure that having some Rust around won't bother anyone or upset any existing CI workflows and the easiest way to ensure that is by having that `if not rust: return` logic in a single place.

Also commenting here that Cargo.lock should be deleted and added to a `wherever/oxidized/.gitignore` which should include the other standard ignores for Rust workspaces.

I'd also like to pull in our standard `.rustfmt.toml` configuration to limit the column width on Rust code.
state: CHANGES_REQUESTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-29T17:02:38Z
user: login: rroelke
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-29T17:09:43Z
user: login: rroelke
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-29T17:10:33Z
user: login: rroelke
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-29T17:17:37Z
user: login: rroelke
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-29T17:20:04Z
user: login: rroelke
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-29T17:22:19Z
user: login: rroelke
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-29T17:23:15Z
user: login: davisp
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-29T17:24:08Z
user: login: rroelke
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-29T17:24:36Z
user: login: davisp
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-29T17:26:14Z
user: login: rroelke
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-29T17:27:58Z
user: login: davisp
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-29T17:33:35Z
user: login: davisp
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-29T17:40:58Z
user: login: davisp
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-29T17:46:11Z
user: login: davisp
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-29T17:48:59Z
user: login: davisp
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-29T17:49:53Z
user: login: rroelke
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-29T18:00:35Z
user: login: rroelke
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-29T20:17:06Z
user: login: rroelke
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-29T20:18:25Z
user: login: rroelke
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-29T20:20:33Z
user: login: rroelke
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-29T20:22:02Z
user: login: rroelke
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-29T20:22:24Z
user: login: rroelke
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-29T20:23:16Z
user: login: rroelke
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-29T20:23:44Z
user: login: rroelke
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-29T20:24:42Z
user: login: rroelke
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-29T20:25:00Z
user: login: rroelke
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-30T01:50:42Z
user: login: rroelke
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-30T02:00:24Z
user: login: rroelke
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-30T02:02:26Z
user: login: rroelke
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-30T02:02:55Z
user: login: rroelke
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-30T15:39:33Z
user: login: teo-tsirpanis
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-05-05T11:35:47Z
user: login: rroelke
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-06T14:18:33Z
user: login: rroelke
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-06T14:19:06Z
user: login: rroelke
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-06T14:21:38Z
user: login: teo-tsirpanis
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-05-06T16:17:56Z
user: login: teo-tsirpanis
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-05-06T17:12:32Z
user: login: rroelke
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-06T17:49:36Z
user: login: teo-tsirpanis
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-05-06T18:10:42Z
user: login: rroelke
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-06T19:26:01Z
user: login: teo-tsirpanis
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-05-06T20:27:59Z
user: login: teo-tsirpanis
body: Thanks!
state: APPROVED
author_association: MEMBER
submitted_at: 2025-05-06T20:29:29Z
user: login: davisp
body: +1. I've managed to convince myself I understand how all of the math works which was not a short process. Not gonna lie, I was a lot more confident when there was a Rust version to check against (I completely understand why that got removed though). Still, this all looks good to me.
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-05-08T14:34:04Z
user: login: rroelke
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-08T14:37:07Z

Review comments:
diff_hunk: @@ -0,0 +1,79 @@
+set(TILEDB_CRATES
+    arithmetic
+)
+
+if (NOT TILEDB_TESTS)
+  message(FATAL_ERROR "Building Rust without tests is not supported.")
+endif()
+
+find_program(CARGO cargo)
+
+if (CMAKE_BUILD_TYPE STREQUAL "Debug")
+    set(CARGO_PROFILE "dev")
+    set(CARGO_TARGET "debug")
+else ()
+    set(CARGO_PROFILE "release")
+    set(CARGO_TARGET "release")
+endif ()
+
+set(CARGO_MANIFEST_PATH "${CMAKE_CURRENT_SOURCE_DIR}/Cargo.toml")
+
+set(CARGO_INCLUDE_DIR "${CMAKE_CURRENT_BINARY_DIR}/target")
+set(CARGO_LIB_DIR "${CMAKE_CURRENT_BINARY_DIR}/target/${CARGO_TARGET}")
+
+if (TILEDB_RUST)
+    foreach(crate IN LISTS TILEDB_CRATES)
+        add_custom_command(
+            OUTPUT
+                "${CMAKE_CURRENT_BINARY_DIR}/${crate}.h"
+                "${CARGO_INCLUDE_DIR}/cxxbridge/${crate}/src/lib.rs.cc"
+                "${CARGO_LIB_DIR}/${CMAKE_STATIC_LIBRARY_PREFIX}${crate}${CMAKE_STATIC_LIBRARY_SUFFIX}"
+            COMMAND
+                "MACOSX_DEPLOYMENT_TARGET=${CMAKE_OSX_DEPLOYMENT_TARGET}"
+                "CARGO_TARGET_DIR=${CMAKE_CURRENT_BINARY_DIR}/target"
+                ${CARGO}
+                build
+                "-q"
+                "--profile" ${CARGO_PROFILE}
+                "--manifest-path" ${CARGO_MANIFEST_PATH}
+                "-p" ${crate}
+            COMMAND
+                cp "${CARGO_INCLUDE_DIR}/cxxbridge/${crate}/src/lib.rs.h" "${CMAKE_CURRENT_BINARY_DIR}/${crate}.h"
+            USES_TERMINAL
+            DEPENDS
+                ${CMAKE_CURRENT_SOURCE_DIR}/${crate}/src/lib.rs
+            WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}
+        )
+    endforeach()
+else ()
+    foreach(crate IN LISTS TILEDB_CRATES)
+        add_custom_command(
+            OUTPUT
+                "${CMAKE_CURRENT_BINARY_DIR}/${crate}.h"
+                "${CARGO_INCLUDE_DIR}/cxxbridge/${crate}/src/lib.rs.cc"
+                "${CARGO_LIB_DIR}/${CMAKE_STATIC_LIBRARY_PREFIX}${crate}${CMAKE_STATIC_LIBRARY_SUFFIX}"
+            COMMAND
+                echo "This build is not configured with Rust support enabled."
+            COMMAND
+                echo "To build this target, configure with '-DTILEDB_RUST=ON'."
+            COMMAND
+                exit 1
+        )
+    endforeach()
path: test/support/oxidize/CMakeLists.txt
user: login: davisp
body: Why'd you go this route instead of the `if not rust: return` logic? I'm pretty sure this is broken since we'll always attempt to create these targets (and the object libraries below) when Rust isn't enabled which will break things.
created_at: 2025-04-29T16:07:39Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,79 @@
+set(TILEDB_CRATES
+    arithmetic
+)
+
+if (NOT TILEDB_TESTS)
+  message(FATAL_ERROR "Building Rust without tests is not supported.")
+endif()
+
+find_program(CARGO cargo)
+
+if (CMAKE_BUILD_TYPE STREQUAL "Debug")
+    set(CARGO_PROFILE "dev")
+    set(CARGO_TARGET "debug")
+else ()
+    set(CARGO_PROFILE "release")
+    set(CARGO_TARGET "release")
+endif ()
+
+set(CARGO_MANIFEST_PATH "${CMAKE_CURRENT_SOURCE_DIR}/Cargo.toml")
+
+set(CARGO_INCLUDE_DIR "${CMAKE_CURRENT_BINARY_DIR}/target")
+set(CARGO_LIB_DIR "${CMAKE_CURRENT_BINARY_DIR}/target/${CARGO_TARGET}")
+
+if (TILEDB_RUST)
+    foreach(crate IN LISTS TILEDB_CRATES)
path: test/support/oxidize/CMakeLists.txt
user: login: davisp
body: Why'd you swap from creating lists of targets with one Cargo invocation to having one per crate? The list did have one sad thing where the CMake log message listed all of them out in a not super awesome format so I wouldn't be surprised if you found something else.
created_at: 2025-04-29T16:10:19Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,79 @@
+set(TILEDB_CRATES
+    arithmetic
+)
+
+if (NOT TILEDB_TESTS)
+  message(FATAL_ERROR "Building Rust without tests is not supported.")
+endif()
+
+find_program(CARGO cargo)
+
+if (CMAKE_BUILD_TYPE STREQUAL "Debug")
+    set(CARGO_PROFILE "dev")
+    set(CARGO_TARGET "debug")
+else ()
+    set(CARGO_PROFILE "release")
+    set(CARGO_TARGET "release")
+endif ()
+
+set(CARGO_MANIFEST_PATH "${CMAKE_CURRENT_SOURCE_DIR}/Cargo.toml")
+
+set(CARGO_INCLUDE_DIR "${CMAKE_CURRENT_BINARY_DIR}/target")
+set(CARGO_LIB_DIR "${CMAKE_CURRENT_BINARY_DIR}/target/${CARGO_TARGET}")
+
+if (TILEDB_RUST)
+    foreach(crate IN LISTS TILEDB_CRATES)
+        add_custom_command(
+            OUTPUT
+                "${CMAKE_CURRENT_BINARY_DIR}/${crate}.h"
+                "${CARGO_INCLUDE_DIR}/cxxbridge/${crate}/src/lib.rs.cc"
+                "${CARGO_LIB_DIR}/${CMAKE_STATIC_LIBRARY_PREFIX}${crate}${CMAKE_STATIC_LIBRARY_SUFFIX}"
+            COMMAND
+                "MACOSX_DEPLOYMENT_TARGET=${CMAKE_OSX_DEPLOYMENT_TARGET}"
+                "CARGO_TARGET_DIR=${CMAKE_CURRENT_BINARY_DIR}/target"
+                ${CARGO}
+                build
+                "-q"
+                "--profile" ${CARGO_PROFILE}
+                "--manifest-path" ${CARGO_MANIFEST_PATH}
+                "-p" ${crate}
+            COMMAND
+                cp "${CARGO_INCLUDE_DIR}/cxxbridge/${crate}/src/lib.rs.h" "${CMAKE_CURRENT_BINARY_DIR}/${crate}.h"
path: test/support/oxidize/CMakeLists.txt
user: login: davisp
body: This seems to be covering something up. You should be able to just `#include "cxxbridge/${crate}/src/lib.rs.h"` in C++.
created_at: 2025-04-29T16:11:46Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,79 @@
+set(TILEDB_CRATES
+    arithmetic
+)
+
+if (NOT TILEDB_TESTS)
+  message(FATAL_ERROR "Building Rust without tests is not supported.")
+endif()
+
+find_program(CARGO cargo)
+
+if (CMAKE_BUILD_TYPE STREQUAL "Debug")
+    set(CARGO_PROFILE "dev")
+    set(CARGO_TARGET "debug")
+else ()
+    set(CARGO_PROFILE "release")
+    set(CARGO_TARGET "release")
+endif ()
+
+set(CARGO_MANIFEST_PATH "${CMAKE_CURRENT_SOURCE_DIR}/Cargo.toml")
+
+set(CARGO_INCLUDE_DIR "${CMAKE_CURRENT_BINARY_DIR}/target")
+set(CARGO_LIB_DIR "${CMAKE_CURRENT_BINARY_DIR}/target/${CARGO_TARGET}")
+
+if (TILEDB_RUST)
+    foreach(crate IN LISTS TILEDB_CRATES)
+        add_custom_command(
+            OUTPUT
+                "${CMAKE_CURRENT_BINARY_DIR}/${crate}.h"
+                "${CARGO_INCLUDE_DIR}/cxxbridge/${crate}/src/lib.rs.cc"
+                "${CARGO_LIB_DIR}/${CMAKE_STATIC_LIBRARY_PREFIX}${crate}${CMAKE_STATIC_LIBRARY_SUFFIX}"
+            COMMAND
+                "MACOSX_DEPLOYMENT_TARGET=${CMAKE_OSX_DEPLOYMENT_TARGET}"
+                "CARGO_TARGET_DIR=${CMAKE_CURRENT_BINARY_DIR}/target"
+                ${CARGO}
+                build
+                "-q"
+                "--profile" ${CARGO_PROFILE}
+                "--manifest-path" ${CARGO_MANIFEST_PATH}
+                "-p" ${crate}
+            COMMAND
+                cp "${CARGO_INCLUDE_DIR}/cxxbridge/${crate}/src/lib.rs.h" "${CMAKE_CURRENT_BINARY_DIR}/${crate}.h"
+            USES_TERMINAL
+            DEPENDS
+                ${CMAKE_CURRENT_SOURCE_DIR}/${crate}/src/lib.rs
path: test/support/oxidize/CMakeLists.txt
user: login: davisp
body: I assume you moved this from the separate manually maintained list of source files specifically to avoid manually maintaining that list? I'd probably go back to a manual list or folks are going to have a bad time when working on anything that's not `lib.rs` and then not getting the Rust static library created.

That feels a bit vague, so to be more specific, if someone adds `mod utils;` to their `lib.rs`, then modifying `utils.rs` won't trigger a rebuild of the static library for that crate. The bindings will be correctly updated (assuming we always only have a `cxx::bridge` in `lib.rs`, but that still leaves out the rebuild of Rust logic being triggered on non-`cxx::bridge` changes.
created_at: 2025-04-29T16:16:02Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,79 @@
+set(TILEDB_CRATES
+    arithmetic
+)
+
+if (NOT TILEDB_TESTS)
+  message(FATAL_ERROR "Building Rust without tests is not supported.")
+endif()
+
+find_program(CARGO cargo)
+
+if (CMAKE_BUILD_TYPE STREQUAL "Debug")
+    set(CARGO_PROFILE "dev")
+    set(CARGO_TARGET "debug")
+else ()
+    set(CARGO_PROFILE "release")
+    set(CARGO_TARGET "release")
+endif ()
+
+set(CARGO_MANIFEST_PATH "${CMAKE_CURRENT_SOURCE_DIR}/Cargo.toml")
+
+set(CARGO_INCLUDE_DIR "${CMAKE_CURRENT_BINARY_DIR}/target")
+set(CARGO_LIB_DIR "${CMAKE_CURRENT_BINARY_DIR}/target/${CARGO_TARGET}")
+
+if (TILEDB_RUST)
+    foreach(crate IN LISTS TILEDB_CRATES)
+        add_custom_command(
+            OUTPUT
+                "${CMAKE_CURRENT_BINARY_DIR}/${crate}.h"
+                "${CARGO_INCLUDE_DIR}/cxxbridge/${crate}/src/lib.rs.cc"
+                "${CARGO_LIB_DIR}/${CMAKE_STATIC_LIBRARY_PREFIX}${crate}${CMAKE_STATIC_LIBRARY_SUFFIX}"
+            COMMAND
+                "MACOSX_DEPLOYMENT_TARGET=${CMAKE_OSX_DEPLOYMENT_TARGET}"
+                "CARGO_TARGET_DIR=${CMAKE_CURRENT_BINARY_DIR}/target"
+                ${CARGO}
+                build
+                "-q"
+                "--profile" ${CARGO_PROFILE}
+                "--manifest-path" ${CARGO_MANIFEST_PATH}
+                "-p" ${crate}
+            COMMAND
+                cp "${CARGO_INCLUDE_DIR}/cxxbridge/${crate}/src/lib.rs.h" "${CMAKE_CURRENT_BINARY_DIR}/${crate}.h"
+            USES_TERMINAL
+            DEPENDS
+                ${CMAKE_CURRENT_SOURCE_DIR}/${crate}/src/lib.rs
+            WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}
+        )
+    endforeach()
+else ()
+    foreach(crate IN LISTS TILEDB_CRATES)
+        add_custom_command(
+            OUTPUT
+                "${CMAKE_CURRENT_BINARY_DIR}/${crate}.h"
+                "${CARGO_INCLUDE_DIR}/cxxbridge/${crate}/src/lib.rs.cc"
+                "${CARGO_LIB_DIR}/${CMAKE_STATIC_LIBRARY_PREFIX}${crate}${CMAKE_STATIC_LIBRARY_SUFFIX}"
+            COMMAND
+                echo "This build is not configured with Rust support enabled."
+            COMMAND
+                echo "To build this target, configure with '-DTILEDB_RUST=ON'."
+            COMMAND
+                exit 1
+        )
+    endforeach()
path: test/support/oxidize/CMakeLists.txt
user: login: davisp
body: Actually, this appears to the be reason all of the CI runs failed so my initial read seems correct here. Generally speaking I'd go back to the "if not rust: return" logic towards the top of this file (the list of crates before return is something I'd keep though).
created_at: 2025-04-29T16:19:36Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,79 @@
+set(TILEDB_CRATES
+    arithmetic
+)
+
+if (NOT TILEDB_TESTS)
+  message(FATAL_ERROR "Building Rust without tests is not supported.")
path: test/support/oxidize/CMakeLists.txt
user: login: davisp
body: Not tying this to the `TILEDB_RUST` option will fail all builds that try and run without building tests (notably, TileDB-Server does this). 
created_at: 2025-04-29T16:20:15Z
author_association: CONTRIBUTOR
diff_hunk: @@ -115,4 +115,6 @@ commence(object_library mem_helpers)
   this_target_object_libraries(baseline)
 conclude(object_library)
 
+add_subdirectory(oxidize)
path: test/support/CMakeLists.txt
user: login: davisp
body: I'm a bit undecided on whether I prefer having Rust code in `tiledb/oxidize` vs `test/support/oxidize`. I know we're currently only intending to enable Rust for tests, but eventually if we start allowing Rust in other places it'd be a bit awkward to have that code in the test directory. Granted we could have two, but then any common dependencies are going to be compiled twice.
created_at: 2025-04-29T16:22:10Z
author_association: CONTRIBUTOR
diff_hunk: @@ -46,6 +46,11 @@ on:
         description: 'Enable Address Sanitizer'
         required: false
         type: boolean
+      rust:
+        default: false
+        description: 'Enable Rust'
+        required: false
+        type: boolean
path: .github/workflows/ci-linux_mac.yml
user: login: davisp
body: Am I missing somewhere that this is actually enabled?
created_at: 2025-04-29T16:23:08Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,18 @@
+[workspace]
+resolver = "2"
+members = [
+  "arithmetic"
+]
+default-members = [
path: test/support/oxidize/Cargo.toml
user: login: davisp
body: I'd probably avoid declaring this until we actually have crates that are only intended to be compiled in special places like we have for the CLI and wheels. Until then its just extra overhead for folks that might be learning Rust.
created_at: 2025-04-29T16:24:34Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,15 @@
+[package]
+name = "arithmetic"
+edition.workspace = true
+rust-version.workspace = true
+version.workspace = true
+
+[dependencies]
+cxx = { workspace = true }
path: test/support/oxidize/arithmetic/Cargo.toml
user: login: davisp
body: Super silly nit cause I'm still not sure if I have an actual preference, but I think I'm starting to prefer the `crate.workspace = true` like you've got for the package keys above.
created_at: 2025-04-29T16:25:42Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,89 @@
+//! Exports capabilities from the Rust standard library
+//! for use in C++ code.
+
+#[cxx::bridge]
+mod ffi {
+    #[namespace = "rust::std"]
path: test/support/oxidize/arithmetic/src/lib.rs
user: login: davisp
body: Hurms. I noticed this while reading the C++ tests. I think I see why you went with `rust::std` here, but I'm not sure that's a great idea. I get that the checked arithmetic implementations are from Rust's stdlib, but for someone reading this while learning Rust I think that could end up being fairly confusing. I'd think `tiledb::rust::math` or something might avoid any "so this magically exposes rust's stdlib" thoughts for anyone new to this code/setup.
created_at: 2025-04-29T16:31:09Z
author_association: CONTRIBUTOR
diff_hunk: @@ -64,7 +65,7 @@ void DoubleDelta::compress(
     case Datatype::BLOB:
     case Datatype::GEOM_WKB:
     case Datatype::GEOM_WKT:
-      return DoubleDelta::compress<std::byte>(input_buffer, output_buffer);
+      return DoubleDelta::compress<uint8_t>(input_buffer, output_buffer);
path: tiledb/sm/compressors/dd_compressor.cc
user: login: davisp
body: Do we have to worry about signed-ness here? I think not according to [cppreference](https://en.cppreference.com/w/cpp/types/byte) but it feels a bit underspecified.
created_at: 2025-04-29T16:33:11Z
author_association: CONTRIBUTOR
diff_hunk: @@ -134,10 +136,10 @@ class DoubleDelta {
    * @tparam The datatype of the values.
    * @param in The input buffer.
    * @param num The number of values in the buffer.
-   * @param bitsize The bitsize of the double deltas to be retrieved.
+   * @return The bitsize of the double deltas to be retrieved.
    */
   template <class T>
-  static void compute_bitsize(T* in, uint64_t num, unsigned int* bitsize);
+  static unsigned compute_bitsize(const T* in, uint64_t num);
path: tiledb/sm/compressors/dd_compressor.h
user: login: davisp
body: Can we swap this to `uint32_t`? I'm old enough to have the scars from chasing down implicitly sized types during the 32bit to 64bit transition so now I shudder at the implicitly sized types.
created_at: 2025-04-29T16:36:19Z
author_association: CONTRIBUTOR
diff_hunk: @@ -38,3 +38,25 @@ jobs:
 
       - name: Check formatting
         run: $GITHUB_WORKSPACE/scripts/ci/check_formatting_linux.sh
+
+  rust-lint:
+    runs-on: ubuntu-latest
+    if: ${{ startsWith(github.ref , 'refs/tags') != true && startsWith(github.ref , 'build-') != true }}
+    timeout-minutes: 90
+    name: Check Rust Format
+    steps:
+      - name: Install Rust Stable
path: .github/workflows/check-formatting.yml
user: login: davisp
body: This workflow failed because you forgot to checkout TileDB here.
created_at: 2025-04-29T16:39:32Z
author_association: CONTRIBUTOR
diff_hunk: @@ -115,4 +115,6 @@ commence(object_library mem_helpers)
   this_target_object_libraries(baseline)
 conclude(object_library)
 
+add_subdirectory(oxidize)
path: test/support/CMakeLists.txt
user: login: davisp
body: Actually, now that I've read things closer, I think I'm fairly convinced that this should live in `tiledb/oxidized` since the tests that are using this are unit tests under the `tiledb` directory.
created_at: 2025-04-29T16:43:30Z
author_association: CONTRIBUTOR
diff_hunk: @@ -26,6 +26,15 @@
 
 include(unit_test)
 
+if (TILEDB_RUST)
path: tiledb/common/test/CMakeLists.txt
user: login: davisp
body: I'm not sure I'm in love with this. For the time being I'd lean towards all C++ code referencing Rust to be in the `tiledb/wherever/oxidize` directory in case we ever decide to abandon Rust in core for some unforeseen reason.
created_at: 2025-04-29T16:45:25Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,304 @@
+/**
+ * @file unit_arithmetic.cc
+ *
+ * @section LICENSE
+ *
+ * The MIT License
+ *
+ * @copyright Copyright (c) 2025 TileDB, Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to deal
+ * in the Software without restriction, including without limitation the rights
+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+ * copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+ * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+ * THE SOFTWARE.
+ *
+ * @section DESCRIPTION
+ *
+ * This file tests arithmetic functions defined in `arithmetic.h`.
+ */
+
+#include "build/test/support/oxidize/arithmetic.h"
+#include "tiledb/common/arithmetic.h"
+
+#include <test/support/assert_helpers.h>
+#include <test/support/tdb_catch.h>
+#include <test/support/tdb_rapidcheck.h>
+
+using namespace tiledb::common;
+
+TEST_CASE(
+    "Arithmetic checked_arithmetic uint32_t add", "[arithmetic][rapidcheck]") {
+  auto doit = []<typename Asserter = tiledb::test::AsserterCatch>(
+                  uint32_t a, uint32_t b)
+                  ->std::optional<uint32_t> {
+    const auto cpp = checked_arithmetic<uint32_t>::add(a, b);
+    std::optional<uint32_t> rs;
+    {
+      uint32_t value;
+      if (rust::std::u32_checked_add(a, b, value)) {
+        rs.emplace(value);
+      }
+    }
+    ASSERTER(cpp == rs);
+
+    return cpp;
+  };
+
+  SECTION("Rapidcheck") {
+    rc::prop(
+        "checked_arithmetic<uint32_t>::add", [doit](uint32_t a, uint32_t b) {
+          doit.operator()<tiledb::test::AsserterRapidcheck>(a, b);
+        });
path: tiledb/common/test/unit_arithmetic.cc
user: login: davisp
body: Was there a reason to not drive these tests from Rust? The idea here is that we're verifying our own checked arithmentic using Rust's stdlib so it seems like it'd be slightly more obvious if we were checking from the Rust side rather than just asserting we match. I'm not really asking you to change anything here since they're functionally equivalent, just curious if you hit something funky trying the other direction.
created_at: 2025-04-29T16:50:54Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,304 @@
+/**
+ * @file unit_arithmetic.cc
+ *
+ * @section LICENSE
+ *
+ * The MIT License
+ *
+ * @copyright Copyright (c) 2025 TileDB, Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to deal
+ * in the Software without restriction, including without limitation the rights
+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+ * copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+ * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+ * THE SOFTWARE.
+ *
+ * @section DESCRIPTION
+ *
+ * This file tests arithmetic functions defined in `arithmetic.h`.
+ */
+
+#include "build/test/support/oxidize/arithmetic.h"
+#include "tiledb/common/arithmetic.h"
+
+#include <test/support/assert_helpers.h>
+#include <test/support/tdb_catch.h>
+#include <test/support/tdb_rapidcheck.h>
+
+using namespace tiledb::common;
+
+TEST_CASE(
+    "Arithmetic checked_arithmetic uint32_t add", "[arithmetic][rapidcheck]") {
+  auto doit = []<typename Asserter = tiledb::test::AsserterCatch>(
+                  uint32_t a, uint32_t b)
+                  ->std::optional<uint32_t> {
+    const auto cpp = checked_arithmetic<uint32_t>::add(a, b);
+    std::optional<uint32_t> rs;
+    {
+      uint32_t value;
+      if (rust::std::u32_checked_add(a, b, value)) {
+        rs.emplace(value);
+      }
+    }
+    ASSERTER(cpp == rs);
+
+    return cpp;
+  };
+
+  SECTION("Rapidcheck") {
+    rc::prop(
+        "checked_arithmetic<uint32_t>::add", [doit](uint32_t a, uint32_t b) {
+          doit.operator()<tiledb::test::AsserterRapidcheck>(a, b);
+        });
+  }
+}
+
+TEST_CASE(
+    "Arithmetic checked_arithmetic int32_t add", "[arithmetic][rapidcheck]") {
+  auto doit =
+      []<typename Asserter = tiledb::test::AsserterCatch>(int32_t a, int32_t b)
+          ->std::optional<int32_t> {
+    const auto cpp = checked_arithmetic<int32_t>::add(a, b);
+    std::optional<int32_t> rs;
+    {
+      int32_t value;
+      if (rust::std::i32_checked_add(a, b, value)) {
path: tiledb/common/test/unit_arithmetic.cc
user: login: davisp
body: Huh. I could have sworn that `Option<T>` was a supported conversion but I guess I just assumed that because `Result<T, E>` is supported. Looks like there's ongoing work for it though: https://github.com/dtolnay/cxx/issues/87
created_at: 2025-04-29T16:53:46Z
author_association: CONTRIBUTOR
diff_hunk: @@ -257,35 +257,54 @@ void DoubleDelta::compress(ConstBuffer* input_buffer, Buffer* output_buffer) {
 }
 
 template <class T>
-void DoubleDelta::compute_bitsize(T* in, uint64_t num, unsigned int* bitsize) {
+unsigned DoubleDelta::compute_bitsize(const T* in, uint64_t num) {
   // Find maximum absolute double delta
-  *bitsize = 0;
-  int64_t max = 0;
   if (num <= 2) {
-    *bitsize = 0;
-    return;
-  }
-  int64_t prev_delta = int64_t(in[1]) - int64_t(in[0]);
-  char delta_out_of_bounds = 0;
-  for (uint64_t i = 2; i < num; ++i) {
-    int64_t cur_delta = int64_t(in[i]) - int64_t(in[i - 1]);
-    int64_t dd = cur_delta - prev_delta;
-    delta_out_of_bounds |= (char)(cur_delta < 0 && prev_delta > 0 && dd > 0);
-    delta_out_of_bounds |= (char)(cur_delta > 0 && prev_delta < 0 && dd < 0);
-    max = std::max(std::abs(dd), max);
-    prev_delta = cur_delta;
+    return 0;
   }
-  // Handle error
-  if (delta_out_of_bounds) {
-    throw DoubleDeltaException(
-        "Cannot compress with DoubleDelta; Some negative double delta is out "
-        "of bounds");
+
+  using DeltaType = common::integral_64<T>::type;
+
+  uint64_t max_delta = 0;
+  std::optional<int64_t> prev_delta;
+  for (uint64_t i = 1; i < num; ++i) {
+    const DeltaType cur = (DeltaType)(in[i]);
+    const DeltaType prev = (DeltaType)(in[i - 1]);
+
+    const std::optional<int64_t> cur_delta =
+        common::checked_arithmetic<DeltaType>::sub_signed(cur, prev);
+    if (!cur_delta.has_value()) {
+      throw DoubleDeltaException(
+          "Cannot compress with DoubleDelta: delta exceeds range of int64_t");
+    }
+
+    if (prev_delta.has_value()) {
+      const std::optional<int64_t> dd =
+          common::checked_arithmetic<int64_t>::sub(
+              cur_delta.value(), prev_delta.value());
+      if (!dd.has_value()) {
+        throw DoubleDeltaException(
+            "Cannot compress with DoubleDelta; Some negative double delta is "
+            "out "
+            "of bounds");
+      }
+      max_delta =
+          std::max(static_cast<uint64_t>(std::abs(dd.value())), max_delta);
+    } else {
+      max_delta = std::abs(cur_delta.value());
+    }
+
+    prev_delta = cur_delta.value();
   }
+
   // Calculate bitsize of the maximum absolute double delta
+  unsigned bitsize = 0;
path: tiledb/sm/compressors/dd_compressor.cc
user: login: davisp
body: Another implicitly sized type if you agree with that earlier suggestion.
created_at: 2025-04-29T16:55:47Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,79 @@
+set(TILEDB_CRATES
+    arithmetic
+)
+
+if (NOT TILEDB_TESTS)
+  message(FATAL_ERROR "Building Rust without tests is not supported.")
+endif()
+
+find_program(CARGO cargo)
+
+if (CMAKE_BUILD_TYPE STREQUAL "Debug")
+    set(CARGO_PROFILE "dev")
+    set(CARGO_TARGET "debug")
+else ()
+    set(CARGO_PROFILE "release")
+    set(CARGO_TARGET "release")
+endif ()
+
+set(CARGO_MANIFEST_PATH "${CMAKE_CURRENT_SOURCE_DIR}/Cargo.toml")
+
+set(CARGO_INCLUDE_DIR "${CMAKE_CURRENT_BINARY_DIR}/target")
+set(CARGO_LIB_DIR "${CMAKE_CURRENT_BINARY_DIR}/target/${CARGO_TARGET}")
+
+if (TILEDB_RUST)
+    foreach(crate IN LISTS TILEDB_CRATES)
path: test/support/oxidize/CMakeLists.txt
user: login: rroelke
body: My main reason was that if I build a particular unit test target (which is usually what I find myself doing) then I only want to build the crates which that particular target needs.

The single crate doing `--all-targets` is definitely not what I want for that
created_at: 2025-04-29T17:09:43Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,79 @@
+set(TILEDB_CRATES
+    arithmetic
+)
+
+if (NOT TILEDB_TESTS)
+  message(FATAL_ERROR "Building Rust without tests is not supported.")
+endif()
+
+find_program(CARGO cargo)
+
+if (CMAKE_BUILD_TYPE STREQUAL "Debug")
+    set(CARGO_PROFILE "dev")
+    set(CARGO_TARGET "debug")
+else ()
+    set(CARGO_PROFILE "release")
+    set(CARGO_TARGET "release")
+endif ()
+
+set(CARGO_MANIFEST_PATH "${CMAKE_CURRENT_SOURCE_DIR}/Cargo.toml")
+
+set(CARGO_INCLUDE_DIR "${CMAKE_CURRENT_BINARY_DIR}/target")
+set(CARGO_LIB_DIR "${CMAKE_CURRENT_BINARY_DIR}/target/${CARGO_TARGET}")
+
+if (TILEDB_RUST)
+    foreach(crate IN LISTS TILEDB_CRATES)
+        add_custom_command(
+            OUTPUT
+                "${CMAKE_CURRENT_BINARY_DIR}/${crate}.h"
+                "${CARGO_INCLUDE_DIR}/cxxbridge/${crate}/src/lib.rs.cc"
+                "${CARGO_LIB_DIR}/${CMAKE_STATIC_LIBRARY_PREFIX}${crate}${CMAKE_STATIC_LIBRARY_SUFFIX}"
+            COMMAND
+                "MACOSX_DEPLOYMENT_TARGET=${CMAKE_OSX_DEPLOYMENT_TARGET}"
+                "CARGO_TARGET_DIR=${CMAKE_CURRENT_BINARY_DIR}/target"
+                ${CARGO}
+                build
+                "-q"
+                "--profile" ${CARGO_PROFILE}
+                "--manifest-path" ${CARGO_MANIFEST_PATH}
+                "-p" ${crate}
+            COMMAND
+                cp "${CARGO_INCLUDE_DIR}/cxxbridge/${crate}/src/lib.rs.h" "${CMAKE_CURRENT_BINARY_DIR}/${crate}.h"
path: test/support/oxidize/CMakeLists.txt
user: login: rroelke
body: That worked too, this was intentional.
I would much rather see
`#include "build/tiledb/test/support/oxidize/arithmetic.h`
in the C++ code
created_at: 2025-04-29T17:10:32Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,79 @@
+set(TILEDB_CRATES
+    arithmetic
+)
+
+if (NOT TILEDB_TESTS)
+  message(FATAL_ERROR "Building Rust without tests is not supported.")
+endif()
+
+find_program(CARGO cargo)
+
+if (CMAKE_BUILD_TYPE STREQUAL "Debug")
+    set(CARGO_PROFILE "dev")
+    set(CARGO_TARGET "debug")
+else ()
+    set(CARGO_PROFILE "release")
+    set(CARGO_TARGET "release")
+endif ()
+
+set(CARGO_MANIFEST_PATH "${CMAKE_CURRENT_SOURCE_DIR}/Cargo.toml")
+
+set(CARGO_INCLUDE_DIR "${CMAKE_CURRENT_BINARY_DIR}/target")
+set(CARGO_LIB_DIR "${CMAKE_CURRENT_BINARY_DIR}/target/${CARGO_TARGET}")
+
+if (TILEDB_RUST)
+    foreach(crate IN LISTS TILEDB_CRATES)
+        add_custom_command(
+            OUTPUT
+                "${CMAKE_CURRENT_BINARY_DIR}/${crate}.h"
+                "${CARGO_INCLUDE_DIR}/cxxbridge/${crate}/src/lib.rs.cc"
+                "${CARGO_LIB_DIR}/${CMAKE_STATIC_LIBRARY_PREFIX}${crate}${CMAKE_STATIC_LIBRARY_SUFFIX}"
+            COMMAND
+                "MACOSX_DEPLOYMENT_TARGET=${CMAKE_OSX_DEPLOYMENT_TARGET}"
+                "CARGO_TARGET_DIR=${CMAKE_CURRENT_BINARY_DIR}/target"
+                ${CARGO}
+                build
+                "-q"
+                "--profile" ${CARGO_PROFILE}
+                "--manifest-path" ${CARGO_MANIFEST_PATH}
+                "-p" ${crate}
+            COMMAND
+                cp "${CARGO_INCLUDE_DIR}/cxxbridge/${crate}/src/lib.rs.h" "${CMAKE_CURRENT_BINARY_DIR}/${crate}.h"
+            USES_TERMINAL
+            DEPENDS
+                ${CMAKE_CURRENT_SOURCE_DIR}/${crate}/src/lib.rs
path: test/support/oxidize/CMakeLists.txt
user: login: rroelke
body: I did foresee that problem but decided it was a problem for later :)

I imagined that we could use a glob here.  From what I'm reading now, the `cmake` documentation doesn't really recommend this because it doesn't pick up files that are added, but there is an [easy workaround to that](https://www.reddit.com/r/cmake/comments/17j3fmu/comment/k717tlm/).  This workaround is a little more obfuscated so perhaps it is better not to bother.

If we can list files per crate I'm fine with that, but I would like to avoid a single monolithic list that would trigger a re-build of all of the crates
created_at: 2025-04-29T17:17:37Z
author_association: CONTRIBUTOR
diff_hunk: @@ -46,6 +46,11 @@ on:
         description: 'Enable Address Sanitizer'
         required: false
         type: boolean
+      rust:
+        default: false
+        description: 'Enable Rust'
+        required: false
+        type: boolean
path: .github/workflows/ci-linux_mac.yml
user: login: rroelke
body: I don't think I added it for anything yet other than the standalone unit tests, since it wouldn't do anything yet except for those
created_at: 2025-04-29T17:20:04Z
author_association: CONTRIBUTOR
diff_hunk: @@ -115,4 +115,6 @@ commence(object_library mem_helpers)
   this_target_object_libraries(baseline)
 conclude(object_library)
 
+add_subdirectory(oxidize)
path: test/support/CMakeLists.txt
user: login: rroelke
body: There's quite a bit of unit test code under `tiledb` that references test code living in `test` so there is some back and forth.

I was picturing two workspaces, `test/support/oxidized` in addition to `tiledb/oxidized`

I suppose we could have one workspace with `tiledb/oxidized/tiledb` and `tiledb/oxidized/test` sub-directories. It would be nice to have the dependencies compiled just once
created_at: 2025-04-29T17:22:19Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,89 @@
+//! Exports capabilities from the Rust standard library
+//! for use in C++ code.
+
+#[cxx::bridge]
+mod ffi {
+    #[namespace = "rust::std"]
path: test/support/oxidize/arithmetic/src/lib.rs
user: login: rroelke
body: Oops, this was also transitional, I intended to rename this to `rust::arithmetic`
created_at: 2025-04-29T17:23:15Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,79 @@
+set(TILEDB_CRATES
+    arithmetic
+)
+
+if (NOT TILEDB_TESTS)
+  message(FATAL_ERROR "Building Rust without tests is not supported.")
+endif()
+
+find_program(CARGO cargo)
+
+if (CMAKE_BUILD_TYPE STREQUAL "Debug")
+    set(CARGO_PROFILE "dev")
+    set(CARGO_TARGET "debug")
+else ()
+    set(CARGO_PROFILE "release")
+    set(CARGO_TARGET "release")
+endif ()
+
+set(CARGO_MANIFEST_PATH "${CMAKE_CURRENT_SOURCE_DIR}/Cargo.toml")
+
+set(CARGO_INCLUDE_DIR "${CMAKE_CURRENT_BINARY_DIR}/target")
+set(CARGO_LIB_DIR "${CMAKE_CURRENT_BINARY_DIR}/target/${CARGO_TARGET}")
+
+if (TILEDB_RUST)
+    foreach(crate IN LISTS TILEDB_CRATES)
path: test/support/oxidize/CMakeLists.txt
user: login: davisp
body: Fair enough. I tend to just build all the tests regardless but I see your point.
created_at: 2025-04-29T17:24:08Z
author_association: CONTRIBUTOR
diff_hunk: @@ -64,7 +65,7 @@ void DoubleDelta::compress(
     case Datatype::BLOB:
     case Datatype::GEOM_WKB:
     case Datatype::GEOM_WKT:
-      return DoubleDelta::compress<std::byte>(input_buffer, output_buffer);
+      return DoubleDelta::compress<uint8_t>(input_buffer, output_buffer);
path: tiledb/sm/compressors/dd_compressor.cc
user: login: rroelke
body: Since the first thing that happens to the data is cast to `int64_t`, we don't.  The reason I had to change from `std::byte` is because of some of the template code I added in `tiledb/common/arithmetic.h`.  It doesn't feel right to add a specialization for `std::byte` to those since from what I read `std::byte` is specifically not intended for arithmetic.
created_at: 2025-04-29T17:24:36Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,79 @@
+set(TILEDB_CRATES
+    arithmetic
+)
+
+if (NOT TILEDB_TESTS)
+  message(FATAL_ERROR "Building Rust without tests is not supported.")
+endif()
+
+find_program(CARGO cargo)
+
+if (CMAKE_BUILD_TYPE STREQUAL "Debug")
+    set(CARGO_PROFILE "dev")
+    set(CARGO_TARGET "debug")
+else ()
+    set(CARGO_PROFILE "release")
+    set(CARGO_TARGET "release")
+endif ()
+
+set(CARGO_MANIFEST_PATH "${CMAKE_CURRENT_SOURCE_DIR}/Cargo.toml")
+
+set(CARGO_INCLUDE_DIR "${CMAKE_CURRENT_BINARY_DIR}/target")
+set(CARGO_LIB_DIR "${CMAKE_CURRENT_BINARY_DIR}/target/${CARGO_TARGET}")
+
+if (TILEDB_RUST)
+    foreach(crate IN LISTS TILEDB_CRATES)
path: test/support/oxidize/CMakeLists.txt
user: login: davisp
body: Also, it occurs to me that as long as everything shares a single workspace we should still benefit from not recompiling dependencies. I bet we could turn this into a function call as well and then our crate list would look something like:

```cmake
add_crate(
    NAME
        arithmetic
    SOURCES
        src/lib.rs
        src/utils.rs
)
```
created_at: 2025-04-29T17:26:14Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,15 @@
+[package]
+name = "arithmetic"
+edition.workspace = true
+rust-version.workspace = true
+version.workspace = true
+
+[dependencies]
+cxx = { workspace = true }
path: test/support/oxidize/arithmetic/Cargo.toml
user: login: rroelke
body: Hmm, I actually prefer the `{ workspace = true }` style. I am sure we both agree that they should be consistent however!
created_at: 2025-04-29T17:27:58Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,79 @@
+set(TILEDB_CRATES
+    arithmetic
+)
+
+if (NOT TILEDB_TESTS)
+  message(FATAL_ERROR "Building Rust without tests is not supported.")
+endif()
+
+find_program(CARGO cargo)
+
+if (CMAKE_BUILD_TYPE STREQUAL "Debug")
+    set(CARGO_PROFILE "dev")
+    set(CARGO_TARGET "debug")
+else ()
+    set(CARGO_PROFILE "release")
+    set(CARGO_TARGET "release")
+endif ()
+
+set(CARGO_MANIFEST_PATH "${CMAKE_CURRENT_SOURCE_DIR}/Cargo.toml")
+
+set(CARGO_INCLUDE_DIR "${CMAKE_CURRENT_BINARY_DIR}/target")
+set(CARGO_LIB_DIR "${CMAKE_CURRENT_BINARY_DIR}/target/${CARGO_TARGET}")
+
+if (TILEDB_RUST)
+    foreach(crate IN LISTS TILEDB_CRATES)
+        add_custom_command(
+            OUTPUT
+                "${CMAKE_CURRENT_BINARY_DIR}/${crate}.h"
+                "${CARGO_INCLUDE_DIR}/cxxbridge/${crate}/src/lib.rs.cc"
+                "${CARGO_LIB_DIR}/${CMAKE_STATIC_LIBRARY_PREFIX}${crate}${CMAKE_STATIC_LIBRARY_SUFFIX}"
+            COMMAND
+                "MACOSX_DEPLOYMENT_TARGET=${CMAKE_OSX_DEPLOYMENT_TARGET}"
+                "CARGO_TARGET_DIR=${CMAKE_CURRENT_BINARY_DIR}/target"
+                ${CARGO}
+                build
+                "-q"
+                "--profile" ${CARGO_PROFILE}
+                "--manifest-path" ${CARGO_MANIFEST_PATH}
+                "-p" ${crate}
+            COMMAND
+                cp "${CARGO_INCLUDE_DIR}/cxxbridge/${crate}/src/lib.rs.h" "${CMAKE_CURRENT_BINARY_DIR}/${crate}.h"
path: test/support/oxidize/CMakeLists.txt
user: login: davisp
body: Ah, I didn't see the rename part until you pointed it out which is fair as the `#include "path/to/lib.rs.h"` is a bit awkward.

However, for cross platform support, you'll probably want to remove this `COMMAND` and replace it with the [`file` function](https://cmake.org/cmake/help/latest/command/file.html).
created_at: 2025-04-29T17:33:35Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,79 @@
+set(TILEDB_CRATES
+    arithmetic
+)
+
+if (NOT TILEDB_TESTS)
+  message(FATAL_ERROR "Building Rust without tests is not supported.")
+endif()
+
+find_program(CARGO cargo)
+
+if (CMAKE_BUILD_TYPE STREQUAL "Debug")
+    set(CARGO_PROFILE "dev")
+    set(CARGO_TARGET "debug")
+else ()
+    set(CARGO_PROFILE "release")
+    set(CARGO_TARGET "release")
+endif ()
+
+set(CARGO_MANIFEST_PATH "${CMAKE_CURRENT_SOURCE_DIR}/Cargo.toml")
+
+set(CARGO_INCLUDE_DIR "${CMAKE_CURRENT_BINARY_DIR}/target")
+set(CARGO_LIB_DIR "${CMAKE_CURRENT_BINARY_DIR}/target/${CARGO_TARGET}")
+
+if (TILEDB_RUST)
+    foreach(crate IN LISTS TILEDB_CRATES)
+        add_custom_command(
+            OUTPUT
+                "${CMAKE_CURRENT_BINARY_DIR}/${crate}.h"
+                "${CARGO_INCLUDE_DIR}/cxxbridge/${crate}/src/lib.rs.cc"
+                "${CARGO_LIB_DIR}/${CMAKE_STATIC_LIBRARY_PREFIX}${crate}${CMAKE_STATIC_LIBRARY_SUFFIX}"
+            COMMAND
+                "MACOSX_DEPLOYMENT_TARGET=${CMAKE_OSX_DEPLOYMENT_TARGET}"
+                "CARGO_TARGET_DIR=${CMAKE_CURRENT_BINARY_DIR}/target"
+                ${CARGO}
+                build
+                "-q"
+                "--profile" ${CARGO_PROFILE}
+                "--manifest-path" ${CARGO_MANIFEST_PATH}
+                "-p" ${crate}
+            COMMAND
+                cp "${CARGO_INCLUDE_DIR}/cxxbridge/${crate}/src/lib.rs.h" "${CMAKE_CURRENT_BINARY_DIR}/${crate}.h"
+            USES_TERMINAL
+            DEPENDS
+                ${CMAKE_CURRENT_SOURCE_DIR}/${crate}/src/lib.rs
path: test/support/oxidize/CMakeLists.txt
user: login: davisp
body: I attempted the glob approach before switching back to the list of files. I didn't look too hard into it, but using the glob list appeared to unconditionally rerun Cargo on every build rather than only when source files changed.
created_at: 2025-04-29T17:40:57Z
author_association: CONTRIBUTOR
diff_hunk: @@ -115,4 +115,6 @@ commence(object_library mem_helpers)
   this_target_object_libraries(baseline)
 conclude(object_library)
 
+add_subdirectory(oxidize)
path: test/support/CMakeLists.txt
user: login: davisp
body: My main priority is just that all Rust code exists in a single workspace for the dependency building reason. Beyond that I don't mind too much other than I'd find it rather odd to be shipping things that are built in the test directory even though there's no real reason that'd be an issue.
created_at: 2025-04-29T17:46:11Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,15 @@
+[package]
+name = "arithmetic"
+edition.workspace = true
+rust-version.workspace = true
+version.workspace = true
+
+[dependencies]
+cxx = { workspace = true }
path: test/support/oxidize/arithmetic/Cargo.toml
user: login: davisp
body: Definitely one or the other. My preference for `crate.workspace = true` is that I'm mulling over writing a lint that asserts that every crate dependency is exactly `crate.workspace = true` without any other modifications. We have a few places where we have features defined in the `Cargo.toml` of some crates which means that building different sets of crates causes different feature combinations which causes needless rebuilding when switching between crates with `-p`.
created_at: 2025-04-29T17:48:59Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,89 @@
+//! Exports capabilities from the Rust standard library
+//! for use in C++ code.
+
+#[cxx::bridge]
+mod ffi {
+    #[namespace = "rust::std"]
path: test/support/oxidize/arithmetic/src/lib.rs
user: login: davisp
body: I'd prefer to have `tiledb` in the namespace somewhere to indicate that its code we're exposing. Though feel free to do a `using rust = tiledb::rust` or similar.
created_at: 2025-04-29T17:49:53Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,79 @@
+set(TILEDB_CRATES
+    arithmetic
+)
+
+if (NOT TILEDB_TESTS)
+  message(FATAL_ERROR "Building Rust without tests is not supported.")
+endif()
+
+find_program(CARGO cargo)
+
+if (CMAKE_BUILD_TYPE STREQUAL "Debug")
+    set(CARGO_PROFILE "dev")
+    set(CARGO_TARGET "debug")
+else ()
+    set(CARGO_PROFILE "release")
+    set(CARGO_TARGET "release")
+endif ()
+
+set(CARGO_MANIFEST_PATH "${CMAKE_CURRENT_SOURCE_DIR}/Cargo.toml")
+
+set(CARGO_INCLUDE_DIR "${CMAKE_CURRENT_BINARY_DIR}/target")
+set(CARGO_LIB_DIR "${CMAKE_CURRENT_BINARY_DIR}/target/${CARGO_TARGET}")
+
+if (TILEDB_RUST)
+    foreach(crate IN LISTS TILEDB_CRATES)
+        add_custom_command(
+            OUTPUT
+                "${CMAKE_CURRENT_BINARY_DIR}/${crate}.h"
+                "${CARGO_INCLUDE_DIR}/cxxbridge/${crate}/src/lib.rs.cc"
+                "${CARGO_LIB_DIR}/${CMAKE_STATIC_LIBRARY_PREFIX}${crate}${CMAKE_STATIC_LIBRARY_SUFFIX}"
+            COMMAND
+                "MACOSX_DEPLOYMENT_TARGET=${CMAKE_OSX_DEPLOYMENT_TARGET}"
+                "CARGO_TARGET_DIR=${CMAKE_CURRENT_BINARY_DIR}/target"
+                ${CARGO}
+                build
+                "-q"
+                "--profile" ${CARGO_PROFILE}
+                "--manifest-path" ${CARGO_MANIFEST_PATH}
+                "-p" ${crate}
+            COMMAND
+                cp "${CARGO_INCLUDE_DIR}/cxxbridge/${crate}/src/lib.rs.h" "${CMAKE_CURRENT_BINARY_DIR}/${crate}.h"
+            USES_TERMINAL
+            DEPENDS
+                ${CMAKE_CURRENT_SOURCE_DIR}/${crate}/src/lib.rs
+            WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}
+        )
+    endforeach()
+else ()
+    foreach(crate IN LISTS TILEDB_CRATES)
+        add_custom_command(
+            OUTPUT
+                "${CMAKE_CURRENT_BINARY_DIR}/${crate}.h"
+                "${CARGO_INCLUDE_DIR}/cxxbridge/${crate}/src/lib.rs.cc"
+                "${CARGO_LIB_DIR}/${CMAKE_STATIC_LIBRARY_PREFIX}${crate}${CMAKE_STATIC_LIBRARY_SUFFIX}"
+            COMMAND
+                echo "This build is not configured with Rust support enabled."
+            COMMAND
+                echo "To build this target, configure with '-DTILEDB_RUST=ON'."
+            COMMAND
+                exit 1
+        )
+    endforeach()
path: test/support/oxidize/CMakeLists.txt
user: login: rroelke
body: I was thinking about what should happen if someone wrote a target which depends on a Rust crate, but then ran a CI job which has `not TILEDB_RUST`.  This build should certainly fail.  Without an expansion like this it would probably fail with "no object library arithmetic_rs` or something, but I was imagining that a message saying "Hey, you can't build this target without `TILEDB_RUST` more explicitly would be better.

Given that it didn't pan out this way maybe I shouldn't bother.
created_at: 2025-04-29T18:00:35Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,79 @@
+set(TILEDB_CRATES
+    arithmetic
+)
+
+if (NOT TILEDB_TESTS)
+  message(FATAL_ERROR "Building Rust without tests is not supported.")
+endif()
+
+find_program(CARGO cargo)
+
+if (CMAKE_BUILD_TYPE STREQUAL "Debug")
+    set(CARGO_PROFILE "dev")
+    set(CARGO_TARGET "debug")
+else ()
+    set(CARGO_PROFILE "release")
+    set(CARGO_TARGET "release")
+endif ()
+
+set(CARGO_MANIFEST_PATH "${CMAKE_CURRENT_SOURCE_DIR}/Cargo.toml")
+
+set(CARGO_INCLUDE_DIR "${CMAKE_CURRENT_BINARY_DIR}/target")
+set(CARGO_LIB_DIR "${CMAKE_CURRENT_BINARY_DIR}/target/${CARGO_TARGET}")
+
+if (TILEDB_RUST)
+    foreach(crate IN LISTS TILEDB_CRATES)
path: test/support/oxidize/CMakeLists.txt
user: login: rroelke
body: 46503afdf9122d688821b40a4ca79e0c4c76eb6a
created_at: 2025-04-29T20:17:05Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,304 @@
+/**
+ * @file unit_arithmetic.cc
+ *
+ * @section LICENSE
+ *
+ * The MIT License
+ *
+ * @copyright Copyright (c) 2025 TileDB, Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to deal
+ * in the Software without restriction, including without limitation the rights
+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+ * copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+ * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+ * THE SOFTWARE.
+ *
+ * @section DESCRIPTION
+ *
+ * This file tests arithmetic functions defined in `arithmetic.h`.
+ */
+
+#include "build/test/support/oxidize/arithmetic.h"
+#include "tiledb/common/arithmetic.h"
+
+#include <test/support/assert_helpers.h>
+#include <test/support/tdb_catch.h>
+#include <test/support/tdb_rapidcheck.h>
+
+using namespace tiledb::common;
+
+TEST_CASE(
+    "Arithmetic checked_arithmetic uint32_t add", "[arithmetic][rapidcheck]") {
+  auto doit = []<typename Asserter = tiledb::test::AsserterCatch>(
+                  uint32_t a, uint32_t b)
+                  ->std::optional<uint32_t> {
+    const auto cpp = checked_arithmetic<uint32_t>::add(a, b);
+    std::optional<uint32_t> rs;
+    {
+      uint32_t value;
+      if (rust::std::u32_checked_add(a, b, value)) {
+        rs.emplace(value);
+      }
+    }
+    ASSERTER(cpp == rs);
+
+    return cpp;
+  };
+
+  SECTION("Rapidcheck") {
+    rc::prop(
+        "checked_arithmetic<uint32_t>::add", [doit](uint32_t a, uint32_t b) {
+          doit.operator()<tiledb::test::AsserterRapidcheck>(a, b);
+        });
path: tiledb/common/test/unit_arithmetic.cc
user: login: rroelke
body: My assumption is that function calls across the FFI boundary will not be inlined where as the C++ template ones would be
created_at: 2025-04-29T20:18:25Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,304 @@
+/**
+ * @file unit_arithmetic.cc
+ *
+ * @section LICENSE
+ *
+ * The MIT License
+ *
+ * @copyright Copyright (c) 2025 TileDB, Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to deal
+ * in the Software without restriction, including without limitation the rights
+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+ * copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+ * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+ * THE SOFTWARE.
+ *
+ * @section DESCRIPTION
+ *
+ * This file tests arithmetic functions defined in `arithmetic.h`.
+ */
+
+#include "build/test/support/oxidize/arithmetic.h"
+#include "tiledb/common/arithmetic.h"
+
+#include <test/support/assert_helpers.h>
+#include <test/support/tdb_catch.h>
+#include <test/support/tdb_rapidcheck.h>
+
+using namespace tiledb::common;
+
+TEST_CASE(
+    "Arithmetic checked_arithmetic uint32_t add", "[arithmetic][rapidcheck]") {
+  auto doit = []<typename Asserter = tiledb::test::AsserterCatch>(
+                  uint32_t a, uint32_t b)
+                  ->std::optional<uint32_t> {
+    const auto cpp = checked_arithmetic<uint32_t>::add(a, b);
+    std::optional<uint32_t> rs;
+    {
+      uint32_t value;
+      if (rust::std::u32_checked_add(a, b, value)) {
+        rs.emplace(value);
+      }
+    }
+    ASSERTER(cpp == rs);
+
+    return cpp;
+  };
+
+  SECTION("Rapidcheck") {
+    rc::prop(
+        "checked_arithmetic<uint32_t>::add", [doit](uint32_t a, uint32_t b) {
+          doit.operator()<tiledb::test::AsserterRapidcheck>(a, b);
+        });
+  }
+}
+
+TEST_CASE(
+    "Arithmetic checked_arithmetic int32_t add", "[arithmetic][rapidcheck]") {
+  auto doit =
+      []<typename Asserter = tiledb::test::AsserterCatch>(int32_t a, int32_t b)
+          ->std::optional<int32_t> {
+    const auto cpp = checked_arithmetic<int32_t>::add(a, b);
+    std::optional<int32_t> rs;
+    {
+      int32_t value;
+      if (rust::std::i32_checked_add(a, b, value)) {
path: tiledb/common/test/unit_arithmetic.cc
user: login: rroelke
body: [This](https://github.com/dtolnay/cxx/issues/87#issuecomment-2677287152) was a pretty interesting comment about this
created_at: 2025-04-29T20:20:33Z
author_association: CONTRIBUTOR
diff_hunk: @@ -134,10 +136,10 @@ class DoubleDelta {
    * @tparam The datatype of the values.
    * @param in The input buffer.
    * @param num The number of values in the buffer.
-   * @param bitsize The bitsize of the double deltas to be retrieved.
+   * @return The bitsize of the double deltas to be retrieved.
    */
   template <class T>
-  static void compute_bitsize(T* in, uint64_t num, unsigned int* bitsize);
+  static unsigned compute_bitsize(const T* in, uint64_t num);
path: tiledb/sm/compressors/dd_compressor.h
user: login: rroelke
body: 503d2dd3a1278faa6050fe352294fed0299f78cb
created_at: 2025-04-29T20:22:02Z
author_association: CONTRIBUTOR
diff_hunk: @@ -257,35 +257,54 @@ void DoubleDelta::compress(ConstBuffer* input_buffer, Buffer* output_buffer) {
 }
 
 template <class T>
-void DoubleDelta::compute_bitsize(T* in, uint64_t num, unsigned int* bitsize) {
+unsigned DoubleDelta::compute_bitsize(const T* in, uint64_t num) {
   // Find maximum absolute double delta
-  *bitsize = 0;
-  int64_t max = 0;
   if (num <= 2) {
-    *bitsize = 0;
-    return;
-  }
-  int64_t prev_delta = int64_t(in[1]) - int64_t(in[0]);
-  char delta_out_of_bounds = 0;
-  for (uint64_t i = 2; i < num; ++i) {
-    int64_t cur_delta = int64_t(in[i]) - int64_t(in[i - 1]);
-    int64_t dd = cur_delta - prev_delta;
-    delta_out_of_bounds |= (char)(cur_delta < 0 && prev_delta > 0 && dd > 0);
-    delta_out_of_bounds |= (char)(cur_delta > 0 && prev_delta < 0 && dd < 0);
-    max = std::max(std::abs(dd), max);
-    prev_delta = cur_delta;
+    return 0;
   }
-  // Handle error
-  if (delta_out_of_bounds) {
-    throw DoubleDeltaException(
-        "Cannot compress with DoubleDelta; Some negative double delta is out "
-        "of bounds");
+
+  using DeltaType = common::integral_64<T>::type;
+
+  uint64_t max_delta = 0;
+  std::optional<int64_t> prev_delta;
+  for (uint64_t i = 1; i < num; ++i) {
+    const DeltaType cur = (DeltaType)(in[i]);
+    const DeltaType prev = (DeltaType)(in[i - 1]);
+
+    const std::optional<int64_t> cur_delta =
+        common::checked_arithmetic<DeltaType>::sub_signed(cur, prev);
+    if (!cur_delta.has_value()) {
+      throw DoubleDeltaException(
+          "Cannot compress with DoubleDelta: delta exceeds range of int64_t");
+    }
+
+    if (prev_delta.has_value()) {
+      const std::optional<int64_t> dd =
+          common::checked_arithmetic<int64_t>::sub(
+              cur_delta.value(), prev_delta.value());
+      if (!dd.has_value()) {
+        throw DoubleDeltaException(
+            "Cannot compress with DoubleDelta; Some negative double delta is "
+            "out "
+            "of bounds");
+      }
+      max_delta =
+          std::max(static_cast<uint64_t>(std::abs(dd.value())), max_delta);
+    } else {
+      max_delta = std::abs(cur_delta.value());
+    }
+
+    prev_delta = cur_delta.value();
   }
+
   // Calculate bitsize of the maximum absolute double delta
+  unsigned bitsize = 0;
path: tiledb/sm/compressors/dd_compressor.cc
user: login: rroelke
body: I do and just saw this now, will update.
created_at: 2025-04-29T20:22:23Z
author_association: CONTRIBUTOR
diff_hunk: @@ -257,35 +257,54 @@ void DoubleDelta::compress(ConstBuffer* input_buffer, Buffer* output_buffer) {
 }
 
 template <class T>
-void DoubleDelta::compute_bitsize(T* in, uint64_t num, unsigned int* bitsize) {
+unsigned DoubleDelta::compute_bitsize(const T* in, uint64_t num) {
   // Find maximum absolute double delta
-  *bitsize = 0;
-  int64_t max = 0;
   if (num <= 2) {
-    *bitsize = 0;
-    return;
-  }
-  int64_t prev_delta = int64_t(in[1]) - int64_t(in[0]);
-  char delta_out_of_bounds = 0;
-  for (uint64_t i = 2; i < num; ++i) {
-    int64_t cur_delta = int64_t(in[i]) - int64_t(in[i - 1]);
-    int64_t dd = cur_delta - prev_delta;
-    delta_out_of_bounds |= (char)(cur_delta < 0 && prev_delta > 0 && dd > 0);
-    delta_out_of_bounds |= (char)(cur_delta > 0 && prev_delta < 0 && dd < 0);
-    max = std::max(std::abs(dd), max);
-    prev_delta = cur_delta;
+    return 0;
   }
-  // Handle error
-  if (delta_out_of_bounds) {
-    throw DoubleDeltaException(
-        "Cannot compress with DoubleDelta; Some negative double delta is out "
-        "of bounds");
+
+  using DeltaType = common::integral_64<T>::type;
+
+  uint64_t max_delta = 0;
+  std::optional<int64_t> prev_delta;
+  for (uint64_t i = 1; i < num; ++i) {
+    const DeltaType cur = (DeltaType)(in[i]);
+    const DeltaType prev = (DeltaType)(in[i - 1]);
+
+    const std::optional<int64_t> cur_delta =
+        common::checked_arithmetic<DeltaType>::sub_signed(cur, prev);
+    if (!cur_delta.has_value()) {
+      throw DoubleDeltaException(
+          "Cannot compress with DoubleDelta: delta exceeds range of int64_t");
+    }
+
+    if (prev_delta.has_value()) {
+      const std::optional<int64_t> dd =
+          common::checked_arithmetic<int64_t>::sub(
+              cur_delta.value(), prev_delta.value());
+      if (!dd.has_value()) {
+        throw DoubleDeltaException(
+            "Cannot compress with DoubleDelta; Some negative double delta is "
+            "out "
+            "of bounds");
+      }
+      max_delta =
+          std::max(static_cast<uint64_t>(std::abs(dd.value())), max_delta);
+    } else {
+      max_delta = std::abs(cur_delta.value());
+    }
+
+    prev_delta = cur_delta.value();
   }
+
   // Calculate bitsize of the maximum absolute double delta
+  unsigned bitsize = 0;
path: tiledb/sm/compressors/dd_compressor.cc
user: login: rroelke
body: a0690b53a825b0dd230937c4af8f5ff7fee3c3db
created_at: 2025-04-29T20:23:16Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,15 @@
+[package]
+name = "arithmetic"
+edition.workspace = true
+rust-version.workspace = true
+version.workspace = true
+
+[dependencies]
+cxx = { workspace = true }
path: test/support/oxidize/arithmetic/Cargo.toml
user: login: rroelke
body: c18a374a5f5051deaea66c5549fe824531c85e92
created_at: 2025-04-29T20:23:44Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,79 @@
+set(TILEDB_CRATES
+    arithmetic
+)
+
+if (NOT TILEDB_TESTS)
+  message(FATAL_ERROR "Building Rust without tests is not supported.")
+endif()
+
+find_program(CARGO cargo)
+
+if (CMAKE_BUILD_TYPE STREQUAL "Debug")
+    set(CARGO_PROFILE "dev")
+    set(CARGO_TARGET "debug")
+else ()
+    set(CARGO_PROFILE "release")
+    set(CARGO_TARGET "release")
+endif ()
+
+set(CARGO_MANIFEST_PATH "${CMAKE_CURRENT_SOURCE_DIR}/Cargo.toml")
+
+set(CARGO_INCLUDE_DIR "${CMAKE_CURRENT_BINARY_DIR}/target")
+set(CARGO_LIB_DIR "${CMAKE_CURRENT_BINARY_DIR}/target/${CARGO_TARGET}")
+
+if (TILEDB_RUST)
+    foreach(crate IN LISTS TILEDB_CRATES)
+        add_custom_command(
+            OUTPUT
+                "${CMAKE_CURRENT_BINARY_DIR}/${crate}.h"
+                "${CARGO_INCLUDE_DIR}/cxxbridge/${crate}/src/lib.rs.cc"
+                "${CARGO_LIB_DIR}/${CMAKE_STATIC_LIBRARY_PREFIX}${crate}${CMAKE_STATIC_LIBRARY_SUFFIX}"
+            COMMAND
+                "MACOSX_DEPLOYMENT_TARGET=${CMAKE_OSX_DEPLOYMENT_TARGET}"
+                "CARGO_TARGET_DIR=${CMAKE_CURRENT_BINARY_DIR}/target"
+                ${CARGO}
+                build
+                "-q"
+                "--profile" ${CARGO_PROFILE}
+                "--manifest-path" ${CARGO_MANIFEST_PATH}
+                "-p" ${crate}
+            COMMAND
+                cp "${CARGO_INCLUDE_DIR}/cxxbridge/${crate}/src/lib.rs.h" "${CMAKE_CURRENT_BINARY_DIR}/${crate}.h"
+            USES_TERMINAL
+            DEPENDS
+                ${CMAKE_CURRENT_SOURCE_DIR}/${crate}/src/lib.rs
path: test/support/oxidize/CMakeLists.txt
user: login: rroelke
body: 46503afdf9122d688821b40a4ca79e0c4c76eb6a implements the `add_crate` function with a manual list of source files. I think that should be sufficient to resolve this
created_at: 2025-04-29T20:24:42Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,79 @@
+set(TILEDB_CRATES
+    arithmetic
+)
+
+if (NOT TILEDB_TESTS)
+  message(FATAL_ERROR "Building Rust without tests is not supported.")
+endif()
+
+find_program(CARGO cargo)
+
+if (CMAKE_BUILD_TYPE STREQUAL "Debug")
+    set(CARGO_PROFILE "dev")
+    set(CARGO_TARGET "debug")
+else ()
+    set(CARGO_PROFILE "release")
+    set(CARGO_TARGET "release")
+endif ()
+
+set(CARGO_MANIFEST_PATH "${CMAKE_CURRENT_SOURCE_DIR}/Cargo.toml")
+
+set(CARGO_INCLUDE_DIR "${CMAKE_CURRENT_BINARY_DIR}/target")
+set(CARGO_LIB_DIR "${CMAKE_CURRENT_BINARY_DIR}/target/${CARGO_TARGET}")
+
+if (TILEDB_RUST)
+    foreach(crate IN LISTS TILEDB_CRATES)
+        add_custom_command(
+            OUTPUT
+                "${CMAKE_CURRENT_BINARY_DIR}/${crate}.h"
+                "${CARGO_INCLUDE_DIR}/cxxbridge/${crate}/src/lib.rs.cc"
+                "${CARGO_LIB_DIR}/${CMAKE_STATIC_LIBRARY_PREFIX}${crate}${CMAKE_STATIC_LIBRARY_SUFFIX}"
+            COMMAND
+                "MACOSX_DEPLOYMENT_TARGET=${CMAKE_OSX_DEPLOYMENT_TARGET}"
+                "CARGO_TARGET_DIR=${CMAKE_CURRENT_BINARY_DIR}/target"
+                ${CARGO}
+                build
+                "-q"
+                "--profile" ${CARGO_PROFILE}
+                "--manifest-path" ${CARGO_MANIFEST_PATH}
+                "-p" ${crate}
+            COMMAND
+                cp "${CARGO_INCLUDE_DIR}/cxxbridge/${crate}/src/lib.rs.h" "${CMAKE_CURRENT_BINARY_DIR}/${crate}.h"
path: test/support/oxidize/CMakeLists.txt
user: login: rroelke
body: fa19829d71de25fc8e649d7abfd027f84909e53d
created_at: 2025-04-29T20:25:00Z
author_association: CONTRIBUTOR
diff_hunk: @@ -115,4 +115,6 @@ commence(object_library mem_helpers)
   this_target_object_libraries(baseline)
 conclude(object_library)
 
+add_subdirectory(oxidize)
path: test/support/CMakeLists.txt
user: login: rroelke
body: 11e4103ac616e10a5dcba009107c040855d614f2 moves to repository root `oxidize` directory
created_at: 2025-04-30T01:50:42Z
author_association: CONTRIBUTOR
diff_hunk: @@ -38,3 +38,25 @@ jobs:
 
       - name: Check formatting
         run: $GITHUB_WORKSPACE/scripts/ci/check_formatting_linux.sh
+
+  rust-lint:
+    runs-on: ubuntu-latest
+    if: ${{ startsWith(github.ref , 'refs/tags') != true && startsWith(github.ref , 'build-') != true }}
+    timeout-minutes: 90
+    name: Check Rust Format
+    steps:
+      - name: Install Rust Stable
path: .github/workflows/check-formatting.yml
user: login: rroelke
body: 20280b42ad5223145ecc86a8f0f2484f469e9425 hopefully fixes this...
created_at: 2025-04-30T02:00:24Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,79 @@
+set(TILEDB_CRATES
+    arithmetic
+)
+
+if (NOT TILEDB_TESTS)
+  message(FATAL_ERROR "Building Rust without tests is not supported.")
path: test/support/oxidize/CMakeLists.txt
user: login: rroelke
body: Looking at your original branch it was the same way there.  But I'm not sure we need this at all?  If there's no tests then I expect there just aren't any dependencies on these targets.

I removed this in 5fae3b8f75fa07f5860f80dcd16dce538c9bfb00 but we can add something back if we do need it
created_at: 2025-04-30T02:02:26Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,89 @@
+//! Exports capabilities from the Rust standard library
+//! for use in C++ code.
+
+#[cxx::bridge]
+mod ffi {
+    #[namespace = "rust::std"]
path: test/support/oxidize/arithmetic/src/lib.rs
user: login: rroelke
body: Changed to `tiledb::rust::arithmetic` in 7e9cd08462d0734ac0d67eb5edae0238758b7360
created_at: 2025-04-30T02:02:55Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,18 @@
+[workspace]
+resolver = "2"
+members = [
+  "arithmetic"
+]
+default-members = [
path: test/support/oxidize/Cargo.toml
user: login: rroelke
body: d186de573142633fcf265b2a5a7c31d8afb7b050
created_at: 2025-04-30T15:39:33Z
author_association: CONTRIBUTOR
diff_hunk: @@ -35,6 +35,8 @@
 
 #include "tiledb/common/common.h"
 
+#include <type_traits>
+
path: tiledb/sm/compressors/dd_compressor.h
user: login: teo-tsirpanis
body: Why is this needed?
created_at: 2025-05-05T11:17:11Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,251 @@
+/**
+ * @file   arithmetic.h
+ *
+ * @section LICENSE
+ *
+ * The MIT License
+ *
+ * @copyright Copyright (c) 2025 TileDB, Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to deal
+ * in the Software without restriction, including without limitation the rights
+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+ * copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+ * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+ * THE SOFTWARE.
+ *
+ * @section DESCRIPTION
+ *
+ * This file defines common non-primitive arithmetic functions.
+ */
+#ifndef TILEDB_ARITHMETIC_H
+#define TILEDB_ARITHMETIC_H
+
+#include "tiledb/common/unreachable.h"
+
+#include <cstdint>
+#include <limits>
+#include <optional>
+
+namespace tiledb::common {
+
+/**
+ * Provides the member typedef `type` which is the 64-bit integral type
+ * with the same signed-ness as the integral type `T`,
+ * if `T` itself is narrower than 64 bits.
+ */
+template <typename T>
+struct extended_integral_type;
+
+template <>
+struct extended_integral_type<int8_t> {
+  using type = int64_t;
+};
+
+template <>
+struct extended_integral_type<int16_t> {
+  using type = int64_t;
+};
+
+template <>
+struct extended_integral_type<int32_t> {
+  using type = int64_t;
+};
+
+template <>
+struct extended_integral_type<uint8_t> {
+  using type = uint64_t;
+};
+
+template <>
+struct extended_integral_type<uint16_t> {
+  using type = uint64_t;
+};
+
+template <>
+struct extended_integral_type<uint32_t> {
+  using type = uint64_t;
+};
+
path: tiledb/common/arithmetic.h
user: login: teo-tsirpanis
body: ```suggestion

template <class T> using extended_integral_type_t = typename extended_integral_type<T>::type;

```

Using `extended_integral_type_t` will slightly shorten some code below.
created_at: 2025-05-05T11:20:15Z
author_association: MEMBER
diff_hunk: @@ -257,35 +257,54 @@ void DoubleDelta::compress(ConstBuffer* input_buffer, Buffer* output_buffer) {
 }
 
 template <class T>
-void DoubleDelta::compute_bitsize(T* in, uint64_t num, unsigned int* bitsize) {
+uint32_t DoubleDelta::compute_bitsize(const T* in, uint64_t num) {
   // Find maximum absolute double delta
-  *bitsize = 0;
-  int64_t max = 0;
   if (num <= 2) {
-    *bitsize = 0;
-    return;
-  }
-  int64_t prev_delta = int64_t(in[1]) - int64_t(in[0]);
-  char delta_out_of_bounds = 0;
-  for (uint64_t i = 2; i < num; ++i) {
-    int64_t cur_delta = int64_t(in[i]) - int64_t(in[i - 1]);
-    int64_t dd = cur_delta - prev_delta;
-    delta_out_of_bounds |= (char)(cur_delta < 0 && prev_delta > 0 && dd > 0);
-    delta_out_of_bounds |= (char)(cur_delta > 0 && prev_delta < 0 && dd < 0);
-    max = std::max(std::abs(dd), max);
-    prev_delta = cur_delta;
+    return 0;
   }
-  // Handle error
-  if (delta_out_of_bounds) {
-    throw DoubleDeltaException(
-        "Cannot compress with DoubleDelta; Some negative double delta is out "
-        "of bounds");
+
+  using DeltaType = common::integral_64<T>::type;
+
+  uint64_t max_delta = 0;
+  std::optional<int64_t> prev_delta;
+  for (uint64_t i = 1; i < num; ++i) {
+    const DeltaType cur = (DeltaType)(in[i]);
+    const DeltaType prev = (DeltaType)(in[i - 1]);
+
+    const std::optional<int64_t> cur_delta =
+        common::checked_arithmetic<DeltaType>::sub_signed(cur, prev);
+    if (!cur_delta.has_value()) {
+      throw DoubleDeltaException(
+          "Cannot compress with DoubleDelta: delta exceeds range of int64_t");
+    }
+
+    if (prev_delta.has_value()) {
+      const std::optional<int64_t> dd =
+          common::checked_arithmetic<int64_t>::sub(
+              cur_delta.value(), prev_delta.value());
+      if (!dd.has_value()) {
+        throw DoubleDeltaException(
+            "Cannot compress with DoubleDelta; Some negative double delta is "
+            "out "
+            "of bounds");
+      }
path: tiledb/sm/compressors/dd_compressor.cc
user: login: teo-tsirpanis
body: Can't we wrap around the result? It is weird for a filter to be unable to process some inputs.
created_at: 2025-05-05T11:34:01Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,251 @@
+/**
+ * @file   arithmetic.h
+ *
+ * @section LICENSE
+ *
+ * The MIT License
+ *
+ * @copyright Copyright (c) 2025 TileDB, Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to deal
+ * in the Software without restriction, including without limitation the rights
+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+ * copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+ * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+ * THE SOFTWARE.
+ *
+ * @section DESCRIPTION
+ *
+ * This file defines common non-primitive arithmetic functions.
+ */
+#ifndef TILEDB_ARITHMETIC_H
+#define TILEDB_ARITHMETIC_H
+
+#include "tiledb/common/unreachable.h"
+
+#include <cstdint>
+#include <limits>
+#include <optional>
+
+namespace tiledb::common {
+
+/**
+ * Provides the member typedef `type` which is the 64-bit integral type
+ * with the same signed-ness as the integral type `T`,
+ * if `T` itself is narrower than 64 bits.
+ */
+template <typename T>
+struct extended_integral_type;
+
+template <>
+struct extended_integral_type<int8_t> {
+  using type = int64_t;
+};
+
+template <>
+struct extended_integral_type<int16_t> {
+  using type = int64_t;
+};
+
+template <>
+struct extended_integral_type<int32_t> {
+  using type = int64_t;
+};
+
+template <>
+struct extended_integral_type<uint8_t> {
+  using type = uint64_t;
+};
+
+template <>
+struct extended_integral_type<uint16_t> {
+  using type = uint64_t;
+};
+
+template <>
+struct extended_integral_type<uint32_t> {
+  using type = uint64_t;
+};
+
path: tiledb/common/arithmetic.h
user: login: rroelke
body: 756c11d
created_at: 2025-05-06T14:18:33Z
author_association: CONTRIBUTOR
diff_hunk: @@ -35,6 +35,8 @@
 
 #include "tiledb/common/common.h"
 
+#include <type_traits>
+
path: tiledb/sm/compressors/dd_compressor.h
user: login: rroelke
body: Vestigial from an intermediate state. Removed e212f5f5
created_at: 2025-05-06T14:19:05Z
author_association: CONTRIBUTOR
diff_hunk: @@ -257,35 +257,54 @@ void DoubleDelta::compress(ConstBuffer* input_buffer, Buffer* output_buffer) {
 }
 
 template <class T>
-void DoubleDelta::compute_bitsize(T* in, uint64_t num, unsigned int* bitsize) {
+uint32_t DoubleDelta::compute_bitsize(const T* in, uint64_t num) {
   // Find maximum absolute double delta
-  *bitsize = 0;
-  int64_t max = 0;
   if (num <= 2) {
-    *bitsize = 0;
-    return;
-  }
-  int64_t prev_delta = int64_t(in[1]) - int64_t(in[0]);
-  char delta_out_of_bounds = 0;
-  for (uint64_t i = 2; i < num; ++i) {
-    int64_t cur_delta = int64_t(in[i]) - int64_t(in[i - 1]);
-    int64_t dd = cur_delta - prev_delta;
-    delta_out_of_bounds |= (char)(cur_delta < 0 && prev_delta > 0 && dd > 0);
-    delta_out_of_bounds |= (char)(cur_delta > 0 && prev_delta < 0 && dd < 0);
-    max = std::max(std::abs(dd), max);
-    prev_delta = cur_delta;
+    return 0;
   }
-  // Handle error
-  if (delta_out_of_bounds) {
-    throw DoubleDeltaException(
-        "Cannot compress with DoubleDelta; Some negative double delta is out "
-        "of bounds");
+
+  using DeltaType = common::integral_64<T>::type;
+
+  uint64_t max_delta = 0;
+  std::optional<int64_t> prev_delta;
+  for (uint64_t i = 1; i < num; ++i) {
+    const DeltaType cur = (DeltaType)(in[i]);
+    const DeltaType prev = (DeltaType)(in[i - 1]);
+
+    const std::optional<int64_t> cur_delta =
+        common::checked_arithmetic<DeltaType>::sub_signed(cur, prev);
+    if (!cur_delta.has_value()) {
+      throw DoubleDeltaException(
+          "Cannot compress with DoubleDelta: delta exceeds range of int64_t");
+    }
+
+    if (prev_delta.has_value()) {
+      const std::optional<int64_t> dd =
+          common::checked_arithmetic<int64_t>::sub(
+              cur_delta.value(), prev_delta.value());
+      if (!dd.has_value()) {
+        throw DoubleDeltaException(
+            "Cannot compress with DoubleDelta; Some negative double delta is "
+            "out "
+            "of bounds");
+      }
path: tiledb/sm/compressors/dd_compressor.cc
user: login: rroelke
body: There is precedent here in the previous code.
```
  // Handle error
  if (delta_out_of_bounds) {
    throw DoubleDeltaException(
        "Cannot compress with DoubleDelta; Some negative double delta is out "
        "of bounds");
  }
```
i.e. it's not a new notion that this filter cannot process all input data.
created_at: 2025-05-06T14:21:38Z
author_association: CONTRIBUTOR
diff_hunk: @@ -257,35 +257,54 @@ void DoubleDelta::compress(ConstBuffer* input_buffer, Buffer* output_buffer) {
 }
 
 template <class T>
-void DoubleDelta::compute_bitsize(T* in, uint64_t num, unsigned int* bitsize) {
+uint32_t DoubleDelta::compute_bitsize(const T* in, uint64_t num) {
   // Find maximum absolute double delta
-  *bitsize = 0;
-  int64_t max = 0;
   if (num <= 2) {
-    *bitsize = 0;
-    return;
-  }
-  int64_t prev_delta = int64_t(in[1]) - int64_t(in[0]);
-  char delta_out_of_bounds = 0;
-  for (uint64_t i = 2; i < num; ++i) {
-    int64_t cur_delta = int64_t(in[i]) - int64_t(in[i - 1]);
-    int64_t dd = cur_delta - prev_delta;
-    delta_out_of_bounds |= (char)(cur_delta < 0 && prev_delta > 0 && dd > 0);
-    delta_out_of_bounds |= (char)(cur_delta > 0 && prev_delta < 0 && dd < 0);
-    max = std::max(std::abs(dd), max);
-    prev_delta = cur_delta;
+    return 0;
   }
-  // Handle error
-  if (delta_out_of_bounds) {
-    throw DoubleDeltaException(
-        "Cannot compress with DoubleDelta; Some negative double delta is out "
-        "of bounds");
+
+  using DeltaType = common::integral_64<T>::type;
+
+  uint64_t max_delta = 0;
+  std::optional<int64_t> prev_delta;
+  for (uint64_t i = 1; i < num; ++i) {
+    const DeltaType cur = (DeltaType)(in[i]);
+    const DeltaType prev = (DeltaType)(in[i - 1]);
+
+    const std::optional<int64_t> cur_delta =
+        common::checked_arithmetic<DeltaType>::sub_signed(cur, prev);
+    if (!cur_delta.has_value()) {
+      throw DoubleDeltaException(
+          "Cannot compress with DoubleDelta: delta exceeds range of int64_t");
+    }
+
+    if (prev_delta.has_value()) {
+      const std::optional<int64_t> dd =
+          common::checked_arithmetic<int64_t>::sub(
+              cur_delta.value(), prev_delta.value());
+      if (!dd.has_value()) {
+        throw DoubleDeltaException(
+            "Cannot compress with DoubleDelta; Some negative double delta is "
+            "out "
+            "of bounds");
+      }
path: tiledb/sm/compressors/dd_compressor.cc
user: login: teo-tsirpanis
body: ~~Can we fix it now?~~ Maybe not, it might require a format version bump. 😩
created_at: 2025-05-06T16:17:56Z
author_association: MEMBER
diff_hunk: @@ -257,35 +257,54 @@ void DoubleDelta::compress(ConstBuffer* input_buffer, Buffer* output_buffer) {
 }
 
 template <class T>
-void DoubleDelta::compute_bitsize(T* in, uint64_t num, unsigned int* bitsize) {
+uint32_t DoubleDelta::compute_bitsize(const T* in, uint64_t num) {
   // Find maximum absolute double delta
-  *bitsize = 0;
-  int64_t max = 0;
   if (num <= 2) {
-    *bitsize = 0;
-    return;
-  }
-  int64_t prev_delta = int64_t(in[1]) - int64_t(in[0]);
-  char delta_out_of_bounds = 0;
-  for (uint64_t i = 2; i < num; ++i) {
-    int64_t cur_delta = int64_t(in[i]) - int64_t(in[i - 1]);
-    int64_t dd = cur_delta - prev_delta;
-    delta_out_of_bounds |= (char)(cur_delta < 0 && prev_delta > 0 && dd > 0);
-    delta_out_of_bounds |= (char)(cur_delta > 0 && prev_delta < 0 && dd < 0);
-    max = std::max(std::abs(dd), max);
-    prev_delta = cur_delta;
+    return 0;
   }
-  // Handle error
-  if (delta_out_of_bounds) {
-    throw DoubleDeltaException(
-        "Cannot compress with DoubleDelta; Some negative double delta is out "
-        "of bounds");
+
+  using DeltaType = common::integral_64<T>::type;
+
+  uint64_t max_delta = 0;
+  std::optional<int64_t> prev_delta;
+  for (uint64_t i = 1; i < num; ++i) {
+    const DeltaType cur = (DeltaType)(in[i]);
+    const DeltaType prev = (DeltaType)(in[i - 1]);
+
+    const std::optional<int64_t> cur_delta =
+        common::checked_arithmetic<DeltaType>::sub_signed(cur, prev);
+    if (!cur_delta.has_value()) {
+      throw DoubleDeltaException(
+          "Cannot compress with DoubleDelta: delta exceeds range of int64_t");
+    }
+
+    if (prev_delta.has_value()) {
+      const std::optional<int64_t> dd =
+          common::checked_arithmetic<int64_t>::sub(
+              cur_delta.value(), prev_delta.value());
+      if (!dd.has_value()) {
+        throw DoubleDeltaException(
+            "Cannot compress with DoubleDelta; Some negative double delta is "
+            "out "
+            "of bounds");
+      }
path: tiledb/sm/compressors/dd_compressor.cc
user: login: teo-tsirpanis
body: But since we already change the logic (especially in the bit width reduction filter), a format version bump might be warranted either way, and we should seize the opportunity to get it right.
created_at: 2025-05-06T17:12:32Z
author_association: MEMBER
diff_hunk: @@ -257,35 +257,54 @@ void DoubleDelta::compress(ConstBuffer* input_buffer, Buffer* output_buffer) {
 }
 
 template <class T>
-void DoubleDelta::compute_bitsize(T* in, uint64_t num, unsigned int* bitsize) {
+uint32_t DoubleDelta::compute_bitsize(const T* in, uint64_t num) {
   // Find maximum absolute double delta
-  *bitsize = 0;
-  int64_t max = 0;
   if (num <= 2) {
-    *bitsize = 0;
-    return;
-  }
-  int64_t prev_delta = int64_t(in[1]) - int64_t(in[0]);
-  char delta_out_of_bounds = 0;
-  for (uint64_t i = 2; i < num; ++i) {
-    int64_t cur_delta = int64_t(in[i]) - int64_t(in[i - 1]);
-    int64_t dd = cur_delta - prev_delta;
-    delta_out_of_bounds |= (char)(cur_delta < 0 && prev_delta > 0 && dd > 0);
-    delta_out_of_bounds |= (char)(cur_delta > 0 && prev_delta < 0 && dd < 0);
-    max = std::max(std::abs(dd), max);
-    prev_delta = cur_delta;
+    return 0;
   }
-  // Handle error
-  if (delta_out_of_bounds) {
-    throw DoubleDeltaException(
-        "Cannot compress with DoubleDelta; Some negative double delta is out "
-        "of bounds");
+
+  using DeltaType = common::integral_64<T>::type;
+
+  uint64_t max_delta = 0;
+  std::optional<int64_t> prev_delta;
+  for (uint64_t i = 1; i < num; ++i) {
+    const DeltaType cur = (DeltaType)(in[i]);
+    const DeltaType prev = (DeltaType)(in[i - 1]);
+
+    const std::optional<int64_t> cur_delta =
+        common::checked_arithmetic<DeltaType>::sub_signed(cur, prev);
+    if (!cur_delta.has_value()) {
+      throw DoubleDeltaException(
+          "Cannot compress with DoubleDelta: delta exceeds range of int64_t");
+    }
+
+    if (prev_delta.has_value()) {
+      const std::optional<int64_t> dd =
+          common::checked_arithmetic<int64_t>::sub(
+              cur_delta.value(), prev_delta.value());
+      if (!dd.has_value()) {
+        throw DoubleDeltaException(
+            "Cannot compress with DoubleDelta; Some negative double delta is "
+            "out "
+            "of bounds");
+      }
path: tiledb/sm/compressors/dd_compressor.cc
user: login: rroelke
body: RE: format version bump

I think we don't actually need this since we changed only the forward filter logic. Since un-filtering is not changed, any bytes that were written by prior versions of this filter will be un-filtered in the same way.

RE: passing data which hits the overflow through the filter

We should not bother implementing this.
1) this filter is designed to be used with offsets of similar-size strings, where it gets the best encoding results.  It is meant to be used at the beginning of the filter pipeline on non-descending non-negative integers.  It doesn't really matter how it behaves on other streams because that's not where it provides business value.
2) if there is an arithmetic overflow then the double delta produces a value with a high bit width which it will then use to encode each of the deltas in the stream.  This will result in data which is not compacted relative to the input (and is possibly larger) but also requires additional CPU overhead to decode.  Hence streams which have this overflow in them are better off not applying this encoding at all.
3) You could perhaps work around (2) by using windows (as the bit width reduction filter does).  But this will produce worse results against the target use case of (1), whether it would be a worse encoding due to window padding, or a performance hit to identify a dynamic window size, or both.
created_at: 2025-05-06T17:49:36Z
author_association: CONTRIBUTOR
diff_hunk: @@ -257,35 +257,54 @@ void DoubleDelta::compress(ConstBuffer* input_buffer, Buffer* output_buffer) {
 }
 
 template <class T>
-void DoubleDelta::compute_bitsize(T* in, uint64_t num, unsigned int* bitsize) {
+uint32_t DoubleDelta::compute_bitsize(const T* in, uint64_t num) {
   // Find maximum absolute double delta
-  *bitsize = 0;
-  int64_t max = 0;
   if (num <= 2) {
-    *bitsize = 0;
-    return;
-  }
-  int64_t prev_delta = int64_t(in[1]) - int64_t(in[0]);
-  char delta_out_of_bounds = 0;
-  for (uint64_t i = 2; i < num; ++i) {
-    int64_t cur_delta = int64_t(in[i]) - int64_t(in[i - 1]);
-    int64_t dd = cur_delta - prev_delta;
-    delta_out_of_bounds |= (char)(cur_delta < 0 && prev_delta > 0 && dd > 0);
-    delta_out_of_bounds |= (char)(cur_delta > 0 && prev_delta < 0 && dd < 0);
-    max = std::max(std::abs(dd), max);
-    prev_delta = cur_delta;
+    return 0;
   }
-  // Handle error
-  if (delta_out_of_bounds) {
-    throw DoubleDeltaException(
-        "Cannot compress with DoubleDelta; Some negative double delta is out "
-        "of bounds");
+
+  using DeltaType = common::integral_64<T>::type;
+
+  uint64_t max_delta = 0;
+  std::optional<int64_t> prev_delta;
+  for (uint64_t i = 1; i < num; ++i) {
+    const DeltaType cur = (DeltaType)(in[i]);
+    const DeltaType prev = (DeltaType)(in[i - 1]);
+
+    const std::optional<int64_t> cur_delta =
+        common::checked_arithmetic<DeltaType>::sub_signed(cur, prev);
+    if (!cur_delta.has_value()) {
+      throw DoubleDeltaException(
+          "Cannot compress with DoubleDelta: delta exceeds range of int64_t");
+    }
+
+    if (prev_delta.has_value()) {
+      const std::optional<int64_t> dd =
+          common::checked_arithmetic<int64_t>::sub(
+              cur_delta.value(), prev_delta.value());
+      if (!dd.has_value()) {
+        throw DoubleDeltaException(
+            "Cannot compress with DoubleDelta; Some negative double delta is "
+            "out "
+            "of bounds");
+      }
path: tiledb/sm/compressors/dd_compressor.cc
user: login: teo-tsirpanis
body: > It doesn't really matter how it behaves on other streams because that's not where it provides business value.

I don't believe filters have the "right" to reject input, even if they are documented to work on special forms of data. Not hard-failing when writing data does arguably have business value, and I believe somebody will inevitably hit this error one day, and fixing it then would be more urgent than fixing it now.

> This will result in data which is not compacted relative to the input (and is possibly larger) but also requires additional CPU overhead to decode.

This is not an unusual thing with filters; if you compress high-entropy data with zstd, you will not get size savings but the operation will still succeed, which is the improtant thing IMO.

> You could perhaps work around (2) by using windows (as the bit width reduction filter does).

We already split data before filtering them, both at the tile and the chunk level (I'm not very happy that the BWR filter does it again but that's another story) so the impact of filtering locally suboptimal data is not globalized over the whole fragment or tile.
created_at: 2025-05-06T18:10:41Z
author_association: MEMBER
diff_hunk: @@ -257,35 +257,54 @@ void DoubleDelta::compress(ConstBuffer* input_buffer, Buffer* output_buffer) {
 }
 
 template <class T>
-void DoubleDelta::compute_bitsize(T* in, uint64_t num, unsigned int* bitsize) {
+uint32_t DoubleDelta::compute_bitsize(const T* in, uint64_t num) {
   // Find maximum absolute double delta
-  *bitsize = 0;
-  int64_t max = 0;
   if (num <= 2) {
-    *bitsize = 0;
-    return;
-  }
-  int64_t prev_delta = int64_t(in[1]) - int64_t(in[0]);
-  char delta_out_of_bounds = 0;
-  for (uint64_t i = 2; i < num; ++i) {
-    int64_t cur_delta = int64_t(in[i]) - int64_t(in[i - 1]);
-    int64_t dd = cur_delta - prev_delta;
-    delta_out_of_bounds |= (char)(cur_delta < 0 && prev_delta > 0 && dd > 0);
-    delta_out_of_bounds |= (char)(cur_delta > 0 && prev_delta < 0 && dd < 0);
-    max = std::max(std::abs(dd), max);
-    prev_delta = cur_delta;
+    return 0;
   }
-  // Handle error
-  if (delta_out_of_bounds) {
-    throw DoubleDeltaException(
-        "Cannot compress with DoubleDelta; Some negative double delta is out "
-        "of bounds");
+
+  using DeltaType = common::integral_64<T>::type;
+
+  uint64_t max_delta = 0;
+  std::optional<int64_t> prev_delta;
+  for (uint64_t i = 1; i < num; ++i) {
+    const DeltaType cur = (DeltaType)(in[i]);
+    const DeltaType prev = (DeltaType)(in[i - 1]);
+
+    const std::optional<int64_t> cur_delta =
+        common::checked_arithmetic<DeltaType>::sub_signed(cur, prev);
+    if (!cur_delta.has_value()) {
+      throw DoubleDeltaException(
+          "Cannot compress with DoubleDelta: delta exceeds range of int64_t");
+    }
+
+    if (prev_delta.has_value()) {
+      const std::optional<int64_t> dd =
+          common::checked_arithmetic<int64_t>::sub(
+              cur_delta.value(), prev_delta.value());
+      if (!dd.has_value()) {
+        throw DoubleDeltaException(
+            "Cannot compress with DoubleDelta; Some negative double delta is "
+            "out "
+            "of bounds");
+      }
path: tiledb/sm/compressors/dd_compressor.cc
user: login: rroelke
body: > Not hard-failing when writing data does arguably have business value, and I believe somebody will inevitably hit this error one day, and fixing it then would be more urgent than fixing it now

If someone hits this error when using the double delta filter then that's a good sign that they should not be using the double delta filter, and if this does show up in the field then that's the guidance we should provide.

(I'll note that for the story in question the double delta was actually fine, the problem was the overflow detection in the bit width reduction filter applied after the double delta filter... another example of something that shouldn't really be done)

> This is not an unusual thing with filters

It certainly *can* happen, but it's not desirable, and what we're discussing here amounts to a request to enable it.


With respect to this PR it's all moot.  The unresolved question here is about the format version.

> I think we don't actually need this since we changed only the forward filter logic. Since un-filtering is not changed, any bytes that were written by prior versions of this filter will be un-filtered in the same way.

Do you agree?
created_at: 2025-05-06T19:26:01Z
author_association: CONTRIBUTOR
diff_hunk: @@ -257,35 +257,54 @@ void DoubleDelta::compress(ConstBuffer* input_buffer, Buffer* output_buffer) {
 }
 
 template <class T>
-void DoubleDelta::compute_bitsize(T* in, uint64_t num, unsigned int* bitsize) {
+uint32_t DoubleDelta::compute_bitsize(const T* in, uint64_t num) {
   // Find maximum absolute double delta
-  *bitsize = 0;
-  int64_t max = 0;
   if (num <= 2) {
-    *bitsize = 0;
-    return;
-  }
-  int64_t prev_delta = int64_t(in[1]) - int64_t(in[0]);
-  char delta_out_of_bounds = 0;
-  for (uint64_t i = 2; i < num; ++i) {
-    int64_t cur_delta = int64_t(in[i]) - int64_t(in[i - 1]);
-    int64_t dd = cur_delta - prev_delta;
-    delta_out_of_bounds |= (char)(cur_delta < 0 && prev_delta > 0 && dd > 0);
-    delta_out_of_bounds |= (char)(cur_delta > 0 && prev_delta < 0 && dd < 0);
-    max = std::max(std::abs(dd), max);
-    prev_delta = cur_delta;
+    return 0;
   }
-  // Handle error
-  if (delta_out_of_bounds) {
-    throw DoubleDeltaException(
-        "Cannot compress with DoubleDelta; Some negative double delta is out "
-        "of bounds");
+
+  using DeltaType = common::integral_64<T>::type;
+
+  uint64_t max_delta = 0;
+  std::optional<int64_t> prev_delta;
+  for (uint64_t i = 1; i < num; ++i) {
+    const DeltaType cur = (DeltaType)(in[i]);
+    const DeltaType prev = (DeltaType)(in[i - 1]);
+
+    const std::optional<int64_t> cur_delta =
+        common::checked_arithmetic<DeltaType>::sub_signed(cur, prev);
+    if (!cur_delta.has_value()) {
+      throw DoubleDeltaException(
+          "Cannot compress with DoubleDelta: delta exceeds range of int64_t");
+    }
+
+    if (prev_delta.has_value()) {
+      const std::optional<int64_t> dd =
+          common::checked_arithmetic<int64_t>::sub(
+              cur_delta.value(), prev_delta.value());
+      if (!dd.has_value()) {
+        throw DoubleDeltaException(
+            "Cannot compress with DoubleDelta; Some negative double delta is "
+            "out "
+            "of bounds");
+      }
path: tiledb/sm/compressors/dd_compressor.cc
user: login: teo-tsirpanis
body: I agree.
created_at: 2025-05-06T20:27:59Z
author_association: MEMBER
diff_hunk: @@ -257,35 +257,54 @@ void DoubleDelta::compress(ConstBuffer* input_buffer, Buffer* output_buffer) {
 }
 
 template <class T>
-void DoubleDelta::compute_bitsize(T* in, uint64_t num, unsigned int* bitsize) {
+uint32_t DoubleDelta::compute_bitsize(const T* in, uint64_t num) {
   // Find maximum absolute double delta
-  *bitsize = 0;
-  int64_t max = 0;
   if (num <= 2) {
-    *bitsize = 0;
-    return;
-  }
-  int64_t prev_delta = int64_t(in[1]) - int64_t(in[0]);
-  char delta_out_of_bounds = 0;
-  for (uint64_t i = 2; i < num; ++i) {
-    int64_t cur_delta = int64_t(in[i]) - int64_t(in[i - 1]);
-    int64_t dd = cur_delta - prev_delta;
-    delta_out_of_bounds |= (char)(cur_delta < 0 && prev_delta > 0 && dd > 0);
-    delta_out_of_bounds |= (char)(cur_delta > 0 && prev_delta < 0 && dd < 0);
-    max = std::max(std::abs(dd), max);
-    prev_delta = cur_delta;
+    return 0;
   }
-  // Handle error
-  if (delta_out_of_bounds) {
-    throw DoubleDeltaException(
-        "Cannot compress with DoubleDelta; Some negative double delta is out "
-        "of bounds");
+
+  using DeltaType = common::integral_64<T>::type;
+
+  uint64_t max_delta = 0;
+  std::optional<int64_t> prev_delta;
+  for (uint64_t i = 1; i < num; ++i) {
+    const DeltaType cur = (DeltaType)(in[i]);
+    const DeltaType prev = (DeltaType)(in[i - 1]);
+
+    const std::optional<int64_t> cur_delta =
+        common::checked_arithmetic<DeltaType>::sub_signed(cur, prev);
+    if (!cur_delta.has_value()) {
+      throw DoubleDeltaException(
+          "Cannot compress with DoubleDelta: delta exceeds range of int64_t");
+    }
+
+    if (prev_delta.has_value()) {
+      const std::optional<int64_t> dd =
+          common::checked_arithmetic<int64_t>::sub(
+              cur_delta.value(), prev_delta.value());
+      if (!dd.has_value()) {
+        throw DoubleDeltaException(
+            "Cannot compress with DoubleDelta; Some negative double delta is "
+            "out "
+            "of bounds");
path: tiledb/sm/compressors/dd_compressor.cc
user: login: davisp
body: Seems like an odd line break here.
created_at: 2025-05-08T14:02:30Z
author_association: CONTRIBUTOR
diff_hunk: @@ -257,35 +257,54 @@ void DoubleDelta::compress(ConstBuffer* input_buffer, Buffer* output_buffer) {
 }
 
 template <class T>
-void DoubleDelta::compute_bitsize(T* in, uint64_t num, unsigned int* bitsize) {
+uint32_t DoubleDelta::compute_bitsize(const T* in, uint64_t num) {
   // Find maximum absolute double delta
-  *bitsize = 0;
-  int64_t max = 0;
   if (num <= 2) {
-    *bitsize = 0;
-    return;
-  }
-  int64_t prev_delta = int64_t(in[1]) - int64_t(in[0]);
-  char delta_out_of_bounds = 0;
-  for (uint64_t i = 2; i < num; ++i) {
-    int64_t cur_delta = int64_t(in[i]) - int64_t(in[i - 1]);
-    int64_t dd = cur_delta - prev_delta;
-    delta_out_of_bounds |= (char)(cur_delta < 0 && prev_delta > 0 && dd > 0);
-    delta_out_of_bounds |= (char)(cur_delta > 0 && prev_delta < 0 && dd < 0);
-    max = std::max(std::abs(dd), max);
-    prev_delta = cur_delta;
+    return 0;
   }
-  // Handle error
-  if (delta_out_of_bounds) {
-    throw DoubleDeltaException(
-        "Cannot compress with DoubleDelta; Some negative double delta is out "
-        "of bounds");
+
+  using DeltaType = common::integral_64<T>::type;
+
+  uint64_t max_delta = 0;
+  std::optional<int64_t> prev_delta;
+  for (uint64_t i = 1; i < num; ++i) {
+    const DeltaType cur = (DeltaType)(in[i]);
+    const DeltaType prev = (DeltaType)(in[i - 1]);
+
+    const std::optional<int64_t> cur_delta =
+        common::checked_arithmetic<DeltaType>::sub_signed(cur, prev);
+    if (!cur_delta.has_value()) {
+      throw DoubleDeltaException(
+          "Cannot compress with DoubleDelta: delta exceeds range of int64_t");
+    }
+
+    if (prev_delta.has_value()) {
+      const std::optional<int64_t> dd =
+          common::checked_arithmetic<int64_t>::sub(
+              cur_delta.value(), prev_delta.value());
+      if (!dd.has_value()) {
+        throw DoubleDeltaException(
+            "Cannot compress with DoubleDelta; Some negative double delta is "
+            "out "
+            "of bounds");
path: tiledb/sm/compressors/dd_compressor.cc
user: login: rroelke
body: Good catch!  I think `clang-format` will split strings up to multiple lines, but not consolidate them back

0c035c8fc
created_at: 2025-05-08T14:37:07Z
author_association: CONTRIBUTOR

PR comments:
user: login: rroelke
created_at: 2025-05-02T15:18:37Z
body: @davisp I removed the Rust components here, after discussing with @ihnorton we agreed that we want the filter bug fix to land in 2.28 if possible but without introducing any Rust build onto that branch.

So I've updated this to just be the overflow detection, filter fixes, and unit tests for those.

Rust will follow once this is merged
author_association: CONTRIBUTOR
user: login: rroelke
created_at: 2025-05-08T14:37:53Z
body: > Not gonna lie, I was a lot more confident when there was a Rust version to check against 

When I have it in me to deal with CI again I'll put up a new PR which adds the Rust tests back
author_association: CONTRIBUTOR