Repository: single-cell-data/TileDB-SOMA
Pull request #4066: [c++/python] Fix domain getter memory leak on C++/Python layer
Author: login: XanthosXanthopoulos
Created: 2025-05-19T18:06:22Z | Merged: 2025-05-21T18:58:37Z

**Issue and/or context:** [SOMA-169](https://linear.app/tiledb/issue/SOMA-169/c-memory-leak-in-somaarray-get-core-domainish)

**Changes:**

**Notes for Reviewer:**



Diff:
diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
index 356455965e..bbea6bee49 100644
--- a/.pre-commit-config.yaml
+++ b/.pre-commit-config.yaml
@@ -24,12 +24,12 @@ repos:
       args: ["--config-file=apis/python/pyproject.toml", "apis/python/src", "apis/python/devtools"]
       pass_filenames: false
 
-  - repo: https://github.com/hukkin/mdformat
-    rev: 0.7.22  # Use the ref you want to point at
-    hooks:
-    - id: mdformat
-      files: .*\.md|^apis/r/vignettes/.*\.Rmd
-      types: [file, text]
-      exclude: ^.github/
-      additional_dependencies:
-        - "mdformat-frontmatter"
+#  - repo: https://github.com/hukkin/mdformat
+#    rev: 0.7.22  # Use the ref you want to point at
+#    hooks:
+#    - id: mdformat
+#      files: .*\.md|^apis/r/vignettes/.*\.Rmd
+#      types: [file, text]
+#      exclude: ^.github/
+#      additional_dependencies:
+#        - "mdformat-frontmatter"
diff --git a/apis/python/HISTORY.md b/apis/python/HISTORY.md
index 0a80256d6f..d90ed84e5e 100644
--- a/apis/python/HISTORY.md
+++ b/apis/python/HISTORY.md
@@ -36,6 +36,7 @@ The primary change in 1.17.0 is the upgrade to TileDB 2.28.
 - \[[#4040](https://github.com/single-cell-data/TileDB-SOMA/pull/4040)\] [python] suppress insignificant overflow warning from numpy.
 - \[[#4050](https://github.com/single-cell-data/TileDB-SOMA/pull/4050)\] DataFrame `count` and SparseNDArray `nnz` fix - report correct number of cells in array in the case where a delete query had been previously applied.
 - \[[#4055](https://github.com/single-cell-data/TileDB-SOMA/pull/4055)\] Various `open()` code paths failed to check the SOMA encoding version number, and would fail with cryptic errors.
+- \[[#4066](https://github.com/single-cell-data/TileDB-SOMA/pull/4066)\] Fix various memory leaks related to releasing Arrow structures when transfering ownership between C++ and Python and vise versa.
 
 ## [Release prior to 1.17.0]
 
diff --git a/apis/python/src/tiledbsoma/soma_array.cc b/apis/python/src/tiledbsoma/soma_array.cc
index 92f2f7ba21..ff5e46783e 100644
--- a/apis/python/src/tiledbsoma/soma_array.cc
+++ b/apis/python/src/tiledbsoma/soma_array.cc
@@ -28,22 +28,17 @@ py::list domainish_to_list(ArrowArray* arrow_array, ArrowSchema* arrow_schema) {
 
     py::list array_list;
     for (int i = 0; i < arrow_array->n_children; i++) {
-        // Note that this runs the release callbacks within the ArrowArray and
-        // the ArrowSchema, freeing memory.
+        // "_import_from_c" implements Arrow move semantics, meaning
+        // release is set to NULL for each array imported. Release
+        // should be called on the parent array.
         auto array = pa_array_import(
             py::capsule(arrow_array->children[i]),
             py::capsule(arrow_schema->children[i]));
         array_list.append(array);
-
-        // Already released: ensure there is no attempt at second free.
-        arrow_array->children[i] = nullptr;
-        arrow_schema->children[i] = nullptr;
     }
-    // Already released: ensure there is no attempt at second free.
-    arrow_array->n_children = 0;
-    arrow_array->children = nullptr;
-    arrow_schema->n_children = 0;
-    arrow_schema->children = nullptr;
+
+    arrow_array->release(arrow_array);
+    arrow_schema->release(arrow_schema);
 
     return array_list;
 }
@@ -140,6 +135,7 @@ void load_soma_array(py::module& m) {
                 auto pa = py::module::import("pyarrow");
                 auto pa_schema_import = pa.attr("Schema").attr(
                     "_import_from_c");
+
                 try {
                     return pa_schema_import(
                         py::capsule(array.arrow_schema().get()));
diff --git a/apis/python/src/tiledbsoma/soma_dataframe.cc b/apis/python/src/tiledbsoma/soma_dataframe.cc
index f1821238c8..a1f51473c2 100644
--- a/apis/python/src/tiledbsoma/soma_dataframe.cc
+++ b/apis/python/src/tiledbsoma/soma_dataframe.cc
@@ -95,6 +95,8 @@ void load_soma_dataframe(py::module& m) {
                     TPY_ERROR_LOC(e.what());
                 }
                 schema.release(&schema);
+                index_column_array.release(&index_column_array);
+                index_column_schema.release(&index_column_schema);
             },
             "uri"_a,
             py::kw_only(),
@@ -137,6 +139,10 @@ void load_soma_dataframe(py::module& m) {
                     py::gil_scoped_acquire acquire;
 
                     auto ncol = t.second->n_children;
+                    ScopedExecutor cleanup([&]() {
+                        t.first->release(t.first.get());
+                        t.second->release(t.second.get());
+                    });
 
                     py::dict retval;
                     for (auto i = 0; i < ncol; i++) {
@@ -152,6 +158,7 @@ void load_soma_dataframe(py::module& m) {
                             py::capsule(column_arrow_schema));
                         retval[py::str(column_name)] = pa_array;
                     }
+
                     return retval;
                 } catch (const std::exception& e) {
                     throw TileDBSOMAError(e.what());
@@ -309,6 +316,11 @@ void load_soma_dataframe(py::module& m) {
                 } catch (const std::exception& e) {
                     throw TileDBSOMAError(e.what());
                 }
+
+                nanoarrow_domain_table.first->release(
+                    nanoarrow_domain_table.first.get());
+                nanoarrow_domain_table.second->release(
+                    nanoarrow_domain_table.second.get());
             },
             "pyarrow_domain_table"_a,
             "function_name_for_messages"_a)
@@ -335,6 +347,11 @@ void load_soma_dataframe(py::module& m) {
                 } catch (const std::exception& e) {
                     throw TileDBSOMAError(e.what());
                 }
+
+                nanoarrow_domain_table.first->release(
+                    nanoarrow_domain_table.first.get());
+                nanoarrow_domain_table.second->release(
+                    nanoarrow_domain_table.second.get());
             },
             "pyarrow_domain_table"_a,
             "function_name_for_messages"_a)
@@ -361,6 +378,11 @@ void load_soma_dataframe(py::module& m) {
                 } catch (const std::exception& e) {
                     throw TileDBSOMAError(e.what());
                 }
+
+                nanoarrow_domain_table.first->release(
+                    nanoarrow_domain_table.first.get());
+                nanoarrow_domain_table.second->release(
+                    nanoarrow_domain_table.second.get());
             },
             "pyarrow_domain_table"_a,
             "function_name_for_messages"_a)
@@ -387,6 +409,11 @@ void load_soma_dataframe(py::module& m) {
                 } catch (const std::exception& e) {
                     throw TileDBSOMAError(e.what());
                 }
+
+                nanoarrow_domain_table.first->release(
+                    nanoarrow_domain_table.first.get());
+                nanoarrow_domain_table.second->release(
+                    nanoarrow_domain_table.second.get());
             },
             "pyarrow_domain_table"_a,
             "function_name_for_messages"_a)
diff --git a/apis/python/src/tiledbsoma/soma_geometry_dataframe.cc b/apis/python/src/tiledbsoma/soma_geometry_dataframe.cc
index f49f6a0628..21f2e65384 100644
--- a/apis/python/src/tiledbsoma/soma_geometry_dataframe.cc
+++ b/apis/python/src/tiledbsoma/soma_geometry_dataframe.cc
@@ -100,7 +100,10 @@ void load_soma_geometry_dataframe(py::module& m) {
                 } catch (const std::exception& e) {
                     TPY_ERROR_LOC(e.what());
                 }
+
                 schema.release(&schema);
+                index_column_array.release(&index_column_array);
+                index_column_schema.release(&index_column_schema);
             },
             "uri"_a,
             py::kw_only(),
diff --git a/apis/python/src/tiledbsoma/soma_point_cloud_dataframe.cc b/apis/python/src/tiledbsoma/soma_point_cloud_dataframe.cc
index 233362f213..e08ae4c96d 100644
--- a/apis/python/src/tiledbsoma/soma_point_cloud_dataframe.cc
+++ b/apis/python/src/tiledbsoma/soma_point_cloud_dataframe.cc
@@ -100,7 +100,10 @@ void load_soma_point_cloud_dataframe(py::module& m) {
                 } catch (const std::exception& e) {
                     TPY_ERROR_LOC(e.what());
                 }
+
                 schema.release(&schema);
+                index_column_array.release(&index_column_array);
+                index_column_schema.release(&index_column_schema);
             },
             "uri"_a,
             py::kw_only(),
diff --git a/apis/python/src/tiledbsoma/transformer.cc b/apis/python/src/tiledbsoma/transformer.cc
index 2424050e81..72d7f09f98 100644
--- a/apis/python/src/tiledbsoma/transformer.cc
+++ b/apis/python/src/tiledbsoma/transformer.cc
@@ -70,6 +70,9 @@ void load_transformers(py::module& m) {
                 array_list.append(pa_array);
             }
 
+            array->release(array.get());
+            schema->release(schema.get());
+
             return pa_table_from_arrays(array_list, names);
         });
 
diff --git a/libtiledbsoma/src/soma/soma_array.cc b/libtiledbsoma/src/soma/soma_array.cc
index 5567e79f89..c2a1f19feb 100644
--- a/libtiledbsoma/src/soma/soma_array.cc
+++ b/libtiledbsoma/src/soma/soma_array.cc
@@ -941,6 +941,8 @@ void SOMAArray::_set_soma_joinid_shape_helper(
         }
 
         new_current_domain.set_ndrectangle(ndrect);
+        soma_domain.first->release(soma_domain.first.get());
+        soma_domain.second->release(soma_domain.second.get());
 
     } else {
         // For resize: copy from the existing current domain except for the
diff --git a/libtiledbsoma/src/soma/soma_geometry_column.cc b/libtiledbsoma/src/soma/soma_geometry_column.cc
index b064965370..b85ad3a2ae 100644
--- a/libtiledbsoma/src/soma/soma_geometry_column.cc
+++ b/libtiledbsoma/src/soma/soma_geometry_column.cc
@@ -480,15 +480,37 @@ std::pair<ArrowArray*, ArrowSchema*> SOMAGeometryColumn::arrow_domain_slot(
     const SOMAContext& ctx, Array& array, enum Domainish kind) const {
     switch (domain_type().value()) {
         case TILEDB_FLOAT64: {
-            auto parent_schema = ArrowAdapter::make_arrow_schema_parent(
-                TDB_DIM_PER_SPATIAL_AXIS, name());
-            auto parent_array = ArrowAdapter::make_arrow_array_parent(
+            ArrowSchema* parent_schema = static_cast<ArrowSchema*>(
+                malloc(sizeof(ArrowSchema)));
+            parent_schema->name = strdup(name().c_str());
+            parent_schema->format = strdup("+s");
+            parent_schema->n_children = static_cast<int64_t>(
                 TDB_DIM_PER_SPATIAL_AXIS);
+            parent_schema->metadata = nullptr;
+            parent_schema->private_data = nullptr;
+            parent_schema->flags = 0;
+            parent_schema->dictionary = nullptr;
+            parent_schema->children = static_cast<ArrowSchema**>(
+                malloc(TDB_DIM_PER_SPATIAL_AXIS * sizeof(ArrowSchema*)));
+            parent_schema->release = &ArrowAdapter::release_schema;
+
+            ArrowArray* parent_array = static_cast<ArrowArray*>(
+                malloc(sizeof(ArrowArray)));
 
             parent_array->length = 2;
+            parent_array->null_count = 0;
+            parent_array->offset = 0;
             parent_array->n_buffers = 1;
+            parent_array->n_children = static_cast<int64_t>(
+                TDB_DIM_PER_SPATIAL_AXIS);
             parent_array->buffers = (const void**)malloc(sizeof(void*));
             parent_array->buffers[0] = nullptr;
+            parent_array->dictionary = nullptr;
+            parent_array->release = &ArrowAdapter::release_array;
+            parent_array->private_data = nullptr;
+
+            parent_array->children = static_cast<ArrowArray**>(
+                malloc(TDB_DIM_PER_SPATIAL_AXIS * sizeof(ArrowArray*)));
 
             auto kind_domain = domain_slot<std::vector<double_t>>(
                 ctx, array, kind);
@@ -516,8 +538,7 @@ std::pair<ArrowArray*, ArrowSchema*> SOMAGeometryColumn::arrow_domain_slot(
                         {kind_domain.first[i], kind_domain.second[i]}));
             }
 
-            return std::make_pair(
-                parent_array.release(), parent_schema.release());
+            return std::make_pair(parent_array, parent_schema);
         } break;
         default:
             throw TileDBSOMAError(fmt::format(
diff --git a/libtiledbsoma/src/utils/arrow_adapter.cc b/libtiledbsoma/src/utils/arrow_adapter.cc
index 8111c591bd..8fbd7bb35d 100644
--- a/libtiledbsoma/src/utils/arrow_adapter.cc
+++ b/libtiledbsoma/src/utils/arrow_adapter.cc
@@ -99,7 +99,7 @@ void ArrowAdapter::release_schema(struct ArrowSchema* schema) {
             LOG_TRACE(fmt::format(
                 "[ArrowAdapter] release_schema name {} schema->dict release",
                 name_for_log));
-            release_schema(schema->dictionary);
+            schema->dictionary->release(schema->dictionary);
         }
         LOG_TRACE(fmt::format(
             "[ArrowAdapter] release_schema name {} schema->dict free",
@@ -536,8 +536,6 @@ ArrowSchema* ArrowAdapter::arrow_schema_from_tiledb_attribute(
         auto enmr = ArrayExperimental::get_enumeration(
             ctx, tiledb_array, *enmr_name);
         auto dict = (ArrowSchema*)malloc(sizeof(ArrowSchema));
-        dict->format = strdup(
-            ArrowAdapter::to_arrow_format(enmr.type(), false).data());
         if (enmr.type() == TILEDB_STRING_ASCII || enmr.type() == TILEDB_CHAR) {
             dict->format = strdup("z");
         } else {


Reviews:
user: login: bkmartinjr
body: Couple of items:
* is this still WIP?  (PR title)
* I built the branch and ran valgrind - still see multiple leaks in this call chain. Left details in a comment

If you were intending to do a PR for each specific leak, LMK and I can re-review. No concerns with the actual changes in this PR.

state: CHANGES_REQUESTED
author_association: MEMBER
submitted_at: 2025-05-19T18:55:31Z
user: login: johnkerl
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-21T13:25:00Z
user: login: bkmartinjr
body: am doing final testing in parallel, but LGTM
state: APPROVED
author_association: MEMBER
submitted_at: 2025-05-21T15:52:27Z

Review comments:
diff_hunk: @@ -28,22 +28,17 @@ py::list domainish_to_list(ArrowArray* arrow_array, ArrowSchema* arrow_schema) {
 
     py::list array_list;
     for (int i = 0; i < arrow_array->n_children; i++) {
-        // Note that this runs the release callbacks within the ArrowArray and
-        // the ArrowSchema, freeing memory.
+        // "_import_from_c" implements Arrow move semantic menaing
path: apis/python/src/tiledbsoma/soma_array.cc
user: login: johnkerl
body: ```suggestion
        // "_import_from_c" implements Arrow move semantics, meaning
```
created_at: 2025-05-21T13:23:41Z
author_association: CONTRIBUTOR
diff_hunk: @@ -28,22 +28,17 @@ py::list domainish_to_list(ArrowArray* arrow_array, ArrowSchema* arrow_schema) {
 
     py::list array_list;
     for (int i = 0; i < arrow_array->n_children; i++) {
-        // Note that this runs the release callbacks within the ArrowArray and
-        // the ArrowSchema, freeing memory.
+        // "_import_from_c" implements Arrow move semantic menaing
+        // release is set to NULL for each array imported. Release
+        // should be called to the parent array
path: apis/python/src/tiledbsoma/soma_array.cc
user: login: johnkerl
body: ```suggestion
        // should be called on the parent array.
```
created_at: 2025-05-21T13:24:16Z
author_association: CONTRIBUTOR

PR comments:
user: login: codecov[bot]
created_at: 2025-05-19T18:25:47Z
body: ## [Codecov](https://app.codecov.io/gh/single-cell-data/TileDB-SOMA/pull/4066?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=single-cell-data) Report
Attention: Patch coverage is `0%` with `25 lines` in your changes missing coverage. Please review.
> Project coverage is 66.01%. Comparing base [(`f29839d`)](https://app.codecov.io/gh/single-cell-data/TileDB-SOMA/commit/f29839d1cd73cf6bdbfc63afd7085d6154ba6c42?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=single-cell-data) to head [(`b38fc54`)](https://app.codecov.io/gh/single-cell-data/TileDB-SOMA/commit/b38fc54c93201e5fe179bb6a9c538cfff2ed17f9?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=single-cell-data).
> Report is 7 commits behind head on main.

<details><summary>Additional details and impacted files</summary>


```diff
@@             Coverage Diff             @@
##             main    #4066       +/-   ##
===========================================
- Coverage   88.97%   66.01%   -22.96%     
===========================================
  Files          59      158       +99     
  Lines        7073    21049    +13976     
  Branches        0     1236     +1236     
===========================================
+ Hits         6293    13895     +7602     
- Misses        780     6742     +5962     
- Partials        0      412      +412     
```

| [Flag](https://app.codecov.io/gh/single-cell-data/TileDB-SOMA/pull/4066/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=single-cell-data) | Coverage Δ | |
|---|---|---|
| [libtiledbsoma](https://app.codecov.io/gh/single-cell-data/TileDB-SOMA/pull/4066/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=single-cell-data) | `54.25% <0.00%> (?)` | |
| [python](https://app.codecov.io/gh/single-cell-data/TileDB-SOMA/pull/4066/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=single-cell-data) | `89.26% <ø> (+0.29%)` | :arrow_up: |

Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=single-cell-data#carryforward-flags-in-the-pull-request-comment) to find out more.

| [Components](https://app.codecov.io/gh/single-cell-data/TileDB-SOMA/pull/4066/components?src=pr&el=components&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=single-cell-data) | Coverage Δ | |
|---|---|---|
| [python_api](https://app.codecov.io/gh/single-cell-data/TileDB-SOMA/pull/4066/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=single-cell-data) | `89.26% <ø> (+0.29%)` | :arrow_up: |
| [libtiledbsoma](https://app.codecov.io/gh/single-cell-data/TileDB-SOMA/pull/4066/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=single-cell-data) | `44.66% <0.00%> (∅)` | |

</details>
<details><summary> :rocket: New features to boost your workflow: </summary>

- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
author_association: NONE
user: login: bkmartinjr
created_at: 2025-05-19T18:52:59Z
body: I built this branch and ran under valgrind.  Am still seeing quite a few leaks in this call chain (i.e., `_get_core_domainish`)

My test case:
> valgrind --max-threads=4096 --leak-check=full pytest -s -x -v apis/python/tests/test_shape.py 

Leaks:
```
==361621== 12 bytes in 4 blocks are definitely lost in loss record 1,151 of 7,631
==361621==    at 0x4848899: malloc (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==361621==    by 0x4A0E58E: strdup (strdup.c:42)
==361621==    by 0x873DDB4D: tiledbsoma::ArrowAdapter::make_arrow_schema_parent(unsigned long, std::basic_string_view<char, std::char_traits<char> >) (in /home/bruce/projects/TileDB-SOMA/dist/lib/libtiledbsoma.so)
==361621==    by 0x8731B0C0: tiledbsoma::SOMAArray::_get_core_domainish(Domainish) (in /home/bruce/projects/TileDB-SOMA/dist/lib/libtiledbsoma.so)
==361621==    by 0x8732A218: tiledbsoma::SOMAArray::_set_soma_joinid_shape_helper(long, bool, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >) (in /home/bruce/projects/TileDB-SOMA/dist/lib/libtiledbsoma.so)
==361621==    by 0x87009C08: pybind11::cpp_function::initialize<libtiledbsomacpp::load_soma_dataframe(pybind11::module_&)::{lambda(tiledbsoma::SOMADataFrame&, long, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >)#6}, void, tiledbsoma::SOMADataFrame&, long, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, pybind11::name, pybind11::is_method, pybind11::sibling, pybind11::arg, pybind11::arg>(libtiledbsomacpp::load_soma_dataframe(pybind11::module_&)::{lambda(tiledbsoma::SOMADataFrame&, long, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >)#6}&&, void (*)(tiledbsoma::SOMADataFrame&, long, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >), pybind11::name const&, pybind11::is_method const&, pybind11::sibling const&, pybind11::arg const&, pybind11::arg const&)::{lambda(pybind11::detail::function_call&)#3}::_FUN(pybind11::detail::function_call&) (in /home/bruce/projects/TileDB-SOMA/apis/python/src/tiledbsoma/pytiledbsoma.cpython-312-x86_64-linux-gnu.so)
==361621==    by 0x86D05613: pybind11::cpp_function::dispatcher(_object*, _object*, _object*) (in /home/bruce/projects/TileDB-SOMA/apis/python/src/tiledbsoma/pytiledbsoma.cpython-312-x86_64-linux-gnu.so)
==361621==    by 0x323D37: cfunction_call (methodobject.c:537)
==361621==    by 0x30726B: _PyObject_MakeTpCall (call.c:240)
==361621==    by 0x21AB4E: _PyEval_EvalFrameDefault.cold (bytecodes.c:2715)
==361621==    by 0x309CF1: _PyObject_FastCallDictTstate (call.c:144)
==361621==    by 0x3327A8: _PyObject_Call_Prepend (call.c:508)

==361621== 28 bytes in 4 blocks are definitely lost in loss record 1,941 of 7,631
==361621==    at 0x4848899: malloc (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==361621==    by 0x4A0E58E: strdup (strdup.c:42)
==361621==    by 0x873DDB5C: tiledbsoma::ArrowAdapter::make_arrow_schema_parent(unsigned long, std::basic_string_view<char, std::char_traits<char> >) (in /home/bruce/projects/TileDB-SOMA/dist/lib/libtiledbsoma.so)
==361621==    by 0x8731B0C0: tiledbsoma::SOMAArray::_get_core_domainish(Domainish) (in /home/bruce/projects/TileDB-SOMA/dist/lib/libtiledbsoma.so)
==361621==    by 0x8732A218: tiledbsoma::SOMAArray::_set_soma_joinid_shape_helper(long, bool, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >) (in /home/bruce/projects/TileDB-SOMA/dist/lib/libtiledbsoma.so)
==361621==    by 0x87009C08: pybind11::cpp_function::initialize<libtiledbsomacpp::load_soma_dataframe(pybind11::module_&)::{lambda(tiledbsoma::SOMADataFrame&, long, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >)#6}, void, tiledbsoma::SOMADataFrame&, long, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, pybind11::name, pybind11::is_method, pybind11::sibling, pybind11::arg, pybind11::arg>(libtiledbsomacpp::load_soma_dataframe(pybind11::module_&)::{lambda(tiledbsoma::SOMADataFrame&, long, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >)#6}&&, void (*)(tiledbsoma::SOMADataFrame&, long, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >), pybind11::name const&, pybind11::is_method const&, pybind11::sibling const&, pybind11::arg const&, pybind11::arg const&)::{lambda(pybind11::detail::function_call&)#3}::_FUN(pybind11::detail::function_call&) (in /home/bruce/projects/TileDB-SOMA/apis/python/src/tiledbsoma/pytiledbsoma.cpython-312-x86_64-linux-gnu.so)
==361621==    by 0x86D05613: pybind11::cpp_function::dispatcher(_object*, _object*, _object*) (in /home/bruce/projects/TileDB-SOMA/apis/python/src/tiledbsoma/pytiledbsoma.cpython-312-x86_64-linux-gnu.so)
==361621==    by 0x323D37: cfunction_call (methodobject.c:537)
==361621==    by 0x30726B: _PyObject_MakeTpCall (call.c:240)
==361621==    by 0x21AB4E: _PyEval_EvalFrameDefault.cold (bytecodes.c:2715)
==361621==    by 0x309CF1: _PyObject_FastCallDictTstate (call.c:144)
==361621==    by 0x3327A8: _PyObject_Call_Prepend (call.c:508)

==361621== 555 (48 direct, 507 indirect) bytes in 4 blocks are definitely lost in loss record 6,649 of 7,631
==361621==    at 0x4848899: malloc (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==361621==    by 0x873DDB85: tiledbsoma::ArrowAdapter::make_arrow_schema_parent(unsigned long, std::basic_string_view<char, std::char_traits<char> >) (in /home/bruce/projects/TileDB-SOMA/dist/lib/libtiledbsoma.so)
==361621==    by 0x8731B0C0: tiledbsoma::SOMAArray::_get_core_domainish(Domainish) (in /home/bruce/projects/TileDB-SOMA/dist/lib/libtiledbsoma.so)
==361621==    by 0x8732A218: tiledbsoma::SOMAArray::_set_soma_joinid_shape_helper(long, bool, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >) (in /home/bruce/projects/TileDB-SOMA/dist/lib/libtiledbsoma.so)
==361621==    by 0x87009C08: pybind11::cpp_function::initialize<libtiledbsomacpp::load_soma_dataframe(pybind11::module_&)::{lambda(tiledbsoma::SOMADataFrame&, long, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >)#6}, void, tiledbsoma::SOMADataFrame&, long, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, pybind11::name, pybind11::is_method, pybind11::sibling, pybind11::arg, pybind11::arg>(libtiledbsomacpp::load_soma_dataframe(pybind11::module_&)::{lambda(tiledbsoma::SOMADataFrame&, long, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >)#6}&&, void (*)(tiledbsoma::SOMADataFrame&, long, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >), pybind11::name const&, pybind11::is_method const&, pybind11::sibling const&, pybind11::arg const&, pybind11::arg const&)::{lambda(pybind11::detail::function_call&)#3}::_FUN(pybind11::detail::function_call&) (in /home/bruce/projects/TileDB-SOMA/apis/python/src/tiledbsoma/pytiledbsoma.cpython-312-x86_64-linux-gnu.so)
==361621==    by 0x86D05613: pybind11::cpp_function::dispatcher(_object*, _object*, _object*) (in /home/bruce/projects/TileDB-SOMA/apis/python/src/tiledbsoma/pytiledbsoma.cpython-312-x86_64-linux-gnu.so)
==361621==    by 0x323D37: cfunction_call (methodobject.c:537)
==361621==    by 0x30726B: _PyObject_MakeTpCall (call.c:240)
==361621==    by 0x21AB4E: _PyEval_EvalFrameDefault.cold (bytecodes.c:2715)
==361621==    by 0x309CF1: _PyObject_FastCallDictTstate (call.c:144)
==361621==    by 0x3327A8: _PyObject_Call_Prepend (call.c:508)
==361621==    by 0x3FF2FA: slot_tp_call (typeobject.c:8791)

==361621== 728 (48 direct, 680 indirect) bytes in 4 blocks are definitely lost in loss record 6,751 of 7,631
==361621==    at 0x4848899: malloc (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==361621==    by 0x873DE03A: tiledbsoma::ArrowAdapter::make_arrow_array_parent(unsigned long) (in /home/bruce/projects/TileDB-SOMA/dist/lib/libtiledbsoma.so)
==361621==    by 0x8731B0CD: tiledbsoma::SOMAArray::_get_core_domainish(Domainish) (in /home/bruce/projects/TileDB-SOMA/dist/lib/libtiledbsoma.so)
==361621==    by 0x8732A218: tiledbsoma::SOMAArray::_set_soma_joinid_shape_helper(long, bool, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >) (in /home/bruce/projects/TileDB-SOMA/dist/lib/libtiledbsoma.so)
==361621==    by 0x87009C08: pybind11::cpp_function::initialize<libtiledbsomacpp::load_soma_dataframe(pybind11::module_&)::{lambda(tiledbsoma::SOMADataFrame&, long, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >)#6}, void, tiledbsoma::SOMADataFrame&, long, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, pybind11::name, pybind11::is_method, pybind11::sibling, pybind11::arg, pybind11::arg>(libtiledbsomacpp::load_soma_dataframe(pybind11::module_&)::{lambda(tiledbsoma::SOMADataFrame&, long, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >)#6}&&, void (*)(tiledbsoma::SOMADataFrame&, long, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >), pybind11::name const&, pybind11::is_method const&, pybind11::sibling const&, pybind11::arg const&, pybind11::arg const&)::{lambda(pybind11::detail::function_call&)#3}::_FUN(pybind11::detail::function_call&) (in /home/bruce/projects/TileDB-SOMA/apis/python/src/tiledbsoma/pytiledbsoma.cpython-312-x86_64-linux-gnu.so)
==361621==    by 0x86D05613: pybind11::cpp_function::dispatcher(_object*, _object*, _object*) (in /home/bruce/projects/TileDB-SOMA/apis/python/src/tiledbsoma/pytiledbsoma.cpython-312-x86_64-linux-gnu.so)
==361621==    by 0x323D37: cfunction_call (methodobject.c:537)
==361621==    by 0x30726B: _PyObject_MakeTpCall (call.c:240)
==361621==    by 0x21AB4E: _PyEval_EvalFrameDefault.cold (bytecodes.c:2715)
==361621==    by 0x309CF1: _PyObject_FastCallDictTstate (call.c:144)
==361621==    by 0x3327A8: _PyObject_Call_Prepend (call.c:508)
==361621==    by 0x3FF2FA: slot_tp_call (typeobject.c:8791)
```
author_association: MEMBER
user: login: johnkerl
created_at: 2025-05-19T20:45:55Z
body: @XanthosXanthopoulos #4067 may help?
author_association: CONTRIBUTOR
user: login: bkmartinjr
created_at: 2025-05-20T14:41:37Z
body: Update after latest commit.  Ran:

> valgrind --max-threads=4096 --leak-check=full pytest -s -x -v apis/python/tests/test_shape.py -k test_canned_experiments[False]

And leaked:

```
==2066522== 4 bytes in 2 blocks are definitely lost in loss record 241 of 7,551
==2066522==    at 0x4848899: malloc (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==2066522==    by 0x4A0E58E: strdup (strdup.c:42)
==2066522==    by 0x873E0309: tiledbsoma::ArrowAdapter::arrow_schema_from_tiledb_attribute(tiledb::Attribute const&, tiledb::Context const&, tiledb::Array const&) (in /home/bruce/projects/TileDB-SOMA/dist/lib/libtiledbsoma.so)
==2066522==    by 0x873579A2: tiledbsoma::SOMAAttribute::arrow_schema_slot(tiledbsoma::SOMAContext const&, tiledb::Array&) const (in /home/bruce/projects/TileDB-SOMA/dist/lib/libtiledbsoma.so)
==2066522==    by 0x86FB7348: libtiledbsomacpp::load_soma_array(pybind11::module_&)::{lambda(tiledbsoma::SOMAArray&)#7}::operator()(tiledbsoma::SOMAArray&) const [clone .constprop.0] (in /home/bruce/projects/TileDB-SOMA/apis/python/src/tiledbsoma/pytiledbsoma.cpython-312-x86_64-linux-gnu.so)
==2066522==    by 0x86FB8452: pybind11::cpp_function::initialize<libtiledbsomacpp::load_soma_array(pybind11::module_&)::{lambda(tiledbsoma::SOMAArray&)#7} const&, pybind11::object, tiledbsoma::SOMAArray&>(libtiledbsomacpp::load_soma_array(pybind11::module_&)::{lambda(tiledbsoma::SOMAArray&)#7} const&, pybind11::object (*)(tiledbsoma::SOMAArray&))::{lambda(pybind11::detail::function_call&)#3}::_FUN(pybind11::detail::function_call&) (in /home/bruce/projects/TileDB-SOMA/apis/python/src/tiledbsoma/pytiledbsoma.cpython-312-x86_64-linux-gnu.so)
==2066522==    by 0x86D055C3: pybind11::cpp_function::dispatcher(_object*, _object*, _object*) (in /home/bruce/projects/TileDB-SOMA/apis/python/src/tiledbsoma/pytiledbsoma.cpython-312-x86_64-linux-gnu.so)
==2066522==    by 0x323D37: cfunction_call (methodobject.c:537)
==2066522==    by 0x30726B: _PyObject_MakeTpCall (call.c:240)
==2066522==    by 0x330577: UnknownInlinedFun (pycore_call.h:90)
==2066522==    by 0x330577: UnknownInlinedFun (pycore_call.h:77)
==2066522==    by 0x330577: PyObject_CallOneArg (call.c:401)
==2066522==    by 0x322A55: _PyObject_GenericGetAttrWithDict (object.c:1447)
==2066522==    by 0x309720: UnknownInlinedFun (object.c:1538)


==2066522== 9 bytes in 3 blocks are definitely lost in loss record 742 of 7,551
==2066522==    at 0x4848899: malloc (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==2066522==    by 0x4A0E58E: strdup (strdup.c:42)
==2066522==    by 0x873DDB4D: tiledbsoma::ArrowAdapter::make_arrow_schema_parent(unsigned long, std::basic_string_view<char, std::char_traits<char> >) (in /home/bruce/projects/TileDB-SOMA/dist/lib/libtiledbsoma.so)
==2066522==    by 0x8731B0C0: tiledbsoma::SOMAArray::_get_core_domainish(Domainish) (in /home/bruce/projects/TileDB-SOMA/dist/lib/libtiledbsoma.so)
==2066522==    by 0x8732A218: tiledbsoma::SOMAArray::_set_soma_joinid_shape_helper(long, bool, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >) (in /home/bruce/projects/TileDB-SOMA/dist/lib/libtiledbsoma.so)
==2066522==    by 0x87009678: pybind11::cpp_function::initialize<libtiledbsomacpp::load_soma_dataframe(pybind11::module_&)::{lambda(tiledbsoma::SOMADataFrame&, long, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >)#6}, void, tiledbsoma::SOMADataFrame&, long, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, pybind11::name, pybind11::is_method, pybind11::sibling, pybind11::arg, pybind11::arg>(libtiledbsomacpp::load_soma_dataframe(pybind11::module_&)::{lambda(tiledbsoma::SOMADataFrame&, long, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >)#6}&&, void (*)(tiledbsoma::SOMADataFrame&, long, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >), pybind11::name const&, pybind11::is_method const&, pybind11::sibling const&, pybind11::arg const&, pybind11::arg const&)::{lambda(pybind11::detail::function_call&)#3}::_FUN(pybind11::detail::function_call&) (in /home/bruce/projects/TileDB-SOMA/apis/python/src/tiledbsoma/pytiledbsoma.cpython-312-x86_64-linux-gnu.so)
==2066522==    by 0x86D055C3: pybind11::cpp_function::dispatcher(_object*, _object*, _object*) (in /home/bruce/projects/TileDB-SOMA/apis/python/src/tiledbsoma/pytiledbsoma.cpython-312-x86_64-linux-gnu.so)
==2066522==    by 0x323D37: cfunction_call (methodobject.c:537)
==2066522==    by 0x30726B: _PyObject_MakeTpCall (call.c:240)
==2066522==    by 0x21AB4E: _PyEval_EvalFrameDefault.cold (bytecodes.c:2715)
==2066522==    by 0x309CF1: _PyObject_FastCallDictTstate (call.c:144)
==2066522==    by 0x3327A8: _PyObject_Call_Prepend (call.c:508)

==2066522== 21 bytes in 3 blocks are definitely lost in loss record 1,380 of 7,551
==2066522==    at 0x4848899: malloc (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==2066522==    by 0x4A0E58E: strdup (strdup.c:42)
==2066522==    by 0x873DDB5C: tiledbsoma::ArrowAdapter::make_arrow_schema_parent(unsigned long, std::basic_string_view<char, std::char_traits<char> >) (in /home/bruce/projects/TileDB-SOMA/dist/lib/libtiledbsoma.so)
==2066522==    by 0x8731B0C0: tiledbsoma::SOMAArray::_get_core_domainish(Domainish) (in /home/bruce/projects/TileDB-SOMA/dist/lib/libtiledbsoma.so)
==2066522==    by 0x8732A218: tiledbsoma::SOMAArray::_set_soma_joinid_shape_helper(long, bool, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >) (in /home/bruce/projects/TileDB-SOMA/dist/lib/libtiledbsoma.so)
==2066522==    by 0x87009678: pybind11::cpp_function::initialize<libtiledbsomacpp::load_soma_dataframe(pybind11::module_&)::{lambda(tiledbsoma::SOMADataFrame&, long, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >)#6}, void, tiledbsoma::SOMADataFrame&, long, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, pybind11::name, pybind11::is_method, pybind11::sibling, pybind11::arg, pybind11::arg>(libtiledbsomacpp::load_soma_dataframe(pybind11::module_&)::{lambda(tiledbsoma::SOMADataFrame&, long, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >)#6}&&, void (*)(tiledbsoma::SOMADataFrame&, long, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >), pybind11::name const&, pybind11::is_method const&, pybind11::sibling const&, pybind11::arg const&, pybind11::arg const&)::{lambda(pybind11::detail::function_call&)#3}::_FUN(pybind11::detail::function_call&) (in /home/bruce/projects/TileDB-SOMA/apis/python/src/tiledbsoma/pytiledbsoma.cpython-312-x86_64-linux-gnu.so)
==2066522==    by 0x86D055C3: pybind11::cpp_function::dispatcher(_object*, _object*, _object*) (in /home/bruce/projects/TileDB-SOMA/apis/python/src/tiledbsoma/pytiledbsoma.cpython-312-x86_64-linux-gnu.so)
==2066522==    by 0x323D37: cfunction_call (methodobject.c:537)
==2066522==    by 0x30726B: _PyObject_MakeTpCall (call.c:240)
==2066522==    by 0x21AB4E: _PyEval_EvalFrameDefault.cold (bytecodes.c:2715)
==2066522==    by 0x309CF1: _PyObject_FastCallDictTstate (call.c:144)
==2066522==    by 0x3327A8: _PyObject_Call_Prepend (call.c:508)
```
author_association: MEMBER
user: login: johnkerl
created_at: 2025-05-20T16:01:13Z
body: > And leaked:

This is because of
https://github.com/single-cell-data/TileDB-SOMA/blob/xanthosxanthopoulos/soma-169-c-memory-leak-in-somaarray_get_core_domainish/libtiledbsoma/src/utils/arrow_adapter.cc#L539-L546

* `dict->format` is `strdup`'ed outside the `if`
* then again inside the `if`
author_association: CONTRIBUTOR
user: login: johnkerl
created_at: 2025-05-20T16:10:05Z
body: With this mod compiled in my sandbox I re-ran
```
valgrind --error-limit=no --trace-children=yes -s --leak-check=full $(which python) ~/130.py
```
as detailed on #4065 and #4067, and I no longer see the `strdup` leak appearing in the output from `valgrind`
author_association: CONTRIBUTOR
user: login: bkmartinjr
created_at: 2025-05-21T15:19:40Z
body: I ran a couple of quick unit tests for validation, and for those I tested, things are clean.
author_association: MEMBER