Repository: single-cell-data/TileDB-SOMA (language: python)

Issue:
[python/r/c++] Revisit `shape` for component arrays
## PRs

**Merged PRs:**

<details>

* #2909 `kerl/schevo-timestamp-methodize`
* #2913 `kerl/name-neaten`
* #2908 `kerl/ut-soma-exc-simplify`
* #2910 `kerl/test-common-parameterize`
* #2918 `kerl/cpp-test-deadstrip`
* #2919 `kerl/minor-unit-test-helper-mod`
* #2936 `kerl/cpp-ut-helper-neaten`
* #2938 `kerl/more-cur-dom-parameterize`
* #2915 `kerl/cpp-strict-int64-shape`
* #2911 `kerl/arrow-util-current-domain-optional`
* #2939 `kerl/step-two-temp`
* #2947 `kerl/cpp-ndarray-resize-testing`
* #2945 `kerl/dataframe-test-fixture`
* #2944 `kerl/cpp-variant-indexed-dataframes`
* #2916 `kerl/sdf-shape`
* #2917 `kerl/cpp-resizes`
* #2948 `kerl/upgrade-shape-int64`
* #2951 `kerl/sdf-test-accessors`
* #2953 `kerl/py-r-accessor-plumbing`
* #2957 `kerl/sdf-domain-accessors`
* #2960 `kerl/dense-link`
* #2963 `kerl/nightly-fix`
* #2970 `kerl/dense-writeable-after-create`
* #2968 `kerl/minor-trim`
* #2969 `kerl/more-py-domain-name-neaten`
* #2972 `kerl/libtiledbsoma-env-logging-level`
* #2962 `kerl/py-r-creation-paths`
* #2950 `kerl/py-r-test-2`
* #2994 `kerl/nanoarrow-helpers`
* #3011 `kerl/polydom3`
* #3017 `kerl/polydom5`
* #3018 `kerl/polydom6`
* #2990 `kerl/variant-nnz-bug`
* #3019 `kerl/index-swap`
* #3020 `kerl/ut-max-shape`
* #3012 `kerl/polydom4`
* #3025 `kerl/fix-3020-merge`
* #3026 `kerl/one-more-rename`
* #3028 `kerl/ff-not`
* #3029 `kerl/ut-vg`
* #3030 `kerl/table-utils-memory`
* #3067 `kerl/improve-sdf-test-field-names`
* #3068 `kerl/ut-generate`
* #3069 `kerl/cpp-sdf-domain-at-create`
* #3027 `kerl/hll-domainish`
* #3088 `kerl/max-domain-int64`
* #3090 `kerl/maybe-resize-soma-joinid-cpp-tweak`
* #3032 `kerl/sdf-domain-at-create` -- fixes #2967
* #3091 `kerl/maybe-resize-soma-joinid-py-r`
* #3095 `kerl/cpp-exp-resize-prep`
* #3089 `kerl/r-dataframe-shapeable`
* https://github.com/single-cell-data/SOMA/pull/233
* #3125 `kerl/cpp-ut-name-shortens`
* #3127 `kerl/helper-rename`
* #3130 `kerl/cpp-can-resizers-names`
* #3132 `kerl/cpp-dataframe-sizing-helpers`
* #3139 `kerl/cpp-dataframe-upgrade-test`
* #3140 `kerl/py-resizer-connects`
* #3151 `kerl/py-can-upgrade-shape`
* #3152 `kerl/registration-shape-acceessors`
* #3156 `kerl/py-exp-shaping`
* #3157 `kerl/py-exp-shaping2`
* #3148 `kerl/py-exp-resize`
* #3191 `kerl/py-domain-at-create-ut-1`
* #3190 `kerl/py-domain-at-create-ut-2`
* #3192 `kerl/py-domain-at-create-ut-3`
* #3193 `kerl/py-domain-at-create-ut-4`
* #3194 `kerl/py-domain-at-create-ut-5`
* #3203 `kerl/min-size-2`
* #3208 `kerl/r-min-sizing`
* #3211 `kerl/cpp-ugr-dom`
* #3232 `kerl/ff-interop`
* #3230 `kerl/ffon`
* #3234 `kerl/docstring-prune`
* #3236 `kerl/prefixing`
* #3241 `kerl/fix-bad-merge`
* #3235 `kerl/py-r-ugr-dom`
* #3237 `kerl/py-r-ugr-dom-2`
* #3238 `kerl/py-r-ugr-dom-3`
* #3253 `kerl/set-coords-rename`
* #3261 `kerl/pybind11-nda-sizing`
* #3265 `kerl/dense-227-a`
* #3263 `kerl/dense-range-trim`
* #3268 `kerl/dim-explosion`
* #3269 `kerl/python-227-dense-ned-read`
* #3270 `kerl/r-227-dense-fixes`
* #3280 `kerl/r-dense-227-more`
* #3286 `kerl/more-fn4m`
* #3283 `kerl/readthedocs-pre-1.15`
* #3288 `kerl/dense-ugrsh`
* #3289 `kerl/notebook-shape-upgrade`
* #3285 `kerl/new-shape-doc-updates`
* #3295 `kerl/notebook-data-refresh`
* #3290 `kerl/notebook-new-shape-refresh`
* #3301 `kerl/ffena`
* #3303 `kerl/r-data-refresh`
* #3300 `kerl/sdf-sjid-lower-zero`
* #3296 `kerl/dense-example-data-refresh`
* #3294 `kerl/new-shape-notebook-and-vignette`
* #3308 `kerl/upgrade-experiment-resources`
* #3309 `kerl/fix-notebook-merge`
* #3321 `kerl/more-use-shape`
* #3358 `kerl/revert-3300`
* #3368 `kerl/227a`
* #3369 `kerl/ucd1`
* #3370 `kerl/ucd2`
* #3371 `kerl/ucd3`
* #3372 `kerl/ucd4`
* #3396 `kerl/domain-at-create-docstrings`
* #3302 `kerl/new-shape-vignette`
* https://github.com/single-cell-data/SOMA/pull/250
* #3399 `kerl/new-shape-more-docstrings`
* #3400 `kerl/check-only-r`
* https://github.com/TileDB-Inc/somacore-feedstock/pull/27

</details>

**Closed/abandoned PRs:**

<details>

* #2785 -- This was only dogfooding for the core 2.25 release -- not to be merged
* #2952 `kerl/feature-flag-temp` -- folded into 2962
* #2995 `kerl/polydom`
* #2964 `kerl/tiledbsoma-io-test`
* #3189 `kerl/min-size`
* #3220 `kerl/cpp-ugr-dom-2`
* #3244 `kerl/dense-227-fixes`
* #3276 `kerl/dataframe-shape`

</details>

**Issues which are related but non-blocking:**

* https://github.com/single-cell-data/SOMA/pull/216
* https://github.com/TileDB-Inc/TileDB/pull/5303
* #2966
* #3081
* Note: R append mode does not exist yet -- see #1630 -- so an experiment-level resizer is not a priority in R
* #3271 
* #3272
* #3273

**See also:** [[sc-51048]](https://app.shortcut.com/tiledb-inc/story/51048).

## Problem to be solved

Users want to know the `shape` of an array, in the SciPy sense:

* Reads and writes are bounds-checked against the shape
* This retains its value regardless of which values of a sparse array are or are not actually occupied
* Users can `resize`.
  * Some users need the ability to grow their datasets later, using either `tiledbsoma.io`'s append mode, or subsequent writes using the `tiledbsoma` API.
  * Note that the cellxgene census doesn't need this: eact week's published census has fixed shape, and any updates will happen in new storage, on a new week.

Using TileDB-SOMA up until the present:

* The TIleDB `domain` is immutable after array creation
  * This does bounds-checking for reads and writes, which is good
  * To leverage this to function as a `shape`, users would need to set the `domain` at array-creation time. However, users lose the ability to grow their datasets later.
* There is a `non_empty_domain` accessor
  * This only indicates min/max coordinates _at which data exists_. Consider an `X` array for 100 cells and 200 genes. If non-zero expression counts exist only for cell join IDs 2-17, then the `non_empty_domain` will indicate `(2,17)` along `soma_dim_0`.
  * Consider an `obms["X_pca"]` within the same experiment. This may be 100 cells by 50 PCA components: we need a placd to store the number 50.
  * Therefore users cannot leverage this to function as a `shape` accessor.
* We have offered a `used_shape` accessor since TileDB-SOMA 1.5.
  * This functions as a `shape` accessor, in the SciPy sense, but it is not multi-writer safe.

New feature for TileDB-SOMA 1.15:

* Arrays will have a  `shape`
* Reads and writes are bounds-checked against the shape
* This retains its value regardless of which values of a sparse array are or are not actually occupied
* Users can `resize`
* The `used_shape` accessor will be deprecated in TileDB-SOMA 1.13, and slated for removal in TileDB-SOMA 1.14.

Compatiblity:

This will now require users to do an explicit `resize` before appending/growing TileDB-SOMA Experiments. Guidance in the form of example notebooks will be provided.

##  Tracking

See also: [[sc-41074]](https://app.shortcut.com/tiledb-inc/story/41074) and [[sc-51048]](https://app.shortcut.com/tiledb-inc/story/51048).

## Scheduling

Support arrives in TileDB Core 2.25. Deprecations for TileDB-SOMA will be released with 1.13. Full support within TileDB-SOMA will be release in 1.14.

## Details

SOMA API mods as we've discussed in a Google doc are as follows.

### `SOMADataFrame`

* `create`: Retain the `domain` argument
  * Issue:
    * Core has a `(lo, hi)` tuple per dim, e.g. `(0,99)` or `(10,19)`
    * SOMA has count per dim, with 0 implicit: e.g. 100 or 20
    * For `SparseNDArray` and `DenseNDArray` core can have `(lo, hi)` and SOMA can have `count`
    * For `DataFrame` there can be multiple dims --- default is a single `soma_joinid`
    * That could be treated either in `(lo, hi)` fashion or `count` fashion
    * However additional dims (e.g. `cell_type`) can be on any type, including strings, floats, etc. where there is no implicit lo=0
    * Therefore we need to keep the current SOMA API wherein `DataFrame` takes a `domain` argument (in `(lo, hi)` fashion) and not a `shape` argument (in `count` fashion)

### `SparseNDArray and DenseNDArray`

* `create`
  * Have an optional shape argument which is of type `Tuple[Int,...]` where each element is the cell count of the corresponding dimension
    * If unsupplied, or if supplied but None in any slot: use the minimum 0 in each slot – nothing larger makes sense since we will not support downsize
  * User guidance should make clear that it will not be possible to create an ‘old’ style array with the ‘new style’ API. (See also the upgrade logic below.)

### All three of `SOMADataFrame`, `SparseNDArray`, `DenseNDArray`

* `write`
  * For new arrays, created with the new shape feature:
    * Core will bounds-check that coordinates provided at `write` time are within the current shape
    * Core will raise `tiledb.cc.TileDBError` to TileDB-SOMA, which will catch and raise `IndexError`, and R-standard behavior on the R side
  * For old arrays created before this feature:
    * Core will not bounds-check that coordinates provided at write time are within the current shape
* Existing `used_shape` accessor
  * TileDB-SOMA will deprecate this over a release cycle.
  * For new arrays: raise `NotImplementedError`
  * For old arrays: return what’s currently returned, with a deprecation warning.
  * Mechanism for determining old vs. new: `array.schema.version` (the core storage version).
* Existing `shape` accessor
  * For new arrays:
    * Have this return the new shape as proposed by core, no longer returning the TileDB domain.
  * For old arrays created before this feature:
    * Return the TileDB domain as now.
* Existing `non_empty_domain` accessor
  * Same behavior for old and new arrays (unaffected by this proposal).
  * Keep this accessor supported, but, with user notes that it’s generally non-useful
  * This should return None (or R equivalent) when there is a schema but no data have been written.
* New `maxshape` accessor
  * Maps the core-level `(lo, hi)` accessor for domain to count-style accessor hi+1. E.g. if the core domain is either `(0,99)` or `(50,99)` then TileDB-SOMA `maxshape` will say 100.
  * Same behavior for old and new arrays.
  * Let users query for what the TileDB domain is, with user notes that it’s the maximum that users can reshape to.
  * Issac suggests: maybe `domain` or `maxshape` (see h5py).
* New `resize` mutator
  * Note: `reshape` means something else in the community (numpy, zarr, h5py), e.g. a 5x20 (total 100 cells) being reinterpreted as 4x25 (still 100 cells). The standard name for changing cell-count is `resize`.
  * For old arrays created before this feature: raise `NotImplementedError`.
  * For new arrays:
    * Will raise `ValueError` if the new shape is smaller on any dim than currently in storage
    * Regardless of whether any data have been written whatsoever
    * Will raise `ValueError` if the new shape exceeds the TileDB domain from create time, which will serve TileDB-SOMA in a role of “max possible shape the user can reshape to”
    * Otherwise, any calls to write from this point will bounds-check writes within this new shape
    * **We don’t expect resize to be multi-writer safe with regard to write ; user notes must be clear on this point**
* New `tiledbsoma_upgrade_shape` method for SparseNDArray and DenseNDArray
  * This will leverage `array.schema.version` to see if an upgrade is needed
  * Leverage core support for storage-version updates
  * This will take a shape argument as in `create`
  * For arrays created with “just-right” size: this will succeed
  * For arrays created with “room-for-growth” / “two billion-ish” size: this will succeed
  * If the user passes a shape which exceeds the current TileDB domain: this will fail
* New `tiledbsoma_upgrade_domain method` for `DataFrame`
  * Same as for `SparseNDArray`/`DenseNDArray` except it will take a domain at the SOMA-API level just as `DataFrame`'s create method

### `tiledbsoma.io`

* The user-facing API has no shape arguments and thus won’t need changing.
* Internally to `tiledbsoma.io`, we’ll still ask the tiledbsoma API for the “big domain” (2 billionish)
* Append mode:
  * Will need a new `resize` method at the `Experiment` level
  * Users will need to:
    * Register as now
    * Call the experiment-level `resize`
      * Could be `exp.resize(...)`, or (better) this could be `tiledbsoma.io.reshape_experiment`
  * In either case: this method will take the new `obs` and `var` counts as inputs:
    * `exp.obs.reshape` to new `obs` count
    * `exp.ms[name].var.reshape` to new `var` count
    * `exp.ms[name].X[name].reshape` to new `obs` count x `var` count
    * `exp.ms[name].obsm[name].reshape` to new `obs` count x same width
    * `exp.ms[name].obsp[name].reshape` to new `obs` count x `obs` count
    * `exp.ms[name].varm[name].reshape` to new `var` count x same width
    * `exp.ms[name].varp[name].reshape` to new `var` count x `var` count
  * Do the individual append-mode writes as now

[c++] Pre-neatens for polymorphic domainish accessors
**Issue and/or context:** As tracked on issue #2407 / [[sc-51048]](https://app.shortcut.com/tiledb-inc/story/51048).

This is split out from #2995 to make 2995 simpler.

Note that the intended Python and R API changes are all agreed on and finalized as described in #2407.

These are higher-line-count mods, which would distract from the more substantive mods on 2995.

**Changes:**

**Notes for Reviewer:**


[c++] Push polymorphic domain-ish accessors down to C++
**Issue and/or context:** As tracked on issue #2407 / [[sc-51048]](https://app.shortcut.com/tiledb-inc/story/51048).

**Changes:**

Please see the description of #2994. This leverages 2994 to begin pushdown to `libtiledbsoma`.

Note this PR will only introduce the pushed-down logic within `libtiledbsoma`. R and Python PRs will be atop this.

**Notes for Reviewer:**

This PR is a work in progress. It is not ready for review.

* This needs a code-reuse hook for domain, current domain, and non-empty domain
* It needs unit tests
[python/r/c++] Revisit `shape` for component arrays
## PRs

**Merged PRs:**

<details>

* #2909 `kerl/schevo-timestamp-methodize`
* #2913 `kerl/name-neaten`
* #2908 `kerl/ut-soma-exc-simplify`
* #2910 `kerl/test-common-parameterize`
* #2918 `kerl/cpp-test-deadstrip`
* #2919 `kerl/minor-unit-test-helper-mod`
* #2936 `kerl/cpp-ut-helper-neaten`
* #2938 `kerl/more-cur-dom-parameterize`
* #2915 `kerl/cpp-strict-int64-shape`
* #2911 `kerl/arrow-util-current-domain-optional`
* #2939 `kerl/step-two-temp`
* #2947 `kerl/cpp-ndarray-resize-testing`
* #2945 `kerl/dataframe-test-fixture`
* #2944 `kerl/cpp-variant-indexed-dataframes`
* #2916 `kerl/sdf-shape`
* #2917 `kerl/cpp-resizes`
* #2948 `kerl/upgrade-shape-int64`
* #2951 `kerl/sdf-test-accessors`
* #2953 `kerl/py-r-accessor-plumbing`
* #2957 `kerl/sdf-domain-accessors`
* #2960 `kerl/dense-link`
* #2963 `kerl/nightly-fix`
* #2970 `kerl/dense-writeable-after-create`
* #2968 `kerl/minor-trim`
* #2969 `kerl/more-py-domain-name-neaten`
* #2972 `kerl/libtiledbsoma-env-logging-level`
* #2962 `kerl/py-r-creation-paths`
* #2950 `kerl/py-r-test-2`
* #2994 `kerl/nanoarrow-helpers`
* #3011 `kerl/polydom3`
* #3017 `kerl/polydom5`
* #3018 `kerl/polydom6`
* #2990 `kerl/variant-nnz-bug`
* #3019 `kerl/index-swap`
* #3020 `kerl/ut-max-shape`
* #3012 `kerl/polydom4`
* #3025 `kerl/fix-3020-merge`
* #3026 `kerl/one-more-rename`
* #3028 `kerl/ff-not`
* #3029 `kerl/ut-vg`
* #3030 `kerl/table-utils-memory`
* #3067 `kerl/improve-sdf-test-field-names`
* #3068 `kerl/ut-generate`
* #3069 `kerl/cpp-sdf-domain-at-create`
* #3027 `kerl/hll-domainish`
* #3088 `kerl/max-domain-int64`
* #3090 `kerl/maybe-resize-soma-joinid-cpp-tweak`
* #3032 `kerl/sdf-domain-at-create` -- fixes #2967
* #3091 `kerl/maybe-resize-soma-joinid-py-r`
* #3095 `kerl/cpp-exp-resize-prep`
* #3089 `kerl/r-dataframe-shapeable`
* https://github.com/single-cell-data/SOMA/pull/233
* #3125 `kerl/cpp-ut-name-shortens`
* #3127 `kerl/helper-rename`
* #3130 `kerl/cpp-can-resizers-names`
* #3132 `kerl/cpp-dataframe-sizing-helpers`
* #3139 `kerl/cpp-dataframe-upgrade-test`
* #3140 `kerl/py-resizer-connects`
* #3151 `kerl/py-can-upgrade-shape`
* #3152 `kerl/registration-shape-acceessors`
* #3156 `kerl/py-exp-shaping`
* #3157 `kerl/py-exp-shaping2`
* #3148 `kerl/py-exp-resize`
* #3191 `kerl/py-domain-at-create-ut-1`
* #3190 `kerl/py-domain-at-create-ut-2`
* #3192 `kerl/py-domain-at-create-ut-3`
* #3193 `kerl/py-domain-at-create-ut-4`
* #3194 `kerl/py-domain-at-create-ut-5`
* #3203 `kerl/min-size-2`
* #3208 `kerl/r-min-sizing`
* #3211 `kerl/cpp-ugr-dom`
* #3232 `kerl/ff-interop`
* #3230 `kerl/ffon`
* #3234 `kerl/docstring-prune`
* #3236 `kerl/prefixing`
* #3241 `kerl/fix-bad-merge`
* #3235 `kerl/py-r-ugr-dom`
* #3237 `kerl/py-r-ugr-dom-2`
* #3238 `kerl/py-r-ugr-dom-3`
* #3253 `kerl/set-coords-rename`
* #3261 `kerl/pybind11-nda-sizing`
* #3265 `kerl/dense-227-a`
* #3263 `kerl/dense-range-trim`
* #3268 `kerl/dim-explosion`
* #3269 `kerl/python-227-dense-ned-read`
* #3270 `kerl/r-227-dense-fixes`
* #3280 `kerl/r-dense-227-more`
* #3286 `kerl/more-fn4m`
* #3283 `kerl/readthedocs-pre-1.15`
* #3288 `kerl/dense-ugrsh`
* #3289 `kerl/notebook-shape-upgrade`
* #3285 `kerl/new-shape-doc-updates`
* #3295 `kerl/notebook-data-refresh`
* #3290 `kerl/notebook-new-shape-refresh`
* #3301 `kerl/ffena`
* #3303 `kerl/r-data-refresh`
* #3300 `kerl/sdf-sjid-lower-zero`
* #3296 `kerl/dense-example-data-refresh`
* #3294 `kerl/new-shape-notebook-and-vignette`
* #3308 `kerl/upgrade-experiment-resources`
* #3309 `kerl/fix-notebook-merge`
* #3321 `kerl/more-use-shape`
* #3358 `kerl/revert-3300`
* #3368 `kerl/227a`
* #3369 `kerl/ucd1`
* #3370 `kerl/ucd2`
* #3371 `kerl/ucd3`
* #3372 `kerl/ucd4`
* #3396 `kerl/domain-at-create-docstrings`
* #3302 `kerl/new-shape-vignette`
* https://github.com/single-cell-data/SOMA/pull/250
* #3399 `kerl/new-shape-more-docstrings`
* #3400 `kerl/check-only-r`
* https://github.com/TileDB-Inc/somacore-feedstock/pull/27

</details>

**Closed/abandoned PRs:**

<details>

* #2785 -- This was only dogfooding for the core 2.25 release -- not to be merged
* #2952 `kerl/feature-flag-temp` -- folded into 2962
* #2995 `kerl/polydom`
* #2964 `kerl/tiledbsoma-io-test`
* #3189 `kerl/min-size`
* #3220 `kerl/cpp-ugr-dom-2`
* #3244 `kerl/dense-227-fixes`
* #3276 `kerl/dataframe-shape`

</details>

**Issues which are related but non-blocking:**

* https://github.com/single-cell-data/SOMA/pull/216
* https://github.com/TileDB-Inc/TileDB/pull/5303
* #2966
* #3081
* Note: R append mode does not exist yet -- see #1630 -- so an experiment-level resizer is not a priority in R
* #3271 
* #3272
* #3273

**See also:** [[sc-51048]](https://app.shortcut.com/tiledb-inc/story/51048).

## Problem to be solved

Users want to know the `shape` of an array, in the SciPy sense:

* Reads and writes are bounds-checked against the shape
* This retains its value regardless of which values of a sparse array are or are not actually occupied
* Users can `resize`.
  * Some users need the ability to grow their datasets later, using either `tiledbsoma.io`'s append mode, or subsequent writes using the `tiledbsoma` API.
  * Note that the cellxgene census doesn't need this: eact week's published census has fixed shape, and any updates will happen in new storage, on a new week.

Using TileDB-SOMA up until the present:

* The TIleDB `domain` is immutable after array creation
  * This does bounds-checking for reads and writes, which is good
  * To leverage this to function as a `shape`, users would need to set the `domain` at array-creation time. However, users lose the ability to grow their datasets later.
* There is a `non_empty_domain` accessor
  * This only indicates min/max coordinates _at which data exists_. Consider an `X` array for 100 cells and 200 genes. If non-zero expression counts exist only for cell join IDs 2-17, then the `non_empty_domain` will indicate `(2,17)` along `soma_dim_0`.
  * Consider an `obms["X_pca"]` within the same experiment. This may be 100 cells by 50 PCA components: we need a placd to store the number 50.
  * Therefore users cannot leverage this to function as a `shape` accessor.
* We have offered a `used_shape` accessor since TileDB-SOMA 1.5.
  * This functions as a `shape` accessor, in the SciPy sense, but it is not multi-writer safe.

New feature for TileDB-SOMA 1.15:

* Arrays will have a  `shape`
* Reads and writes are bounds-checked against the shape
* This retains its value regardless of which values of a sparse array are or are not actually occupied
* Users can `resize`
* The `used_shape` accessor will be deprecated in TileDB-SOMA 1.13, and slated for removal in TileDB-SOMA 1.14.

Compatiblity:

This will now require users to do an explicit `resize` before appending/growing TileDB-SOMA Experiments. Guidance in the form of example notebooks will be provided.

##  Tracking

See also: [[sc-41074]](https://app.shortcut.com/tiledb-inc/story/41074) and [[sc-51048]](https://app.shortcut.com/tiledb-inc/story/51048).

## Scheduling

Support arrives in TileDB Core 2.25. Deprecations for TileDB-SOMA will be released with 1.13. Full support within TileDB-SOMA will be release in 1.14.

## Details

SOMA API mods as we've discussed in a Google doc are as follows.

### `SOMADataFrame`

* `create`: Retain the `domain` argument
  * Issue:
    * Core has a `(lo, hi)` tuple per dim, e.g. `(0,99)` or `(10,19)`
    * SOMA has count per dim, with 0 implicit: e.g. 100 or 20
    * For `SparseNDArray` and `DenseNDArray` core can have `(lo, hi)` and SOMA can have `count`
    * For `DataFrame` there can be multiple dims --- default is a single `soma_joinid`
    * That could be treated either in `(lo, hi)` fashion or `count` fashion
    * However additional dims (e.g. `cell_type`) can be on any type, including strings, floats, etc. where there is no implicit lo=0
    * Therefore we need to keep the current SOMA API wherein `DataFrame` takes a `domain` argument (in `(lo, hi)` fashion) and not a `shape` argument (in `count` fashion)

### `SparseNDArray and DenseNDArray`

* `create`
  * Have an optional shape argument which is of type `Tuple[Int,...]` where each element is the cell count of the corresponding dimension
    * If unsupplied, or if supplied but None in any slot: use the minimum 0 in each slot – nothing larger makes sense since we will not support downsize
  * User guidance should make clear that it will not be possible to create an ‘old’ style array with the ‘new style’ API. (See also the upgrade logic below.)

### All three of `SOMADataFrame`, `SparseNDArray`, `DenseNDArray`

* `write`
  * For new arrays, created with the new shape feature:
    * Core will bounds-check that coordinates provided at `write` time are within the current shape
    * Core will raise `tiledb.cc.TileDBError` to TileDB-SOMA, which will catch and raise `IndexError`, and R-standard behavior on the R side
  * For old arrays created before this feature:
    * Core will not bounds-check that coordinates provided at write time are within the current shape
* Existing `used_shape` accessor
  * TileDB-SOMA will deprecate this over a release cycle.
  * For new arrays: raise `NotImplementedError`
  * For old arrays: return what’s currently returned, with a deprecation warning.
  * Mechanism for determining old vs. new: `array.schema.version` (the core storage version).
* Existing `shape` accessor
  * For new arrays:
    * Have this return the new shape as proposed by core, no longer returning the TileDB domain.
  * For old arrays created before this feature:
    * Return the TileDB domain as now.
* Existing `non_empty_domain` accessor
  * Same behavior for old and new arrays (unaffected by this proposal).
  * Keep this accessor supported, but, with user notes that it’s generally non-useful
  * This should return None (or R equivalent) when there is a schema but no data have been written.
* New `maxshape` accessor
  * Maps the core-level `(lo, hi)` accessor for domain to count-style accessor hi+1. E.g. if the core domain is either `(0,99)` or `(50,99)` then TileDB-SOMA `maxshape` will say 100.
  * Same behavior for old and new arrays.
  * Let users query for what the TileDB domain is, with user notes that it’s the maximum that users can reshape to.
  * Issac suggests: maybe `domain` or `maxshape` (see h5py).
* New `resize` mutator
  * Note: `reshape` means something else in the community (numpy, zarr, h5py), e.g. a 5x20 (total 100 cells) being reinterpreted as 4x25 (still 100 cells). The standard name for changing cell-count is `resize`.
  * For old arrays created before this feature: raise `NotImplementedError`.
  * For new arrays:
    * Will raise `ValueError` if the new shape is smaller on any dim than currently in storage
    * Regardless of whether any data have been written whatsoever
    * Will raise `ValueError` if the new shape exceeds the TileDB domain from create time, which will serve TileDB-SOMA in a role of “max possible shape the user can reshape to”
    * Otherwise, any calls to write from this point will bounds-check writes within this new shape
    * **We don’t expect resize to be multi-writer safe with regard to write ; user notes must be clear on this point**
* New `tiledbsoma_upgrade_shape` method for SparseNDArray and DenseNDArray
  * This will leverage `array.schema.version` to see if an upgrade is needed
  * Leverage core support for storage-version updates
  * This will take a shape argument as in `create`
  * For arrays created with “just-right” size: this will succeed
  * For arrays created with “room-for-growth” / “two billion-ish” size: this will succeed
  * If the user passes a shape which exceeds the current TileDB domain: this will fail
* New `tiledbsoma_upgrade_domain method` for `DataFrame`
  * Same as for `SparseNDArray`/`DenseNDArray` except it will take a domain at the SOMA-API level just as `DataFrame`'s create method

### `tiledbsoma.io`

* The user-facing API has no shape arguments and thus won’t need changing.
* Internally to `tiledbsoma.io`, we’ll still ask the tiledbsoma API for the “big domain” (2 billionish)
* Append mode:
  * Will need a new `resize` method at the `Experiment` level
  * Users will need to:
    * Register as now
    * Call the experiment-level `resize`
      * Could be `exp.resize(...)`, or (better) this could be `tiledbsoma.io.reshape_experiment`
  * In either case: this method will take the new `obs` and `var` counts as inputs:
    * `exp.obs.reshape` to new `obs` count
    * `exp.ms[name].var.reshape` to new `var` count
    * `exp.ms[name].X[name].reshape` to new `obs` count x `var` count
    * `exp.ms[name].obsm[name].reshape` to new `obs` count x same width
    * `exp.ms[name].obsp[name].reshape` to new `obs` count x `obs` count
    * `exp.ms[name].varm[name].reshape` to new `var` count x same width
    * `exp.ms[name].varp[name].reshape` to new `var` count x `var` count
  * Do the individual append-mode writes as now



Discussion / hints:
#2785 is a quick-and-dirty concept-prover -- its sole function is to flush out any API misunderstandings we might have, in prep for 2.25.0 core release.
## [Codecov](https://app.codecov.io/gh/single-cell-data/TileDB-SOMA/pull/3011?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=single-cell-data) Report
All modified and coverable lines are covered by tests :white_check_mark:
> Project coverage is 89.79%. Comparing base [(`29be284`)](https://app.codecov.io/gh/single-cell-data/TileDB-SOMA/commit/29be28498d19f74af903add64df35b01676f20dd?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=single-cell-data) to head [(`fba692e`)](https://app.codecov.io/gh/single-cell-data/TileDB-SOMA/commit/fba692efe2745ea79296df09056c419067adfbbd?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=single-cell-data).
> Report is 1 commits behind head on main.

<details><summary>Additional details and impacted files</summary>


```diff
@@            Coverage Diff             @@
##             main    #3011      +/-   ##
==========================================
+ Coverage   89.64%   89.79%   +0.14%     
==========================================
  Files          40       40              
  Lines        4096     4096              
==========================================
+ Hits         3672     3678       +6     
+ Misses        424      418       -6     
```

| [Flag](https://app.codecov.io/gh/single-cell-data/TileDB-SOMA/pull/3011/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=single-cell-data) | Coverage Δ | |
|---|---|---|
| [python](https://app.codecov.io/gh/single-cell-data/TileDB-SOMA/pull/3011/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=single-cell-data) | `89.79% <ø> (+0.14%)` | :arrow_up: |

Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=single-cell-data#carryforward-flags-in-the-pull-request-comment) to find out more.

| [Components](https://app.codecov.io/gh/single-cell-data/TileDB-SOMA/pull/3011/components?src=pr&el=components&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=single-cell-data) | Coverage Δ | |
|---|---|---|
| [python_api](https://app.codecov.io/gh/single-cell-data/TileDB-SOMA/pull/3011/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=single-cell-data) | `89.79% <ø> (+0.14%)` | :arrow_up: |
| [libtiledbsoma](https://app.codecov.io/gh/single-cell-data/TileDB-SOMA/pull/3011/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=single-cell-data) | `∅ <ø> (∅)` | |

</details>

#2785 is a quick-and-dirty concept-prover -- its sole function is to flush out any API misunderstandings we might have, in prep for 2.25.0 core release.

Fix patch:
diff --git a/libtiledbsoma/src/soma/soma_array.cc b/libtiledbsoma/src/soma/soma_array.cc
index 00ba2224e6..fdfdfa9308 100644
--- a/libtiledbsoma/src/soma/soma_array.cc
+++ b/libtiledbsoma/src/soma/soma_array.cc
@@ -1,5 +1,4 @@
-/**
- * @file   soma_array.cc
+/** @file   soma_array.cc
  *
  * @section LICENSE
  *
@@ -1141,6 +1140,102 @@ std::optional<TimestampRange> SOMAArray::timestamp() {
     return timestamp_;
 }
 
+// Note that ArrowTable is simply our libtiledbsoma pairing of ArrowArray and
+// ArrowSchema from nanoarrow.
+//
+// The domainish enum simply lets us re-use code which is common across
+// core domain, core current domain, and core non-empty domain.
+ArrowTable SOMAArray::_get_core_domainish(enum Domainish which_kind) {
+    int array_ndim = this->ndim();
+    auto dimensions = tiledb_schema()->domain().dimensions();
+
+    // Create the schema for the info we return
+    std::vector<std::string> names(array_ndim);
+    std::vector<tiledb_datatype_t> tiledb_datatypes(array_ndim);
+
+    for (int i = 0; i < (int)array_ndim; i++) {
+        const Dimension& core_dim = dimensions[i];
+        names[i] = core_dim.name();
+        tiledb_datatypes[i] = core_dim.type();
+    }
+
+    auto arrow_schema = ArrowAdapter::make_arrow_schema(
+        names, tiledb_datatypes);
+
+    // Create the data for the info we return
+    auto arrow_array = ArrowAdapter::make_arrow_array_parent(array_ndim);
+
+    for (int i = 0; i < array_ndim; i++) {
+        auto core_dim = dimensions[i];
+        auto core_type_code = core_dim.type();
+
+        ArrowArray* child = nullptr;
+
+        switch (core_type_code) {
+            case TILEDB_INT64:
+                child = ArrowAdapter::make_arrow_array_child(
+                    _core_domainish_slot<int64_t>(core_dim.name(), which_kind));
+                break;
+            case TILEDB_UINT64:
+                child = ArrowAdapter::make_arrow_array_child(
+                    _core_domainish_slot<uint64_t>(
+                        core_dim.name(), which_kind));
+                break;
+            case TILEDB_INT32:
+                child = ArrowAdapter::make_arrow_array_child(
+                    _core_domainish_slot<int32_t>(core_dim.name(), which_kind));
+                break;
+            case TILEDB_UINT32:
+                child = ArrowAdapter::make_arrow_array_child(
+                    _core_domainish_slot<uint32_t>(
+                        core_dim.name(), which_kind));
+                break;
+            case TILEDB_INT16:
+                child = ArrowAdapter::make_arrow_array_child(
+                    _core_domainish_slot<int16_t>(core_dim.name(), which_kind));
+                break;
+            case TILEDB_UINT16:
+                child = ArrowAdapter::make_arrow_array_child(
+                    _core_domainish_slot<uint16_t>(
+                        core_dim.name(), which_kind));
+                break;
+            case TILEDB_INT8:
+                child = ArrowAdapter::make_arrow_array_child(
+                    _core_domainish_slot<int8_t>(core_dim.name(), which_kind));
+                break;
+            case TILEDB_UINT8:
+                child = ArrowAdapter::make_arrow_array_child(
+                    _core_domainish_slot<uint8_t>(core_dim.name(), which_kind));
+                break;
+
+            case TILEDB_FLOAT64:
+                child = ArrowAdapter::make_arrow_array_child(
+                    _core_domainish_slot<double>(core_dim.name(), which_kind));
+                break;
+            case TILEDB_FLOAT32:
+                child = ArrowAdapter::make_arrow_array_child(
+                    _core_domainish_slot<float>(core_dim.name(), which_kind));
+                break;
+
+            case TILEDB_STRING_ASCII:
+            case TILEDB_CHAR:
+                child = ArrowAdapter::make_arrow_array_child_string(
+                    _core_domainish_slot_string(core_dim.name(), which_kind));
+                break;
+
+            default:
+                throw TileDBSOMAError(fmt::format(
+                    "SOMAArray::_get_core_domainish:dim {} has unhandled type "
+                    "{}",
+                    core_dim.name(),
+                    tiledb::impl::type_to_str(core_type_code)));
+        }
+        arrow_array->children[i] = child;
+    }
+
+    return ArrowTable(std::move(arrow_array), std::move(arrow_schema));
+}
+
 uint64_t SOMAArray::nnz() {
     // Verify array is sparse
     if (mq_->schema()->array_type() != TILEDB_SPARSE) {
diff --git a/libtiledbsoma/src/soma/soma_array.h b/libtiledbsoma/src/soma/soma_array.h
index 8d5038aae2..f3adc794f6 100644
--- a/libtiledbsoma/src/soma/soma_array.h
+++ b/libtiledbsoma/src/soma/soma_array.h
@@ -48,6 +48,14 @@
 namespace tiledbsoma {
 using namespace tiledb;
 
+// This enables some code deduplication between core domain, core current
+// domain, and core non-empty domain.
+enum class Domainish {
+    kind_core_domain = 0,
+    kind_core_current_domain = 1,
+    kind_non_empty_domain = 2
+};
+
 class SOMAArray : public SOMAObject {
    public:
     //===================================================================
@@ -703,7 +711,7 @@ class SOMAArray : public SOMAObject {
      * non-empty domains of the array fragments.
      */
     template <typename T>
-    std::pair<T, T> non_empty_domain_slot(const std::string& name) {
+    std::pair<T, T> non_empty_domain_slot(const std::string& name) const {
         try {
             return arr_->non_empty_domain<T>(name);
         } catch (const std::exception& e) {
@@ -717,7 +725,7 @@ class SOMAArray : public SOMAObject {
      * Applicable only to var-sized dimensions.
      */
     std::pair<std::string, std::string> non_empty_domain_slot_var(
-        const std::string& name) {
+        const std::string& name) const {
         try {
             return arr_->non_empty_domain_var(name);
         } catch (const std::exception& e) {
@@ -745,6 +753,70 @@ class SOMAArray : public SOMAObject {
         return !_get_current_domain().is_empty();
     }
 
+    /**
+     * Returns the core current domain at the given dimension.
+     *
+     * o For arrays with core current-domain support:
+     *   - soma domain is core current domain
+     *   - soma maxdomain is core domain
+     * o For arrays without core current-domain support:
+     *   - soma domain is core domain
+     *   - soma maxdomain is core domain
+     *   - core current domain is not accessed at the soma level
+     *
+     * @tparam T Domain datatype
+     * @return Pair of [lower, upper] inclusive bounds.
+     */
+    template <typename T>
+    std::pair<T, T> _core_current_domain_slot(const std::string& name) const {
+        CurrentDomain current_domain = _get_current_domain();
+        if (current_domain.is_empty()) {
+            throw TileDBSOMAError(
+                "_core_current_domain_slot: internal coding error");
+        }
+        if (current_domain.type() != TILEDB_NDRECTANGLE) {
+            throw TileDBSOMAError(
+                "_core_current_domain_slot: found non-rectangle type");
+        }
+        NDRectangle ndrect = current_domain.ndrectangle();
+
+        // Convert from two-element array (core API) to pair (tiledbsoma API)
+        std::array<T, 2> arr = ndrect.range<T>(name);
+        return std::pair<T, T>(arr[0], arr[1]);
+    }
+
+    /**
+     * Returns the core domain at the given dimension.
+     *
+     * o For arrays with core current-domain support:
+     *   - soma domain is core current domain
+     *   - soma maxdomain is core domain
+     * o For arrays without core current-domain support:
+     *   - soma domain is core domain
+     *   - soma maxdomain is core domain
+     *   - core current domain is not accessed at the soma level
+     *
+     * @tparam T Domain datatype
+     * @return Pair of [lower, upper] inclusive bounds.
+     */
+    template <typename T>
+    std::pair<T, T> _core_domain_slot(const std::string& name) const {
+        if (std::is_same_v<T, std::string>) {
+            throw std::runtime_error(
+                "SOMAArray::_core_domain_slot: template-specialization "
+                "failure.");
+        }
+        return arr_->schema().domain().dimension(name).domain<T>();
+    }
+
+    std::pair<std::string, std::string> _core_domain_slot_string(
+        const std::string&) const {
+        // Core domain for string dims is always a nullptr pair at the C++
+        // level.  We follow the convention started by TileDB-Py which is to
+        // report these as an empty-string pair.
+        return std::pair<std::string, std::string>("", "");
+    }
+
     /**
      * Returns the SOMA domain at the given dimension.
      *
@@ -775,6 +847,105 @@ class SOMAArray : public SOMAObject {
         return _core_domain_slot<T>(name);
     }
 
+    /**
+     * Returns the SOMA domain in its entirety, as an Arrow table for return to
+     * Python/R.
+     *
+     * o For arrays with core current-domain support:
+     *   - soma domain is core current domain
+     *   - soma maxdomain is core domain
+     * o For arrays without core current-domain support:
+     *   - soma domain is core domain
+     *   - soma maxdomain is core domain
+     *   - core current domain is not accessed at the soma level
+     *
+     * @tparam T Domain datatype
+     * @return Pair of [lower, upper] inclusive bounds.
+     */
+    ArrowTable get_soma_domain() {
+        if (has_current_domain()) {
+            return _get_core_current_domain();
+        } else {
+            return _get_core_domain();
+        }
+    }
+
+    /**
+     * Returns the SOMA maxdomain in its entirety, as an Arrow table for return
+     * to Python/R.
+     *
+     * o For arrays with core current-domain support:
+     *   - soma domain is core current domain
+     *   - soma maxdomain is core domain
+     * o For arrays without core current-domain support:
+     *   - soma domain is core domain
+     *   - soma maxdomain is core domain
+     *   - core current domain is not accessed at the soma level
+     *
+     * @tparam T Domain datatype
+     * @return Pair of [lower, upper] inclusive bounds.
+     */
+    ArrowTable get_soma_maxdomain() {
+        return _get_core_domain();
+    }
+
+    /**
+     * Returns the core non-empty domain in its entirety, as an Arrow
+     * table for return to Python/R.
+     */
+    ArrowTable get_non_empty_domain() {
+        return _get_core_domainish(Domainish::kind_non_empty_domain);
+    }
+
+    /**
+     * Code-dedupe helper for core domain, core current domain, and core
+     * non-empty domain.
+     */
+    ArrowTable _get_core_domainish(enum Domainish which_kind);
+
+    /**
+     * This enables some code deduplication between core domain, core current
+     * domain, and core non-empty domain.
+     */
+    template <typename T>
+    std::pair<T, T> _core_domainish_slot(
+        const std::string& name, enum Domainish which_kind) const {
+        if (std::is_same_v<T, std::string>) {
+            throw std::runtime_error(
+                "SOMAArray::_core_domainish_slot: template-specialization "
+                "failure.");
+        }
+        switch (which_kind) {
+            case Domainish::kind_core_domain:
+                return _core_domain_slot<T>(name);
+            case Domainish::kind_core_current_domain:
+                return _core_current_domain_slot<T>(name);
+            case Domainish::kind_non_empty_domain:
+                return non_empty_domain_slot<T>(name);
+            default:
+                throw std::runtime_error(
+                    "internal coding error in SOMAArray::_core_domainish_slot: "
+                    "unknown kind");
+        }
+    }
+
+    std::pair<std::string, std::string> _core_domainish_slot_string(
+        const std::string& name, enum Domainish which_kind) const {
+        switch (which_kind) {
+            case Domainish::kind_core_domain:
+                return _core_domain_slot_string(name);
+            case Domainish::kind_core_current_domain:
+                return _core_current_domain_slot<std::string>(name);
+            case Domainish::kind_non_empty_domain:
+                return non_empty_domain_slot_var(name);
+            default:
+                throw std::runtime_error(
+                    "internal coding error in "
+                    "SOMAArray::_core_domainish_slot_string: "
+                    "unknown kind");
+        }
+    }
+
     /**
      * @brief Get the total number of unique cells in the array.
      *
@@ -896,54 +1067,19 @@ class SOMAArray : public SOMAObject {
     }
 
     /**
-     * Returns the core current domain at the given dimension.
-     *
-     * o For arrays with core current-domain support:
-     *   - soma domain is core current domain
-     *   - soma maxdomain is core domain
-     * o For arrays without core current-domain support:
-     *   - soma domain is core domain
-     *   - soma maxdomain is core domain
-     *   - core current domain is not accessed at the soma level
-     *
-     * @tparam T Domain datatype
-     * @return Pair of [lower, upper] inclusive bounds.
+     * Returns the core current domain in its entirety, as an Arrow
+     * table for return to Python/R.
      */
-    template <typename T>
-    std::pair<T, T> _core_current_domain_slot(const std::string& name) const {
-        CurrentDomain current_domain = _get_current_domain();
-        if (current_domain.is_empty()) {
-            throw TileDBSOMAError(
-                "_core_current_domain_slot: internal coding error");
-        }
-        if (current_domain.type() != TILEDB_NDRECTANGLE) {
-            throw TileDBSOMAError(
-                "_core_current_domain_slot: found non-rectangle type");
-        }
-        NDRectangle ndrect = current_domain.ndrectangle();
-
-        // Convert from two-element array (core API) to pair (tiledbsoma API)
-        std::array<T, 2> arr = ndrect.range<T>(name);
-        return std::pair<T, T>(arr[0], arr[1]);
+    ArrowTable _get_core_current_domain() {
+        return _get_core_domainish(Domainish::kind_core_current_domain);
     }
 
     /**
-     * Returns the core current domain at the given dimension.
-     *
-     * o For arrays with core current-domain support:
-     *   - soma domain is core current domain
-     *   - soma maxdomain is core domain
-     * o For arrays without core current-domain support:
-     *   - soma domain is core domain
-     *   - soma maxdomain is core domain
-     *   - core current domain is not accessed at the soma level
-     *
-     * @tparam T Domain datatype
-     * @return Pair of [lower, upper] inclusive bounds.
+     * Returns the core domain in its entirety, as an Arrow
+     * table for return to Python/R.
      */
-    template <typename T>
-    std::pair<T, T> _core_domain_slot(const std::string& name) const {
-        return arr_->schema().domain().dimension(name).domain<T>();
+    ArrowTable _get_core_domain() {
+        return _get_core_domainish(Domainish::kind_core_domain);
     }
 
     /**


Test patch:
diff --git a/libtiledbsoma/test/unit_soma_dataframe.cc b/libtiledbsoma/test/unit_soma_dataframe.cc
index 774a1287ef..a1926390ab 100644
--- a/libtiledbsoma/test/unit_soma_dataframe.cc
+++ b/libtiledbsoma/test/unit_soma_dataframe.cc
@@ -522,19 +522,60 @@ TEST_CASE_METHOD(
             REQUIRE(i64_range[1] == (int64_t)dim_infos[0].dim_max);
         }
 
-        // Check shape before write
+        // Check shape before resize
         int64_t expect = dim_infos[0].dim_max + 1;
         std::optional<int64_t> actual = soma_dataframe
                                             ->maybe_soma_joinid_shape();
         REQUIRE(actual.has_value());
         REQUIRE(actual.value() == expect);
 
-        soma_dataframe->close();
-
         REQUIRE(soma_dataframe->nnz() == 0);
 
+        soma_dataframe->close();
+
+        // Write data
         write_sjid_u32_str_data_from(0);
 
+        // Check shape after write
+        soma_dataframe->open(OpenMode::read);
+
+        REQUIRE(soma_dataframe->nnz() == 2);
+
+        expect = dim_infos[0].dim_max + 1;
+        actual = soma_dataframe->maybe_soma_joinid_shape();
+        REQUIRE(actual.has_value());
+        REQUIRE(actual.value() == expect);
+
+        // Check domainish accessors before resize
+        ArrowTable non_empty_domain = soma_dataframe->get_non_empty_domain();
+        std::vector<int64_t>
+            ned_sjid = ArrowAdapter::get_table_column_by_name<int64_t>(
+                non_empty_domain, "soma_joinid");
+
+        ArrowTable soma_domain = soma_dataframe->get_soma_domain();
+        std::vector<int64_t>
+            dom_sjid = ArrowAdapter::get_table_column_by_name<int64_t>(
+                soma_domain, "soma_joinid");
+
+        ArrowTable soma_maxdomain = soma_dataframe->get_soma_maxdomain();
+        std::vector<int64_t>
+            maxdom_sjid = ArrowAdapter::get_table_column_by_name<int64_t>(
+                soma_maxdomain, "soma_joinid");
+
+        REQUIRE(ned_sjid == std::vector<int64_t>({1, 2}));
+
+        REQUIRE(dom_sjid == std::vector<int64_t>({0, 100}));
+
+        REQUIRE(maxdom_sjid.size() == 2);
+        REQUIRE(maxdom_sjid[0] == 0);
+        if (!use_current_domain) {
+            REQUIRE(maxdom_sjid[1] == 100);
+        } else {
+            REQUIRE(maxdom_sjid[1] > 2000000000);
+        }
+
+        soma_dataframe->close();
+
         REQUIRE(soma_dataframe->nnz() == 2);
         write_sjid_u32_str_data_from(8);
         REQUIRE(soma_dataframe->nnz() == 4);
@@ -588,6 +629,35 @@ TEST_CASE_METHOD(
             // Implicitly we expect no throw
             write_sjid_u32_str_data_from(SOMA_JOINID_DIM_MAX);
         }
+
+        // Check domainish accessors after resize
+        soma_dataframe->open(OpenMode::read);
+
+        non_empty_domain = soma_dataframe->get_non_empty_domain();
+        ned_sjid = ArrowAdapter::get_table_column_by_name<int64_t>(
+            non_empty_domain, "soma_joinid");
+
+        soma_domain = soma_dataframe->get_soma_domain();
+        dom_sjid = ArrowAdapter::get_table_column_by_name<int64_t>(
+            soma_domain, "soma_joinid");
+
+        soma_maxdomain = soma_dataframe->get_soma_maxdomain();
+        maxdom_sjid = ArrowAdapter::get_table_column_by_name<int64_t>(
+            soma_maxdomain, "soma_joinid");
+
+        if (!use_current_domain) {
+            REQUIRE(ned_sjid == std::vector<int64_t>({1, 10}));
+            REQUIRE(dom_sjid == std::vector<int64_t>({0, 100}));
+            REQUIRE(maxdom_sjid == std::vector<int64_t>({0, 100}));
+        } else {
+            REQUIRE(ned_sjid == std::vector<int64_t>({1, 102}));
+            REQUIRE(dom_sjid == std::vector<int64_t>({0, 199}));
+            REQUIRE(maxdom_sjid.size() == 2);
+            REQUIRE(maxdom_sjid[0] == 0);
+            REQUIRE(maxdom_sjid[1] > 2000000000);
+        }
+
+        soma_dataframe->close();
     }
 }
 
@@ -641,6 +711,7 @@ TEST_CASE_METHOD(
                                             ->maybe_soma_joinid_shape();
         REQUIRE(actual.has_value());
         REQUIRE(actual.value() == expect);
+
         soma_dataframe->close();
 
         REQUIRE(soma_dataframe->nnz() == 0);
@@ -652,6 +723,51 @@ TEST_CASE_METHOD(
         write_sjid_u32_str_data_from(8);
         REQUIRE(soma_dataframe->nnz() == 4);
 
+        // Check domainish accessors before resize
+        soma_dataframe->open(OpenMode::read);
+
+        ArrowTable non_empty_domain = soma_dataframe->get_non_empty_domain();
+        std::vector<int64_t>
+            ned_sjid = ArrowAdapter::get_table_column_by_name<int64_t>(
+                non_empty_domain, "soma_joinid");
+        std::vector<uint32_t>
+            ned_u32 = ArrowAdapter::get_table_column_by_name<uint32_t>(
+                non_empty_domain, "myuint32");
+
+        ArrowTable soma_domain = soma_dataframe->get_soma_domain();
+        std::vector<int64_t>
+            dom_sjid = ArrowAdapter::get_table_column_by_name<int64_t>(
+                soma_domain, "soma_joinid");
+        std::vector<uint32_t>
+            dom_u32 = ArrowAdapter::get_table_column_by_name<uint32_t>(
+                soma_domain, "myuint32");
+
+        ArrowTable soma_maxdomain = soma_dataframe->get_soma_maxdomain();
+        std::vector<int64_t>
+            maxdom_sjid = ArrowAdapter::get_table_column_by_name<int64_t>(
+                soma_maxdomain, "soma_joinid");
+        std::vector<uint32_t>
+            maxdom_u32 = ArrowAdapter::get_table_column_by_name<uint32_t>(
+                soma_maxdomain, "myuint32");
+
+        REQUIRE(ned_sjid == std::vector<int64_t>({1, 10}));
+        REQUIRE(ned_u32 == std::vector<uint32_t>({1234, 5678}));
+
+        REQUIRE(dom_sjid == std::vector<int64_t>({0, 100}));
+        REQUIRE(dom_u32 == std::vector<uint32_t>({0, 10000}));
+
+        REQUIRE(maxdom_sjid.size() == 2);
+        REQUIRE(maxdom_u32.size() == 2);
+
+        REQUIRE(maxdom_u32[0] == 0);
+        if (!use_current_domain) {
+            REQUIRE(maxdom_u32[1] == 10000);
+        } else {
+            REQUIRE(maxdom_u32[1] > 2000000000);
+        }
+
+        soma_dataframe->close();
+
         // Check shape after write
         soma_dataframe = open(OpenMode::read);
         expect = dim_infos[0].dim_max + 1;
@@ -707,6 +823,55 @@ TEST_CASE_METHOD(
             // Implicitly we expect no throw
             write_sjid_u32_str_data_from(SOMA_JOINID_DIM_MAX);
         }
+
+        // Check domainish accessors after resize
+        soma_dataframe->open(OpenMode::read);
+
+        non_empty_domain = soma_dataframe->get_non_empty_domain();
+        ned_sjid = ArrowAdapter::get_table_column_by_name<int64_t>(
+            non_empty_domain, "soma_joinid");
+        ned_u32 = ArrowAdapter::get_table_column_by_name<uint32_t>(
+            non_empty_domain, "myuint32");
+
+        soma_domain = soma_dataframe->get_soma_domain();
+        dom_sjid = ArrowAdapter::get_table_column_by_name<int64_t>(
+            soma_domain, "soma_joinid");
+        dom_u32 = ArrowAdapter::get_table_column_by_name<uint32_t>(
+            soma_domain, "myuint32");
+
+        soma_maxdomain = soma_dataframe->get_soma_maxdomain();
+        maxdom_sjid = ArrowAdapter::get_table_column_by_name<int64_t>(
+            soma_maxdomain, "soma_joinid");
+        maxdom_u32 = ArrowAdapter::get_table_column_by_name<uint32_t>(
+            soma_maxdomain, "myuint32");
+
+        if (!use_current_domain) {
+            REQUIRE(ned_sjid == std::vector<int64_t>({1, 10}));
+            REQUIRE(ned_u32 == std::vector<uint32_t>({1234, 5678}));
+
+            REQUIRE(dom_sjid == std::vector<int64_t>({0, 100}));
+            REQUIRE(dom_u32 == std::vector<uint32_t>({0, 10000}));
+
+            REQUIRE(maxdom_sjid == std::vector<int64_t>({0, 100}));
+            REQUIRE(maxdom_u32 == std::vector<uint32_t>({0, 10000}));
+
+        } else {
+            REQUIRE(ned_sjid == std::vector<int64_t>({1, 102}));
+            REQUIRE(ned_u32 == std::vector<uint32_t>({1234, 5678}));
+
+            REQUIRE(dom_sjid == std::vector<int64_t>({0, 199}));
+            REQUIRE(dom_u32 == std::vector<uint32_t>({0, 10000}));
+
+            REQUIRE(maxdom_sjid.size() == 2);
+            REQUIRE(maxdom_sjid[0] == 0);
+            REQUIRE(maxdom_sjid[1] > 2000000000);
+
+            REQUIRE(maxdom_u32.size() == 2);
+            REQUIRE(maxdom_u32[0] == 0);
+            REQUIRE(maxdom_u32[1] > 2000000000);
+        }
+
+        soma_dataframe->close();
     }
 }
 
@@ -782,6 +947,41 @@ TEST_CASE_METHOD(
         actual = soma_dataframe->maybe_soma_joinid_shape();
         REQUIRE(actual.has_value());
         REQUIRE(actual.value() == expect);
+
+        // Check domainish accessors before resize
+        ArrowTable non_empty_domain = soma_dataframe->get_non_empty_domain();
+        std::vector<int64_t>
+            ned_sjid = ArrowAdapter::get_table_column_by_name<int64_t>(
+                non_empty_domain, "soma_joinid");
+        std::vector<std::string>
+            ned_str = ArrowAdapter::get_table_string_column_by_name(
+                non_empty_domain, "mystring");
+
+        ArrowTable soma_domain = soma_dataframe->get_soma_domain();
+        std::vector<int64_t>
+            dom_sjid = ArrowAdapter::get_table_column_by_name<int64_t>(
+                soma_domain, "soma_joinid");
+        std::vector<std::string>
+            dom_str = ArrowAdapter::get_table_string_column_by_name(
+                soma_domain, "mystring");
+
+        ArrowTable soma_maxdomain = soma_dataframe->get_soma_maxdomain();
+        std::vector<int64_t>
+            maxdom_sjid = ArrowAdapter::get_table_column_by_name<int64_t>(
+                soma_maxdomain, "soma_joinid");
+        std::vector<std::string>
+            maxdom_str = ArrowAdapter::get_table_string_column_by_name(
+                soma_maxdomain, "mystring");
+
+        REQUIRE(ned_sjid == std::vector<int64_t>({1, 10}));
+        REQUIRE(ned_str == std::vector<std::string>({"apple", "bat"}));
+
+        REQUIRE(dom_sjid == std::vector<int64_t>({0, 100}));
+        REQUIRE(dom_str == std::vector<std::string>({"", ""}));
+
+        REQUIRE(maxdom_sjid == std::vector<int64_t>({0, 100}));
+        REQUIRE(maxdom_str == std::vector<std::string>({"", ""}));
+
         soma_dataframe->close();
 
         // Resize
@@ -831,6 +1031,41 @@ TEST_CASE_METHOD(
             // Implicitly we expect no throw
             write_sjid_u32_str_data_from(SOMA_JOINID_DIM_MAX);
         }
+
+        // Check domainish accessors after resize
+        soma_dataframe->open(OpenMode::read, TimestampRange(0, 2));
+
+        non_empty_domain = soma_dataframe->get_non_empty_domain();
+        ned_sjid = ArrowAdapter::get_table_column_by_name<int64_t>(
+            non_empty_domain, "soma_joinid");
+        ned_str = ArrowAdapter::get_table_string_column_by_name(
+            non_empty_domain, "mystring");
+
+        soma_domain = soma_dataframe->get_soma_domain();
+        dom_sjid = ArrowAdapter::get_table_column_by_name<int64_t>(
+            soma_domain, "soma_joinid");
+        dom_str = ArrowAdapter::get_table_string_column_by_name(
+            soma_domain, "mystring");
+
+        soma_maxdomain = soma_dataframe->get_soma_maxdomain();
+        maxdom_sjid = ArrowAdapter::get_table_column_by_name<int64_t>(
+            soma_maxdomain, "soma_joinid");
+        maxdom_str = ArrowAdapter::get_table_string_column_by_name(
+            soma_maxdomain, "mystring");
+
+        REQUIRE(ned_sjid == std::vector<int64_t>({0, 0}));
+        REQUIRE(ned_str == std::vector<std::string>({"", ""}));
+
+        REQUIRE(dom_sjid == std::vector<int64_t>({0, 100}));
+        REQUIRE(dom_str == std::vector<std::string>({"", ""}));
+
+        REQUIRE(maxdom_sjid == std::vector<int64_t>({0, 100}));
+        REQUIRE(maxdom_str == std::vector<std::string>({"", ""}));
+
+        REQUIRE(ned_str == std::vector<std::string>({"", ""}));
+        REQUIRE(dom_str == std::vector<std::string>({"", ""}));
+
+        soma_dataframe->close();
     }
 }
 
@@ -886,6 +1121,33 @@ TEST_CASE_METHOD(
         std::optional<int64_t> actual = soma_dataframe
                                             ->maybe_soma_joinid_shape();
         REQUIRE(!actual.has_value());
+
+        // Check domainish accessors before resize
+        ArrowTable non_empty_domain = soma_dataframe->get_non_empty_domain();
+        std::vector<std::string>
+            ned_str = ArrowAdapter::get_table_string_column_by_name(
+                non_empty_domain, "mystring");
+
+        ArrowTable soma_domain = soma_dataframe->get_soma_domain();
+        std::vector<std::string>
+            dom_str = ArrowAdapter::get_table_string_column_by_name(
+                soma_domain, "mystring");
+
+        ArrowTable soma_maxdomain = soma_dataframe->get_soma_maxdomain();
+        std::vector<std::string>
+            maxdom_str = ArrowAdapter::get_table_string_column_by_name(
+                soma_maxdomain, "mystring");
+
+        REQUIRE(ned_str == std::vector<std::string>({"", ""}));
+
+        if (!use_current_domain) {
+            REQUIRE(dom_str == std::vector<std::string>({"", ""}));
+            REQUIRE(maxdom_str == std::vector<std::string>({"", ""}));
+        } else {
+            REQUIRE(dom_str == std::vector<std::string>({"", "\xff"}));
+            REQUIRE(maxdom_str == std::vector<std::string>({"", ""}));
+        }
+
         soma_dataframe->close();
 
         REQUIRE(soma_dataframe->nnz() == 0);
@@ -945,5 +1207,32 @@ TEST_CASE_METHOD(
             // Implicitly we expect no throw
             write_sjid_u32_str_data_from(SOMA_JOINID_DIM_MAX);
         }
+
+        // Check domainish accessors after resize
+        soma_dataframe->open(OpenMode::read, TimestampRange(0, 2));
+
+        non_empty_domain = soma_dataframe->get_non_empty_domain();
+        ned_str = ArrowAdapter::get_table_string_column_by_name(
+            non_empty_domain, "mystring");
+
+        soma_domain = soma_dataframe->get_soma_domain();
+        dom_str = ArrowAdapter::get_table_string_column_by_name(
+            soma_domain, "mystring");
+
+        soma_maxdomain = soma_dataframe->get_soma_maxdomain();
+        maxdom_str = ArrowAdapter::get_table_string_column_by_name(
+            soma_maxdomain, "mystring");
+
+        REQUIRE(ned_str == std::vector<std::string>({"", ""}));
+
+        if (!use_current_domain) {
+            REQUIRE(dom_str == std::vector<std::string>({"", ""}));
+            REQUIRE(maxdom_str == std::vector<std::string>({"", ""}));
+        } else {
+            REQUIRE(dom_str == std::vector<std::string>({"", "\xff"}));
+            REQUIRE(maxdom_str == std::vector<std::string>({"", ""}));
+        }
+
+        soma_dataframe->close();
     }
 }