Repository: single-cell-data/TileDB-SOMA (language: python)

Issue:
[python/r/c++] Revisit `shape` for component arrays
## PRs

**Merged PRs:**

<details>

* #2909 `kerl/schevo-timestamp-methodize`
* #2913 `kerl/name-neaten`
* #2908 `kerl/ut-soma-exc-simplify`
* #2910 `kerl/test-common-parameterize`
* #2918 `kerl/cpp-test-deadstrip`
* #2919 `kerl/minor-unit-test-helper-mod`
* #2936 `kerl/cpp-ut-helper-neaten`
* #2938 `kerl/more-cur-dom-parameterize`
* #2915 `kerl/cpp-strict-int64-shape`
* #2911 `kerl/arrow-util-current-domain-optional`
* #2939 `kerl/step-two-temp`
* #2947 `kerl/cpp-ndarray-resize-testing`
* #2945 `kerl/dataframe-test-fixture`
* #2944 `kerl/cpp-variant-indexed-dataframes`
* #2916 `kerl/sdf-shape`
* #2917 `kerl/cpp-resizes`
* #2948 `kerl/upgrade-shape-int64`
* #2951 `kerl/sdf-test-accessors`
* #2953 `kerl/py-r-accessor-plumbing`
* #2957 `kerl/sdf-domain-accessors`
* #2960 `kerl/dense-link`
* #2963 `kerl/nightly-fix`
* #2970 `kerl/dense-writeable-after-create`
* #2968 `kerl/minor-trim`
* #2969 `kerl/more-py-domain-name-neaten`
* #2972 `kerl/libtiledbsoma-env-logging-level`
* #2962 `kerl/py-r-creation-paths`
* #2950 `kerl/py-r-test-2`
* #2994 `kerl/nanoarrow-helpers`
* #3011 `kerl/polydom3`
* #3017 `kerl/polydom5`
* #3018 `kerl/polydom6`
* #2990 `kerl/variant-nnz-bug`
* #3019 `kerl/index-swap`
* #3020 `kerl/ut-max-shape`
* #3012 `kerl/polydom4`
* #3025 `kerl/fix-3020-merge`
* #3026 `kerl/one-more-rename`
* #3028 `kerl/ff-not`
* #3029 `kerl/ut-vg`
* #3030 `kerl/table-utils-memory`
* #3067 `kerl/improve-sdf-test-field-names`
* #3068 `kerl/ut-generate`
* #3069 `kerl/cpp-sdf-domain-at-create`
* #3027 `kerl/hll-domainish`
* #3088 `kerl/max-domain-int64`
* #3090 `kerl/maybe-resize-soma-joinid-cpp-tweak`
* #3032 `kerl/sdf-domain-at-create` -- fixes #2967
* #3091 `kerl/maybe-resize-soma-joinid-py-r`
* #3095 `kerl/cpp-exp-resize-prep`
* #3089 `kerl/r-dataframe-shapeable`
* https://github.com/single-cell-data/SOMA/pull/233
* #3125 `kerl/cpp-ut-name-shortens`
* #3127 `kerl/helper-rename`
* #3130 `kerl/cpp-can-resizers-names`
* #3132 `kerl/cpp-dataframe-sizing-helpers`
* #3139 `kerl/cpp-dataframe-upgrade-test`
* #3140 `kerl/py-resizer-connects`
* #3151 `kerl/py-can-upgrade-shape`
* #3152 `kerl/registration-shape-acceessors`
* #3156 `kerl/py-exp-shaping`
* #3157 `kerl/py-exp-shaping2`
* #3148 `kerl/py-exp-resize`
* #3191 `kerl/py-domain-at-create-ut-1`
* #3190 `kerl/py-domain-at-create-ut-2`
* #3192 `kerl/py-domain-at-create-ut-3`
* #3193 `kerl/py-domain-at-create-ut-4`
* #3194 `kerl/py-domain-at-create-ut-5`
* #3203 `kerl/min-size-2`
* #3208 `kerl/r-min-sizing`
* #3211 `kerl/cpp-ugr-dom`
* #3232 `kerl/ff-interop`
* #3230 `kerl/ffon`
* #3234 `kerl/docstring-prune`
* #3236 `kerl/prefixing`
* #3241 `kerl/fix-bad-merge`
* #3235 `kerl/py-r-ugr-dom`
* #3237 `kerl/py-r-ugr-dom-2`
* #3238 `kerl/py-r-ugr-dom-3`
* #3253 `kerl/set-coords-rename`
* #3261 `kerl/pybind11-nda-sizing`
* #3265 `kerl/dense-227-a`
* #3263 `kerl/dense-range-trim`
* #3268 `kerl/dim-explosion`
* #3269 `kerl/python-227-dense-ned-read`
* #3270 `kerl/r-227-dense-fixes`
* #3280 `kerl/r-dense-227-more`
* #3286 `kerl/more-fn4m`
* #3283 `kerl/readthedocs-pre-1.15`
* #3288 `kerl/dense-ugrsh`
* #3289 `kerl/notebook-shape-upgrade`
* #3285 `kerl/new-shape-doc-updates`
* #3295 `kerl/notebook-data-refresh`
* #3290 `kerl/notebook-new-shape-refresh`
* #3301 `kerl/ffena`
* #3303 `kerl/r-data-refresh`
* #3300 `kerl/sdf-sjid-lower-zero`
* #3296 `kerl/dense-example-data-refresh`
* #3294 `kerl/new-shape-notebook-and-vignette`
* #3308 `kerl/upgrade-experiment-resources`
* #3309 `kerl/fix-notebook-merge`
* #3321 `kerl/more-use-shape`
* #3358 `kerl/revert-3300`
* #3368 `kerl/227a`
* #3369 `kerl/ucd1`
* #3370 `kerl/ucd2`
* #3371 `kerl/ucd3`
* #3372 `kerl/ucd4`
* #3396 `kerl/domain-at-create-docstrings`
* #3302 `kerl/new-shape-vignette`
* https://github.com/single-cell-data/SOMA/pull/250
* #3399 `kerl/new-shape-more-docstrings`
* #3400 `kerl/check-only-r`
* https://github.com/TileDB-Inc/somacore-feedstock/pull/27

</details>

**Closed/abandoned PRs:**

<details>

* #2785 -- This was only dogfooding for the core 2.25 release -- not to be merged
* #2952 `kerl/feature-flag-temp` -- folded into 2962
* #2995 `kerl/polydom`
* #2964 `kerl/tiledbsoma-io-test`
* #3189 `kerl/min-size`
* #3220 `kerl/cpp-ugr-dom-2`
* #3244 `kerl/dense-227-fixes`
* #3276 `kerl/dataframe-shape`

</details>

**Issues which are related but non-blocking:**

* https://github.com/single-cell-data/SOMA/pull/216
* https://github.com/TileDB-Inc/TileDB/pull/5303
* #2966
* #3081
* Note: R append mode does not exist yet -- see #1630 -- so an experiment-level resizer is not a priority in R
* #3271 
* #3272
* #3273

**See also:** [[sc-51048]](https://app.shortcut.com/tiledb-inc/story/51048).

## Problem to be solved

Users want to know the `shape` of an array, in the SciPy sense:

* Reads and writes are bounds-checked against the shape
* This retains its value regardless of which values of a sparse array are or are not actually occupied
* Users can `resize`.
  * Some users need the ability to grow their datasets later, using either `tiledbsoma.io`'s append mode, or subsequent writes using the `tiledbsoma` API.
  * Note that the cellxgene census doesn't need this: eact week's published census has fixed shape, and any updates will happen in new storage, on a new week.

Using TileDB-SOMA up until the present:

* The TIleDB `domain` is immutable after array creation
  * This does bounds-checking for reads and writes, which is good
  * To leverage this to function as a `shape`, users would need to set the `domain` at array-creation time. However, users lose the ability to grow their datasets later.
* There is a `non_empty_domain` accessor
  * This only indicates min/max coordinates _at which data exists_. Consider an `X` array for 100 cells and 200 genes. If non-zero expression counts exist only for cell join IDs 2-17, then the `non_empty_domain` will indicate `(2,17)` along `soma_dim_0`.
  * Consider an `obms["X_pca"]` within the same experiment. This may be 100 cells by 50 PCA components: we need a placd to store the number 50.
  * Therefore users cannot leverage this to function as a `shape` accessor.
* We have offered a `used_shape` accessor since TileDB-SOMA 1.5.
  * This functions as a `shape` accessor, in the SciPy sense, but it is not multi-writer safe.

New feature for TileDB-SOMA 1.15:

* Arrays will have a  `shape`
* Reads and writes are bounds-checked against the shape
* This retains its value regardless of which values of a sparse array are or are not actually occupied
* Users can `resize`
* The `used_shape` accessor will be deprecated in TileDB-SOMA 1.13, and slated for removal in TileDB-SOMA 1.14.

Compatiblity:

This will now require users to do an explicit `resize` before appending/growing TileDB-SOMA Experiments. Guidance in the form of example notebooks will be provided.

##  Tracking

See also: [[sc-41074]](https://app.shortcut.com/tiledb-inc/story/41074) and [[sc-51048]](https://app.shortcut.com/tiledb-inc/story/51048).

## Scheduling

Support arrives in TileDB Core 2.25. Deprecations for TileDB-SOMA will be released with 1.13. Full support within TileDB-SOMA will be release in 1.14.

## Details

SOMA API mods as we've discussed in a Google doc are as follows.

### `SOMADataFrame`

* `create`: Retain the `domain` argument
  * Issue:
    * Core has a `(lo, hi)` tuple per dim, e.g. `(0,99)` or `(10,19)`
    * SOMA has count per dim, with 0 implicit: e.g. 100 or 20
    * For `SparseNDArray` and `DenseNDArray` core can have `(lo, hi)` and SOMA can have `count`
    * For `DataFrame` there can be multiple dims --- default is a single `soma_joinid`
    * That could be treated either in `(lo, hi)` fashion or `count` fashion
    * However additional dims (e.g. `cell_type`) can be on any type, including strings, floats, etc. where there is no implicit lo=0
    * Therefore we need to keep the current SOMA API wherein `DataFrame` takes a `domain` argument (in `(lo, hi)` fashion) and not a `shape` argument (in `count` fashion)

### `SparseNDArray and DenseNDArray`

* `create`
  * Have an optional shape argument which is of type `Tuple[Int,...]` where each element is the cell count of the corresponding dimension
    * If unsupplied, or if supplied but None in any slot: use the minimum 0 in each slot – nothing larger makes sense since we will not support downsize
  * User guidance should make clear that it will not be possible to create an ‘old’ style array with the ‘new style’ API. (See also the upgrade logic below.)

### All three of `SOMADataFrame`, `SparseNDArray`, `DenseNDArray`

* `write`
  * For new arrays, created with the new shape feature:
    * Core will bounds-check that coordinates provided at `write` time are within the current shape
    * Core will raise `tiledb.cc.TileDBError` to TileDB-SOMA, which will catch and raise `IndexError`, and R-standard behavior on the R side
  * For old arrays created before this feature:
    * Core will not bounds-check that coordinates provided at write time are within the current shape
* Existing `used_shape` accessor
  * TileDB-SOMA will deprecate this over a release cycle.
  * For new arrays: raise `NotImplementedError`
  * For old arrays: return what’s currently returned, with a deprecation warning.
  * Mechanism for determining old vs. new: `array.schema.version` (the core storage version).
* Existing `shape` accessor
  * For new arrays:
    * Have this return the new shape as proposed by core, no longer returning the TileDB domain.
  * For old arrays created before this feature:
    * Return the TileDB domain as now.
* Existing `non_empty_domain` accessor
  * Same behavior for old and new arrays (unaffected by this proposal).
  * Keep this accessor supported, but, with user notes that it’s generally non-useful
  * This should return None (or R equivalent) when there is a schema but no data have been written.
* New `maxshape` accessor
  * Maps the core-level `(lo, hi)` accessor for domain to count-style accessor hi+1. E.g. if the core domain is either `(0,99)` or `(50,99)` then TileDB-SOMA `maxshape` will say 100.
  * Same behavior for old and new arrays.
  * Let users query for what the TileDB domain is, with user notes that it’s the maximum that users can reshape to.
  * Issac suggests: maybe `domain` or `maxshape` (see h5py).
* New `resize` mutator
  * Note: `reshape` means something else in the community (numpy, zarr, h5py), e.g. a 5x20 (total 100 cells) being reinterpreted as 4x25 (still 100 cells). The standard name for changing cell-count is `resize`.
  * For old arrays created before this feature: raise `NotImplementedError`.
  * For new arrays:
    * Will raise `ValueError` if the new shape is smaller on any dim than currently in storage
    * Regardless of whether any data have been written whatsoever
    * Will raise `ValueError` if the new shape exceeds the TileDB domain from create time, which will serve TileDB-SOMA in a role of “max possible shape the user can reshape to”
    * Otherwise, any calls to write from this point will bounds-check writes within this new shape
    * **We don’t expect resize to be multi-writer safe with regard to write ; user notes must be clear on this point**
* New `tiledbsoma_upgrade_shape` method for SparseNDArray and DenseNDArray
  * This will leverage `array.schema.version` to see if an upgrade is needed
  * Leverage core support for storage-version updates
  * This will take a shape argument as in `create`
  * For arrays created with “just-right” size: this will succeed
  * For arrays created with “room-for-growth” / “two billion-ish” size: this will succeed
  * If the user passes a shape which exceeds the current TileDB domain: this will fail
* New `tiledbsoma_upgrade_domain method` for `DataFrame`
  * Same as for `SparseNDArray`/`DenseNDArray` except it will take a domain at the SOMA-API level just as `DataFrame`'s create method

### `tiledbsoma.io`

* The user-facing API has no shape arguments and thus won’t need changing.
* Internally to `tiledbsoma.io`, we’ll still ask the tiledbsoma API for the “big domain” (2 billionish)
* Append mode:
  * Will need a new `resize` method at the `Experiment` level
  * Users will need to:
    * Register as now
    * Call the experiment-level `resize`
      * Could be `exp.resize(...)`, or (better) this could be `tiledbsoma.io.reshape_experiment`
  * In either case: this method will take the new `obs` and `var` counts as inputs:
    * `exp.obs.reshape` to new `obs` count
    * `exp.ms[name].var.reshape` to new `var` count
    * `exp.ms[name].X[name].reshape` to new `obs` count x `var` count
    * `exp.ms[name].obsm[name].reshape` to new `obs` count x same width
    * `exp.ms[name].obsp[name].reshape` to new `obs` count x `obs` count
    * `exp.ms[name].varm[name].reshape` to new `var` count x same width
    * `exp.ms[name].varp[name].reshape` to new `var` count x `var` count
  * Do the individual append-mode writes as now

[python/r/c++] Revisit `shape` for component arrays
## PRs

**Merged PRs:**

<details>

* #2909 `kerl/schevo-timestamp-methodize`
* #2913 `kerl/name-neaten`
* #2908 `kerl/ut-soma-exc-simplify`
* #2910 `kerl/test-common-parameterize`
* #2918 `kerl/cpp-test-deadstrip`
* #2919 `kerl/minor-unit-test-helper-mod`
* #2936 `kerl/cpp-ut-helper-neaten`
* #2938 `kerl/more-cur-dom-parameterize`
* #2915 `kerl/cpp-strict-int64-shape`
* #2911 `kerl/arrow-util-current-domain-optional`
* #2939 `kerl/step-two-temp`
* #2947 `kerl/cpp-ndarray-resize-testing`
* #2945 `kerl/dataframe-test-fixture`
* #2944 `kerl/cpp-variant-indexed-dataframes`
* #2916 `kerl/sdf-shape`
* #2917 `kerl/cpp-resizes`
* #2948 `kerl/upgrade-shape-int64`
* #2951 `kerl/sdf-test-accessors`
* #2953 `kerl/py-r-accessor-plumbing`
* #2957 `kerl/sdf-domain-accessors`
* #2960 `kerl/dense-link`
* #2963 `kerl/nightly-fix`
* #2970 `kerl/dense-writeable-after-create`
* #2968 `kerl/minor-trim`
* #2969 `kerl/more-py-domain-name-neaten`
* #2972 `kerl/libtiledbsoma-env-logging-level`
* #2962 `kerl/py-r-creation-paths`
* #2950 `kerl/py-r-test-2`
* #2994 `kerl/nanoarrow-helpers`
* #3011 `kerl/polydom3`
* #3017 `kerl/polydom5`
* #3018 `kerl/polydom6`
* #2990 `kerl/variant-nnz-bug`
* #3019 `kerl/index-swap`
* #3020 `kerl/ut-max-shape`
* #3012 `kerl/polydom4`
* #3025 `kerl/fix-3020-merge`
* #3026 `kerl/one-more-rename`
* #3028 `kerl/ff-not`
* #3029 `kerl/ut-vg`
* #3030 `kerl/table-utils-memory`
* #3067 `kerl/improve-sdf-test-field-names`
* #3068 `kerl/ut-generate`
* #3069 `kerl/cpp-sdf-domain-at-create`
* #3027 `kerl/hll-domainish`
* #3088 `kerl/max-domain-int64`
* #3090 `kerl/maybe-resize-soma-joinid-cpp-tweak`
* #3032 `kerl/sdf-domain-at-create` -- fixes #2967
* #3091 `kerl/maybe-resize-soma-joinid-py-r`
* #3095 `kerl/cpp-exp-resize-prep`
* #3089 `kerl/r-dataframe-shapeable`
* https://github.com/single-cell-data/SOMA/pull/233
* #3125 `kerl/cpp-ut-name-shortens`
* #3127 `kerl/helper-rename`
* #3130 `kerl/cpp-can-resizers-names`
* #3132 `kerl/cpp-dataframe-sizing-helpers`
* #3139 `kerl/cpp-dataframe-upgrade-test`
* #3140 `kerl/py-resizer-connects`
* #3151 `kerl/py-can-upgrade-shape`
* #3152 `kerl/registration-shape-acceessors`
* #3156 `kerl/py-exp-shaping`
* #3157 `kerl/py-exp-shaping2`
* #3148 `kerl/py-exp-resize`
* #3191 `kerl/py-domain-at-create-ut-1`
* #3190 `kerl/py-domain-at-create-ut-2`
* #3192 `kerl/py-domain-at-create-ut-3`
* #3193 `kerl/py-domain-at-create-ut-4`
* #3194 `kerl/py-domain-at-create-ut-5`
* #3203 `kerl/min-size-2`
* #3208 `kerl/r-min-sizing`
* #3211 `kerl/cpp-ugr-dom`
* #3232 `kerl/ff-interop`
* #3230 `kerl/ffon`
* #3234 `kerl/docstring-prune`
* #3236 `kerl/prefixing`
* #3241 `kerl/fix-bad-merge`
* #3235 `kerl/py-r-ugr-dom`
* #3237 `kerl/py-r-ugr-dom-2`
* #3238 `kerl/py-r-ugr-dom-3`
* #3253 `kerl/set-coords-rename`
* #3261 `kerl/pybind11-nda-sizing`
* #3265 `kerl/dense-227-a`
* #3263 `kerl/dense-range-trim`
* #3268 `kerl/dim-explosion`
* #3269 `kerl/python-227-dense-ned-read`
* #3270 `kerl/r-227-dense-fixes`
* #3280 `kerl/r-dense-227-more`
* #3286 `kerl/more-fn4m`
* #3283 `kerl/readthedocs-pre-1.15`
* #3288 `kerl/dense-ugrsh`
* #3289 `kerl/notebook-shape-upgrade`
* #3285 `kerl/new-shape-doc-updates`
* #3295 `kerl/notebook-data-refresh`
* #3290 `kerl/notebook-new-shape-refresh`
* #3301 `kerl/ffena`
* #3303 `kerl/r-data-refresh`
* #3300 `kerl/sdf-sjid-lower-zero`
* #3296 `kerl/dense-example-data-refresh`
* #3294 `kerl/new-shape-notebook-and-vignette`
* #3308 `kerl/upgrade-experiment-resources`
* #3309 `kerl/fix-notebook-merge`
* #3321 `kerl/more-use-shape`
* #3358 `kerl/revert-3300`
* #3368 `kerl/227a`
* #3369 `kerl/ucd1`
* #3370 `kerl/ucd2`
* #3371 `kerl/ucd3`
* #3372 `kerl/ucd4`
* #3396 `kerl/domain-at-create-docstrings`
* #3302 `kerl/new-shape-vignette`
* https://github.com/single-cell-data/SOMA/pull/250
* #3399 `kerl/new-shape-more-docstrings`
* #3400 `kerl/check-only-r`
* https://github.com/TileDB-Inc/somacore-feedstock/pull/27

</details>

**Closed/abandoned PRs:**

<details>

* #2785 -- This was only dogfooding for the core 2.25 release -- not to be merged
* #2952 `kerl/feature-flag-temp` -- folded into 2962
* #2995 `kerl/polydom`
* #2964 `kerl/tiledbsoma-io-test`
* #3189 `kerl/min-size`
* #3220 `kerl/cpp-ugr-dom-2`
* #3244 `kerl/dense-227-fixes`
* #3276 `kerl/dataframe-shape`

</details>

**Issues which are related but non-blocking:**

* https://github.com/single-cell-data/SOMA/pull/216
* https://github.com/TileDB-Inc/TileDB/pull/5303
* #2966
* #3081
* Note: R append mode does not exist yet -- see #1630 -- so an experiment-level resizer is not a priority in R
* #3271 
* #3272
* #3273

**See also:** [[sc-51048]](https://app.shortcut.com/tiledb-inc/story/51048).

## Problem to be solved

Users want to know the `shape` of an array, in the SciPy sense:

* Reads and writes are bounds-checked against the shape
* This retains its value regardless of which values of a sparse array are or are not actually occupied
* Users can `resize`.
  * Some users need the ability to grow their datasets later, using either `tiledbsoma.io`'s append mode, or subsequent writes using the `tiledbsoma` API.
  * Note that the cellxgene census doesn't need this: eact week's published census has fixed shape, and any updates will happen in new storage, on a new week.

Using TileDB-SOMA up until the present:

* The TIleDB `domain` is immutable after array creation
  * This does bounds-checking for reads and writes, which is good
  * To leverage this to function as a `shape`, users would need to set the `domain` at array-creation time. However, users lose the ability to grow their datasets later.
* There is a `non_empty_domain` accessor
  * This only indicates min/max coordinates _at which data exists_. Consider an `X` array for 100 cells and 200 genes. If non-zero expression counts exist only for cell join IDs 2-17, then the `non_empty_domain` will indicate `(2,17)` along `soma_dim_0`.
  * Consider an `obms["X_pca"]` within the same experiment. This may be 100 cells by 50 PCA components: we need a placd to store the number 50.
  * Therefore users cannot leverage this to function as a `shape` accessor.
* We have offered a `used_shape` accessor since TileDB-SOMA 1.5.
  * This functions as a `shape` accessor, in the SciPy sense, but it is not multi-writer safe.

New feature for TileDB-SOMA 1.15:

* Arrays will have a  `shape`
* Reads and writes are bounds-checked against the shape
* This retains its value regardless of which values of a sparse array are or are not actually occupied
* Users can `resize`
* The `used_shape` accessor will be deprecated in TileDB-SOMA 1.13, and slated for removal in TileDB-SOMA 1.14.

Compatiblity:

This will now require users to do an explicit `resize` before appending/growing TileDB-SOMA Experiments. Guidance in the form of example notebooks will be provided.

##  Tracking

See also: [[sc-41074]](https://app.shortcut.com/tiledb-inc/story/41074) and [[sc-51048]](https://app.shortcut.com/tiledb-inc/story/51048).

## Scheduling

Support arrives in TileDB Core 2.25. Deprecations for TileDB-SOMA will be released with 1.13. Full support within TileDB-SOMA will be release in 1.14.

## Details

SOMA API mods as we've discussed in a Google doc are as follows.

### `SOMADataFrame`

* `create`: Retain the `domain` argument
  * Issue:
    * Core has a `(lo, hi)` tuple per dim, e.g. `(0,99)` or `(10,19)`
    * SOMA has count per dim, with 0 implicit: e.g. 100 or 20
    * For `SparseNDArray` and `DenseNDArray` core can have `(lo, hi)` and SOMA can have `count`
    * For `DataFrame` there can be multiple dims --- default is a single `soma_joinid`
    * That could be treated either in `(lo, hi)` fashion or `count` fashion
    * However additional dims (e.g. `cell_type`) can be on any type, including strings, floats, etc. where there is no implicit lo=0
    * Therefore we need to keep the current SOMA API wherein `DataFrame` takes a `domain` argument (in `(lo, hi)` fashion) and not a `shape` argument (in `count` fashion)

### `SparseNDArray and DenseNDArray`

* `create`
  * Have an optional shape argument which is of type `Tuple[Int,...]` where each element is the cell count of the corresponding dimension
    * If unsupplied, or if supplied but None in any slot: use the minimum 0 in each slot – nothing larger makes sense since we will not support downsize
  * User guidance should make clear that it will not be possible to create an ‘old’ style array with the ‘new style’ API. (See also the upgrade logic below.)

### All three of `SOMADataFrame`, `SparseNDArray`, `DenseNDArray`

* `write`
  * For new arrays, created with the new shape feature:
    * Core will bounds-check that coordinates provided at `write` time are within the current shape
    * Core will raise `tiledb.cc.TileDBError` to TileDB-SOMA, which will catch and raise `IndexError`, and R-standard behavior on the R side
  * For old arrays created before this feature:
    * Core will not bounds-check that coordinates provided at write time are within the current shape
* Existing `used_shape` accessor
  * TileDB-SOMA will deprecate this over a release cycle.
  * For new arrays: raise `NotImplementedError`
  * For old arrays: return what’s currently returned, with a deprecation warning.
  * Mechanism for determining old vs. new: `array.schema.version` (the core storage version).
* Existing `shape` accessor
  * For new arrays:
    * Have this return the new shape as proposed by core, no longer returning the TileDB domain.
  * For old arrays created before this feature:
    * Return the TileDB domain as now.
* Existing `non_empty_domain` accessor
  * Same behavior for old and new arrays (unaffected by this proposal).
  * Keep this accessor supported, but, with user notes that it’s generally non-useful
  * This should return None (or R equivalent) when there is a schema but no data have been written.
* New `maxshape` accessor
  * Maps the core-level `(lo, hi)` accessor for domain to count-style accessor hi+1. E.g. if the core domain is either `(0,99)` or `(50,99)` then TileDB-SOMA `maxshape` will say 100.
  * Same behavior for old and new arrays.
  * Let users query for what the TileDB domain is, with user notes that it’s the maximum that users can reshape to.
  * Issac suggests: maybe `domain` or `maxshape` (see h5py).
* New `resize` mutator
  * Note: `reshape` means something else in the community (numpy, zarr, h5py), e.g. a 5x20 (total 100 cells) being reinterpreted as 4x25 (still 100 cells). The standard name for changing cell-count is `resize`.
  * For old arrays created before this feature: raise `NotImplementedError`.
  * For new arrays:
    * Will raise `ValueError` if the new shape is smaller on any dim than currently in storage
    * Regardless of whether any data have been written whatsoever
    * Will raise `ValueError` if the new shape exceeds the TileDB domain from create time, which will serve TileDB-SOMA in a role of “max possible shape the user can reshape to”
    * Otherwise, any calls to write from this point will bounds-check writes within this new shape
    * **We don’t expect resize to be multi-writer safe with regard to write ; user notes must be clear on this point**
* New `tiledbsoma_upgrade_shape` method for SparseNDArray and DenseNDArray
  * This will leverage `array.schema.version` to see if an upgrade is needed
  * Leverage core support for storage-version updates
  * This will take a shape argument as in `create`
  * For arrays created with “just-right” size: this will succeed
  * For arrays created with “room-for-growth” / “two billion-ish” size: this will succeed
  * If the user passes a shape which exceeds the current TileDB domain: this will fail
* New `tiledbsoma_upgrade_domain method` for `DataFrame`
  * Same as for `SparseNDArray`/`DenseNDArray` except it will take a domain at the SOMA-API level just as `DataFrame`'s create method

### `tiledbsoma.io`

* The user-facing API has no shape arguments and thus won’t need changing.
* Internally to `tiledbsoma.io`, we’ll still ask the tiledbsoma API for the “big domain” (2 billionish)
* Append mode:
  * Will need a new `resize` method at the `Experiment` level
  * Users will need to:
    * Register as now
    * Call the experiment-level `resize`
      * Could be `exp.resize(...)`, or (better) this could be `tiledbsoma.io.reshape_experiment`
  * In either case: this method will take the new `obs` and `var` counts as inputs:
    * `exp.obs.reshape` to new `obs` count
    * `exp.ms[name].var.reshape` to new `var` count
    * `exp.ms[name].X[name].reshape` to new `obs` count x `var` count
    * `exp.ms[name].obsm[name].reshape` to new `obs` count x same width
    * `exp.ms[name].obsp[name].reshape` to new `obs` count x `obs` count
    * `exp.ms[name].varm[name].reshape` to new `var` count x same width
    * `exp.ms[name].varp[name].reshape` to new `var` count x `var` count
  * Do the individual append-mode writes as now

[python/r] Implement `resize` and `tiledbsoma_upgrade_shape`
**Issue and/or context:** As tracked on issue #2407 / [[sc-51048]](https://app.shortcut.com/tiledb-inc/story/51048). Here we implement `resize` and `upgrade_shape` for ND arrays. The dataframe mutators are deferred to a separate PR.

Note that the intended Python and R API changes are all agreed on and finalized as described in #2407.

**Changes:**

Implements and tests the mutators.

**Notes for Reviewer:**

For a separate PR: needs unit-test cases, including some saved-off arrays (in a tar file, say) lacking the new-shape feature which we can apply upgrade-shape to in unit-test cases.



Discussion / hints:
#2785 is a quick-and-dirty concept-prover -- its sole function is to flush out any API misunderstandings we might have, in prep for 2.25.0 core release.
#2785 is a quick-and-dirty concept-prover -- its sole function is to flush out any API misunderstandings we might have, in prep for 2.25.0 core release.


Fix patch:
diff --git a/libtiledbsoma/src/soma/soma_array.cc b/libtiledbsoma/src/soma/soma_array.cc
index d9818d6497..97e0f2196b 100644
--- a/libtiledbsoma/src/soma/soma_array.cc
+++ b/libtiledbsoma/src/soma/soma_array.cc
@@ -1030,6 +1030,19 @@ void SOMAArray::_cast_bit_to_uint8(
     }
 }
 
+uint64_t SOMAArray::ndim() const {
+    return tiledb_schema()->domain().ndim();
+}
+
+std::vector<std::string> SOMAArray::dimension_names() const {
+    std::vector<std::string> result;
+    auto dimensions = tiledb_schema()->domain().dimensions();
+    for (const auto& dim : dimensions) {
+        result.push_back(dim.name());
+    }
+    return result;
+}
+
 void SOMAArray::write(bool sort_coords) {
     if (mq_->query_type() != TILEDB_WRITE) {
         throw TileDBSOMAError("[SOMAArray] array must be opened in write mode");
@@ -1049,6 +1062,116 @@ void SOMAArray::consolidate_and_vacuum(std::vector<std::string> modes) {
     }
 }
 
+std::map<std::string, Enumeration> SOMAArray::get_attr_to_enum_mapping() {
+    std::map<std::string, Enumeration> result;
+    for (uint32_t i = 0; i < arr_->schema().attribute_num(); ++i) {
+        auto attr = arr_->schema().attribute(i);
+        if (attr_has_enum(attr.name())) {
+            auto enmr_label = *get_enum_label_on_attr(attr.name());
+            auto enmr = ArrayExperimental::get_enumeration(
+                *ctx_->tiledb_ctx(), *arr_, enmr_label);
+            result.insert({attr.name(), enmr});
+        }
+    }
+    return result;
+}
+
+std::optional<std::string> SOMAArray::get_enum_label_on_attr(
+    std::string attr_name) {
+    auto attr = arr_->schema().attribute(attr_name);
+    return AttributeExperimental::get_enumeration_name(
+        *ctx_->tiledb_ctx(), attr);
+}
+
+bool SOMAArray::attr_has_enum(std::string attr_name) {
+    return get_enum_label_on_attr(attr_name).has_value();
+}
+
+void SOMAArray::set_metadata(
+    const std::string& key,
+    tiledb_datatype_t value_type,
+    uint32_t value_num,
+    const void* value,
+    bool force) {
+    if (!force && key.compare(SOMA_OBJECT_TYPE_KEY) == 0)
+        throw TileDBSOMAError(SOMA_OBJECT_TYPE_KEY + " cannot be modified.");
+
+    if (!force && key.compare(ENCODING_VERSION_KEY) == 0)
+        throw TileDBSOMAError(ENCODING_VERSION_KEY + " cannot be modified.");
+
+    arr_->put_metadata(key, value_type, value_num, value);
+
+    MetadataValue mdval(value_type, value_num, value);
+    std::pair<std::string, const MetadataValue> mdpair(key, mdval);
+    metadata_.insert(mdpair);
+}
+
+void SOMAArray::delete_metadata(const std::string& key) {
+    if (key.compare(SOMA_OBJECT_TYPE_KEY) == 0) {
+        throw TileDBSOMAError(SOMA_OBJECT_TYPE_KEY + " cannot be deleted.");
+    }
+
+    if (key.compare(ENCODING_VERSION_KEY) == 0) {
+        throw TileDBSOMAError(ENCODING_VERSION_KEY + " cannot be deleted.");
+    }
+
+    arr_->delete_metadata(key);
+    metadata_.erase(key);
+}
+
+std::optional<MetadataValue> SOMAArray::get_metadata(const std::string& key) {
+    if (metadata_.count(key) == 0) {
+        return std::nullopt;
+    }
+
+    return metadata_[key];
+}
+
+std::map<std::string, MetadataValue> SOMAArray::get_metadata() {
+    return metadata_;
+}
+
+bool SOMAArray::has_metadata(const std::string& key) {
+    return metadata_.count(key) != 0;
+}
+
+uint64_t SOMAArray::metadata_num() const {
+    return metadata_.size();
+}
+
+void SOMAArray::validate(
+    OpenMode mode,
+    std::string_view name,
+    std::optional<TimestampRange> timestamp) {
+    // Validate parameters
+    auto tdb_mode = mode == OpenMode::read ? TILEDB_READ : TILEDB_WRITE;
+
+    try {
+        LOG_DEBUG(fmt::format("[SOMAArray] opening array '{}'", uri_));
+        if (timestamp) {
+            arr_ = std::make_shared<Array>(
+                *ctx_->tiledb_ctx(),
+                uri_,
+                tdb_mode,
+                TemporalPolicy(
+                    TimestampStartEnd, timestamp->first, timestamp->second));
+        } else {
+            arr_ = std::make_shared<Array>(*ctx_->tiledb_ctx(), uri_, tdb_mode);
+        }
+        LOG_TRACE(fmt::format("[SOMAArray] loading enumerations"));
+        ArrayExperimental::load_all_enumerations(
+            *ctx_->tiledb_ctx(), *(arr_.get()));
+        mq_ = std::make_unique<ManagedQuery>(arr_, ctx_->tiledb_ctx(), name);
+    } catch (const std::exception& e) {
+        throw TileDBSOMAError(
+            fmt::format("Error opening array: '{}'\n  {}", uri_, e.what()));
+    }
+}
+
+std::optional<TimestampRange> SOMAArray::timestamp() {
+    return timestamp_;
+}
+
 uint64_t SOMAArray::nnz() {
     // Verify array is sparse
     if (mq_->schema()->array_type() != TILEDB_SPARSE) {
@@ -1080,7 +1203,7 @@ uint64_t SOMAArray::nnz() {
                          frag_ts.second <= timestamp_->second)) {
                 // fragment overlaps read timestamp range, but isn't fully
                 // contained within: fall back to count_cells to sort that out.
-                return nnz_slow();
+                return _nnz_slow();
             }
         }
         // fall through: fragment is fully contained within the read timestamp
@@ -1093,7 +1216,7 @@ uint64_t SOMAArray::nnz() {
         // application's job to otherwise ensure uniqueness), then
         // sum-over-fragments is the right thing to do.
         if (!mq_->schema()->allows_dups() && frag_ts.first != frag_ts.second) {
-            return nnz_slow();
+            return _nnz_slow();
         }
     }
 
@@ -1149,10 +1272,10 @@ uint64_t SOMAArray::nnz() {
         return total_cell_num;
     }
     // Found relevant fragments with overlap, count cells
-    return nnz_slow();
+    return _nnz_slow();
 }
 
-uint64_t SOMAArray::nnz_slow() {
+uint64_t SOMAArray::_nnz_slow() {
     LOG_DEBUG(
         "[SOMAArray] nnz() found consolidated or overlapping fragments, "
         "counting cells...");
@@ -1182,98 +1305,14 @@ std::vector<int64_t> SOMAArray::shape() {
     // * Even after the new-shape feature is fully released, there will be old
     //   arrays on disk that were created before this feature existed.
     // So this is long-term code.
-    auto current_domain = _get_current_domain();
-    return current_domain.is_empty() ? _tiledb_domain() :
-                                       _tiledb_current_domain();
+    return _get_current_domain().is_empty() ? _tiledb_domain() :
+                                              _tiledb_current_domain();
 }
 
 std::vector<int64_t> SOMAArray::maxshape() {
     return _tiledb_domain();
 }
 
-std::optional<int64_t> SOMAArray::_shape_slot_if_soma_joinid_dim() {
-    const std::string dim_name = "soma_joinid";
-
-    if (!arr_->schema().domain().has_dimension(dim_name)) {
-        return std::nullopt;
-    }
-
-    auto current_domain = _get_current_domain();
-    if (current_domain.is_empty()) {
-        return std::nullopt;
-    }
-
-    auto t = current_domain.type();
-    if (t != TILEDB_NDRECTANGLE) {
-        throw TileDBSOMAError("current_domain type is not NDRECTANGLE");
-    }
-
-    NDRectangle ndrect = current_domain.ndrectangle();
-
-    auto range = ndrect.range<int64_t>(dim_name);
-    auto max = range[1] + 1;
-    return std::optional<int64_t>(max);
-}
-
-std::vector<int64_t> SOMAArray::_tiledb_domain() {
-    // Variant-indexed dataframes must use a separate path
-    _assert_dims_are_int64();
-
-    std::vector<int64_t> result;
-    auto dimensions = mq_->schema()->domain().dimensions();
-
-    for (const auto& dim : dimensions) {
-        // Callers inquiring about non-int64 shapes should not be here.
-        //
-        // In the SOMA data model:
-        // * SparseNDArray has dims which are all necessarily int64_t
-        // * DenseNDArray has dims which are all necessarily int64_t
-        // * DataFrame _default_ indexing is one dim named "soma_dim_0" of type
-        //   int64_t, however:
-        //   * Users can (and do) add other additional dims
-        //   * The SOMA data model requires that soma_joinid be present in each
-        //     DataFrame either as a dim or an attr -- and there are DataFrame
-        //     objects for which soma_joinid is not a dim at all
-        //   * These cases are all actively unit-tested within apis/python/tests
-        if (dim.type() != TILEDB_INT64) {
-            throw TileDBSOMAError("Found unexpected non-int64 dimension type.");
-        }
-        result.push_back(
-            dim.domain<int64_t>().second - dim.domain<int64_t>().first + 1);
-    }
-
-    return result;
-}
-
-std::vector<int64_t> SOMAArray::_tiledb_current_domain() {
-    // Variant-indexed dataframes must use a separate path
-    _assert_dims_are_int64();
-
-    std::vector<int64_t> result;
-
-    auto current_domain = tiledb::ArraySchemaExperimental::current_domain(
-        *ctx_->tiledb_ctx(), arr_->schema());
-
-    if (current_domain.is_empty()) {
-        throw TileDBSOMAError(
-            "Internal error: current domain requested for an array which does "
-            "not support it");
-    }
-
-    auto t = current_domain.type();
-    if (t != TILEDB_NDRECTANGLE) {
-        throw TileDBSOMAError("current_domain type is not NDRECTANGLE");
-    }
-
-    NDRectangle ndrect = current_domain.ndrectangle();
-
-    for (auto dimension_name : dimension_names()) {
-        auto range = ndrect.range<int64_t>(dimension_name);
-        result.push_back(range[1] + 1);
-    }
-    return result;
-}
-
 void SOMAArray::resize(const std::vector<int64_t>& newshape) {
     if (_get_current_domain().is_empty()) {
         throw TileDBSOMAError(
@@ -1298,7 +1337,7 @@ void SOMAArray::_set_current_domain_from_shape(
     }
 
     // Variant-indexed dataframes must use a separate path
-    _assert_dims_are_int64();
+    _check_dims_are_int64();
 
     if (_get_current_domain().is_empty()) {
         throw TileDBSOMAError(
@@ -1332,26 +1371,7 @@ void SOMAArray::_set_current_domain_from_shape(
     schema_evolution.array_evolve(uri_);
 }
 
-bool SOMAArray::_dims_are_int64() {
-    ArraySchema schema = arr_->schema();
-    Domain domain = schema.domain();
-    for (auto dimension : domain.dimensions()) {
-        if (dimension.type() != TILEDB_INT64) {
-            return false;
-        }
-    }
-    return true;
-}
-
-void SOMAArray::_assert_dims_are_int64() {
-    if (!_dims_are_int64()) {
-        throw TileDBSOMAError(
-            "[SOMAArray] internal coding error: expected all dims to be int64");
-    }
-}
-
-void SOMAArray::resize_soma_joinid_if_dim(
-    const std::vector<int64_t>& newshape) {
+void SOMAArray::maybe_resize_soma_joinid(const std::vector<int64_t>& newshape) {
     if (mq_->query_type() != TILEDB_WRITE) {
         throw TileDBSOMAError(
             "[SOMAArray::resize] array must be opened in write mode");
@@ -1362,8 +1382,7 @@ void SOMAArray::resize_soma_joinid_if_dim(
     unsigned ndim = domain.ndim();
     if (newshape.size() != 1) {
         throw TileDBSOMAError(fmt::format(
-            "[SOMAArray::resize]: newshape has dimension count {}; needed "
-            "1",
+            "[SOMAArray::resize]: newshape has dimension count {}; needed 1",
             newshape.size(),
             ndim));
     }
@@ -1388,127 +1407,132 @@ void SOMAArray::resize_soma_joinid_if_dim(
     schema_evolution.array_evolve(uri_);
 }
 
-uint64_t SOMAArray::ndim() const {
-    return tiledb_schema()->domain().ndim();
-}
+std::vector<int64_t> SOMAArray::_tiledb_current_domain() {
+    // Variant-indexed dataframes must use a separate path
+    _check_dims_are_int64();
 
-std::vector<std::string> SOMAArray::dimension_names() const {
-    std::vector<std::string> result;
-    auto dimensions = tiledb_schema()->domain().dimensions();
-    for (const auto& dim : dimensions) {
-        result.push_back(dim.name());
+    std::vector<int64_t> result;
+
+    auto current_domain = tiledb::ArraySchemaExperimental::current_domain(
+        *ctx_->tiledb_ctx(), arr_->schema());
+
+    if (current_domain.is_empty()) {
+        throw TileDBSOMAError(
+            "Internal error: current domain requested for an array which does "
+            "not support it");
+    }
+
+    auto t = current_domain.type();
+    if (t != TILEDB_NDRECTANGLE) {
+        throw TileDBSOMAError("current_domain type is not NDRECTANGLE");
+    }
+
+    NDRectangle ndrect = current_domain.ndrectangle();
+
+    for (auto dimension_name : dimension_names()) {
+        auto range = ndrect.range<int64_t>(dimension_name);
+        result.push_back(range[1] + 1);
     }
     return result;
 }
 
-std::map<std::string, Enumeration> SOMAArray::get_attr_to_enum_mapping() {
-    std::map<std::string, Enumeration> result;
-    for (uint32_t i = 0; i < arr_->schema().attribute_num(); ++i) {
-        auto attr = arr_->schema().attribute(i);
-        if (attr_has_enum(attr.name())) {
-            auto enmr_label = *get_enum_label_on_attr(attr.name());
-            auto enmr = ArrayExperimental::get_enumeration(
-                *ctx_->tiledb_ctx(), *arr_, enmr_label);
-            result.insert({attr.name(), enmr});
-        }
+std::vector<int64_t> SOMAArray::_tiledb_domain() {
+    // Variant-indexed dataframes must use a separate path
+    _check_dims_are_int64();
+
+    std::vector<int64_t> result;
+    auto dimensions = mq_->schema()->domain().dimensions();
+
+    for (const auto& dim : dimensions) {
+        result.push_back(
+            dim.domain<int64_t>().second - dim.domain<int64_t>().first + 1);
     }
+
     return result;
 }
 
-std::optional<std::string> SOMAArray::get_enum_label_on_attr(
-    std::string attr_name) {
-    auto attr = arr_->schema().attribute(attr_name);
-    return AttributeExperimental::get_enumeration_name(
-        *ctx_->tiledb_ctx(), attr);
+std::optional<int64_t> SOMAArray::_maybe_soma_joinid_shape() {
+    return _get_current_domain().is_empty() ?
+               _maybe_soma_joinid_tiledb_domain() :
+               _maybe_soma_joinid_tiledb_current_domain();
 }
 
-bool SOMAArray::attr_has_enum(std::string attr_name) {
-    return get_enum_label_on_attr(attr_name).has_value();
+std::optional<int64_t> SOMAArray::_maybe_soma_joinid_maxshape() {
+    return _maybe_soma_joinid_tiledb_domain();
 }
 
-void SOMAArray::set_metadata(
-    const std::string& key,
-    tiledb_datatype_t value_type,
-    uint32_t value_num,
-    const void* value,
-    bool force) {
-    if (!force && key.compare(SOMA_OBJECT_TYPE_KEY) == 0)
-        throw TileDBSOMAError(SOMA_OBJECT_TYPE_KEY + " cannot be modified.");
-
-    if (!force && key.compare(ENCODING_VERSION_KEY) == 0)
-        throw TileDBSOMAError(ENCODING_VERSION_KEY + " cannot be modified.");
+std::optional<int64_t> SOMAArray::_maybe_soma_joinid_tiledb_current_domain() {
+    const std::string dim_name = "soma_joinid";
 
-    arr_->put_metadata(key, value_type, value_num, value);
+    auto dom = arr_->schema().domain();
+    if (!dom.has_dimension(dim_name)) {
+        return std::nullopt;
+    }
 
-    MetadataValue mdval(value_type, value_num, value);
-    std::pair<std::string, const MetadataValue> mdpair(key, mdval);
-    metadata_.insert(mdpair);
-}
+    auto current_domain = _get_current_domain();
+    if (current_domain.is_empty()) {
+        throw TileDBSOMAError("internal coding error");
+    }
 
-void SOMAArray::delete_metadata(const std::string& key) {
-    if (key.compare(SOMA_OBJECT_TYPE_KEY) == 0) {
-        throw TileDBSOMAError(SOMA_OBJECT_TYPE_KEY + " cannot be deleted.");
+    auto t = current_domain.type();
+    if (t != TILEDB_NDRECTANGLE) {
+        throw TileDBSOMAError("current_domain type is not NDRECTANGLE");
     }
 
-    if (key.compare(ENCODING_VERSION_KEY) == 0) {
-        throw TileDBSOMAError(ENCODING_VERSION_KEY + " cannot be deleted.");
+    NDRectangle ndrect = current_domain.ndrectangle();
+
+    auto dim = dom.dimension(dim_name);
+    if (dim.type() != TILEDB_INT64) {
+        throw TileDBSOMAError(fmt::format(
+            "expected {} dim to be {}; got {}",
+            dim_name,
+            tiledb::impl::type_to_str(TILEDB_INT64),
+            tiledb::impl::type_to_str(dim.type())));
     }
 
-    arr_->delete_metadata(key);
-    metadata_.erase(key);
+    auto range = ndrect.range<int64_t>(dim_name);
+    auto max = range[1] + 1;
+    return std::optional<int64_t>(max);
 }
 
-std::optional<MetadataValue> SOMAArray::get_metadata(const std::string& key) {
-    if (metadata_.count(key) == 0) {
+std::optional<int64_t> SOMAArray::_maybe_soma_joinid_tiledb_domain() {
+    const std::string dim_name = "soma_joinid";
+
+    auto dom = arr_->schema().domain();
+    if (!dom.has_dimension(dim_name)) {
         return std::nullopt;
     }
 
-    return metadata_[key];
-}
-
-std::map<std::string, MetadataValue> SOMAArray::get_metadata() {
-    return metadata_;
-}
+    auto dim = dom.dimension(dim_name);
+    if (dim.type() != TILEDB_INT64) {
+        throw TileDBSOMAError(fmt::format(
+            "expected {} dim to be {}; got {}",
+            dim_name,
+            tiledb::impl::type_to_str(TILEDB_INT64),
+            tiledb::impl::type_to_str(dim.type())));
+    }
 
-bool SOMAArray::has_metadata(const std::string& key) {
-    return metadata_.count(key) != 0;
-}
+    auto max = dim.domain<int64_t>().second + 1;
 
-uint64_t SOMAArray::metadata_num() const {
-    return metadata_.size();
+    return std::optional<int64_t>(max);
 }
 
-void SOMAArray::validate(
-    OpenMode mode,
-    std::string_view name,
-    std::optional<TimestampRange> timestamp) {
-    // Validate parameters
-    auto tdb_mode = mode == OpenMode::read ? TILEDB_READ : TILEDB_WRITE;
-
-    try {
-        LOG_DEBUG(fmt::format("[SOMAArray] opening array '{}'", uri_));
-        if (timestamp) {
-            arr_ = std::make_shared<Array>(
-                *ctx_->tiledb_ctx(),
-                uri_,
-                tdb_mode,
-                TemporalPolicy(
-                    TimestampStartEnd, timestamp->first, timestamp->second));
-        } else {
-            arr_ = std::make_shared<Array>(*ctx_->tiledb_ctx(), uri_, tdb_mode);
+bool SOMAArray::_dims_are_int64() {
+    ArraySchema schema = arr_->schema();
+    Domain domain = schema.domain();
+    for (auto dimension : domain.dimensions()) {
+        if (dimension.type() != TILEDB_INT64) {
+            return false;
         }
-        LOG_TRACE(fmt::format("[SOMAArray] loading enumerations"));
-        ArrayExperimental::load_all_enumerations(
-            *ctx_->tiledb_ctx(), *(arr_.get()));
-        mq_ = std::make_unique<ManagedQuery>(arr_, ctx_->tiledb_ctx(), name);
-    } catch (const std::exception& e) {
-        throw TileDBSOMAError(
-            fmt::format("Error opening array: '{}'\n  {}", uri_, e.what()));
     }
+    return true;
 }
 
-std::optional<TimestampRange> SOMAArray::timestamp() {
-    return timestamp_;
+void SOMAArray::_check_dims_are_int64() {
+    if (!_dims_are_int64()) {
+        throw TileDBSOMAError(
+            "[SOMAArray] internal coding error: expected all dims to be int64");
+    }
 }
 
 }  // namespace tiledbsoma
diff --git a/libtiledbsoma/src/soma/soma_array.h b/libtiledbsoma/src/soma/soma_array.h
index 4d08e59a47..69d0494c4f 100644
--- a/libtiledbsoma/src/soma/soma_array.h
+++ b/libtiledbsoma/src/soma/soma_array.h
@@ -269,6 +269,20 @@ class SOMAArray : public SOMAObject {
         std::string_view batch_size = "auto",
         ResultOrder result_order = ResultOrder::automatic);
 
+    /**
+     * @brief Get the number of dimensions.
+     *
+     * @return uint64_t Number of dimensions.
+     */
+    uint64_t ndim() const;
+
+    /**
+     * @brief Get the name of each dimensions.
+     *
+     * @return std::vector<std::string> Name of each dimensions.
+     */
+    std::vector<std::string> dimension_names() const;
+
     /**
      * @brief Set the dimension slice using one point
      *
@@ -543,13 +557,6 @@ class SOMAArray : public SOMAObject {
         return first_read_next_;
     }
 
-    /**
-     * @brief Get the total number of unique cells in the array.
-     *
-     * @return uint64_t Total number of unique cells
-     */
-    uint64_t nnz();
-
     /**
      * @brief Get the TileDB ArraySchema. This should eventually
      * be removed in lieu of arrow_schema below.
@@ -570,131 +577,6 @@ class SOMAArray : public SOMAObject {
             ctx_->tiledb_ctx(), arr_);
     }
 
-    /**
-     * @brief Get the current capacity of each dimension.
-     *
-     * This applies to arrays all of whose dims are of type int64_t: this
-     * includes SOMASparseNDArray and SOMADenseNDArray, and default-indexed
-     * SOMADataFrame.
-     *
-     * At the TileDB-SOMA level we call this "shape". At the TileDB Core
-     * storage level this maps to "current domain".
-     *
-     * Further, we map this single n to the pair (0, n-1) since core permits a
-     * doubly inclusive pair (lo, hi) on each dimension slot.
-     *
-     * @return A vector with length equal to the number of dimensions; each
-     * value in the vector is the capacity of each dimension.
-     */
-    std::vector<int64_t> shape();
-
-    /**
-     * @brief Get the maximum resizable capacity of each dimension.
-     *
-     * This applies to arrays all of whose dims are of type int64_t: this
-     * includes SOMASparseNDArray and SOMADenseNDArray, and default-indexed
-     * SOMADataFrame.
-     *
-     * At the TileDB-SOMA level we call this "maxshape". At the TileDB Core
-     * storage level this maps to "domain".
-     *
-     * Further, we map this single n to the pair (0, n-1) since core permits a
-     * doubly inclusive pair (lo, hi) on each dimension slot.
-     *
-     * @return A vector with length equal to the number of dimensions; each
-     * value in the vector is the maximum capacity of each dimension.
-     */
-    std::vector<int64_t> maxshape();
-
-    /**
-     * @brief Resize the shape (what core calls "current domain") up to the
-     * maxshape (what core calls "domain").
-     *
-     * This applies to arrays all of whose dims are of type int64_t: this
-     * includes SOMASparseNDArray and SOMADenseNDArray, and default-indexed
-     * SOMADataFrame.
-     *
-     * @return Nothing. Raises an exception if the resize would be a downsize,
-     * which is not supported.
-     */
-    void resize(const std::vector<int64_t>& newshape);
-
-    /**
-     * @brief Given an old-style array without current domain, sets its
-     * current domain. This is applicable only to arrays having all dims
-     * of int64 type. Namely, all SparseNDArray/DenseNDArray, and
-     * default-indexed DataFrame.
-     */
-    void upgrade_shape(const std::vector<int64_t>& newshape);
-
-    /**
-     * @brief Increases the tiledbsoma shape up to at most the maxshape,
-     * resizing the soma_joinid dimension if it is a dimension.
-     *
-     * While SOMA SparseNDArray and DenseNDArray, along with default-indexed
-     * DataFrame, have int64_t dims, non-default-indexed DataFrame objects need
-     * not: it is only required that they have a dim _or_ an attr called
-     * soma_joinid. If soma_joinid is one of the dims, it will be resized while
-     * the others will be preserved. If soma_joinid is not one of the dims,
-     * nothing will be changed, as nothing _needs_ to be changed.
-     *
-     * @return Throws if the requested shape exceeds the array's create-time
-     * maxshape. Throws if the array does not have current-domain support.
-     */
-    void resize_soma_joinid_if_dim(const std::vector<int64_t>& newshape);
-
-    /**
-     * @brief Get the number of dimensions.
-     *
-     * @return uint64_t Number of dimensions.
-     */
-    uint64_t ndim() const;
-
-    /**
-     * Retrieves the non-empty domain from the array. This is the union of the
-     * non-empty domains of the array fragments.
-     */
-    template <typename T>
-    std::pair<T, T> non_empty_domain(const std::string& name) {
-        try {
-            return arr_->non_empty_domain<T>(name);
-        } catch (const std::exception& e) {
-            throw TileDBSOMAError(e.what());
-        }
-    }
-
-    /**
-     * Retrieves the non-empty domain from the array on the given dimension.
-     * This is the union of the non-empty domains of the array fragments.
-     * Applicable only to var-sized dimensions.
-     */
-    std::pair<std::string, std::string> non_empty_domain_var(
-        const std::string& name) {
-        try {
-            return arr_->non_empty_domain_var(name);
-        } catch (const std::exception& e) {
-            throw TileDBSOMAError(e.what());
-        }
-    }
-
-    /**
-     * Returns the domain of the given dimension.
-     *
-     * @tparam T Domain datatype
-     * @return Pair of [lower, upper] inclusive bounds.
-     */
-    template <typename T>
-    std::pair<T, T> domain(const std::string& name) const {
-        return arr_->schema().domain().dimension(name).domain<T>();
-    }
-
-    /**
-     * @brief Get the name of each dimensions.
-     *
-     * @return std::vector<std::string> Name of each dimensions.
-     */
-    std::vector<std::string> dimension_names() const;
-
     /**
      * @brief Get the mapping of attributes to Enumerations.
      *
@@ -755,12 +637,10 @@ class SOMAArray : public SOMAObject {
     /**
      * @brief Given a key, get the associated value datatype, number of
      * values, and value in binary form. The array must be opened in READ
-     mode,
-     * otherwise the function will error out.
+     * mode, otherwise the function will error out.
      *
      * The value may consist of more than one items of the same datatype.
-     Keys
-     * that do not exist in the metadata will be return NULL for the value.
+     * Keys that do not exist in the metadata will be return NULL for the value.
      *
      * **Example:**
      * @code{.cpp}
@@ -776,8 +656,7 @@ class SOMAArray : public SOMAObject {
      * @endcode
      *
      * @param key The key of the metadata item to be retrieved. UTF-8
-     encodings
-     *     are acceptable.
+     * encodings are acceptable.
      * @return MetadataValue (std::tuple<std::string, tiledb_datatype_t,
      * uint32_t, const void*>)
      */
@@ -820,6 +699,124 @@ class SOMAArray : public SOMAObject {
      */
     std::optional<TimestampRange> timestamp();
 
+    /**
+     * Retrieves the non-empty domain from the array. This is the union of the
+     * non-empty domains of the array fragments.
+     */
+    template <typename T>
+    std::pair<T, T> non_empty_domain(const std::string& name) {
+        try {
+            return arr_->non_empty_domain<T>(name);
+        } catch (const std::exception& e) {
+            throw TileDBSOMAError(e.what());
+        }
+    }
+
+    /**
+     * Retrieves the non-empty domain from the array on the given dimension.
+     * This is the union of the non-empty domains of the array fragments.
+     * Applicable only to var-sized dimensions.
+     */
+    std::pair<std::string, std::string> non_empty_domain_var(
+        const std::string& name) {
+        try {
+            return arr_->non_empty_domain_var(name);
+        } catch (const std::exception& e) {
+            throw TileDBSOMAError(e.what());
+        }
+    }
+
+    /**
+     * Returns the domain of the given dimension.
+     *
+     * @tparam T Domain datatype
+     * @return Pair of [lower, upper] inclusive bounds.
+     */
+    template <typename T>
+    std::pair<T, T> domain(const std::string& name) const {
+        return arr_->schema().domain().dimension(name).domain<T>();
+    }
+
+    /**
+     * @brief Get the total number of unique cells in the array.
+     *
+     * @return uint64_t Total number of unique cells
+     */
+    uint64_t nnz();
+
+    /**
+     * @brief Get the current capacity of each dimension.
+     *
+     * This applies to arrays all of whose dims are of type int64_t: this
+     * includes SOMASparseNDArray and SOMADenseNDArray, and default-indexed
+     * SOMADataFrame.
+     *
+     * At the TileDB-SOMA level we call this "shape". At the TileDB Core
+     * storage level this maps to "current domain".
+     *
+     * Further, we map this single n to the pair (0, n-1) since core permits a
+     * doubly inclusive pair (lo, hi) on each dimension slot.
+     *
+     * @return A vector with length equal to the number of dimensions; each
+     * value in the vector is the capacity of each dimension.
+     */
+    std::vector<int64_t> shape();
+
+    /**
+     * @brief Get the maximum resizable capacity of each dimension.
+     *
+     * This applies to arrays all of whose dims are of type int64_t: this
+     * includes SOMASparseNDArray and SOMADenseNDArray, and default-indexed
+     * SOMADataFrame.
+     *
+     * At the TileDB-SOMA level we call this "maxshape". At the TileDB Core
+     * storage level this maps to "domain".
+     *
+     * Further, we map this single n to the pair (0, n-1) since core permits a
+     * doubly inclusive pair (lo, hi) on each dimension slot.
+     *
+     * @return A vector with length equal to the number of dimensions; each
+     * value in the vector is the maximum capacity of each dimension.
+     */
+    std::vector<int64_t> maxshape();
+
+    /**
+     * @brief Resize the shape (what core calls "current domain") up to the
+     * maxshape (what core calls "domain").
+     *
+     * This applies to arrays all of whose dims are of type int64_t: this
+     * includes SOMASparseNDArray and SOMADenseNDArray, and default-indexed
+     * SOMADataFrame.
+     *
+     * @return Nothing. Raises an exception if the resize would be a downsize,
+     * which is not supported.
+     */
+    void resize(const std::vector<int64_t>& newshape);
+
+    /**
+     * @brief Given an old-style array without current domain, sets its
+     * current domain. This is applicable only to arrays having all dims
+     * of int64 type. Namely, all SparseNDArray/DenseNDArray, and
+     * default-indexed DataFrame.
+     */
+    void upgrade_shape(const std::vector<int64_t>& newshape);
+
+    /**
+     * @brief Increases the tiledbsoma shape up to at most the maxshape,
+     * resizing the soma_joinid dimension if it is a dimension.
+     *
+     * While SOMA SparseNDArray and DenseNDArray, along with default-indexed
+     * DataFrame, have int64_t dims, non-default-indexed DataFrame objects need
+     * not: it is only required that they have a dim _or_ an attr called
+     * soma_joinid. If soma_joinid is one of the dims, it will be resized while
+     * the others will be preserved. If soma_joinid is not one of the dims,
+     * nothing will be changed, as nothing _needs_ to be changed.
+     *
+     * @return Throws if the requested shape exceeds the array's create-time
+     * maxshape. Throws if the array does not have current-domain support.
+     */
+    void maybe_resize_soma_joinid(const std::vector<int64_t>& newshape);
+
     /**
      * Exposed for testing purposes.
      */
@@ -828,12 +825,18 @@ class SOMAArray : public SOMAObject {
     }
 
    protected:
-    // For use nominally by SOMADataFrame. This could be moved in its entirety
-    // to SOMADataFrame, but it would entail moving several SOMAArray attributes
-    // from private to protected, which has knock-on effects on the order of
-    // constructor initializers, etc.: in total it's simplest to place this
-    // here and have SOMADataFrame invoke it.
-    std::optional<int64_t> _shape_slot_if_soma_joinid_dim();
+    // These two are for use nominally by SOMADataFrame. This could be moved in
+    // its entirety to SOMADataFrame, but it would entail moving several
+    // SOMAArray attributes from private to protected, which has knock-on
+    // effects on the order of constructor initializers, etc.: in total it's
+    // simplest to place this here and have SOMADataFrame invoke it.
+    //
+    // They return the shape and maxshape slots for the soma_joinid dim, if
+    // the array has one. These are important test-points and dev-internal
+    // access-points, in particular, for the tiledbsoma-io experiment-level
+    // resizer.
+    std::optional<int64_t> _maybe_soma_joinid_shape();
+    std::optional<int64_t> _maybe_soma_joinid_maxshape();
 
    private:
     //===================================================================
@@ -876,7 +879,7 @@ class SOMAArray : public SOMAObject {
     /**
      * Same, but throws.
      */
-    void _assert_dims_are_int64();
+    void _check_dims_are_int64();
 
     /**
      * With old shape: core domain mapped to tiledbsoma shape; core current
@@ -889,6 +892,8 @@ class SOMAArray : public SOMAObject {
      */
     std::vector<int64_t> _tiledb_domain();
     std::vector<int64_t> _tiledb_current_domain();
+    std::optional<int64_t> _maybe_soma_joinid_tiledb_current_domain();
+    std::optional<int64_t> _maybe_soma_joinid_tiledb_domain();
 
     bool _extend_enumeration(
         ArrowSchema* value_schema,
@@ -1379,7 +1384,7 @@ class SOMAArray : public SOMAObject {
     bool submitted_ = false;
 
     // Unoptimized method for computing nnz() (issue `count_cells` query)
-    uint64_t nnz_slow();
+    uint64_t _nnz_slow();
 
     // ArrayBuffers to hold ColumnBuffers alive when submitting to write
     // query
diff --git a/libtiledbsoma/src/soma/soma_dataframe.cc b/libtiledbsoma/src/soma/soma_dataframe.cc
index 431b843c08..a65773a81e 100644
--- a/libtiledbsoma/src/soma/soma_dataframe.cc
+++ b/libtiledbsoma/src/soma/soma_dataframe.cc
@@ -94,10 +94,12 @@ uint64_t SOMADataFrame::count() {
     return this->nnz();
 }
 
-std::vector<int64_t> SOMADataFrame::shape() {
-    std::optional<int64_t> attempt = _shape_slot_if_soma_joinid_dim();
-    int64_t max = attempt.has_value() ? attempt.value() : this->nnz();
-    return std::vector<int64_t>({max});
+std::optional<int64_t> SOMADataFrame::maybe_soma_joinid_shape() {
+    return _maybe_soma_joinid_shape();
+}
+
+std::optional<int64_t> SOMADataFrame::maybe_soma_joinid_maxshape() {
+    return _maybe_soma_joinid_maxshape();
 }
 
 }  // namespace tiledbsoma
diff --git a/libtiledbsoma/src/soma/soma_dataframe.h b/libtiledbsoma/src/soma/soma_dataframe.h
index 6bf50ac97a..231e578baf 100644
--- a/libtiledbsoma/src/soma/soma_dataframe.h
+++ b/libtiledbsoma/src/soma/soma_dataframe.h
@@ -165,20 +165,23 @@ class SOMADataFrame : public SOMAArray {
     uint64_t count();
 
     /**
-     * For DataFrame with default indexing, namely, a single int64_t
-     * soma_joinid, returns the same as SOMAArray. For DataFrame with
-     * soma_joinid being a dim along with other dims (optional behavior), return
-     * the slot along that dim. For DataFrame with soma_joinid being an attr,
-     * not a dim at all, returns nnz().
+     * While application-level SOMA DataFrame doesn't have shape
+     * and maxshape, these are important test-point accessors,
+     * as well as crucial for experiment-level resize within tiledbsoma.io.
      *
      * Note that the SOMA spec for SOMADataFrame mandates a .domain() accessor,
-     * which is distinct, and type-polymorphic. This shape accessor exists
-     * because people can and do call .shape() on SOMA DataFrames, and we have
-     * to keep letting them do that.
+     * which is distinct, and type-polymorphic.
      *
-     * @return int64_t
+     * @return std::optional<int64_t>
+     */
+    std::optional<int64_t> maybe_soma_joinid_shape();
+
+    /**
+     * See comments for maybe_soma_joinid_shape.
+     *
+     * @return std::optional<int64_t>
      */
-    std::vector<int64_t> shape();
+    std::optional<int64_t> maybe_soma_joinid_maxshape();
 };
 
 }  // namespace tiledbsoma


Test patch:
diff --git a/libtiledbsoma/test/unit_soma_dataframe.cc b/libtiledbsoma/test/unit_soma_dataframe.cc
index dbe3abe1e9..f160a0221b 100644
--- a/libtiledbsoma/test/unit_soma_dataframe.cc
+++ b/libtiledbsoma/test/unit_soma_dataframe.cc
@@ -461,7 +461,7 @@ TEST_CASE_METHOD(
     soma_dataframe->close();
 
     soma_dataframe = open(OpenMode::write);
-    soma_dataframe->resize_soma_joinid_if_dim(std::vector<int64_t>({new_max}));
+    soma_dataframe->maybe_resize_soma_joinid(std::vector<int64_t>({new_max}));
     soma_dataframe->close();
 
     soma_dataframe = open(OpenMode::write);
@@ -512,8 +512,11 @@ TEST_CASE_METHOD(
         }
 
         // Check shape before write
-        int64_t expect = use_current_domain ? dim_infos[0].dim_max + 1 : 0;
-        REQUIRE(soma_dataframe->shape() == std::vector<int64_t>({expect}));
+        int64_t expect = dim_infos[0].dim_max + 1;
+        std::optional<int64_t> actual = soma_dataframe
+                                            ->maybe_soma_joinid_shape();
+        REQUIRE(actual.has_value());
+        REQUIRE(actual.value() == expect);
 
         soma_dataframe->close();
 
@@ -531,8 +534,7 @@ TEST_CASE_METHOD(
 
             soma_dataframe = open(OpenMode::write);
             // Array not resizeable if it has not already been sized
-            REQUIRE_THROWS(
-                soma_dataframe->resize_soma_joinid_if_dim(new_shape));
+            REQUIRE_THROWS(soma_dataframe->maybe_resize_soma_joinid(new_shape));
             soma_dataframe->close();
 
         } else {
@@ -542,22 +544,28 @@ TEST_CASE_METHOD(
             // Check shape after write
             soma_dataframe = open(OpenMode::read);
             expect = dim_infos[0].dim_max + 1;
-            REQUIRE(soma_dataframe->shape() == std::vector<int64_t>({expect}));
+
+            std::optional<int64_t> actual = soma_dataframe
+                                                ->maybe_soma_joinid_shape();
+            REQUIRE(actual.has_value());
+            REQUIRE(actual.value() == expect);
             soma_dataframe->close();
 
             soma_dataframe = open(OpenMode::read);
-            REQUIRE_THROWS(
-                soma_dataframe->resize_soma_joinid_if_dim(new_shape));
+            REQUIRE_THROWS(soma_dataframe->maybe_resize_soma_joinid(new_shape));
             soma_dataframe->close();
 
             soma_dataframe = open(OpenMode::write);
-            soma_dataframe->resize_soma_joinid_if_dim(new_shape);
+            soma_dataframe->maybe_resize_soma_joinid(new_shape);
             soma_dataframe->close();
 
             // Check shape after resize
             soma_dataframe = open(OpenMode::read);
-            expect = SOMA_JOINID_RESIZE_DIM_MAX;
-            REQUIRE(soma_dataframe->shape() == std::vector<int64_t>({expect}));
+            expect = SOMA_JOINID_RESIZE_DIM_MAX;  // XXX MISSING A + 1 SOMEWHERE
+            actual = soma_dataframe->maybe_soma_joinid_shape();
+            REQUIRE(actual.has_value());
+            REQUIRE(actual.value() == expect);
+
             soma_dataframe->close();
 
             // Implicitly we expect no throw
@@ -610,9 +618,11 @@ TEST_CASE_METHOD(
         }
 
         // Check shape before write
-        int64_t expect = use_current_domain ? dim_infos[0].dim_max + 1 : 0;
-        REQUIRE(soma_dataframe->shape() == std::vector<int64_t>({expect}));
-
+        int64_t expect = dim_infos[0].dim_max + 1;
+        std::optional<int64_t> actual = soma_dataframe
+                                            ->maybe_soma_joinid_shape();
+        REQUIRE(actual.has_value());
+        REQUIRE(actual.value() == expect);
         soma_dataframe->close();
 
         // Write
@@ -620,8 +630,10 @@ TEST_CASE_METHOD(
 
         // Check shape after write
         soma_dataframe = open(OpenMode::read);
-        expect = use_current_domain ? dim_infos[0].dim_max + 1 : 2;
-        REQUIRE(soma_dataframe->shape() == std::vector<int64_t>({expect}));
+        expect = dim_infos[0].dim_max + 1;
+        actual = soma_dataframe->maybe_soma_joinid_shape();
+        REQUIRE(actual.has_value());
+        REQUIRE(actual.value() == expect);
         soma_dataframe->close();
 
         // Resize
@@ -636,8 +648,7 @@ TEST_CASE_METHOD(
 
             soma_dataframe = open(OpenMode::write);
             // Array not resizeable if it has not already been sized
-            REQUIRE_THROWS(
-                soma_dataframe->resize_soma_joinid_if_dim(new_shape));
+            REQUIRE_THROWS(soma_dataframe->maybe_resize_soma_joinid(new_shape));
             soma_dataframe->close();
 
         } else {
@@ -647,22 +658,26 @@ TEST_CASE_METHOD(
             // Check shape after write
             soma_dataframe = open(OpenMode::read);
             expect = dim_infos[0].dim_max + 1;
-            REQUIRE(soma_dataframe->shape() == std::vector<int64_t>({expect}));
+            std::optional<int64_t> actual = soma_dataframe
+                                                ->maybe_soma_joinid_shape();
+            REQUIRE(actual.has_value());
+            REQUIRE(actual.value() == expect);
             soma_dataframe->close();
 
             soma_dataframe = open(OpenMode::read);
-            REQUIRE_THROWS(
-                soma_dataframe->resize_soma_joinid_if_dim(new_shape));
+            REQUIRE_THROWS(soma_dataframe->maybe_resize_soma_joinid(new_shape));
             soma_dataframe->close();
 
             soma_dataframe = open(OpenMode::write);
-            soma_dataframe->resize_soma_joinid_if_dim(new_shape);
+            soma_dataframe->maybe_resize_soma_joinid(new_shape);
             soma_dataframe->close();
 
             // Check shape after resize
             soma_dataframe = open(OpenMode::read);
             expect = SOMA_JOINID_RESIZE_DIM_MAX;
-            REQUIRE(soma_dataframe->shape() == std::vector<int64_t>({expect}));
+            actual = soma_dataframe->maybe_soma_joinid_shape();
+            REQUIRE(actual.has_value());
+            REQUIRE(actual.value() == expect);
             soma_dataframe->close();
 
             // Implicitly we expect no throw
@@ -675,7 +690,8 @@ TEST_CASE_METHOD(
     VariouslyIndexedDataFrameFixture,
     "SOMADataFrame: variant-indexed dataframe dim-sjid-str attr-u32",
     "[SOMADataFrame]") {
-    auto use_current_domain = GENERATE(false, true);
+    // auto use_current_domain = GENERATE(false, true);
+    auto use_current_domain = GENERATE(false);
     std::ostringstream section;
     section << "- use_current_domain=" << use_current_domain;
     SECTION(section.str()) {
@@ -719,9 +735,11 @@ TEST_CASE_METHOD(
         }
 
         // Check shape before write
-        int64_t expect = use_current_domain ? dim_infos[0].dim_max + 1 : 0;
-        REQUIRE(soma_dataframe->shape() == std::vector<int64_t>({expect}));
-
+        int64_t expect = dim_infos[0].dim_max + 1;
+        std::optional<int64_t> actual = soma_dataframe
+                                            ->maybe_soma_joinid_shape();
+        REQUIRE(actual.has_value());
+        REQUIRE(actual.value() == expect);
         soma_dataframe->close();
 
         // Write
@@ -729,8 +747,10 @@ TEST_CASE_METHOD(
 
         // Check shape after write
         soma_dataframe = open(OpenMode::read);
-        expect = use_current_domain ? dim_infos[0].dim_max + 1 : 2;
-        REQUIRE(soma_dataframe->shape() == std::vector<int64_t>({expect}));
+        expect = dim_infos[0].dim_max + 1;
+        actual = soma_dataframe->maybe_soma_joinid_shape();
+        REQUIRE(actual.has_value());
+        REQUIRE(actual.value() == expect);
         soma_dataframe->close();
 
         // Resize
@@ -745,8 +765,7 @@ TEST_CASE_METHOD(
 
             soma_dataframe = open(OpenMode::write);
             // Array not resizeable if it has not already been sized
-            REQUIRE_THROWS(
-                soma_dataframe->resize_soma_joinid_if_dim(new_shape));
+            REQUIRE_THROWS(soma_dataframe->maybe_resize_soma_joinid(new_shape));
             soma_dataframe->close();
 
         } else {
@@ -756,22 +775,26 @@ TEST_CASE_METHOD(
             // Check shape after write
             soma_dataframe = open(OpenMode::read);
             expect = dim_infos[0].dim_max + 1;
-            REQUIRE(soma_dataframe->shape() == std::vector<int64_t>({expect}));
+            std::optional<int64_t> actual = soma_dataframe
+                                                ->maybe_soma_joinid_shape();
+            REQUIRE(actual.has_value());
+            REQUIRE(actual.value() == expect);
             soma_dataframe->close();
 
             soma_dataframe = open(OpenMode::read);
-            REQUIRE_THROWS(
-                soma_dataframe->resize_soma_joinid_if_dim(new_shape));
+            REQUIRE_THROWS(soma_dataframe->maybe_resize_soma_joinid(new_shape));
             soma_dataframe->close();
 
             soma_dataframe = open(OpenMode::write);
-            soma_dataframe->resize_soma_joinid_if_dim(new_shape);
+            soma_dataframe->maybe_resize_soma_joinid(new_shape);
             soma_dataframe->close();
 
             // Check shape after resize
             soma_dataframe = open(OpenMode::read);
             expect = SOMA_JOINID_RESIZE_DIM_MAX;
-            REQUIRE(soma_dataframe->shape() == std::vector<int64_t>({expect}));
+            actual = soma_dataframe->maybe_soma_joinid_shape();
+            REQUIRE(actual.has_value());
+            REQUIRE(actual.value() == expect);
             soma_dataframe->close();
 
             // Implicitly we expect no throw
@@ -828,9 +851,9 @@ TEST_CASE_METHOD(
         }
 
         // Check shape before write
-        int64_t expect = 0;
-        REQUIRE(soma_dataframe->shape() == std::vector<int64_t>({expect}));
-
+        std::optional<int64_t> actual = soma_dataframe
+                                            ->maybe_soma_joinid_shape();
+        REQUIRE(!actual.has_value());
         soma_dataframe->close();
 
         // Write
@@ -838,8 +861,8 @@ TEST_CASE_METHOD(
 
         // Check shape after write
         soma_dataframe = open(OpenMode::read);
-        expect = 2;
-        REQUIRE(soma_dataframe->shape() == std::vector<int64_t>({expect}));
+        actual = soma_dataframe->maybe_soma_joinid_shape();
+        REQUIRE(!actual.has_value());
         soma_dataframe->close();
 
         // Resize
@@ -854,30 +877,29 @@ TEST_CASE_METHOD(
 
             soma_dataframe = open(OpenMode::write);
             // Array not resizeable if it has not already been sized
-            REQUIRE_THROWS(
-                soma_dataframe->resize_soma_joinid_if_dim(new_shape));
+            REQUIRE_THROWS(soma_dataframe->maybe_resize_soma_joinid(new_shape));
             soma_dataframe->close();
 
         } else {
             // Check shape after write
             soma_dataframe = open(OpenMode::read);
-            expect = 2;
-            REQUIRE(soma_dataframe->shape() == std::vector<int64_t>({expect}));
+            std::optional<int64_t> actual = soma_dataframe
+                                                ->maybe_soma_joinid_shape();
+            REQUIRE(!actual.has_value());
             soma_dataframe->close();
 
             soma_dataframe = open(OpenMode::read);
-            REQUIRE_THROWS(
-                soma_dataframe->resize_soma_joinid_if_dim(new_shape));
+            REQUIRE_THROWS(soma_dataframe->maybe_resize_soma_joinid(new_shape));
             soma_dataframe->close();
 
             soma_dataframe = open(OpenMode::write);
-            soma_dataframe->resize_soma_joinid_if_dim(new_shape);
+            soma_dataframe->maybe_resize_soma_joinid(new_shape);
             soma_dataframe->close();
 
             // Check shape after resize -- noting soma_joinid is not a dim here
             soma_dataframe = open(OpenMode::read);
-            expect = 2;
-            REQUIRE(soma_dataframe->shape() == std::vector<int64_t>({expect}));
+            actual = soma_dataframe->maybe_soma_joinid_shape();
+            REQUIRE(!actual.has_value());
             soma_dataframe->close();
 
             // Implicitly we expect no throw