Repository: single-cell-data/TileDB-SOMA (language: python)

Issue:
[python/r/c++] Revisit `shape` for component arrays
## PRs

**Merged PRs:**

<details>

* #2909 `kerl/schevo-timestamp-methodize`
* #2913 `kerl/name-neaten`
* #2908 `kerl/ut-soma-exc-simplify`
* #2910 `kerl/test-common-parameterize`
* #2918 `kerl/cpp-test-deadstrip`
* #2919 `kerl/minor-unit-test-helper-mod`
* #2936 `kerl/cpp-ut-helper-neaten`
* #2938 `kerl/more-cur-dom-parameterize`
* #2915 `kerl/cpp-strict-int64-shape`
* #2911 `kerl/arrow-util-current-domain-optional`
* #2939 `kerl/step-two-temp`
* #2947 `kerl/cpp-ndarray-resize-testing`
* #2945 `kerl/dataframe-test-fixture`
* #2944 `kerl/cpp-variant-indexed-dataframes`
* #2916 `kerl/sdf-shape`
* #2917 `kerl/cpp-resizes`
* #2948 `kerl/upgrade-shape-int64`
* #2951 `kerl/sdf-test-accessors`
* #2953 `kerl/py-r-accessor-plumbing`
* #2957 `kerl/sdf-domain-accessors`
* #2960 `kerl/dense-link`
* #2963 `kerl/nightly-fix`
* #2970 `kerl/dense-writeable-after-create`
* #2968 `kerl/minor-trim`
* #2969 `kerl/more-py-domain-name-neaten`
* #2972 `kerl/libtiledbsoma-env-logging-level`
* #2962 `kerl/py-r-creation-paths`
* #2950 `kerl/py-r-test-2`
* #2994 `kerl/nanoarrow-helpers`
* #3011 `kerl/polydom3`
* #3017 `kerl/polydom5`
* #3018 `kerl/polydom6`
* #2990 `kerl/variant-nnz-bug`
* #3019 `kerl/index-swap`
* #3020 `kerl/ut-max-shape`
* #3012 `kerl/polydom4`
* #3025 `kerl/fix-3020-merge`
* #3026 `kerl/one-more-rename`
* #3028 `kerl/ff-not`
* #3029 `kerl/ut-vg`
* #3030 `kerl/table-utils-memory`
* #3067 `kerl/improve-sdf-test-field-names`
* #3068 `kerl/ut-generate`
* #3069 `kerl/cpp-sdf-domain-at-create`
* #3027 `kerl/hll-domainish`
* #3088 `kerl/max-domain-int64`
* #3090 `kerl/maybe-resize-soma-joinid-cpp-tweak`
* #3032 `kerl/sdf-domain-at-create` -- fixes #2967
* #3091 `kerl/maybe-resize-soma-joinid-py-r`
* #3095 `kerl/cpp-exp-resize-prep`
* #3089 `kerl/r-dataframe-shapeable`
* https://github.com/single-cell-data/SOMA/pull/233
* #3125 `kerl/cpp-ut-name-shortens`
* #3127 `kerl/helper-rename`
* #3130 `kerl/cpp-can-resizers-names`
* #3132 `kerl/cpp-dataframe-sizing-helpers`
* #3139 `kerl/cpp-dataframe-upgrade-test`
* #3140 `kerl/py-resizer-connects`
* #3151 `kerl/py-can-upgrade-shape`
* #3152 `kerl/registration-shape-acceessors`
* #3156 `kerl/py-exp-shaping`
* #3157 `kerl/py-exp-shaping2`
* #3148 `kerl/py-exp-resize`
* #3191 `kerl/py-domain-at-create-ut-1`
* #3190 `kerl/py-domain-at-create-ut-2`
* #3192 `kerl/py-domain-at-create-ut-3`
* #3193 `kerl/py-domain-at-create-ut-4`
* #3194 `kerl/py-domain-at-create-ut-5`
* #3203 `kerl/min-size-2`
* #3208 `kerl/r-min-sizing`
* #3211 `kerl/cpp-ugr-dom`
* #3232 `kerl/ff-interop`
* #3230 `kerl/ffon`
* #3234 `kerl/docstring-prune`
* #3236 `kerl/prefixing`
* #3241 `kerl/fix-bad-merge`
* #3235 `kerl/py-r-ugr-dom`
* #3237 `kerl/py-r-ugr-dom-2`
* #3238 `kerl/py-r-ugr-dom-3`
* #3253 `kerl/set-coords-rename`
* #3261 `kerl/pybind11-nda-sizing`
* #3265 `kerl/dense-227-a`
* #3263 `kerl/dense-range-trim`
* #3268 `kerl/dim-explosion`
* #3269 `kerl/python-227-dense-ned-read`
* #3270 `kerl/r-227-dense-fixes`
* #3280 `kerl/r-dense-227-more`
* #3286 `kerl/more-fn4m`
* #3283 `kerl/readthedocs-pre-1.15`
* #3288 `kerl/dense-ugrsh`
* #3289 `kerl/notebook-shape-upgrade`
* #3285 `kerl/new-shape-doc-updates`
* #3295 `kerl/notebook-data-refresh`
* #3290 `kerl/notebook-new-shape-refresh`
* #3301 `kerl/ffena`
* #3303 `kerl/r-data-refresh`
* #3300 `kerl/sdf-sjid-lower-zero`
* #3296 `kerl/dense-example-data-refresh`
* #3294 `kerl/new-shape-notebook-and-vignette`
* #3308 `kerl/upgrade-experiment-resources`
* #3309 `kerl/fix-notebook-merge`
* #3321 `kerl/more-use-shape`
* #3358 `kerl/revert-3300`
* #3368 `kerl/227a`
* #3369 `kerl/ucd1`
* #3370 `kerl/ucd2`
* #3371 `kerl/ucd3`
* #3372 `kerl/ucd4`
* #3396 `kerl/domain-at-create-docstrings`
* #3302 `kerl/new-shape-vignette`
* https://github.com/single-cell-data/SOMA/pull/250
* #3399 `kerl/new-shape-more-docstrings`
* #3400 `kerl/check-only-r`
* https://github.com/TileDB-Inc/somacore-feedstock/pull/27

</details>

**Closed/abandoned PRs:**

<details>

* #2785 -- This was only dogfooding for the core 2.25 release -- not to be merged
* #2952 `kerl/feature-flag-temp` -- folded into 2962
* #2995 `kerl/polydom`
* #2964 `kerl/tiledbsoma-io-test`
* #3189 `kerl/min-size`
* #3220 `kerl/cpp-ugr-dom-2`
* #3244 `kerl/dense-227-fixes`
* #3276 `kerl/dataframe-shape`

</details>

**Issues which are related but non-blocking:**

* https://github.com/single-cell-data/SOMA/pull/216
* https://github.com/TileDB-Inc/TileDB/pull/5303
* #2966
* #3081
* Note: R append mode does not exist yet -- see #1630 -- so an experiment-level resizer is not a priority in R
* #3271 
* #3272
* #3273

**See also:** [[sc-51048]](https://app.shortcut.com/tiledb-inc/story/51048).

## Problem to be solved

Users want to know the `shape` of an array, in the SciPy sense:

* Reads and writes are bounds-checked against the shape
* This retains its value regardless of which values of a sparse array are or are not actually occupied
* Users can `resize`.
  * Some users need the ability to grow their datasets later, using either `tiledbsoma.io`'s append mode, or subsequent writes using the `tiledbsoma` API.
  * Note that the cellxgene census doesn't need this: eact week's published census has fixed shape, and any updates will happen in new storage, on a new week.

Using TileDB-SOMA up until the present:

* The TIleDB `domain` is immutable after array creation
  * This does bounds-checking for reads and writes, which is good
  * To leverage this to function as a `shape`, users would need to set the `domain` at array-creation time. However, users lose the ability to grow their datasets later.
* There is a `non_empty_domain` accessor
  * This only indicates min/max coordinates _at which data exists_. Consider an `X` array for 100 cells and 200 genes. If non-zero expression counts exist only for cell join IDs 2-17, then the `non_empty_domain` will indicate `(2,17)` along `soma_dim_0`.
  * Consider an `obms["X_pca"]` within the same experiment. This may be 100 cells by 50 PCA components: we need a placd to store the number 50.
  * Therefore users cannot leverage this to function as a `shape` accessor.
* We have offered a `used_shape` accessor since TileDB-SOMA 1.5.
  * This functions as a `shape` accessor, in the SciPy sense, but it is not multi-writer safe.

New feature for TileDB-SOMA 1.15:

* Arrays will have a  `shape`
* Reads and writes are bounds-checked against the shape
* This retains its value regardless of which values of a sparse array are or are not actually occupied
* Users can `resize`
* The `used_shape` accessor will be deprecated in TileDB-SOMA 1.13, and slated for removal in TileDB-SOMA 1.14.

Compatiblity:

This will now require users to do an explicit `resize` before appending/growing TileDB-SOMA Experiments. Guidance in the form of example notebooks will be provided.

##  Tracking

See also: [[sc-41074]](https://app.shortcut.com/tiledb-inc/story/41074) and [[sc-51048]](https://app.shortcut.com/tiledb-inc/story/51048).

## Scheduling

Support arrives in TileDB Core 2.25. Deprecations for TileDB-SOMA will be released with 1.13. Full support within TileDB-SOMA will be release in 1.14.

## Details

SOMA API mods as we've discussed in a Google doc are as follows.

### `SOMADataFrame`

* `create`: Retain the `domain` argument
  * Issue:
    * Core has a `(lo, hi)` tuple per dim, e.g. `(0,99)` or `(10,19)`
    * SOMA has count per dim, with 0 implicit: e.g. 100 or 20
    * For `SparseNDArray` and `DenseNDArray` core can have `(lo, hi)` and SOMA can have `count`
    * For `DataFrame` there can be multiple dims --- default is a single `soma_joinid`
    * That could be treated either in `(lo, hi)` fashion or `count` fashion
    * However additional dims (e.g. `cell_type`) can be on any type, including strings, floats, etc. where there is no implicit lo=0
    * Therefore we need to keep the current SOMA API wherein `DataFrame` takes a `domain` argument (in `(lo, hi)` fashion) and not a `shape` argument (in `count` fashion)

### `SparseNDArray and DenseNDArray`

* `create`
  * Have an optional shape argument which is of type `Tuple[Int,...]` where each element is the cell count of the corresponding dimension
    * If unsupplied, or if supplied but None in any slot: use the minimum 0 in each slot – nothing larger makes sense since we will not support downsize
  * User guidance should make clear that it will not be possible to create an ‘old’ style array with the ‘new style’ API. (See also the upgrade logic below.)

### All three of `SOMADataFrame`, `SparseNDArray`, `DenseNDArray`

* `write`
  * For new arrays, created with the new shape feature:
    * Core will bounds-check that coordinates provided at `write` time are within the current shape
    * Core will raise `tiledb.cc.TileDBError` to TileDB-SOMA, which will catch and raise `IndexError`, and R-standard behavior on the R side
  * For old arrays created before this feature:
    * Core will not bounds-check that coordinates provided at write time are within the current shape
* Existing `used_shape` accessor
  * TileDB-SOMA will deprecate this over a release cycle.
  * For new arrays: raise `NotImplementedError`
  * For old arrays: return what’s currently returned, with a deprecation warning.
  * Mechanism for determining old vs. new: `array.schema.version` (the core storage version).
* Existing `shape` accessor
  * For new arrays:
    * Have this return the new shape as proposed by core, no longer returning the TileDB domain.
  * For old arrays created before this feature:
    * Return the TileDB domain as now.
* Existing `non_empty_domain` accessor
  * Same behavior for old and new arrays (unaffected by this proposal).
  * Keep this accessor supported, but, with user notes that it’s generally non-useful
  * This should return None (or R equivalent) when there is a schema but no data have been written.
* New `maxshape` accessor
  * Maps the core-level `(lo, hi)` accessor for domain to count-style accessor hi+1. E.g. if the core domain is either `(0,99)` or `(50,99)` then TileDB-SOMA `maxshape` will say 100.
  * Same behavior for old and new arrays.
  * Let users query for what the TileDB domain is, with user notes that it’s the maximum that users can reshape to.
  * Issac suggests: maybe `domain` or `maxshape` (see h5py).
* New `resize` mutator
  * Note: `reshape` means something else in the community (numpy, zarr, h5py), e.g. a 5x20 (total 100 cells) being reinterpreted as 4x25 (still 100 cells). The standard name for changing cell-count is `resize`.
  * For old arrays created before this feature: raise `NotImplementedError`.
  * For new arrays:
    * Will raise `ValueError` if the new shape is smaller on any dim than currently in storage
    * Regardless of whether any data have been written whatsoever
    * Will raise `ValueError` if the new shape exceeds the TileDB domain from create time, which will serve TileDB-SOMA in a role of “max possible shape the user can reshape to”
    * Otherwise, any calls to write from this point will bounds-check writes within this new shape
    * **We don’t expect resize to be multi-writer safe with regard to write ; user notes must be clear on this point**
* New `tiledbsoma_upgrade_shape` method for SparseNDArray and DenseNDArray
  * This will leverage `array.schema.version` to see if an upgrade is needed
  * Leverage core support for storage-version updates
  * This will take a shape argument as in `create`
  * For arrays created with “just-right” size: this will succeed
  * For arrays created with “room-for-growth” / “two billion-ish” size: this will succeed
  * If the user passes a shape which exceeds the current TileDB domain: this will fail
* New `tiledbsoma_upgrade_domain method` for `DataFrame`
  * Same as for `SparseNDArray`/`DenseNDArray` except it will take a domain at the SOMA-API level just as `DataFrame`'s create method

### `tiledbsoma.io`

* The user-facing API has no shape arguments and thus won’t need changing.
* Internally to `tiledbsoma.io`, we’ll still ask the tiledbsoma API for the “big domain” (2 billionish)
* Append mode:
  * Will need a new `resize` method at the `Experiment` level
  * Users will need to:
    * Register as now
    * Call the experiment-level `resize`
      * Could be `exp.resize(...)`, or (better) this could be `tiledbsoma.io.reshape_experiment`
  * In either case: this method will take the new `obs` and `var` counts as inputs:
    * `exp.obs.reshape` to new `obs` count
    * `exp.ms[name].var.reshape` to new `var` count
    * `exp.ms[name].X[name].reshape` to new `obs` count x `var` count
    * `exp.ms[name].obsm[name].reshape` to new `obs` count x same width
    * `exp.ms[name].obsp[name].reshape` to new `obs` count x `obs` count
    * `exp.ms[name].varm[name].reshape` to new `var` count x same width
    * `exp.ms[name].varp[name].reshape` to new `var` count x `var` count
  * Do the individual append-mode writes as now

[python/r/c++] Revisit `shape` for component arrays
## PRs

**Merged PRs:**

<details>

* #2909 `kerl/schevo-timestamp-methodize`
* #2913 `kerl/name-neaten`
* #2908 `kerl/ut-soma-exc-simplify`
* #2910 `kerl/test-common-parameterize`
* #2918 `kerl/cpp-test-deadstrip`
* #2919 `kerl/minor-unit-test-helper-mod`
* #2936 `kerl/cpp-ut-helper-neaten`
* #2938 `kerl/more-cur-dom-parameterize`
* #2915 `kerl/cpp-strict-int64-shape`
* #2911 `kerl/arrow-util-current-domain-optional`
* #2939 `kerl/step-two-temp`
* #2947 `kerl/cpp-ndarray-resize-testing`
* #2945 `kerl/dataframe-test-fixture`
* #2944 `kerl/cpp-variant-indexed-dataframes`
* #2916 `kerl/sdf-shape`
* #2917 `kerl/cpp-resizes`
* #2948 `kerl/upgrade-shape-int64`
* #2951 `kerl/sdf-test-accessors`
* #2953 `kerl/py-r-accessor-plumbing`
* #2957 `kerl/sdf-domain-accessors`
* #2960 `kerl/dense-link`
* #2963 `kerl/nightly-fix`
* #2970 `kerl/dense-writeable-after-create`
* #2968 `kerl/minor-trim`
* #2969 `kerl/more-py-domain-name-neaten`
* #2972 `kerl/libtiledbsoma-env-logging-level`
* #2962 `kerl/py-r-creation-paths`
* #2950 `kerl/py-r-test-2`
* #2994 `kerl/nanoarrow-helpers`
* #3011 `kerl/polydom3`
* #3017 `kerl/polydom5`
* #3018 `kerl/polydom6`
* #2990 `kerl/variant-nnz-bug`
* #3019 `kerl/index-swap`
* #3020 `kerl/ut-max-shape`
* #3012 `kerl/polydom4`
* #3025 `kerl/fix-3020-merge`
* #3026 `kerl/one-more-rename`
* #3028 `kerl/ff-not`
* #3029 `kerl/ut-vg`
* #3030 `kerl/table-utils-memory`
* #3067 `kerl/improve-sdf-test-field-names`
* #3068 `kerl/ut-generate`
* #3069 `kerl/cpp-sdf-domain-at-create`
* #3027 `kerl/hll-domainish`
* #3088 `kerl/max-domain-int64`
* #3090 `kerl/maybe-resize-soma-joinid-cpp-tweak`
* #3032 `kerl/sdf-domain-at-create` -- fixes #2967
* #3091 `kerl/maybe-resize-soma-joinid-py-r`
* #3095 `kerl/cpp-exp-resize-prep`
* #3089 `kerl/r-dataframe-shapeable`
* https://github.com/single-cell-data/SOMA/pull/233
* #3125 `kerl/cpp-ut-name-shortens`
* #3127 `kerl/helper-rename`
* #3130 `kerl/cpp-can-resizers-names`
* #3132 `kerl/cpp-dataframe-sizing-helpers`
* #3139 `kerl/cpp-dataframe-upgrade-test`
* #3140 `kerl/py-resizer-connects`
* #3151 `kerl/py-can-upgrade-shape`
* #3152 `kerl/registration-shape-acceessors`
* #3156 `kerl/py-exp-shaping`
* #3157 `kerl/py-exp-shaping2`
* #3148 `kerl/py-exp-resize`
* #3191 `kerl/py-domain-at-create-ut-1`
* #3190 `kerl/py-domain-at-create-ut-2`
* #3192 `kerl/py-domain-at-create-ut-3`
* #3193 `kerl/py-domain-at-create-ut-4`
* #3194 `kerl/py-domain-at-create-ut-5`
* #3203 `kerl/min-size-2`
* #3208 `kerl/r-min-sizing`
* #3211 `kerl/cpp-ugr-dom`
* #3232 `kerl/ff-interop`
* #3230 `kerl/ffon`
* #3234 `kerl/docstring-prune`
* #3236 `kerl/prefixing`
* #3241 `kerl/fix-bad-merge`
* #3235 `kerl/py-r-ugr-dom`
* #3237 `kerl/py-r-ugr-dom-2`
* #3238 `kerl/py-r-ugr-dom-3`
* #3253 `kerl/set-coords-rename`
* #3261 `kerl/pybind11-nda-sizing`
* #3265 `kerl/dense-227-a`
* #3263 `kerl/dense-range-trim`
* #3268 `kerl/dim-explosion`
* #3269 `kerl/python-227-dense-ned-read`
* #3270 `kerl/r-227-dense-fixes`
* #3280 `kerl/r-dense-227-more`
* #3286 `kerl/more-fn4m`
* #3283 `kerl/readthedocs-pre-1.15`
* #3288 `kerl/dense-ugrsh`
* #3289 `kerl/notebook-shape-upgrade`
* #3285 `kerl/new-shape-doc-updates`
* #3295 `kerl/notebook-data-refresh`
* #3290 `kerl/notebook-new-shape-refresh`
* #3301 `kerl/ffena`
* #3303 `kerl/r-data-refresh`
* #3300 `kerl/sdf-sjid-lower-zero`
* #3296 `kerl/dense-example-data-refresh`
* #3294 `kerl/new-shape-notebook-and-vignette`
* #3308 `kerl/upgrade-experiment-resources`
* #3309 `kerl/fix-notebook-merge`
* #3321 `kerl/more-use-shape`
* #3358 `kerl/revert-3300`
* #3368 `kerl/227a`
* #3369 `kerl/ucd1`
* #3370 `kerl/ucd2`
* #3371 `kerl/ucd3`
* #3372 `kerl/ucd4`
* #3396 `kerl/domain-at-create-docstrings`
* #3302 `kerl/new-shape-vignette`
* https://github.com/single-cell-data/SOMA/pull/250
* #3399 `kerl/new-shape-more-docstrings`
* #3400 `kerl/check-only-r`
* https://github.com/TileDB-Inc/somacore-feedstock/pull/27

</details>

**Closed/abandoned PRs:**

<details>

* #2785 -- This was only dogfooding for the core 2.25 release -- not to be merged
* #2952 `kerl/feature-flag-temp` -- folded into 2962
* #2995 `kerl/polydom`
* #2964 `kerl/tiledbsoma-io-test`
* #3189 `kerl/min-size`
* #3220 `kerl/cpp-ugr-dom-2`
* #3244 `kerl/dense-227-fixes`
* #3276 `kerl/dataframe-shape`

</details>

**Issues which are related but non-blocking:**

* https://github.com/single-cell-data/SOMA/pull/216
* https://github.com/TileDB-Inc/TileDB/pull/5303
* #2966
* #3081
* Note: R append mode does not exist yet -- see #1630 -- so an experiment-level resizer is not a priority in R
* #3271 
* #3272
* #3273

**See also:** [[sc-51048]](https://app.shortcut.com/tiledb-inc/story/51048).

## Problem to be solved

Users want to know the `shape` of an array, in the SciPy sense:

* Reads and writes are bounds-checked against the shape
* This retains its value regardless of which values of a sparse array are or are not actually occupied
* Users can `resize`.
  * Some users need the ability to grow their datasets later, using either `tiledbsoma.io`'s append mode, or subsequent writes using the `tiledbsoma` API.
  * Note that the cellxgene census doesn't need this: eact week's published census has fixed shape, and any updates will happen in new storage, on a new week.

Using TileDB-SOMA up until the present:

* The TIleDB `domain` is immutable after array creation
  * This does bounds-checking for reads and writes, which is good
  * To leverage this to function as a `shape`, users would need to set the `domain` at array-creation time. However, users lose the ability to grow their datasets later.
* There is a `non_empty_domain` accessor
  * This only indicates min/max coordinates _at which data exists_. Consider an `X` array for 100 cells and 200 genes. If non-zero expression counts exist only for cell join IDs 2-17, then the `non_empty_domain` will indicate `(2,17)` along `soma_dim_0`.
  * Consider an `obms["X_pca"]` within the same experiment. This may be 100 cells by 50 PCA components: we need a placd to store the number 50.
  * Therefore users cannot leverage this to function as a `shape` accessor.
* We have offered a `used_shape` accessor since TileDB-SOMA 1.5.
  * This functions as a `shape` accessor, in the SciPy sense, but it is not multi-writer safe.

New feature for TileDB-SOMA 1.15:

* Arrays will have a  `shape`
* Reads and writes are bounds-checked against the shape
* This retains its value regardless of which values of a sparse array are or are not actually occupied
* Users can `resize`
* The `used_shape` accessor will be deprecated in TileDB-SOMA 1.13, and slated for removal in TileDB-SOMA 1.14.

Compatiblity:

This will now require users to do an explicit `resize` before appending/growing TileDB-SOMA Experiments. Guidance in the form of example notebooks will be provided.

##  Tracking

See also: [[sc-41074]](https://app.shortcut.com/tiledb-inc/story/41074) and [[sc-51048]](https://app.shortcut.com/tiledb-inc/story/51048).

## Scheduling

Support arrives in TileDB Core 2.25. Deprecations for TileDB-SOMA will be released with 1.13. Full support within TileDB-SOMA will be release in 1.14.

## Details

SOMA API mods as we've discussed in a Google doc are as follows.

### `SOMADataFrame`

* `create`: Retain the `domain` argument
  * Issue:
    * Core has a `(lo, hi)` tuple per dim, e.g. `(0,99)` or `(10,19)`
    * SOMA has count per dim, with 0 implicit: e.g. 100 or 20
    * For `SparseNDArray` and `DenseNDArray` core can have `(lo, hi)` and SOMA can have `count`
    * For `DataFrame` there can be multiple dims --- default is a single `soma_joinid`
    * That could be treated either in `(lo, hi)` fashion or `count` fashion
    * However additional dims (e.g. `cell_type`) can be on any type, including strings, floats, etc. where there is no implicit lo=0
    * Therefore we need to keep the current SOMA API wherein `DataFrame` takes a `domain` argument (in `(lo, hi)` fashion) and not a `shape` argument (in `count` fashion)

### `SparseNDArray and DenseNDArray`

* `create`
  * Have an optional shape argument which is of type `Tuple[Int,...]` where each element is the cell count of the corresponding dimension
    * If unsupplied, or if supplied but None in any slot: use the minimum 0 in each slot – nothing larger makes sense since we will not support downsize
  * User guidance should make clear that it will not be possible to create an ‘old’ style array with the ‘new style’ API. (See also the upgrade logic below.)

### All three of `SOMADataFrame`, `SparseNDArray`, `DenseNDArray`

* `write`
  * For new arrays, created with the new shape feature:
    * Core will bounds-check that coordinates provided at `write` time are within the current shape
    * Core will raise `tiledb.cc.TileDBError` to TileDB-SOMA, which will catch and raise `IndexError`, and R-standard behavior on the R side
  * For old arrays created before this feature:
    * Core will not bounds-check that coordinates provided at write time are within the current shape
* Existing `used_shape` accessor
  * TileDB-SOMA will deprecate this over a release cycle.
  * For new arrays: raise `NotImplementedError`
  * For old arrays: return what’s currently returned, with a deprecation warning.
  * Mechanism for determining old vs. new: `array.schema.version` (the core storage version).
* Existing `shape` accessor
  * For new arrays:
    * Have this return the new shape as proposed by core, no longer returning the TileDB domain.
  * For old arrays created before this feature:
    * Return the TileDB domain as now.
* Existing `non_empty_domain` accessor
  * Same behavior for old and new arrays (unaffected by this proposal).
  * Keep this accessor supported, but, with user notes that it’s generally non-useful
  * This should return None (or R equivalent) when there is a schema but no data have been written.
* New `maxshape` accessor
  * Maps the core-level `(lo, hi)` accessor for domain to count-style accessor hi+1. E.g. if the core domain is either `(0,99)` or `(50,99)` then TileDB-SOMA `maxshape` will say 100.
  * Same behavior for old and new arrays.
  * Let users query for what the TileDB domain is, with user notes that it’s the maximum that users can reshape to.
  * Issac suggests: maybe `domain` or `maxshape` (see h5py).
* New `resize` mutator
  * Note: `reshape` means something else in the community (numpy, zarr, h5py), e.g. a 5x20 (total 100 cells) being reinterpreted as 4x25 (still 100 cells). The standard name for changing cell-count is `resize`.
  * For old arrays created before this feature: raise `NotImplementedError`.
  * For new arrays:
    * Will raise `ValueError` if the new shape is smaller on any dim than currently in storage
    * Regardless of whether any data have been written whatsoever
    * Will raise `ValueError` if the new shape exceeds the TileDB domain from create time, which will serve TileDB-SOMA in a role of “max possible shape the user can reshape to”
    * Otherwise, any calls to write from this point will bounds-check writes within this new shape
    * **We don’t expect resize to be multi-writer safe with regard to write ; user notes must be clear on this point**
* New `tiledbsoma_upgrade_shape` method for SparseNDArray and DenseNDArray
  * This will leverage `array.schema.version` to see if an upgrade is needed
  * Leverage core support for storage-version updates
  * This will take a shape argument as in `create`
  * For arrays created with “just-right” size: this will succeed
  * For arrays created with “room-for-growth” / “two billion-ish” size: this will succeed
  * If the user passes a shape which exceeds the current TileDB domain: this will fail
* New `tiledbsoma_upgrade_domain method` for `DataFrame`
  * Same as for `SparseNDArray`/`DenseNDArray` except it will take a domain at the SOMA-API level just as `DataFrame`'s create method

### `tiledbsoma.io`

* The user-facing API has no shape arguments and thus won’t need changing.
* Internally to `tiledbsoma.io`, we’ll still ask the tiledbsoma API for the “big domain” (2 billionish)
* Append mode:
  * Will need a new `resize` method at the `Experiment` level
  * Users will need to:
    * Register as now
    * Call the experiment-level `resize`
      * Could be `exp.resize(...)`, or (better) this could be `tiledbsoma.io.reshape_experiment`
  * In either case: this method will take the new `obs` and `var` counts as inputs:
    * `exp.obs.reshape` to new `obs` count
    * `exp.ms[name].var.reshape` to new `var` count
    * `exp.ms[name].X[name].reshape` to new `obs` count x `var` count
    * `exp.ms[name].obsm[name].reshape` to new `obs` count x same width
    * `exp.ms[name].obsp[name].reshape` to new `obs` count x `obs` count
    * `exp.ms[name].varm[name].reshape` to new `var` count x same width
    * `exp.ms[name].varp[name].reshape` to new `var` count x `var` count
  * Do the individual append-mode writes as now

[python/r/c++] Revisit `shape` for component arrays
## PRs

**Merged PRs:**

<details>

* #2909 `kerl/schevo-timestamp-methodize`
* #2913 `kerl/name-neaten`
* #2908 `kerl/ut-soma-exc-simplify`
* #2910 `kerl/test-common-parameterize`
* #2918 `kerl/cpp-test-deadstrip`
* #2919 `kerl/minor-unit-test-helper-mod`
* #2936 `kerl/cpp-ut-helper-neaten`
* #2938 `kerl/more-cur-dom-parameterize`
* #2915 `kerl/cpp-strict-int64-shape`
* #2911 `kerl/arrow-util-current-domain-optional`
* #2939 `kerl/step-two-temp`
* #2947 `kerl/cpp-ndarray-resize-testing`
* #2945 `kerl/dataframe-test-fixture`
* #2944 `kerl/cpp-variant-indexed-dataframes`
* #2916 `kerl/sdf-shape`
* #2917 `kerl/cpp-resizes`
* #2948 `kerl/upgrade-shape-int64`
* #2951 `kerl/sdf-test-accessors`
* #2953 `kerl/py-r-accessor-plumbing`
* #2957 `kerl/sdf-domain-accessors`
* #2960 `kerl/dense-link`
* #2963 `kerl/nightly-fix`
* #2970 `kerl/dense-writeable-after-create`
* #2968 `kerl/minor-trim`
* #2969 `kerl/more-py-domain-name-neaten`
* #2972 `kerl/libtiledbsoma-env-logging-level`
* #2962 `kerl/py-r-creation-paths`
* #2950 `kerl/py-r-test-2`
* #2994 `kerl/nanoarrow-helpers`
* #3011 `kerl/polydom3`
* #3017 `kerl/polydom5`
* #3018 `kerl/polydom6`
* #2990 `kerl/variant-nnz-bug`
* #3019 `kerl/index-swap`
* #3020 `kerl/ut-max-shape`
* #3012 `kerl/polydom4`
* #3025 `kerl/fix-3020-merge`
* #3026 `kerl/one-more-rename`
* #3028 `kerl/ff-not`
* #3029 `kerl/ut-vg`
* #3030 `kerl/table-utils-memory`
* #3067 `kerl/improve-sdf-test-field-names`
* #3068 `kerl/ut-generate`
* #3069 `kerl/cpp-sdf-domain-at-create`
* #3027 `kerl/hll-domainish`
* #3088 `kerl/max-domain-int64`
* #3090 `kerl/maybe-resize-soma-joinid-cpp-tweak`
* #3032 `kerl/sdf-domain-at-create` -- fixes #2967
* #3091 `kerl/maybe-resize-soma-joinid-py-r`
* #3095 `kerl/cpp-exp-resize-prep`
* #3089 `kerl/r-dataframe-shapeable`
* https://github.com/single-cell-data/SOMA/pull/233
* #3125 `kerl/cpp-ut-name-shortens`
* #3127 `kerl/helper-rename`
* #3130 `kerl/cpp-can-resizers-names`
* #3132 `kerl/cpp-dataframe-sizing-helpers`
* #3139 `kerl/cpp-dataframe-upgrade-test`
* #3140 `kerl/py-resizer-connects`
* #3151 `kerl/py-can-upgrade-shape`
* #3152 `kerl/registration-shape-acceessors`
* #3156 `kerl/py-exp-shaping`
* #3157 `kerl/py-exp-shaping2`
* #3148 `kerl/py-exp-resize`
* #3191 `kerl/py-domain-at-create-ut-1`
* #3190 `kerl/py-domain-at-create-ut-2`
* #3192 `kerl/py-domain-at-create-ut-3`
* #3193 `kerl/py-domain-at-create-ut-4`
* #3194 `kerl/py-domain-at-create-ut-5`
* #3203 `kerl/min-size-2`
* #3208 `kerl/r-min-sizing`
* #3211 `kerl/cpp-ugr-dom`
* #3232 `kerl/ff-interop`
* #3230 `kerl/ffon`
* #3234 `kerl/docstring-prune`
* #3236 `kerl/prefixing`
* #3241 `kerl/fix-bad-merge`
* #3235 `kerl/py-r-ugr-dom`
* #3237 `kerl/py-r-ugr-dom-2`
* #3238 `kerl/py-r-ugr-dom-3`
* #3253 `kerl/set-coords-rename`
* #3261 `kerl/pybind11-nda-sizing`
* #3265 `kerl/dense-227-a`
* #3263 `kerl/dense-range-trim`
* #3268 `kerl/dim-explosion`
* #3269 `kerl/python-227-dense-ned-read`
* #3270 `kerl/r-227-dense-fixes`
* #3280 `kerl/r-dense-227-more`
* #3286 `kerl/more-fn4m`
* #3283 `kerl/readthedocs-pre-1.15`
* #3288 `kerl/dense-ugrsh`
* #3289 `kerl/notebook-shape-upgrade`
* #3285 `kerl/new-shape-doc-updates`
* #3295 `kerl/notebook-data-refresh`
* #3290 `kerl/notebook-new-shape-refresh`
* #3301 `kerl/ffena`
* #3303 `kerl/r-data-refresh`
* #3300 `kerl/sdf-sjid-lower-zero`
* #3296 `kerl/dense-example-data-refresh`
* #3294 `kerl/new-shape-notebook-and-vignette`
* #3308 `kerl/upgrade-experiment-resources`
* #3309 `kerl/fix-notebook-merge`
* #3321 `kerl/more-use-shape`
* #3358 `kerl/revert-3300`
* #3368 `kerl/227a`
* #3369 `kerl/ucd1`
* #3370 `kerl/ucd2`
* #3371 `kerl/ucd3`
* #3372 `kerl/ucd4`
* #3396 `kerl/domain-at-create-docstrings`
* #3302 `kerl/new-shape-vignette`
* https://github.com/single-cell-data/SOMA/pull/250
* #3399 `kerl/new-shape-more-docstrings`
* #3400 `kerl/check-only-r`
* https://github.com/TileDB-Inc/somacore-feedstock/pull/27

</details>

**Closed/abandoned PRs:**

<details>

* #2785 -- This was only dogfooding for the core 2.25 release -- not to be merged
* #2952 `kerl/feature-flag-temp` -- folded into 2962
* #2995 `kerl/polydom`
* #2964 `kerl/tiledbsoma-io-test`
* #3189 `kerl/min-size`
* #3220 `kerl/cpp-ugr-dom-2`
* #3244 `kerl/dense-227-fixes`
* #3276 `kerl/dataframe-shape`

</details>

**Issues which are related but non-blocking:**

* https://github.com/single-cell-data/SOMA/pull/216
* https://github.com/TileDB-Inc/TileDB/pull/5303
* #2966
* #3081
* Note: R append mode does not exist yet -- see #1630 -- so an experiment-level resizer is not a priority in R
* #3271 
* #3272
* #3273

**See also:** [[sc-51048]](https://app.shortcut.com/tiledb-inc/story/51048).

## Problem to be solved

Users want to know the `shape` of an array, in the SciPy sense:

* Reads and writes are bounds-checked against the shape
* This retains its value regardless of which values of a sparse array are or are not actually occupied
* Users can `resize`.
  * Some users need the ability to grow their datasets later, using either `tiledbsoma.io`'s append mode, or subsequent writes using the `tiledbsoma` API.
  * Note that the cellxgene census doesn't need this: eact week's published census has fixed shape, and any updates will happen in new storage, on a new week.

Using TileDB-SOMA up until the present:

* The TIleDB `domain` is immutable after array creation
  * This does bounds-checking for reads and writes, which is good
  * To leverage this to function as a `shape`, users would need to set the `domain` at array-creation time. However, users lose the ability to grow their datasets later.
* There is a `non_empty_domain` accessor
  * This only indicates min/max coordinates _at which data exists_. Consider an `X` array for 100 cells and 200 genes. If non-zero expression counts exist only for cell join IDs 2-17, then the `non_empty_domain` will indicate `(2,17)` along `soma_dim_0`.
  * Consider an `obms["X_pca"]` within the same experiment. This may be 100 cells by 50 PCA components: we need a placd to store the number 50.
  * Therefore users cannot leverage this to function as a `shape` accessor.
* We have offered a `used_shape` accessor since TileDB-SOMA 1.5.
  * This functions as a `shape` accessor, in the SciPy sense, but it is not multi-writer safe.

New feature for TileDB-SOMA 1.15:

* Arrays will have a  `shape`
* Reads and writes are bounds-checked against the shape
* This retains its value regardless of which values of a sparse array are or are not actually occupied
* Users can `resize`
* The `used_shape` accessor will be deprecated in TileDB-SOMA 1.13, and slated for removal in TileDB-SOMA 1.14.

Compatiblity:

This will now require users to do an explicit `resize` before appending/growing TileDB-SOMA Experiments. Guidance in the form of example notebooks will be provided.

##  Tracking

See also: [[sc-41074]](https://app.shortcut.com/tiledb-inc/story/41074) and [[sc-51048]](https://app.shortcut.com/tiledb-inc/story/51048).

## Scheduling

Support arrives in TileDB Core 2.25. Deprecations for TileDB-SOMA will be released with 1.13. Full support within TileDB-SOMA will be release in 1.14.

## Details

SOMA API mods as we've discussed in a Google doc are as follows.

### `SOMADataFrame`

* `create`: Retain the `domain` argument
  * Issue:
    * Core has a `(lo, hi)` tuple per dim, e.g. `(0,99)` or `(10,19)`
    * SOMA has count per dim, with 0 implicit: e.g. 100 or 20
    * For `SparseNDArray` and `DenseNDArray` core can have `(lo, hi)` and SOMA can have `count`
    * For `DataFrame` there can be multiple dims --- default is a single `soma_joinid`
    * That could be treated either in `(lo, hi)` fashion or `count` fashion
    * However additional dims (e.g. `cell_type`) can be on any type, including strings, floats, etc. where there is no implicit lo=0
    * Therefore we need to keep the current SOMA API wherein `DataFrame` takes a `domain` argument (in `(lo, hi)` fashion) and not a `shape` argument (in `count` fashion)

### `SparseNDArray and DenseNDArray`

* `create`
  * Have an optional shape argument which is of type `Tuple[Int,...]` where each element is the cell count of the corresponding dimension
    * If unsupplied, or if supplied but None in any slot: use the minimum 0 in each slot – nothing larger makes sense since we will not support downsize
  * User guidance should make clear that it will not be possible to create an ‘old’ style array with the ‘new style’ API. (See also the upgrade logic below.)

### All three of `SOMADataFrame`, `SparseNDArray`, `DenseNDArray`

* `write`
  * For new arrays, created with the new shape feature:
    * Core will bounds-check that coordinates provided at `write` time are within the current shape
    * Core will raise `tiledb.cc.TileDBError` to TileDB-SOMA, which will catch and raise `IndexError`, and R-standard behavior on the R side
  * For old arrays created before this feature:
    * Core will not bounds-check that coordinates provided at write time are within the current shape
* Existing `used_shape` accessor
  * TileDB-SOMA will deprecate this over a release cycle.
  * For new arrays: raise `NotImplementedError`
  * For old arrays: return what’s currently returned, with a deprecation warning.
  * Mechanism for determining old vs. new: `array.schema.version` (the core storage version).
* Existing `shape` accessor
  * For new arrays:
    * Have this return the new shape as proposed by core, no longer returning the TileDB domain.
  * For old arrays created before this feature:
    * Return the TileDB domain as now.
* Existing `non_empty_domain` accessor
  * Same behavior for old and new arrays (unaffected by this proposal).
  * Keep this accessor supported, but, with user notes that it’s generally non-useful
  * This should return None (or R equivalent) when there is a schema but no data have been written.
* New `maxshape` accessor
  * Maps the core-level `(lo, hi)` accessor for domain to count-style accessor hi+1. E.g. if the core domain is either `(0,99)` or `(50,99)` then TileDB-SOMA `maxshape` will say 100.
  * Same behavior for old and new arrays.
  * Let users query for what the TileDB domain is, with user notes that it’s the maximum that users can reshape to.
  * Issac suggests: maybe `domain` or `maxshape` (see h5py).
* New `resize` mutator
  * Note: `reshape` means something else in the community (numpy, zarr, h5py), e.g. a 5x20 (total 100 cells) being reinterpreted as 4x25 (still 100 cells). The standard name for changing cell-count is `resize`.
  * For old arrays created before this feature: raise `NotImplementedError`.
  * For new arrays:
    * Will raise `ValueError` if the new shape is smaller on any dim than currently in storage
    * Regardless of whether any data have been written whatsoever
    * Will raise `ValueError` if the new shape exceeds the TileDB domain from create time, which will serve TileDB-SOMA in a role of “max possible shape the user can reshape to”
    * Otherwise, any calls to write from this point will bounds-check writes within this new shape
    * **We don’t expect resize to be multi-writer safe with regard to write ; user notes must be clear on this point**
* New `tiledbsoma_upgrade_shape` method for SparseNDArray and DenseNDArray
  * This will leverage `array.schema.version` to see if an upgrade is needed
  * Leverage core support for storage-version updates
  * This will take a shape argument as in `create`
  * For arrays created with “just-right” size: this will succeed
  * For arrays created with “room-for-growth” / “two billion-ish” size: this will succeed
  * If the user passes a shape which exceeds the current TileDB domain: this will fail
* New `tiledbsoma_upgrade_domain method` for `DataFrame`
  * Same as for `SparseNDArray`/`DenseNDArray` except it will take a domain at the SOMA-API level just as `DataFrame`'s create method

### `tiledbsoma.io`

* The user-facing API has no shape arguments and thus won’t need changing.
* Internally to `tiledbsoma.io`, we’ll still ask the tiledbsoma API for the “big domain” (2 billionish)
* Append mode:
  * Will need a new `resize` method at the `Experiment` level
  * Users will need to:
    * Register as now
    * Call the experiment-level `resize`
      * Could be `exp.resize(...)`, or (better) this could be `tiledbsoma.io.reshape_experiment`
  * In either case: this method will take the new `obs` and `var` counts as inputs:
    * `exp.obs.reshape` to new `obs` count
    * `exp.ms[name].var.reshape` to new `var` count
    * `exp.ms[name].X[name].reshape` to new `obs` count x `var` count
    * `exp.ms[name].obsm[name].reshape` to new `obs` count x same width
    * `exp.ms[name].obsp[name].reshape` to new `obs` count x `obs` count
    * `exp.ms[name].varm[name].reshape` to new `var` count x same width
    * `exp.ms[name].varp[name].reshape` to new `var` count x `var` count
  * Do the individual append-mode writes as now



Discussion / hints:
#2785 is a quick-and-dirty concept-prover -- its sole function is to flush out any API misunderstandings we might have, in prep for 2.25.0 core release.
#2785 is a quick-and-dirty concept-prover -- its sole function is to flush out any API misunderstandings we might have, in prep for 2.25.0 core release.
#2785 is a quick-and-dirty concept-prover -- its sole function is to flush out any API misunderstandings we might have, in prep for 2.25.0 core release.

Fix patch:
diff --git a/.github/workflows/python-ci-single.yml b/.github/workflows/python-ci-single.yml
index 0c05dd8413..66104372cb 100644
--- a/.github/workflows/python-ci-single.yml
+++ b/.github/workflows/python-ci-single.yml
@@ -129,13 +129,20 @@ jobs:
       env:
         TILEDB_SOMA_INIT_BUFFER_BYTES: 33554432 # accommodate tiny runners
 
-    - name: Run pytests for Python
+    - name: Run pytests for Python without new shape
       shell: bash
       # Setting PYTHONPATH ensures the tests load the in-tree source code under apis/python/src
       # instead of the copy we `pip install`ed to site-packages above. That's needed for the code
       # coverage analysis to work.
       run: PYTHONPATH=$(pwd)/apis/python/src python -m pytest --cov=apis/python/src --cov-report=xml apis/python/tests -v --durations=20
 
+    - name: Run pytests for Python with new shape
+      shell: bash
+      # Setting PYTHONPATH ensures the tests load the in-tree source code under apis/python/src
+      # instead of the copy we `pip install`ed to site-packages above. That's needed for the code
+      # coverage analysis to work.
+      run: export SOMA_PY_NEW_SHAPE=true; PYTHONPATH=$(pwd)/apis/python/src python -m pytest --cov=apis/python/src --cov-report=xml apis/python/tests -v --durations=20
+
     - name: Report coverage to Codecov
       if: inputs.report_codecov
       uses: codecov/codecov-action@v4
diff --git a/.github/workflows/r-ci.yml b/.github/workflows/r-ci.yml
index 2eb93d2de2..5a50f47065 100644
--- a/.github/workflows/r-ci.yml
+++ b/.github/workflows/r-ci.yml
@@ -148,10 +148,15 @@ jobs:
       #      -e "devtools::install(upgrade = FALSE)" \
       #      -e "testthat::test_local('tests/testthat', load_package = 'installed')"
 
-      - name: Test
+      - name: Test without new shape
         if: ${{ matrix.covr == 'no' }}
         run: cd apis/r/tests && Rscript testthat.R
-      
+
+      # https://github.com/single-cell-data/TileDB-SOMA/issues/2407
+      - name: Test with new shape
+        if: ${{ matrix.covr == 'no' }}
+        run: export SOMA_R_NEW_SHAPE=true && cd apis/r/tests && Rscript testthat.R
+
       - name: Coverage
         if: ${{ matrix.os == 'ubuntu-latest' && matrix.covr == 'yes' && github.event_name == 'workflow_dispatch' }}
         run: apis/r/tools/r-ci.sh coverage
diff --git a/apis/python/src/tiledbsoma/__init__.py b/apis/python/src/tiledbsoma/__init__.py
index 491eaa35c4..0bf3b0b43b 100644
--- a/apis/python/src/tiledbsoma/__init__.py
+++ b/apis/python/src/tiledbsoma/__init__.py
@@ -97,6 +97,18 @@
 import os
 import sys
 
+# Temporary for https://github.com/single-cell-data/TileDB-SOMA/issues/2407
+_new_shape_feature_flag = os.getenv("SOMA_PY_NEW_SHAPE") is not None
+
+
+def _new_shape_feature_flag_enabled() -> bool:
+    """
+    This is temporary only and will be removed once
+    https://github.com/single-cell-data/TileDB-SOMA/issues/2407
+    is complete.
+    """
+    return _new_shape_feature_flag
+
 
 # Load native libraries. On wheel builds, we may have a shared library
 # already linked. In this case, we can import directly
diff --git a/apis/python/src/tiledbsoma/_dataframe.py b/apis/python/src/tiledbsoma/_dataframe.py
index 5177d3e97b..efa2a4cc5a 100644
--- a/apis/python/src/tiledbsoma/_dataframe.py
+++ b/apis/python/src/tiledbsoma/_dataframe.py
@@ -14,6 +14,8 @@
 from somacore import options
 from typing_extensions import Self
 
+from tiledbsoma import _new_shape_feature_flag_enabled
+
 from . import _arrow_types, _util
 from . import pytiledbsoma as clib
 from ._constants import SOMA_JOINID
@@ -215,10 +217,33 @@ def create(
         """
         context = _validate_soma_tiledb_context(context)
         schema = _canonicalize_schema(schema, index_column_names)
-        if domain is None:
-            domain = tuple(None for _ in index_column_names)
+
+        # SOMA-to-core mappings:
+        #
+        # Before the current-domain feature was enabled (possible after core 2.25):
+        #
+        # * SOMA domain <-> core domain, AKA "max domain" which is a name we'll use for clarity
+        # * core current domain did not exist
+        #
+        # After the current-domain feature was enabled:
+        #
+        # * SOMA max_domain <-> core domain
+        # * SOMA domain <-> core current domain
+        #
+        # As far as the user is concerned, the SOMA-level domain is the only
+        # thing they see and care about. Before 2.25 support, it was immutable
+        # (since it was implemented by core domain). After 2.25 support, it is
+        # mutable/up-resizeable (since it is implemented by core current domain).
+
+        # At this point shift from API terminology "domain" to specifying a soma_ or core_
+        # prefix for these variables. This is crucial to avoid developer confusion.
+        soma_domain = domain
+        domain = None
+
+        if soma_domain is None:
+            soma_domain = tuple(None for _ in index_column_names)
         else:
-            ndom = len(domain)
+            ndom = len(soma_domain)
             nidx = len(index_column_names)
             if ndom != nidx:
                 raise ValueError(
@@ -228,25 +253,56 @@ def create(
         index_column_schema = []
         index_column_data = {}
 
-        for index_column_name, slot_domain in zip(index_column_names, domain):
+        for index_column_name, slot_soma_domain in zip(index_column_names, soma_domain):
             pa_field = schema.field(index_column_name)
             dtype = _arrow_types.tiledb_type_from_arrow_type(
                 pa_field.type, is_indexed_column=True
             )
 
-            slot_domain = _fill_out_slot_domain(
-                slot_domain, index_column_name, pa_field.type, dtype
+            (slot_core_current_domain, saturated_cd) = _fill_out_slot_soma_domain(
+                slot_soma_domain, index_column_name, pa_field.type, dtype
+            )
+            (slot_core_max_domain, saturated_md) = _fill_out_slot_soma_domain(
+                None, index_column_name, pa_field.type, dtype
             )
 
             extent = _find_extent_for_domain(
                 index_column_name,
                 TileDBCreateOptions.from_platform_config(platform_config),
                 dtype,
-                slot_domain,
+                slot_core_current_domain,
             )
 
+            # Necessary to avoid core array-creation error "Reduce domain max by
+            # 1 tile extent to allow for expansion."
+            slot_core_current_domain = _revise_domain_for_extent(
+                slot_core_current_domain, extent, saturated_cd
+            )
+            slot_core_max_domain = _revise_domain_for_extent(
+                slot_core_max_domain, extent, saturated_md
+            )
+
+            # Here is our Arrow data API for communicating schema info between
+            # Python/R and C++ libtiledbsoma:
+            #
+            # [0] core max domain lo
+            # [1] core max domain hi
+            # [2] core extent parameter
+            # If present, these next two signal to use the current-domain feature:
+            # [3] core current domain lo
+            # [4] core current domain hi
+
             index_column_schema.append(pa_field)
-            index_column_data[pa_field.name] = [*slot_domain, extent]
+            if _new_shape_feature_flag_enabled():
+
+                index_column_data[pa_field.name] = [
+                    *slot_core_max_domain,
+                    extent,
+                    *slot_core_current_domain,
+                ]
+
+            else:
+                index_column_data[pa_field.name] = [*slot_core_current_domain, extent]
 
         index_column_info = pa.RecordBatch.from_pydict(
             index_column_data, schema=pa.schema(index_column_schema)
@@ -718,17 +774,20 @@ def _canonicalize_schema(
     return schema
 
 
-def _fill_out_slot_domain(
+def _fill_out_slot_soma_domain(
     slot_domain: AxisDomain,
     index_column_name: str,
     pa_type: pa.DataType,
     dtype: Any,
-) -> Tuple[Any, Any]:
+) -> Tuple[Tuple[Any, Any], bool]:
     """Helper function for _build_tiledb_schema. Given a user-specified domain for a
     dimension slot -- which may be ``None``, or a two-tuple of which either element
     may be ``None`` -- return either what the user specified (if adequate) or
     sensible type-inferred values appropriate to the datatype.
+
+    Returns a boolean for whether the underlying datatype's max range was used.
     """
+    saturated_range = False
     if slot_domain is not None:
         # User-specified; go with it when possible
         if (
@@ -769,9 +828,12 @@ def _fill_out_slot_domain(
         # The SOMA spec disallows negative soma_joinid.
         if index_column_name == SOMA_JOINID:
             slot_domain = (0, 2**31 - 2)  # R-friendly, which 2**63-1 is not
+        else:
+            saturated_range = True
     elif np.issubdtype(dtype, NPFloating):
         finfo = np.finfo(cast(NPFloating, dtype))
         slot_domain = finfo.min, finfo.max
+        saturated_range = True
 
     # The `iinfo.min+1` is necessary as of tiledb core 2.15 / tiledb-py 0.21.1 since
     # `iinfo.min` maps to `NaT` (not a time), resulting in
@@ -806,7 +868,7 @@ def _fill_out_slot_domain(
     else:
         raise TypeError(f"Unsupported dtype {dtype}")
 
-    return slot_domain
+    return (slot_domain, saturated_range)
 
 
 def _find_extent_for_domain(
@@ -823,7 +885,7 @@ def _find_extent_for_domain(
     # Default 2048 mods to 0 for 8-bit types and 0 is an invalid extent
     extent = tiledb_create_write_options.dim_tile(index_column_name)
     if isinstance(dtype, np.dtype) and dtype.itemsize == 1:
-        extent = 64
+        extent = 1
 
     if isinstance(dtype, str):
         return ""
@@ -860,3 +922,17 @@ def _find_extent_for_domain(
         return np.datetime64(iextent, "ns")
 
     return extent
+
+
+# We need to do this to avoid this error at array-creation time:
+#
+# Error: Tile extent check failed; domain max expanded to multiple of tile
+# extent exceeds max value representable by domain type. Reduce domain max
+# by 1 tile extent to allow for expansion.
+def _revise_domain_for_extent(
+    domain: Tuple[Any, Any], extent: Any, saturated_range: bool
+) -> Tuple[Any, Any]:
+    if saturated_range:
+        return (domain[0], domain[1] - extent)
+    else:
+        return domain
diff --git a/apis/python/src/tiledbsoma/_sparse_nd_array.py b/apis/python/src/tiledbsoma/_sparse_nd_array.py
index 3cb2ccf967..46fa56a0ad 100644
--- a/apis/python/src/tiledbsoma/_sparse_nd_array.py
+++ b/apis/python/src/tiledbsoma/_sparse_nd_array.py
@@ -26,6 +26,8 @@
 from somacore.options import PlatformConfig
 from typing_extensions import Self
 
+from tiledbsoma import _new_shape_feature_flag_enabled
+
 from . import _util
 
 # This package's pybind11 code
@@ -122,18 +124,75 @@ def create(
     ) -> Self:
         context = _validate_soma_tiledb_context(context)
 
+        # SOMA-to-core mappings:
+        #
+        # Before the current-domain feature was enabled (possible after core 2.25):
+        #
+        # * SOMA shape <-> core domain, AKA "max domain" which is a name we'll use for clarity
+        #   o Specifically, (0, SOMA shape minus 1) = core domain
+        # * core current domain did not exist
+        #
+        # After the current-domain feature was enabled:
+        #
+        # * SOMA maxshape <-> core domain, AKA "max domain" which is a name we'll use for clarity
+        #   o Specifically, (0, SOMA maxshape minus 1) = core max domain
+        # * SOMA shape <-> core current domain
+        #   o Specifically, (0, SOMA shape minus 1) = core current domain
+
+        # As far as the user is concerned, the SOMA shape is the _only_ thing they see and care
+        # about. It's resizeable (up to max_domain anyway), reads and writes are bounds-checked
+        # against it, etc.
+
         index_column_schema = []
         index_column_data = {}
+
         for dim_idx, dim_shape in enumerate(shape):
             dim_name = f"soma_dim_{dim_idx}"
+
             pa_field = pa.field(dim_name, pa.int64())
-            dim_capacity, dim_extent = cls._dim_capacity_and_extent(
-                dim_name,
-                dim_shape,
-                TileDBCreateOptions.from_platform_config(platform_config),
-            )
+
             index_column_schema.append(pa_field)
-            index_column_data[pa_field.name] = [0, dim_capacity - 1, dim_extent]
+
+            # Here is our Arrow data API for communicating schema info between
+            # Python/R and C++ libtiledbsoma:
+            #
+            # [0] core max domain lo
+            # [1] core max domain hi
+            # [2] core extent parameter
+            # If present, these next two signal to use the current-domain feature:
+            # [3] core current domain lo
+            # [4] core current domain hi
+
+            if _new_shape_feature_flag_enabled():
+                dim_capacity, dim_extent = cls._dim_capacity_and_extent(
+                    dim_name,
+                    # The user specifies current domain -- this is the max domain
+                    # which is taken from the max ranges for the dim datatype.
+                    # We pass None here to detect those.
+                    None,
+                    TileDBCreateOptions.from_platform_config(platform_config),
+                )
+
+                if dim_shape == 0:
+                    raise ValueError("SparseNDArray shape slots must be at least 1")
+                if dim_shape is None:
+                    dim_shape = dim_capacity
+
+                index_column_data[pa_field.name] = [
+                    0,
+                    dim_capacity - 1,
+                    dim_extent,
+                    0,
+                    dim_shape - 1,
+                ]
+
+            else:
+                dim_capacity, dim_extent = cls._dim_capacity_and_extent(
+                    dim_name,
+                    dim_shape,
+                    TileDBCreateOptions.from_platform_config(platform_config),
+                )
+                index_column_data[pa_field.name] = [0, dim_capacity - 1, dim_extent]
 
         index_column_info = pa.RecordBatch.from_pydict(
             index_column_data, schema=pa.schema(index_column_schema)
diff --git a/apis/python/src/tiledbsoma/soma_array.cc b/apis/python/src/tiledbsoma/soma_array.cc
index f301f0947a..ed2ea777f4 100644
--- a/apis/python/src/tiledbsoma/soma_array.cc
+++ b/apis/python/src/tiledbsoma/soma_array.cc
@@ -626,14 +626,18 @@ void load_soma_array(py::module& m) {
                 py::gil_scoped_release release;
 
                 // Try to read more data
-                auto buffers = array.read_next();
-
-                // If more data was read, convert it to an arrow table and
-                // return
-                if (buffers.has_value()) {
-                    // Acquire python GIL before accessing python objects
-                    py::gil_scoped_acquire acquire;
-                    return to_table(*buffers);
+                try {
+                    auto buffers = array.read_next();
+
+                    // If more data was read, convert it to an arrow table and
+                    // return
+                    if (buffers.has_value()) {
+                        // Acquire python GIL before accessing python objects
+                        py::gil_scoped_acquire acquire;
+                        return to_table(*buffers);
+                    }
+                } catch (const std::exception& e) {
+                    throw TileDBSOMAError(e.what());
                 }
 
                 // No data was read, the query is complete, return nullopt
diff --git a/apis/r/R/Init.R b/apis/r/R/Init.R
index 7ecf1156f9..723c832ee7 100644
--- a/apis/r/R/Init.R
+++ b/apis/r/R/Init.R
@@ -17,6 +17,25 @@
                        sQuote(rpkg_lib_version), sQuote(soma_lib_version))
         stop(msg, call. = FALSE)
     }
+
+    # This is temporary for https://github.com/single-cell-data/TileDB-SOMA/issues/2407
+    # It will be removed once 2407 is complete.
+    if (Sys.getenv("SOMA_R_NEW_SHAPE") != "") {
+      .pkgenv[["use_current_domain_transitional_internal_only"]] <- TRUE
+    } else {
+      .pkgenv[["use_current_domain_transitional_internal_only"]] <- FALSE
+    }
+    if (rpkg_lib_version != soma_lib_version) {
+      msg <- sprintf("TileDB Core version %s used by TileDB-R package, but TileDB-SOMA uses %s [%s]",
+                     sQuote(rpkg_lib_version), sQuote(soma_lib_version), sQuote(cdmsg))
+    }
+}
+
+# This is temporary only. Please see:
+# * https://github.com/single-cell-data/TileDB-SOMA/issues/2407
+# * https://github.com/single-cell-data/TileDB-SOMA/pull/2950
+.new_shape_feature_flag_is_enabled <- function() {
+    .pkgenv[["use_current_domain_transitional_internal_only"]]
 }
 
 ## An .onAttach() function is not allowed to use cat() etc but _must_ communicate via
diff --git a/apis/r/R/SOMANDArrayBase.R b/apis/r/R/SOMANDArrayBase.R
index b2a3adcfb0..4ce5d00372 100644
--- a/apis/r/R/SOMANDArrayBase.R
+++ b/apis/r/R/SOMANDArrayBase.R
@@ -39,7 +39,7 @@ SOMANDArrayBase <- R6::R6Class(
 
       #spdl::warn("[SOMANDArrayBase::create] type cached as {}", private$.type)
 
-      dom_ext_tbl <- get_domain_and_extent_array(shape)
+      dom_ext_tbl <- get_domain_and_extent_array(shape, private$.is_sparse)
 
       # Parse the tiledb/create/ subkeys of the platform_config into a handy,
       # typed, queryable data structure.
@@ -135,6 +135,20 @@ SOMANDArrayBase <- R6::R6Class(
       coords
     },
 
+    #  @description Converts a vector of ints into a vector of int64 in a format
+    #  acceptable for libtiledbsoma
+
+    .convert_shape_argument = function(new_shape) {
+      # ensure new_shape is an integerish vector
+      stopifnot(
+        "'new_shape' must be an integerish vector with the same length as the array's maxshape" = rlang::is_integerish(new_shape, n = self$ndim(), finite = TRUE) ||
+          (bit64::is.integer64(new_shape) && length(new_shape) == self$ndim() && all(is.finite(new_shape)))
+      )
+
+      # convert integer to integer64 to match dimension type
+      return(bit64::as.integer64(new_shape))
+    },
+
     # Internal marking of one or zero based matrices for iterated reads
     zero_based = NA
   )
diff --git a/apis/r/R/utils-arrow.R b/apis/r/R/utils-arrow.R
index ea6518702f..7ee2ebc2ae 100644
--- a/apis/r/R/utils-arrow.R
+++ b/apis/r/R/utils-arrow.R
@@ -360,18 +360,42 @@ get_domain_and_extent_dataframe <- function(tbl_schema, ind_col_names,
         ind_col <- tbl_schema$GetFieldByName(ind_col_name)
         ind_col_type <- ind_col$type
         ind_col_type_name <- ind_col$type$name
-        ind_dom <- arrow_type_unsigned_range(ind_col_type) - c(0,1) ## FIXME
+
+        # TODO: tiledbsoma-r does not accept the domain argument to SOMADataFrame::create, but should
+        # https://github.com/single-cell-data/TileDB-SOMA/issues/2967
         ind_ext <- tdco$dim_tile(ind_col_name)
-        if (ind_col_type_name %in% c("string", "large_utf8", "utf8")) ind_ext <- NA
+
         # Default 2048 mods to 0 for 8-bit types and 0 is an invalid extent
         if (ind_col$type$bit_width %||% 0L == 8L) {
             ind_ext <- 64L
         }
-        if (ind_col_type_name %in% c("string", "utf8", "large_utf8")) {
-            aa <- arrow::arrow_array(c("", "", ""), ind_col_type)
+
+        # We need to do this because if we don't:
+        #
+        # Error: [TileDB::Dimension] Error: Tile extent check failed; domain max
+        # expanded to multiple of tile extent exceeds max value representable by
+        # domain type. Reduce domain max by 1 tile extent to allow for
+        # expansion.
+        ind_max_dom <- arrow_type_unsigned_range(ind_col_type) - c(0,ind_ext)
+
+        ind_cur_dom <- ind_max_dom
+        if (ind_col_type_name %in% c("string", "large_utf8", "utf8")) ind_ext <- NA
+
+        # https://github.com/single-cell-data/TileDB-SOMA/issues/2407
+        if (.new_shape_feature_flag_is_enabled()) {
+            if (ind_col_type_name %in% c("string", "utf8", "large_utf8")) {
+                aa <- arrow::arrow_array(c("", "", "", "", ""), ind_col_type)
+            } else {
+                aa <- arrow::arrow_array(c(ind_max_dom, ind_ext, ind_cur_dom), ind_col_type)
+            }
         } else {
-            aa <- arrow::arrow_array(c(ind_dom, ind_ext), ind_col_type)
+            if (ind_col_type_name %in% c("string", "utf8", "large_utf8")) {
+                aa <- arrow::arrow_array(c("", "", ""), ind_col_type)
+            } else {
+                aa <- arrow::arrow_array(c(ind_max_dom, ind_ext), ind_col_type)
+            }
         }
+
         aa
     })
     names(rl) <- ind_col_names
@@ -382,15 +406,39 @@ get_domain_and_extent_dataframe <- function(tbl_schema, ind_col_names,
 #' Domain and extent table creation helper for array writes returning a Table with
 #' a column per dimension for the given (incoming) arrow schema of a Table
 #' @noRd
-get_domain_and_extent_array <- function(shape) {
+get_domain_and_extent_array <- function(shape, is_sparse) {
     stopifnot("First argument must be vector of positive values" = is.vector(shape) && all(shape > 0))
     indvec <- seq_len(length(shape)) - 1   # sequence 0, ..., length()-1
     rl <- sapply(indvec, \(ind) {
         ind_col <- sprintf("soma_dim_%d", ind)
         ind_col_type <- arrow::int64()
-        ind_dom <- c(0L, shape[ind+1] - 1L)
+
+        # TODO:  this function needs to take a
+        # TileDBCreateOptions$new(PlatformConfig option as
+        # get_domain_and_extent_dataframe does.
+        # https://github.com/single-cell-data/TileDB-SOMA/issues/2966
+        # For now, the core extent is not taken from the platform_config.
         ind_ext <- shape[ind+1]
-        aa <- arrow::arrow_array(c(ind_dom, ind_ext), ind_col_type)
+
+        ind_cur_dom <- c(0L, shape[ind+1] - 1L)
+
+        # We need to do this because if we don't:
+        #
+        # Error: [TileDB::Dimension] Error: Tile extent check failed; domain max
+        # expanded to multiple of tile extent exceeds max value representable by
+        # domain type. Reduce domain max by 1 tile extent to allow for
+        # expansion.
+        ind_max_dom <- arrow_type_unsigned_range(ind_col_type) - c(0,ind_ext)
+
+        # TODO: support current domain for dense arrays once we have that support
+        # from core.
+        # https://github.com/single-cell-data/TileDB-SOMA/issues/2955
+        if (.new_shape_feature_flag_is_enabled() && is_sparse) {
+            aa <- arrow::arrow_array(c(ind_max_dom, ind_ext, ind_cur_dom), ind_col_type)
+        } else {
+            aa <- arrow::arrow_array(c(ind_cur_dom, ind_ext), ind_col_type)
+        }
+
         aa
     })
     names(rl) <- sprintf("soma_dim_%d", indvec)


Test patch:
diff --git a/apis/python/tests/test_dataframe.py b/apis/python/tests/test_dataframe.py
index 1a53c7beb4..c5a9199901 100644
--- a/apis/python/tests/test_dataframe.py
+++ b/apis/python/tests/test_dataframe.py
@@ -91,7 +91,7 @@ def test_dataframe(tmp_path, arrow_schema):
         assert len(sdf) == 5
 
         # More to come on https://github.com/single-cell-data/TileDB-SOMA/issues/2407
-        assert not sdf.has_upgraded_domain
+        assert sdf.has_upgraded_domain == soma._new_shape_feature_flag_enabled()
 
         with pytest.raises(AttributeError):
             assert sdf.shape is None
diff --git a/apis/python/tests/test_shape.py b/apis/python/tests/test_shape.py
index d96821bec6..3a7470f042 100644
--- a/apis/python/tests/test_shape.py
+++ b/apis/python/tests/test_shape.py
@@ -53,7 +53,7 @@ def test_sparse_nd_array_basics(
         assert snda.shape == arg_shape
 
         # More to come on https://github.com/single-cell-data/TileDB-SOMA/issues/2407
-        assert not snda.has_upgraded_shape
+        assert snda.has_upgraded_shape == tiledbsoma._new_shape_feature_flag_enabled()
 
         # Before current-domain support: shape is maxshape.
         #
@@ -62,9 +62,11 @@ def test_sparse_nd_array_basics(
         # involving R compatibility, and leaving room for a single tile
         # capacity, etc ...  we could check for some magic value but it suffices
         # to check that it's over 2 billion.)
-        assert snda.shape == snda.maxshape
-        # for e in snda.maxshape:
-        #    assert e > 2_000_000_000
+        if tiledbsoma._new_shape_feature_flag_enabled():
+            for e in snda.maxshape:
+                assert e > 2_000_000_000
+        else:
+            assert snda.shape == snda.maxshape
 
         # No data have been written for this test case
         assert snda.non_empty_domain() == tuple([(0, 0)] * ndim)
@@ -91,9 +93,11 @@ def test_sparse_nd_array_basics(
     with tiledbsoma.SparseNDArray.open(uri) as snda:
         assert snda.shape == arg_shape
         # This will change with current-domain support
-        assert snda.shape == snda.maxshape
-        # for e in snda.maxshape:
-        #    assert e > 2_000_000_000
+        if tiledbsoma._new_shape_feature_flag_enabled():
+            for e in snda.maxshape:
+                assert e > 2_000_000_000
+        else:
+            assert snda.shape == snda.maxshape
         assert snda.non_empty_domain() == coords
 
     # Test reads out of bounds
@@ -113,11 +117,12 @@ def test_sparse_nd_array_basics(
 
     with tiledbsoma.SparseNDArray.open(uri) as snda:
         assert snda.shape == arg_shape
-        assert snda.shape == snda.maxshape
+        if not tiledbsoma._new_shape_feature_flag_enabled():
+            assert snda.shape == snda.maxshape
 
 
 ## Pending 2.27 timeframe for dense support for current domain, including resize
-## TODO: mark these with a linked GitHub tracking issue
+## https://github.com/single-cell-data/TileDB-SOMA/issues/2955
 def test_dense_nd_array_basics(tmp_path):
     uri = tmp_path.as_posix()
     shape = (100, 200)
@@ -202,9 +207,11 @@ def test_dataframe_basics(tmp_path, soma_joinid_domain, index_column_names):
         has_sjid_dim = "soma_joinid" in index_column_names
         if has_sjid_dim:
             assert sdf._maybe_soma_joinid_shape == 1 + soma_joinid_domain[1]
-            assert sdf._maybe_soma_joinid_maxshape == 1 + soma_joinid_domain[1]
+            if not tiledbsoma._new_shape_feature_flag_enabled():
+                assert sdf._maybe_soma_joinid_maxshape == 1 + soma_joinid_domain[1]
         else:
             assert sdf._maybe_soma_joinid_shape is None
-            assert sdf._maybe_soma_joinid_maxshape is None
+            if not tiledbsoma._new_shape_feature_flag_enabled():
+                assert sdf._maybe_soma_joinid_maxshape is None
 
         assert len(sdf.non_empty_domain()) == len(index_column_names)
diff --git a/apis/python/tests/test_sparse_nd_array.py b/apis/python/tests/test_sparse_nd_array.py
index ce7c859050..e735519d78 100644
--- a/apis/python/tests/test_sparse_nd_array.py
+++ b/apis/python/tests/test_sparse_nd_array.py
@@ -1070,8 +1070,9 @@ def test_bad_coords(tmp_path, bad_coords):
 
 
 def test_tile_extents(tmp_path):
+    uri = tmp_path.as_posix()
     soma.SparseNDArray.create(
-        tmp_path.as_posix(),
+        uri,
         type=pa.float32(),
         shape=(100, 10000),
         platform_config={
@@ -1086,9 +1087,13 @@ def test_tile_extents(tmp_path):
         },
     ).close()
 
-    with tiledb.open(tmp_path.as_posix()) as A:
-        assert A.schema.domain.dim(0).tile == 100
-        assert A.schema.domain.dim(1).tile == 2048
+    with tiledb.open(uri) as A:
+        if soma._new_shape_feature_flag_enabled():
+            assert A.schema.domain.dim(0).tile == 2048
+            assert A.schema.domain.dim(1).tile == 2048
+        else:
+            assert A.schema.domain.dim(0).tile == 100
+            assert A.schema.domain.dim(1).tile == 2048
 
 
 @pytest.mark.parametrize(
diff --git a/apis/r/tests/testthat/test-SOMASparseNDArray.R b/apis/r/tests/testthat/test-SOMASparseNDArray.R
index 62ee669430..d442d0444c 100644
--- a/apis/r/tests/testthat/test-SOMASparseNDArray.R
+++ b/apis/r/tests/testthat/test-SOMASparseNDArray.R
@@ -62,7 +62,11 @@ test_that("SOMASparseNDArray creation", {
   ## maxshape
   # TODO: more testing with current-domain feature integrated
   # https://github.com/single-cell-data/TileDB-SOMA/issues/2407
-  expect_false(ndarray$has_upgraded_shape())
+  if (.new_shape_feature_flag_is_enabled()) {
+    expect_true(ndarray$has_upgraded_shape())
+  } else {
+    expect_false(ndarray$has_upgraded_shape())
+  }
   shape <- ndarray$shape()
   maxshape <- ndarray$maxshape()
   expect_equal(length(shape), length(maxshape))
diff --git a/apis/r/tests/testthat/test-shape.r b/apis/r/tests/testthat/test-shape.r
index 1dc3626577..76a171dc37 100644
--- a/apis/r/tests/testthat/test-shape.r
+++ b/apis/r/tests/testthat/test-shape.r
@@ -30,7 +30,11 @@ test_that("SOMADataFrame shape", {
 
     sdf <- SOMADataFrameOpen(uri)
 
-    expect_false(sdf$has_upgraded_domain())
+    if (.new_shape_feature_flag_is_enabled()) {
+      expect_true(sdf$has_upgraded_domain())
+    } else {
+      expect_false(sdf$has_upgraded_domain())
+    }
     expect_error(sdf$shape(), class = "notYetImplementedError")
     expect_error(sdf$maxshape(), class = "notYetImplementedError")
 
@@ -83,11 +87,9 @@ test_that("SOMASparseNDArray shape", {
       readback_shape <- ndarray$shape()
       readback_maxshape <- ndarray$maxshape()
       expect_equal(length(readback_shape), length(readback_maxshape))
-      for (i in 1:length(shape)) {
-        s <- as.integer(readback_shape[[i]])
-        ms <- as.integer(readback_maxshape[[i]])
-        expect_true(s <= ms)
-      }
+      # We can't as.integer things that don't fit in a 32-bit int -- they come
+      # back as NA. :(
+      expect_true(all(readback_shape <= readback_maxshape))
 
       # Resize tests upcoming on
       # https://github.com/single-cell-data/TileDB-SOMA/issues/2407