Repository: single-cell-data/TileDB-SOMA (language: python)

Issue:
[python/r/c++] Revisit `shape` for component arrays
## PRs

**Merged PRs:**

<details>

* #2909 `kerl/schevo-timestamp-methodize`
* #2913 `kerl/name-neaten`
* #2908 `kerl/ut-soma-exc-simplify`
* #2910 `kerl/test-common-parameterize`
* #2918 `kerl/cpp-test-deadstrip`
* #2919 `kerl/minor-unit-test-helper-mod`
* #2936 `kerl/cpp-ut-helper-neaten`
* #2938 `kerl/more-cur-dom-parameterize`
* #2915 `kerl/cpp-strict-int64-shape`
* #2911 `kerl/arrow-util-current-domain-optional`
* #2939 `kerl/step-two-temp`
* #2947 `kerl/cpp-ndarray-resize-testing`
* #2945 `kerl/dataframe-test-fixture`
* #2944 `kerl/cpp-variant-indexed-dataframes`
* #2916 `kerl/sdf-shape`
* #2917 `kerl/cpp-resizes`
* #2948 `kerl/upgrade-shape-int64`
* #2951 `kerl/sdf-test-accessors`
* #2953 `kerl/py-r-accessor-plumbing`
* #2957 `kerl/sdf-domain-accessors`
* #2960 `kerl/dense-link`
* #2963 `kerl/nightly-fix`
* #2970 `kerl/dense-writeable-after-create`
* #2968 `kerl/minor-trim`
* #2969 `kerl/more-py-domain-name-neaten`
* #2972 `kerl/libtiledbsoma-env-logging-level`
* #2962 `kerl/py-r-creation-paths`
* #2950 `kerl/py-r-test-2`
* #2994 `kerl/nanoarrow-helpers`
* #3011 `kerl/polydom3`
* #3017 `kerl/polydom5`
* #3018 `kerl/polydom6`
* #2990 `kerl/variant-nnz-bug`
* #3019 `kerl/index-swap`
* #3020 `kerl/ut-max-shape`
* #3012 `kerl/polydom4`
* #3025 `kerl/fix-3020-merge`
* #3026 `kerl/one-more-rename`
* #3028 `kerl/ff-not`
* #3029 `kerl/ut-vg`
* #3030 `kerl/table-utils-memory`
* #3067 `kerl/improve-sdf-test-field-names`
* #3068 `kerl/ut-generate`
* #3069 `kerl/cpp-sdf-domain-at-create`
* #3027 `kerl/hll-domainish`
* #3088 `kerl/max-domain-int64`
* #3090 `kerl/maybe-resize-soma-joinid-cpp-tweak`
* #3032 `kerl/sdf-domain-at-create` -- fixes #2967
* #3091 `kerl/maybe-resize-soma-joinid-py-r`
* #3095 `kerl/cpp-exp-resize-prep`
* #3089 `kerl/r-dataframe-shapeable`
* https://github.com/single-cell-data/SOMA/pull/233
* #3125 `kerl/cpp-ut-name-shortens`
* #3127 `kerl/helper-rename`
* #3130 `kerl/cpp-can-resizers-names`
* #3132 `kerl/cpp-dataframe-sizing-helpers`
* #3139 `kerl/cpp-dataframe-upgrade-test`
* #3140 `kerl/py-resizer-connects`
* #3151 `kerl/py-can-upgrade-shape`
* #3152 `kerl/registration-shape-acceessors`
* #3156 `kerl/py-exp-shaping`
* #3157 `kerl/py-exp-shaping2`
* #3148 `kerl/py-exp-resize`
* #3191 `kerl/py-domain-at-create-ut-1`
* #3190 `kerl/py-domain-at-create-ut-2`
* #3192 `kerl/py-domain-at-create-ut-3`
* #3193 `kerl/py-domain-at-create-ut-4`
* #3194 `kerl/py-domain-at-create-ut-5`
* #3203 `kerl/min-size-2`
* #3208 `kerl/r-min-sizing`
* #3211 `kerl/cpp-ugr-dom`
* #3232 `kerl/ff-interop`
* #3230 `kerl/ffon`
* #3234 `kerl/docstring-prune`
* #3236 `kerl/prefixing`
* #3241 `kerl/fix-bad-merge`
* #3235 `kerl/py-r-ugr-dom`
* #3237 `kerl/py-r-ugr-dom-2`
* #3238 `kerl/py-r-ugr-dom-3`
* #3253 `kerl/set-coords-rename`
* #3261 `kerl/pybind11-nda-sizing`
* #3265 `kerl/dense-227-a`
* #3263 `kerl/dense-range-trim`
* #3268 `kerl/dim-explosion`
* #3269 `kerl/python-227-dense-ned-read`
* #3270 `kerl/r-227-dense-fixes`
* #3280 `kerl/r-dense-227-more`
* #3286 `kerl/more-fn4m`
* #3283 `kerl/readthedocs-pre-1.15`
* #3288 `kerl/dense-ugrsh`
* #3289 `kerl/notebook-shape-upgrade`
* #3285 `kerl/new-shape-doc-updates`
* #3295 `kerl/notebook-data-refresh`
* #3290 `kerl/notebook-new-shape-refresh`
* #3301 `kerl/ffena`
* #3303 `kerl/r-data-refresh`
* #3300 `kerl/sdf-sjid-lower-zero`
* #3296 `kerl/dense-example-data-refresh`
* #3294 `kerl/new-shape-notebook-and-vignette`
* #3308 `kerl/upgrade-experiment-resources`
* #3309 `kerl/fix-notebook-merge`
* #3321 `kerl/more-use-shape`
* #3358 `kerl/revert-3300`
* #3368 `kerl/227a`
* #3369 `kerl/ucd1`
* #3370 `kerl/ucd2`
* #3371 `kerl/ucd3`
* #3372 `kerl/ucd4`
* #3396 `kerl/domain-at-create-docstrings`
* #3302 `kerl/new-shape-vignette`
* https://github.com/single-cell-data/SOMA/pull/250
* #3399 `kerl/new-shape-more-docstrings`
* #3400 `kerl/check-only-r`
* https://github.com/TileDB-Inc/somacore-feedstock/pull/27

</details>

**Closed/abandoned PRs:**

<details>

* #2785 -- This was only dogfooding for the core 2.25 release -- not to be merged
* #2952 `kerl/feature-flag-temp` -- folded into 2962
* #2995 `kerl/polydom`
* #2964 `kerl/tiledbsoma-io-test`
* #3189 `kerl/min-size`
* #3220 `kerl/cpp-ugr-dom-2`
* #3244 `kerl/dense-227-fixes`
* #3276 `kerl/dataframe-shape`

</details>

**Issues which are related but non-blocking:**

* https://github.com/single-cell-data/SOMA/pull/216
* https://github.com/TileDB-Inc/TileDB/pull/5303
* #2966
* #3081
* Note: R append mode does not exist yet -- see #1630 -- so an experiment-level resizer is not a priority in R
* #3271 
* #3272
* #3273

**See also:** [[sc-51048]](https://app.shortcut.com/tiledb-inc/story/51048).

## Problem to be solved

Users want to know the `shape` of an array, in the SciPy sense:

* Reads and writes are bounds-checked against the shape
* This retains its value regardless of which values of a sparse array are or are not actually occupied
* Users can `resize`.
  * Some users need the ability to grow their datasets later, using either `tiledbsoma.io`'s append mode, or subsequent writes using the `tiledbsoma` API.
  * Note that the cellxgene census doesn't need this: eact week's published census has fixed shape, and any updates will happen in new storage, on a new week.

Using TileDB-SOMA up until the present:

* The TIleDB `domain` is immutable after array creation
  * This does bounds-checking for reads and writes, which is good
  * To leverage this to function as a `shape`, users would need to set the `domain` at array-creation time. However, users lose the ability to grow their datasets later.
* There is a `non_empty_domain` accessor
  * This only indicates min/max coordinates _at which data exists_. Consider an `X` array for 100 cells and 200 genes. If non-zero expression counts exist only for cell join IDs 2-17, then the `non_empty_domain` will indicate `(2,17)` along `soma_dim_0`.
  * Consider an `obms["X_pca"]` within the same experiment. This may be 100 cells by 50 PCA components: we need a placd to store the number 50.
  * Therefore users cannot leverage this to function as a `shape` accessor.
* We have offered a `used_shape` accessor since TileDB-SOMA 1.5.
  * This functions as a `shape` accessor, in the SciPy sense, but it is not multi-writer safe.

New feature for TileDB-SOMA 1.15:

* Arrays will have a  `shape`
* Reads and writes are bounds-checked against the shape
* This retains its value regardless of which values of a sparse array are or are not actually occupied
* Users can `resize`
* The `used_shape` accessor will be deprecated in TileDB-SOMA 1.13, and slated for removal in TileDB-SOMA 1.14.

Compatiblity:

This will now require users to do an explicit `resize` before appending/growing TileDB-SOMA Experiments. Guidance in the form of example notebooks will be provided.

##  Tracking

See also: [[sc-41074]](https://app.shortcut.com/tiledb-inc/story/41074) and [[sc-51048]](https://app.shortcut.com/tiledb-inc/story/51048).

## Scheduling

Support arrives in TileDB Core 2.25. Deprecations for TileDB-SOMA will be released with 1.13. Full support within TileDB-SOMA will be release in 1.14.

## Details

SOMA API mods as we've discussed in a Google doc are as follows.

### `SOMADataFrame`

* `create`: Retain the `domain` argument
  * Issue:
    * Core has a `(lo, hi)` tuple per dim, e.g. `(0,99)` or `(10,19)`
    * SOMA has count per dim, with 0 implicit: e.g. 100 or 20
    * For `SparseNDArray` and `DenseNDArray` core can have `(lo, hi)` and SOMA can have `count`
    * For `DataFrame` there can be multiple dims --- default is a single `soma_joinid`
    * That could be treated either in `(lo, hi)` fashion or `count` fashion
    * However additional dims (e.g. `cell_type`) can be on any type, including strings, floats, etc. where there is no implicit lo=0
    * Therefore we need to keep the current SOMA API wherein `DataFrame` takes a `domain` argument (in `(lo, hi)` fashion) and not a `shape` argument (in `count` fashion)

### `SparseNDArray and DenseNDArray`

* `create`
  * Have an optional shape argument which is of type `Tuple[Int,...]` where each element is the cell count of the corresponding dimension
    * If unsupplied, or if supplied but None in any slot: use the minimum 0 in each slot – nothing larger makes sense since we will not support downsize
  * User guidance should make clear that it will not be possible to create an ‘old’ style array with the ‘new style’ API. (See also the upgrade logic below.)

### All three of `SOMADataFrame`, `SparseNDArray`, `DenseNDArray`

* `write`
  * For new arrays, created with the new shape feature:
    * Core will bounds-check that coordinates provided at `write` time are within the current shape
    * Core will raise `tiledb.cc.TileDBError` to TileDB-SOMA, which will catch and raise `IndexError`, and R-standard behavior on the R side
  * For old arrays created before this feature:
    * Core will not bounds-check that coordinates provided at write time are within the current shape
* Existing `used_shape` accessor
  * TileDB-SOMA will deprecate this over a release cycle.
  * For new arrays: raise `NotImplementedError`
  * For old arrays: return what’s currently returned, with a deprecation warning.
  * Mechanism for determining old vs. new: `array.schema.version` (the core storage version).
* Existing `shape` accessor
  * For new arrays:
    * Have this return the new shape as proposed by core, no longer returning the TileDB domain.
  * For old arrays created before this feature:
    * Return the TileDB domain as now.
* Existing `non_empty_domain` accessor
  * Same behavior for old and new arrays (unaffected by this proposal).
  * Keep this accessor supported, but, with user notes that it’s generally non-useful
  * This should return None (or R equivalent) when there is a schema but no data have been written.
* New `maxshape` accessor
  * Maps the core-level `(lo, hi)` accessor for domain to count-style accessor hi+1. E.g. if the core domain is either `(0,99)` or `(50,99)` then TileDB-SOMA `maxshape` will say 100.
  * Same behavior for old and new arrays.
  * Let users query for what the TileDB domain is, with user notes that it’s the maximum that users can reshape to.
  * Issac suggests: maybe `domain` or `maxshape` (see h5py).
* New `resize` mutator
  * Note: `reshape` means something else in the community (numpy, zarr, h5py), e.g. a 5x20 (total 100 cells) being reinterpreted as 4x25 (still 100 cells). The standard name for changing cell-count is `resize`.
  * For old arrays created before this feature: raise `NotImplementedError`.
  * For new arrays:
    * Will raise `ValueError` if the new shape is smaller on any dim than currently in storage
    * Regardless of whether any data have been written whatsoever
    * Will raise `ValueError` if the new shape exceeds the TileDB domain from create time, which will serve TileDB-SOMA in a role of “max possible shape the user can reshape to”
    * Otherwise, any calls to write from this point will bounds-check writes within this new shape
    * **We don’t expect resize to be multi-writer safe with regard to write ; user notes must be clear on this point**
* New `tiledbsoma_upgrade_shape` method for SparseNDArray and DenseNDArray
  * This will leverage `array.schema.version` to see if an upgrade is needed
  * Leverage core support for storage-version updates
  * This will take a shape argument as in `create`
  * For arrays created with “just-right” size: this will succeed
  * For arrays created with “room-for-growth” / “two billion-ish” size: this will succeed
  * If the user passes a shape which exceeds the current TileDB domain: this will fail
* New `tiledbsoma_upgrade_domain method` for `DataFrame`
  * Same as for `SparseNDArray`/`DenseNDArray` except it will take a domain at the SOMA-API level just as `DataFrame`'s create method

### `tiledbsoma.io`

* The user-facing API has no shape arguments and thus won’t need changing.
* Internally to `tiledbsoma.io`, we’ll still ask the tiledbsoma API for the “big domain” (2 billionish)
* Append mode:
  * Will need a new `resize` method at the `Experiment` level
  * Users will need to:
    * Register as now
    * Call the experiment-level `resize`
      * Could be `exp.resize(...)`, or (better) this could be `tiledbsoma.io.reshape_experiment`
  * In either case: this method will take the new `obs` and `var` counts as inputs:
    * `exp.obs.reshape` to new `obs` count
    * `exp.ms[name].var.reshape` to new `var` count
    * `exp.ms[name].X[name].reshape` to new `obs` count x `var` count
    * `exp.ms[name].obsm[name].reshape` to new `obs` count x same width
    * `exp.ms[name].obsp[name].reshape` to new `obs` count x `obs` count
    * `exp.ms[name].varm[name].reshape` to new `var` count x same width
    * `exp.ms[name].varp[name].reshape` to new `var` count x `var` count
  * Do the individual append-mode writes as now

[python/r/c++] Revisit `shape` for component arrays
## PRs

**Merged PRs:**

<details>

* #2909 `kerl/schevo-timestamp-methodize`
* #2913 `kerl/name-neaten`
* #2908 `kerl/ut-soma-exc-simplify`
* #2910 `kerl/test-common-parameterize`
* #2918 `kerl/cpp-test-deadstrip`
* #2919 `kerl/minor-unit-test-helper-mod`
* #2936 `kerl/cpp-ut-helper-neaten`
* #2938 `kerl/more-cur-dom-parameterize`
* #2915 `kerl/cpp-strict-int64-shape`
* #2911 `kerl/arrow-util-current-domain-optional`
* #2939 `kerl/step-two-temp`
* #2947 `kerl/cpp-ndarray-resize-testing`
* #2945 `kerl/dataframe-test-fixture`
* #2944 `kerl/cpp-variant-indexed-dataframes`
* #2916 `kerl/sdf-shape`
* #2917 `kerl/cpp-resizes`
* #2948 `kerl/upgrade-shape-int64`
* #2951 `kerl/sdf-test-accessors`
* #2953 `kerl/py-r-accessor-plumbing`
* #2957 `kerl/sdf-domain-accessors`
* #2960 `kerl/dense-link`
* #2963 `kerl/nightly-fix`
* #2970 `kerl/dense-writeable-after-create`
* #2968 `kerl/minor-trim`
* #2969 `kerl/more-py-domain-name-neaten`
* #2972 `kerl/libtiledbsoma-env-logging-level`
* #2962 `kerl/py-r-creation-paths`
* #2950 `kerl/py-r-test-2`
* #2994 `kerl/nanoarrow-helpers`
* #3011 `kerl/polydom3`
* #3017 `kerl/polydom5`
* #3018 `kerl/polydom6`
* #2990 `kerl/variant-nnz-bug`
* #3019 `kerl/index-swap`
* #3020 `kerl/ut-max-shape`
* #3012 `kerl/polydom4`
* #3025 `kerl/fix-3020-merge`
* #3026 `kerl/one-more-rename`
* #3028 `kerl/ff-not`
* #3029 `kerl/ut-vg`
* #3030 `kerl/table-utils-memory`
* #3067 `kerl/improve-sdf-test-field-names`
* #3068 `kerl/ut-generate`
* #3069 `kerl/cpp-sdf-domain-at-create`
* #3027 `kerl/hll-domainish`
* #3088 `kerl/max-domain-int64`
* #3090 `kerl/maybe-resize-soma-joinid-cpp-tweak`
* #3032 `kerl/sdf-domain-at-create` -- fixes #2967
* #3091 `kerl/maybe-resize-soma-joinid-py-r`
* #3095 `kerl/cpp-exp-resize-prep`
* #3089 `kerl/r-dataframe-shapeable`
* https://github.com/single-cell-data/SOMA/pull/233
* #3125 `kerl/cpp-ut-name-shortens`
* #3127 `kerl/helper-rename`
* #3130 `kerl/cpp-can-resizers-names`
* #3132 `kerl/cpp-dataframe-sizing-helpers`
* #3139 `kerl/cpp-dataframe-upgrade-test`
* #3140 `kerl/py-resizer-connects`
* #3151 `kerl/py-can-upgrade-shape`
* #3152 `kerl/registration-shape-acceessors`
* #3156 `kerl/py-exp-shaping`
* #3157 `kerl/py-exp-shaping2`
* #3148 `kerl/py-exp-resize`
* #3191 `kerl/py-domain-at-create-ut-1`
* #3190 `kerl/py-domain-at-create-ut-2`
* #3192 `kerl/py-domain-at-create-ut-3`
* #3193 `kerl/py-domain-at-create-ut-4`
* #3194 `kerl/py-domain-at-create-ut-5`
* #3203 `kerl/min-size-2`
* #3208 `kerl/r-min-sizing`
* #3211 `kerl/cpp-ugr-dom`
* #3232 `kerl/ff-interop`
* #3230 `kerl/ffon`
* #3234 `kerl/docstring-prune`
* #3236 `kerl/prefixing`
* #3241 `kerl/fix-bad-merge`
* #3235 `kerl/py-r-ugr-dom`
* #3237 `kerl/py-r-ugr-dom-2`
* #3238 `kerl/py-r-ugr-dom-3`
* #3253 `kerl/set-coords-rename`
* #3261 `kerl/pybind11-nda-sizing`
* #3265 `kerl/dense-227-a`
* #3263 `kerl/dense-range-trim`
* #3268 `kerl/dim-explosion`
* #3269 `kerl/python-227-dense-ned-read`
* #3270 `kerl/r-227-dense-fixes`
* #3280 `kerl/r-dense-227-more`
* #3286 `kerl/more-fn4m`
* #3283 `kerl/readthedocs-pre-1.15`
* #3288 `kerl/dense-ugrsh`
* #3289 `kerl/notebook-shape-upgrade`
* #3285 `kerl/new-shape-doc-updates`
* #3295 `kerl/notebook-data-refresh`
* #3290 `kerl/notebook-new-shape-refresh`
* #3301 `kerl/ffena`
* #3303 `kerl/r-data-refresh`
* #3300 `kerl/sdf-sjid-lower-zero`
* #3296 `kerl/dense-example-data-refresh`
* #3294 `kerl/new-shape-notebook-and-vignette`
* #3308 `kerl/upgrade-experiment-resources`
* #3309 `kerl/fix-notebook-merge`
* #3321 `kerl/more-use-shape`
* #3358 `kerl/revert-3300`
* #3368 `kerl/227a`
* #3369 `kerl/ucd1`
* #3370 `kerl/ucd2`
* #3371 `kerl/ucd3`
* #3372 `kerl/ucd4`
* #3396 `kerl/domain-at-create-docstrings`
* #3302 `kerl/new-shape-vignette`
* https://github.com/single-cell-data/SOMA/pull/250
* #3399 `kerl/new-shape-more-docstrings`
* #3400 `kerl/check-only-r`
* https://github.com/TileDB-Inc/somacore-feedstock/pull/27

</details>

**Closed/abandoned PRs:**

<details>

* #2785 -- This was only dogfooding for the core 2.25 release -- not to be merged
* #2952 `kerl/feature-flag-temp` -- folded into 2962
* #2995 `kerl/polydom`
* #2964 `kerl/tiledbsoma-io-test`
* #3189 `kerl/min-size`
* #3220 `kerl/cpp-ugr-dom-2`
* #3244 `kerl/dense-227-fixes`
* #3276 `kerl/dataframe-shape`

</details>

**Issues which are related but non-blocking:**

* https://github.com/single-cell-data/SOMA/pull/216
* https://github.com/TileDB-Inc/TileDB/pull/5303
* #2966
* #3081
* Note: R append mode does not exist yet -- see #1630 -- so an experiment-level resizer is not a priority in R
* #3271 
* #3272
* #3273

**See also:** [[sc-51048]](https://app.shortcut.com/tiledb-inc/story/51048).

## Problem to be solved

Users want to know the `shape` of an array, in the SciPy sense:

* Reads and writes are bounds-checked against the shape
* This retains its value regardless of which values of a sparse array are or are not actually occupied
* Users can `resize`.
  * Some users need the ability to grow their datasets later, using either `tiledbsoma.io`'s append mode, or subsequent writes using the `tiledbsoma` API.
  * Note that the cellxgene census doesn't need this: eact week's published census has fixed shape, and any updates will happen in new storage, on a new week.

Using TileDB-SOMA up until the present:

* The TIleDB `domain` is immutable after array creation
  * This does bounds-checking for reads and writes, which is good
  * To leverage this to function as a `shape`, users would need to set the `domain` at array-creation time. However, users lose the ability to grow their datasets later.
* There is a `non_empty_domain` accessor
  * This only indicates min/max coordinates _at which data exists_. Consider an `X` array for 100 cells and 200 genes. If non-zero expression counts exist only for cell join IDs 2-17, then the `non_empty_domain` will indicate `(2,17)` along `soma_dim_0`.
  * Consider an `obms["X_pca"]` within the same experiment. This may be 100 cells by 50 PCA components: we need a placd to store the number 50.
  * Therefore users cannot leverage this to function as a `shape` accessor.
* We have offered a `used_shape` accessor since TileDB-SOMA 1.5.
  * This functions as a `shape` accessor, in the SciPy sense, but it is not multi-writer safe.

New feature for TileDB-SOMA 1.15:

* Arrays will have a  `shape`
* Reads and writes are bounds-checked against the shape
* This retains its value regardless of which values of a sparse array are or are not actually occupied
* Users can `resize`
* The `used_shape` accessor will be deprecated in TileDB-SOMA 1.13, and slated for removal in TileDB-SOMA 1.14.

Compatiblity:

This will now require users to do an explicit `resize` before appending/growing TileDB-SOMA Experiments. Guidance in the form of example notebooks will be provided.

##  Tracking

See also: [[sc-41074]](https://app.shortcut.com/tiledb-inc/story/41074) and [[sc-51048]](https://app.shortcut.com/tiledb-inc/story/51048).

## Scheduling

Support arrives in TileDB Core 2.25. Deprecations for TileDB-SOMA will be released with 1.13. Full support within TileDB-SOMA will be release in 1.14.

## Details

SOMA API mods as we've discussed in a Google doc are as follows.

### `SOMADataFrame`

* `create`: Retain the `domain` argument
  * Issue:
    * Core has a `(lo, hi)` tuple per dim, e.g. `(0,99)` or `(10,19)`
    * SOMA has count per dim, with 0 implicit: e.g. 100 or 20
    * For `SparseNDArray` and `DenseNDArray` core can have `(lo, hi)` and SOMA can have `count`
    * For `DataFrame` there can be multiple dims --- default is a single `soma_joinid`
    * That could be treated either in `(lo, hi)` fashion or `count` fashion
    * However additional dims (e.g. `cell_type`) can be on any type, including strings, floats, etc. where there is no implicit lo=0
    * Therefore we need to keep the current SOMA API wherein `DataFrame` takes a `domain` argument (in `(lo, hi)` fashion) and not a `shape` argument (in `count` fashion)

### `SparseNDArray and DenseNDArray`

* `create`
  * Have an optional shape argument which is of type `Tuple[Int,...]` where each element is the cell count of the corresponding dimension
    * If unsupplied, or if supplied but None in any slot: use the minimum 0 in each slot – nothing larger makes sense since we will not support downsize
  * User guidance should make clear that it will not be possible to create an ‘old’ style array with the ‘new style’ API. (See also the upgrade logic below.)

### All three of `SOMADataFrame`, `SparseNDArray`, `DenseNDArray`

* `write`
  * For new arrays, created with the new shape feature:
    * Core will bounds-check that coordinates provided at `write` time are within the current shape
    * Core will raise `tiledb.cc.TileDBError` to TileDB-SOMA, which will catch and raise `IndexError`, and R-standard behavior on the R side
  * For old arrays created before this feature:
    * Core will not bounds-check that coordinates provided at write time are within the current shape
* Existing `used_shape` accessor
  * TileDB-SOMA will deprecate this over a release cycle.
  * For new arrays: raise `NotImplementedError`
  * For old arrays: return what’s currently returned, with a deprecation warning.
  * Mechanism for determining old vs. new: `array.schema.version` (the core storage version).
* Existing `shape` accessor
  * For new arrays:
    * Have this return the new shape as proposed by core, no longer returning the TileDB domain.
  * For old arrays created before this feature:
    * Return the TileDB domain as now.
* Existing `non_empty_domain` accessor
  * Same behavior for old and new arrays (unaffected by this proposal).
  * Keep this accessor supported, but, with user notes that it’s generally non-useful
  * This should return None (or R equivalent) when there is a schema but no data have been written.
* New `maxshape` accessor
  * Maps the core-level `(lo, hi)` accessor for domain to count-style accessor hi+1. E.g. if the core domain is either `(0,99)` or `(50,99)` then TileDB-SOMA `maxshape` will say 100.
  * Same behavior for old and new arrays.
  * Let users query for what the TileDB domain is, with user notes that it’s the maximum that users can reshape to.
  * Issac suggests: maybe `domain` or `maxshape` (see h5py).
* New `resize` mutator
  * Note: `reshape` means something else in the community (numpy, zarr, h5py), e.g. a 5x20 (total 100 cells) being reinterpreted as 4x25 (still 100 cells). The standard name for changing cell-count is `resize`.
  * For old arrays created before this feature: raise `NotImplementedError`.
  * For new arrays:
    * Will raise `ValueError` if the new shape is smaller on any dim than currently in storage
    * Regardless of whether any data have been written whatsoever
    * Will raise `ValueError` if the new shape exceeds the TileDB domain from create time, which will serve TileDB-SOMA in a role of “max possible shape the user can reshape to”
    * Otherwise, any calls to write from this point will bounds-check writes within this new shape
    * **We don’t expect resize to be multi-writer safe with regard to write ; user notes must be clear on this point**
* New `tiledbsoma_upgrade_shape` method for SparseNDArray and DenseNDArray
  * This will leverage `array.schema.version` to see if an upgrade is needed
  * Leverage core support for storage-version updates
  * This will take a shape argument as in `create`
  * For arrays created with “just-right” size: this will succeed
  * For arrays created with “room-for-growth” / “two billion-ish” size: this will succeed
  * If the user passes a shape which exceeds the current TileDB domain: this will fail
* New `tiledbsoma_upgrade_domain method` for `DataFrame`
  * Same as for `SparseNDArray`/`DenseNDArray` except it will take a domain at the SOMA-API level just as `DataFrame`'s create method

### `tiledbsoma.io`

* The user-facing API has no shape arguments and thus won’t need changing.
* Internally to `tiledbsoma.io`, we’ll still ask the tiledbsoma API for the “big domain” (2 billionish)
* Append mode:
  * Will need a new `resize` method at the `Experiment` level
  * Users will need to:
    * Register as now
    * Call the experiment-level `resize`
      * Could be `exp.resize(...)`, or (better) this could be `tiledbsoma.io.reshape_experiment`
  * In either case: this method will take the new `obs` and `var` counts as inputs:
    * `exp.obs.reshape` to new `obs` count
    * `exp.ms[name].var.reshape` to new `var` count
    * `exp.ms[name].X[name].reshape` to new `obs` count x `var` count
    * `exp.ms[name].obsm[name].reshape` to new `obs` count x same width
    * `exp.ms[name].obsp[name].reshape` to new `obs` count x `obs` count
    * `exp.ms[name].varm[name].reshape` to new `var` count x same width
    * `exp.ms[name].varp[name].reshape` to new `var` count x `var` count
  * Do the individual append-mode writes as now

[python/r/c++] Revisit `shape` for component arrays
## PRs

**Merged PRs:**

<details>

* #2909 `kerl/schevo-timestamp-methodize`
* #2913 `kerl/name-neaten`
* #2908 `kerl/ut-soma-exc-simplify`
* #2910 `kerl/test-common-parameterize`
* #2918 `kerl/cpp-test-deadstrip`
* #2919 `kerl/minor-unit-test-helper-mod`
* #2936 `kerl/cpp-ut-helper-neaten`
* #2938 `kerl/more-cur-dom-parameterize`
* #2915 `kerl/cpp-strict-int64-shape`
* #2911 `kerl/arrow-util-current-domain-optional`
* #2939 `kerl/step-two-temp`
* #2947 `kerl/cpp-ndarray-resize-testing`
* #2945 `kerl/dataframe-test-fixture`
* #2944 `kerl/cpp-variant-indexed-dataframes`
* #2916 `kerl/sdf-shape`
* #2917 `kerl/cpp-resizes`
* #2948 `kerl/upgrade-shape-int64`
* #2951 `kerl/sdf-test-accessors`
* #2953 `kerl/py-r-accessor-plumbing`
* #2957 `kerl/sdf-domain-accessors`
* #2960 `kerl/dense-link`
* #2963 `kerl/nightly-fix`
* #2970 `kerl/dense-writeable-after-create`
* #2968 `kerl/minor-trim`
* #2969 `kerl/more-py-domain-name-neaten`
* #2972 `kerl/libtiledbsoma-env-logging-level`
* #2962 `kerl/py-r-creation-paths`
* #2950 `kerl/py-r-test-2`
* #2994 `kerl/nanoarrow-helpers`
* #3011 `kerl/polydom3`
* #3017 `kerl/polydom5`
* #3018 `kerl/polydom6`
* #2990 `kerl/variant-nnz-bug`
* #3019 `kerl/index-swap`
* #3020 `kerl/ut-max-shape`
* #3012 `kerl/polydom4`
* #3025 `kerl/fix-3020-merge`
* #3026 `kerl/one-more-rename`
* #3028 `kerl/ff-not`
* #3029 `kerl/ut-vg`
* #3030 `kerl/table-utils-memory`
* #3067 `kerl/improve-sdf-test-field-names`
* #3068 `kerl/ut-generate`
* #3069 `kerl/cpp-sdf-domain-at-create`
* #3027 `kerl/hll-domainish`
* #3088 `kerl/max-domain-int64`
* #3090 `kerl/maybe-resize-soma-joinid-cpp-tweak`
* #3032 `kerl/sdf-domain-at-create` -- fixes #2967
* #3091 `kerl/maybe-resize-soma-joinid-py-r`
* #3095 `kerl/cpp-exp-resize-prep`
* #3089 `kerl/r-dataframe-shapeable`
* https://github.com/single-cell-data/SOMA/pull/233
* #3125 `kerl/cpp-ut-name-shortens`
* #3127 `kerl/helper-rename`
* #3130 `kerl/cpp-can-resizers-names`
* #3132 `kerl/cpp-dataframe-sizing-helpers`
* #3139 `kerl/cpp-dataframe-upgrade-test`
* #3140 `kerl/py-resizer-connects`
* #3151 `kerl/py-can-upgrade-shape`
* #3152 `kerl/registration-shape-acceessors`
* #3156 `kerl/py-exp-shaping`
* #3157 `kerl/py-exp-shaping2`
* #3148 `kerl/py-exp-resize`
* #3191 `kerl/py-domain-at-create-ut-1`
* #3190 `kerl/py-domain-at-create-ut-2`
* #3192 `kerl/py-domain-at-create-ut-3`
* #3193 `kerl/py-domain-at-create-ut-4`
* #3194 `kerl/py-domain-at-create-ut-5`
* #3203 `kerl/min-size-2`
* #3208 `kerl/r-min-sizing`
* #3211 `kerl/cpp-ugr-dom`
* #3232 `kerl/ff-interop`
* #3230 `kerl/ffon`
* #3234 `kerl/docstring-prune`
* #3236 `kerl/prefixing`
* #3241 `kerl/fix-bad-merge`
* #3235 `kerl/py-r-ugr-dom`
* #3237 `kerl/py-r-ugr-dom-2`
* #3238 `kerl/py-r-ugr-dom-3`
* #3253 `kerl/set-coords-rename`
* #3261 `kerl/pybind11-nda-sizing`
* #3265 `kerl/dense-227-a`
* #3263 `kerl/dense-range-trim`
* #3268 `kerl/dim-explosion`
* #3269 `kerl/python-227-dense-ned-read`
* #3270 `kerl/r-227-dense-fixes`
* #3280 `kerl/r-dense-227-more`
* #3286 `kerl/more-fn4m`
* #3283 `kerl/readthedocs-pre-1.15`
* #3288 `kerl/dense-ugrsh`
* #3289 `kerl/notebook-shape-upgrade`
* #3285 `kerl/new-shape-doc-updates`
* #3295 `kerl/notebook-data-refresh`
* #3290 `kerl/notebook-new-shape-refresh`
* #3301 `kerl/ffena`
* #3303 `kerl/r-data-refresh`
* #3300 `kerl/sdf-sjid-lower-zero`
* #3296 `kerl/dense-example-data-refresh`
* #3294 `kerl/new-shape-notebook-and-vignette`
* #3308 `kerl/upgrade-experiment-resources`
* #3309 `kerl/fix-notebook-merge`
* #3321 `kerl/more-use-shape`
* #3358 `kerl/revert-3300`
* #3368 `kerl/227a`
* #3369 `kerl/ucd1`
* #3370 `kerl/ucd2`
* #3371 `kerl/ucd3`
* #3372 `kerl/ucd4`
* #3396 `kerl/domain-at-create-docstrings`
* #3302 `kerl/new-shape-vignette`
* https://github.com/single-cell-data/SOMA/pull/250
* #3399 `kerl/new-shape-more-docstrings`
* #3400 `kerl/check-only-r`
* https://github.com/TileDB-Inc/somacore-feedstock/pull/27

</details>

**Closed/abandoned PRs:**

<details>

* #2785 -- This was only dogfooding for the core 2.25 release -- not to be merged
* #2952 `kerl/feature-flag-temp` -- folded into 2962
* #2995 `kerl/polydom`
* #2964 `kerl/tiledbsoma-io-test`
* #3189 `kerl/min-size`
* #3220 `kerl/cpp-ugr-dom-2`
* #3244 `kerl/dense-227-fixes`
* #3276 `kerl/dataframe-shape`

</details>

**Issues which are related but non-blocking:**

* https://github.com/single-cell-data/SOMA/pull/216
* https://github.com/TileDB-Inc/TileDB/pull/5303
* #2966
* #3081
* Note: R append mode does not exist yet -- see #1630 -- so an experiment-level resizer is not a priority in R
* #3271 
* #3272
* #3273

**See also:** [[sc-51048]](https://app.shortcut.com/tiledb-inc/story/51048).

## Problem to be solved

Users want to know the `shape` of an array, in the SciPy sense:

* Reads and writes are bounds-checked against the shape
* This retains its value regardless of which values of a sparse array are or are not actually occupied
* Users can `resize`.
  * Some users need the ability to grow their datasets later, using either `tiledbsoma.io`'s append mode, or subsequent writes using the `tiledbsoma` API.
  * Note that the cellxgene census doesn't need this: eact week's published census has fixed shape, and any updates will happen in new storage, on a new week.

Using TileDB-SOMA up until the present:

* The TIleDB `domain` is immutable after array creation
  * This does bounds-checking for reads and writes, which is good
  * To leverage this to function as a `shape`, users would need to set the `domain` at array-creation time. However, users lose the ability to grow their datasets later.
* There is a `non_empty_domain` accessor
  * This only indicates min/max coordinates _at which data exists_. Consider an `X` array for 100 cells and 200 genes. If non-zero expression counts exist only for cell join IDs 2-17, then the `non_empty_domain` will indicate `(2,17)` along `soma_dim_0`.
  * Consider an `obms["X_pca"]` within the same experiment. This may be 100 cells by 50 PCA components: we need a placd to store the number 50.
  * Therefore users cannot leverage this to function as a `shape` accessor.
* We have offered a `used_shape` accessor since TileDB-SOMA 1.5.
  * This functions as a `shape` accessor, in the SciPy sense, but it is not multi-writer safe.

New feature for TileDB-SOMA 1.15:

* Arrays will have a  `shape`
* Reads and writes are bounds-checked against the shape
* This retains its value regardless of which values of a sparse array are or are not actually occupied
* Users can `resize`
* The `used_shape` accessor will be deprecated in TileDB-SOMA 1.13, and slated for removal in TileDB-SOMA 1.14.

Compatiblity:

This will now require users to do an explicit `resize` before appending/growing TileDB-SOMA Experiments. Guidance in the form of example notebooks will be provided.

##  Tracking

See also: [[sc-41074]](https://app.shortcut.com/tiledb-inc/story/41074) and [[sc-51048]](https://app.shortcut.com/tiledb-inc/story/51048).

## Scheduling

Support arrives in TileDB Core 2.25. Deprecations for TileDB-SOMA will be released with 1.13. Full support within TileDB-SOMA will be release in 1.14.

## Details

SOMA API mods as we've discussed in a Google doc are as follows.

### `SOMADataFrame`

* `create`: Retain the `domain` argument
  * Issue:
    * Core has a `(lo, hi)` tuple per dim, e.g. `(0,99)` or `(10,19)`
    * SOMA has count per dim, with 0 implicit: e.g. 100 or 20
    * For `SparseNDArray` and `DenseNDArray` core can have `(lo, hi)` and SOMA can have `count`
    * For `DataFrame` there can be multiple dims --- default is a single `soma_joinid`
    * That could be treated either in `(lo, hi)` fashion or `count` fashion
    * However additional dims (e.g. `cell_type`) can be on any type, including strings, floats, etc. where there is no implicit lo=0
    * Therefore we need to keep the current SOMA API wherein `DataFrame` takes a `domain` argument (in `(lo, hi)` fashion) and not a `shape` argument (in `count` fashion)

### `SparseNDArray and DenseNDArray`

* `create`
  * Have an optional shape argument which is of type `Tuple[Int,...]` where each element is the cell count of the corresponding dimension
    * If unsupplied, or if supplied but None in any slot: use the minimum 0 in each slot – nothing larger makes sense since we will not support downsize
  * User guidance should make clear that it will not be possible to create an ‘old’ style array with the ‘new style’ API. (See also the upgrade logic below.)

### All three of `SOMADataFrame`, `SparseNDArray`, `DenseNDArray`

* `write`
  * For new arrays, created with the new shape feature:
    * Core will bounds-check that coordinates provided at `write` time are within the current shape
    * Core will raise `tiledb.cc.TileDBError` to TileDB-SOMA, which will catch and raise `IndexError`, and R-standard behavior on the R side
  * For old arrays created before this feature:
    * Core will not bounds-check that coordinates provided at write time are within the current shape
* Existing `used_shape` accessor
  * TileDB-SOMA will deprecate this over a release cycle.
  * For new arrays: raise `NotImplementedError`
  * For old arrays: return what’s currently returned, with a deprecation warning.
  * Mechanism for determining old vs. new: `array.schema.version` (the core storage version).
* Existing `shape` accessor
  * For new arrays:
    * Have this return the new shape as proposed by core, no longer returning the TileDB domain.
  * For old arrays created before this feature:
    * Return the TileDB domain as now.
* Existing `non_empty_domain` accessor
  * Same behavior for old and new arrays (unaffected by this proposal).
  * Keep this accessor supported, but, with user notes that it’s generally non-useful
  * This should return None (or R equivalent) when there is a schema but no data have been written.
* New `maxshape` accessor
  * Maps the core-level `(lo, hi)` accessor for domain to count-style accessor hi+1. E.g. if the core domain is either `(0,99)` or `(50,99)` then TileDB-SOMA `maxshape` will say 100.
  * Same behavior for old and new arrays.
  * Let users query for what the TileDB domain is, with user notes that it’s the maximum that users can reshape to.
  * Issac suggests: maybe `domain` or `maxshape` (see h5py).
* New `resize` mutator
  * Note: `reshape` means something else in the community (numpy, zarr, h5py), e.g. a 5x20 (total 100 cells) being reinterpreted as 4x25 (still 100 cells). The standard name for changing cell-count is `resize`.
  * For old arrays created before this feature: raise `NotImplementedError`.
  * For new arrays:
    * Will raise `ValueError` if the new shape is smaller on any dim than currently in storage
    * Regardless of whether any data have been written whatsoever
    * Will raise `ValueError` if the new shape exceeds the TileDB domain from create time, which will serve TileDB-SOMA in a role of “max possible shape the user can reshape to”
    * Otherwise, any calls to write from this point will bounds-check writes within this new shape
    * **We don’t expect resize to be multi-writer safe with regard to write ; user notes must be clear on this point**
* New `tiledbsoma_upgrade_shape` method for SparseNDArray and DenseNDArray
  * This will leverage `array.schema.version` to see if an upgrade is needed
  * Leverage core support for storage-version updates
  * This will take a shape argument as in `create`
  * For arrays created with “just-right” size: this will succeed
  * For arrays created with “room-for-growth” / “two billion-ish” size: this will succeed
  * If the user passes a shape which exceeds the current TileDB domain: this will fail
* New `tiledbsoma_upgrade_domain method` for `DataFrame`
  * Same as for `SparseNDArray`/`DenseNDArray` except it will take a domain at the SOMA-API level just as `DataFrame`'s create method

### `tiledbsoma.io`

* The user-facing API has no shape arguments and thus won’t need changing.
* Internally to `tiledbsoma.io`, we’ll still ask the tiledbsoma API for the “big domain” (2 billionish)
* Append mode:
  * Will need a new `resize` method at the `Experiment` level
  * Users will need to:
    * Register as now
    * Call the experiment-level `resize`
      * Could be `exp.resize(...)`, or (better) this could be `tiledbsoma.io.reshape_experiment`
  * In either case: this method will take the new `obs` and `var` counts as inputs:
    * `exp.obs.reshape` to new `obs` count
    * `exp.ms[name].var.reshape` to new `var` count
    * `exp.ms[name].X[name].reshape` to new `obs` count x `var` count
    * `exp.ms[name].obsm[name].reshape` to new `obs` count x same width
    * `exp.ms[name].obsp[name].reshape` to new `obs` count x `obs` count
    * `exp.ms[name].varm[name].reshape` to new `var` count x same width
    * `exp.ms[name].varp[name].reshape` to new `var` count x `var` count
  * Do the individual append-mode writes as now

[r] Port append-mode logic
Port #1278 from Python to R.


Discussion / hints:
#2785 is a quick-and-dirty concept-prover -- its sole function is to flush out any API misunderstandings we might have, in prep for 2.25.0 core release.
#2785 is a quick-and-dirty concept-prover -- its sole function is to flush out any API misunderstandings we might have, in prep for 2.25.0 core release.
#2785 is a quick-and-dirty concept-prover -- its sole function is to flush out any API misunderstandings we might have, in prep for 2.25.0 core release.
This will also need to include #2455

Fix patch:
diff --git a/apis/python/src/tiledbsoma/_common_nd_array.py b/apis/python/src/tiledbsoma/_common_nd_array.py
index 5525220012..28632c5503 100644
--- a/apis/python/src/tiledbsoma/_common_nd_array.py
+++ b/apis/python/src/tiledbsoma/_common_nd_array.py
@@ -95,6 +95,28 @@ def shape(self) -> Tuple[int, ...]:
         """
         return cast(Tuple[int, ...], tuple(self._handle.shape))
 
+    @property
+    def maxshape(self) -> Tuple[int, ...]:
+        """Returns the maximum resizable capacity of each dimension, always a list of length
+        ``ndim``.  This will not necessarily match the bounds of occupied cells within the array.
+        It is the upper limit for ``resize`` on the array.
+
+        Lifecycle:
+            Maturing.
+        """
+        return cast(Tuple[int, ...], tuple(self._handle.maxshape))
+
+    @property
+    def has_upgraded_shape(self) -> bool:
+        """Returns true if the array has the upgraded resizeable shape feature
+        from TileDB-SOMA 1.14: the array was created with this support, or it has
+        had ``.upgrade_shape`` applied to it.
+
+        Lifecycle:
+            Maturing.
+        """
+        return self._handle.has_upgraded_shape
+
     @classmethod
     def _dim_capacity_and_extent(
         cls,
diff --git a/apis/python/src/tiledbsoma/_dataframe.py b/apis/python/src/tiledbsoma/_dataframe.py
index 84e0aa34e0..5f6373cb71 100644
--- a/apis/python/src/tiledbsoma/_dataframe.py
+++ b/apis/python/src/tiledbsoma/_dataframe.py
@@ -317,6 +317,40 @@ def count(self) -> int:
         # if is it in read open mode, then it is a DataFrameWrapper
         return cast(DataFrameWrapper, self._handle).count
 
+    @property
+    def _maybe_soma_joinid_shape(self) -> Optional[int]:
+        """An internal helper method that returns the shape
+        value along the ``soma_joinid`` index column, if the ``DataFrame
+        has one, else ``None``.
+
+
+        Lifecycle:
+            Experimental.
+        """
+        return self._handle.maybe_soma_joinid_shape
+
+    @property
+    def _maybe_soma_joinid_maxshape(self) -> Optional[int]:
+        """An internal helper method that returns the maxshape
+        value along the ``soma_joinid`` index column, if the ``DataFrame
+        has one, else ``None``.
+
+        Lifecycle:
+            Experimental.
+        """
+        return self._handle.maybe_soma_joinid_maxshape
+
+    @property
+    def has_upgraded_domain(self) -> bool:
+        """Returns true if the array has the upgraded resizeable domain feature
+        from TileDB-SOMA 1.14: the array was created with this support, or it has
+        had ``.upgrade_domain`` applied to it.
+
+        Lifecycle:
+            Maturing.
+        """
+        return self._handle.has_upgraded_domain
+
     def __len__(self) -> int:
         """Returns the number of rows in the dataframe. Same as ``df.count``."""
         return self.count
diff --git a/apis/python/src/tiledbsoma/_tdb_handles.py b/apis/python/src/tiledbsoma/_tdb_handles.py
index edc2f4ed76..2c02ec2169 100644
--- a/apis/python/src/tiledbsoma/_tdb_handles.py
+++ b/apis/python/src/tiledbsoma/_tdb_handles.py
@@ -406,7 +406,33 @@ def dim_names(self) -> Tuple[str, ...]:
 
     @property
     def shape(self) -> Tuple[int, ...]:
-        return tuple(self._handle.shape)
+        """Not implemented for DataFrame."""
+        return cast(Tuple[int, ...], tuple(self._handle.shape))
+
+    @property
+    def maxshape(self) -> Tuple[int, ...]:
+        """Not implemented for DataFrame."""
+        return cast(Tuple[int, ...], tuple(self._handle.maxshape))
+
+    @property
+    def maybe_soma_joinid_shape(self) -> Optional[int]:
+        """Only implemented for DataFrame."""
+        raise NotImplementedError
+
+    @property
+    def maybe_soma_joinid_maxshape(self) -> Optional[int]:
+        """Only implemented for DataFrame."""
+        raise NotImplementedError
+
+    @property
+    def has_upgraded_shape(self) -> bool:
+        """Not implemented for DataFrame."""
+        raise NotImplementedError
+
+    @property
+    def has_upgraded_domain(self) -> bool:
+        """Only implemented for DataFrame."""
+        raise NotImplementedError
 
 
 class DataFrameWrapper(SOMAArrayWrapper[clib.SOMADataFrame]):
@@ -422,9 +448,37 @@ def write(self, values: pa.RecordBatch) -> None:
         self._handle.write(values)
 
     @property
-    def shape(self) -> Tuple[int, ...]:
-        # Shape is not implemented for DataFrames
-        raise NotImplementedError
+    def maybe_soma_joinid_shape(self) -> Optional[int]:
+        """Return the shape slot for the soma_joinid dim, if the array has one.
+        This is an important test-point and dev-internal access-point,
+        in particular, for the tiledbsoma-io experiment-level resizer.
+
+        Lifecycle:
+            Maturing.
+        """
+        return cast(Optional[int], self._handle.maybe_soma_joinid_shape)
+
+    @property
+    def maybe_soma_joinid_maxshape(self) -> Optional[int]:
+        """Return the maxshape slot for the soma_joinid dim, if the array has one.
+        This is an important test-point and dev-internal access-point,
+        in particular, for the tiledbsoma-io experiment-level resizer.
+
+        Lifecycle:
+            Maturing.
+        """
+        return cast(Optional[int], self._handle.maybe_soma_joinid_maxshape)
+
+    @property
+    def has_upgraded_domain(self) -> bool:
+        """Returns true if the array has the upgraded resizeable domain feature
+        from TileDB-SOMA 1.14: the array was created with this support, or it has
+        had ``.upgrade_domain`` applied to it.
+
+        Lifecycle:
+            Maturing.
+        """
+        return cast(bool, self._handle.has_upgraded_domain)
 
 
 class DenseNDArrayWrapper(SOMAArrayWrapper[clib.SOMADenseNDArray]):
@@ -432,6 +486,17 @@ class DenseNDArrayWrapper(SOMAArrayWrapper[clib.SOMADenseNDArray]):
 
     _ARRAY_WRAPPED_TYPE = clib.SOMADenseNDArray
 
+    @property
+    def has_upgraded_shape(self) -> bool:
+        """Returns true if the array has the upgraded resizeable shape feature
+        from TileDB-SOMA 1.14: the array was created with this support, or it has
+        had ``.upgrade_shape`` applied to it.
+
+        Lifecycle:
+            Maturing.
+        """
+        return cast(bool, self._handle.has_upgraded_shape)
+
 
 class SparseNDArrayWrapper(SOMAArrayWrapper[clib.SOMASparseNDArray]):
     """Wrapper around a Pybind11 SparseNDArrayWrapper handle."""
@@ -442,6 +507,17 @@ class SparseNDArrayWrapper(SOMAArrayWrapper[clib.SOMASparseNDArray]):
     def nnz(self) -> int:
         return int(self._handle.nnz())
 
+    @property
+    def has_upgraded_shape(self) -> bool:
+        """Returns true if the array has the upgraded resizeable shape feature
+        from TileDB-SOMA 1.14: the array was created with this support, or it has
+        had ``.upgrade_shape`` applied to it.
+
+        Lifecycle:
+            Maturing.
+        """
+        return cast(bool, self._handle.has_upgraded_shape)
+
 
 class _DictMod(enum.Enum):
     """State machine to keep track of modifications to a dictionary.
diff --git a/apis/python/src/tiledbsoma/soma_dataframe.cc b/apis/python/src/tiledbsoma/soma_dataframe.cc
index 30ba59541e..0716a419f5 100644
--- a/apis/python/src/tiledbsoma/soma_dataframe.cc
+++ b/apis/python/src/tiledbsoma/soma_dataframe.cc
@@ -144,9 +144,17 @@ void load_soma_dataframe(py::module& m) {
         .def_static("exists", &SOMADataFrame::exists)
         .def_property_readonly(
             "index_column_names", &SOMADataFrame::index_column_names)
+
         .def_property_readonly(
             "count",
             &SOMADataFrame::count,
-            py::call_guard<py::gil_scoped_release>());
+            py::call_guard<py::gil_scoped_release>())
+        .def_property_readonly(
+            "maybe_soma_joinid_shape", &SOMADataFrame::maybe_soma_joinid_shape)
+        .def_property_readonly(
+            "maybe_soma_joinid_maxshape",
+            &SOMADataFrame::maybe_soma_joinid_maxshape)
+        .def_property_readonly(
+            "has_upgraded_domain", &SOMAArray::has_current_domain);
 }
 }  // namespace libtiledbsomacpp
diff --git a/apis/python/src/tiledbsoma/soma_dense_ndarray.cc b/apis/python/src/tiledbsoma/soma_dense_ndarray.cc
index 0a8d8c3555..bd5a3b42b4 100644
--- a/apis/python/src/tiledbsoma/soma_dense_ndarray.cc
+++ b/apis/python/src/tiledbsoma/soma_dense_ndarray.cc
@@ -124,6 +124,11 @@ void load_soma_dense_ndarray(py::module& m) {
 
         .def_static("exists", &SOMADenseNDArray::exists)
 
-        .def("write", write);
+        .def("write", write)
+
+        .def_property_readonly("shape", &SOMADenseNDArray::shape)
+        .def_property_readonly("maxshape", &SOMADenseNDArray::maxshape)
+        .def_property_readonly(
+            "has_upgraded_shape", &SOMAArray::has_current_domain);
 }
 }  // namespace libtiledbsomacpp
diff --git a/apis/python/src/tiledbsoma/soma_sparse_ndarray.cc b/apis/python/src/tiledbsoma/soma_sparse_ndarray.cc
index 40bd4c8b43..bb45120f4e 100644
--- a/apis/python/src/tiledbsoma/soma_sparse_ndarray.cc
+++ b/apis/python/src/tiledbsoma/soma_sparse_ndarray.cc
@@ -110,6 +110,11 @@ void load_soma_sparse_ndarray(py::module& m) {
             "result_order"_a = ResultOrder::automatic,
             "timestamp"_a = py::none())
 
-        .def_static("exists", &SOMASparseNDArray::exists);
+        .def_static("exists", &SOMASparseNDArray::exists)
+
+        .def_property_readonly("shape", &SOMASparseNDArray::shape)
+        .def_property_readonly("maxshape", &SOMASparseNDArray::maxshape)
+        .def_property_readonly(
+            "has_upgraded_shape", &SOMAArray::has_current_domain);
 }
 }  // namespace libtiledbsomacpp
diff --git a/apis/r/R/RcppExports.R b/apis/r/R/RcppExports.R
index 4f20189acf..8b0adbd287 100644
--- a/apis/r/R/RcppExports.R
+++ b/apis/r/R/RcppExports.R
@@ -126,6 +126,22 @@ shape <- function(uri, config = NULL) {
     .Call(`_tiledbsoma_shape`, uri, config)
 }
 
+maxshape <- function(uri, config = NULL) {
+    .Call(`_tiledbsoma_maxshape`, uri, config)
+}
+
+maybe_soma_joinid_shape <- function(uri, config = NULL) {
+    .Call(`_tiledbsoma_maybe_soma_joinid_shape`, uri, config)
+}
+
+maybe_soma_joinid_maxshape <- function(uri, config = NULL) {
+    .Call(`_tiledbsoma_maybe_soma_joinid_maxshape`, uri, config)
+}
+
+has_current_domain <- function(uri, config = NULL) {
+    .Call(`_tiledbsoma_has_current_domain`, uri, config)
+}
+
 #' Iterator-Style Access to SOMA Array via SOMAArray
 #'
 #' The `sr_*` functions provide low-level access to an instance of the SOMAArray
diff --git a/apis/r/R/SOMADataFrame.R b/apis/r/R/SOMADataFrame.R
index 62877e6a5f..3ac205f9d1 100644
--- a/apis/r/R/SOMADataFrame.R
+++ b/apis/r/R/SOMADataFrame.R
@@ -329,7 +329,29 @@ SOMADataFrame <- R6::R6Class(
     shape = function() stop(errorCondition(
       "'SOMADataFrame$shape()' is not implemented yet",
       class = 'notYetImplementedError'
-    ))
+    )),
+
+    #' @description Retrieve the maxshape; as \code{SOMADataFrames} are shapeless,
+    #' simply raises an error
+    #'
+    #' @return None, instead a \code{\link{.NotYetImplemented}()} error is raised
+    #'
+    maxshape = function() stop(errorCondition(
+      "'SOMADataFrame$maxshape()' is not implemented",
+      class = 'notYetImplementedError'
+    )),
+
+    #' @description Returns TRUE if the array has the upgraded resizeable domain
+    #' feature from TileDB-SOMA 1.14: the array was created with this support,
+    #' or it has had ``upgrade_domain`` applied to it.
+    #' (lifecycle: maturing)
+    #' @return Logical
+    has_upgraded_domain = function() {
+      has_current_domain(
+        self$uri,
+        config=as.character(tiledb::config(self$tiledbsoma_ctx$context()))
+      )
+    }
 
   ),
 
diff --git a/apis/r/R/SOMANDArrayBase.R b/apis/r/R/SOMANDArrayBase.R
index a52fa13235..b2a3adcfb0 100644
--- a/apis/r/R/SOMANDArrayBase.R
+++ b/apis/r/R/SOMANDArrayBase.R
@@ -84,7 +84,20 @@ SOMANDArrayBase <- R6::R6Class(
     set_data_type = function(type) {
       spdl::debug("[SOMANDArrayBase::set_data_type] caching type {}", type$ToString())
       private$.type <- type
+    },
+
+    #' @description Returns TRUE if the array has the upgraded resizeable shape
+    #' feature from TileDB-SOMA 1.14: the array was created with this support,
+    #' or it has had ``upgrade_domain`` applied to it.
+    #' (lifecycle: maturing)
+    #' @return Logical
+    has_upgraded_shape = function() {
+      has_current_domain(
+        self$uri,
+        config=as.character(tiledb::config(self$tiledbsoma_ctx$context()))
+      )
     }
+
   ),
 
   private = list(
diff --git a/apis/r/R/TileDBArray.R b/apis/r/R/TileDBArray.R
index 9ebba1a873..84c6bd358a 100644
--- a/apis/r/R/TileDBArray.R
+++ b/apis/r/R/TileDBArray.R
@@ -158,6 +158,17 @@ TileDBArray <- R6::R6Class(
       ))
     },
 
+    #' @description Retrieve the maxshape, i.e. maximum possible that the
+    # shape can be resized up to.
+    #' (lifecycle: maturing)
+    #' @return A named vector of dimension length (and the same type as the dimension)
+    maxshape = function() {
+      as.integer64(maxshape(
+        self$uri,
+        config=as.character(tiledb::config(self$tiledbsoma_ctx$context()))
+      ))
+    },
+
     #' @description Retrieve the range of indexes for a dimension that were
     #'  explicitly written.  This method is deprecated as of TileDB-SOMA 1.13, and will be
     #' removed in TileDB-SOMA 1.14.
diff --git a/apis/r/src/RcppExports.cpp b/apis/r/src/RcppExports.cpp
index 4021362a23..1ffea59301 100644
--- a/apis/r/src/RcppExports.cpp
+++ b/apis/r/src/RcppExports.cpp
@@ -260,6 +260,54 @@ BEGIN_RCPP
     return rcpp_result_gen;
 END_RCPP
 }
+// maxshape
+Rcpp::NumericVector maxshape(const std::string& uri, Rcpp::Nullable<Rcpp::CharacterVector> config);
+RcppExport SEXP _tiledbsoma_maxshape(SEXP uriSEXP, SEXP configSEXP) {
+BEGIN_RCPP
+    Rcpp::RObject rcpp_result_gen;
+    Rcpp::RNGScope rcpp_rngScope_gen;
+    Rcpp::traits::input_parameter< const std::string& >::type uri(uriSEXP);
+    Rcpp::traits::input_parameter< Rcpp::Nullable<Rcpp::CharacterVector> >::type config(configSEXP);
+    rcpp_result_gen = Rcpp::wrap(maxshape(uri, config));
+    return rcpp_result_gen;
+END_RCPP
+}
+// maybe_soma_joinid_shape
+Rcpp::NumericVector maybe_soma_joinid_shape(const std::string& uri, Rcpp::Nullable<Rcpp::CharacterVector> config);
+RcppExport SEXP _tiledbsoma_maybe_soma_joinid_shape(SEXP uriSEXP, SEXP configSEXP) {
+BEGIN_RCPP
+    Rcpp::RObject rcpp_result_gen;
+    Rcpp::RNGScope rcpp_rngScope_gen;
+    Rcpp::traits::input_parameter< const std::string& >::type uri(uriSEXP);
+    Rcpp::traits::input_parameter< Rcpp::Nullable<Rcpp::CharacterVector> >::type config(configSEXP);
+    rcpp_result_gen = Rcpp::wrap(maybe_soma_joinid_shape(uri, config));
+    return rcpp_result_gen;
+END_RCPP
+}
+// maybe_soma_joinid_maxshape
+Rcpp::NumericVector maybe_soma_joinid_maxshape(const std::string& uri, Rcpp::Nullable<Rcpp::CharacterVector> config);
+RcppExport SEXP _tiledbsoma_maybe_soma_joinid_maxshape(SEXP uriSEXP, SEXP configSEXP) {
+BEGIN_RCPP
+    Rcpp::RObject rcpp_result_gen;
+    Rcpp::RNGScope rcpp_rngScope_gen;
+    Rcpp::traits::input_parameter< const std::string& >::type uri(uriSEXP);
+    Rcpp::traits::input_parameter< Rcpp::Nullable<Rcpp::CharacterVector> >::type config(configSEXP);
+    rcpp_result_gen = Rcpp::wrap(maybe_soma_joinid_maxshape(uri, config));
+    return rcpp_result_gen;
+END_RCPP
+}
+// has_current_domain
+Rcpp::LogicalVector has_current_domain(const std::string& uri, Rcpp::Nullable<Rcpp::CharacterVector> config);
+RcppExport SEXP _tiledbsoma_has_current_domain(SEXP uriSEXP, SEXP configSEXP) {
+BEGIN_RCPP
+    Rcpp::RObject rcpp_result_gen;
+    Rcpp::RNGScope rcpp_rngScope_gen;
+    Rcpp::traits::input_parameter< const std::string& >::type uri(uriSEXP);
+    Rcpp::traits::input_parameter< Rcpp::Nullable<Rcpp::CharacterVector> >::type config(configSEXP);
+    rcpp_result_gen = Rcpp::wrap(has_current_domain(uri, config));
+    return rcpp_result_gen;
+END_RCPP
+}
 // sr_setup
 Rcpp::List sr_setup(const std::string& uri, Rcpp::CharacterVector config, Rcpp::Nullable<Rcpp::CharacterVector> colnames, Rcpp::Nullable<Rcpp::XPtr<tiledb::QueryCondition>> qc, Rcpp::Nullable<Rcpp::List> dim_points, Rcpp::Nullable<Rcpp::List> dim_ranges, std::string batch_size, std::string result_order, Rcpp::Nullable<Rcpp::DatetimeVector> timestamprange, const std::string& loglevel);
 RcppExport SEXP _tiledbsoma_sr_setup(SEXP uriSEXP, SEXP configSEXP, SEXP colnamesSEXP, SEXP qcSEXP, SEXP dim_pointsSEXP, SEXP dim_rangesSEXP, SEXP batch_sizeSEXP, SEXP result_orderSEXP, SEXP timestamprangeSEXP, SEXP loglevelSEXP) {
@@ -449,6 +497,10 @@ static const R_CallMethodDef CallEntries[] = {
     {"_tiledbsoma_check_arrow_schema_tag", (DL_FUNC) &_tiledbsoma_check_arrow_schema_tag, 1},
     {"_tiledbsoma_check_arrow_array_tag", (DL_FUNC) &_tiledbsoma_check_arrow_array_tag, 1},
     {"_tiledbsoma_shape", (DL_FUNC) &_tiledbsoma_shape, 2},
+    {"_tiledbsoma_maxshape", (DL_FUNC) &_tiledbsoma_maxshape, 2},
+    {"_tiledbsoma_maybe_soma_joinid_shape", (DL_FUNC) &_tiledbsoma_maybe_soma_joinid_shape, 2},
+    {"_tiledbsoma_maybe_soma_joinid_maxshape", (DL_FUNC) &_tiledbsoma_maybe_soma_joinid_maxshape, 2},
+    {"_tiledbsoma_has_current_domain", (DL_FUNC) &_tiledbsoma_has_current_domain, 2},
     {"_tiledbsoma_sr_setup", (DL_FUNC) &_tiledbsoma_sr_setup, 10},
     {"_tiledbsoma_sr_complete", (DL_FUNC) &_tiledbsoma_sr_complete, 1},
     {"_tiledbsoma_create_empty_arrow_table", (DL_FUNC) &_tiledbsoma_create_empty_arrow_table, 0},
diff --git a/apis/r/src/rinterface.cpp b/apis/r/src/rinterface.cpp
index d5a919c7cc..9437276625 100644
--- a/apis/r/src/rinterface.cpp
+++ b/apis/r/src/rinterface.cpp
@@ -234,3 +234,47 @@ Rcpp::NumericVector shape(const std::string& uri,
     auto sr = tdbs::SOMAArray::open(OpenMode::read, uri, "unnamed", config_vector_to_map(Rcpp::wrap(config)));
     return Rcpp::toInteger64(sr->shape());
 }
+
+// [[Rcpp::export]]
+Rcpp::NumericVector maxshape(const std::string& uri,
+                          Rcpp::Nullable<Rcpp::CharacterVector> config = R_NilValue) {
+    auto sr = tdbs::SOMAArray::open(OpenMode::read, uri, "unnamed", config_vector_to_map(Rcpp::wrap(config)));
+    return Rcpp::toInteger64(sr->maxshape());
+}
+
+// [[Rcpp::export]]
+Rcpp::NumericVector maybe_soma_joinid_shape(const std::string& uri,
+                          Rcpp::Nullable<Rcpp::CharacterVector> config = R_NilValue) {
+    // Pro-tip:
+    // * Open with mode and uri gives a SOMAArray.
+    // * Open with uri and mode gives a SOMADataFrame.
+    // This was done intentionally to resolve an ambiguous-overload compiler error.
+    auto sr = tdbs::SOMADataFrame::open(uri, OpenMode::read, "unnamed", config_vector_to_map(Rcpp::wrap(config)));
+    auto retval = sr->maybe_soma_joinid_shape();
+    if (retval.has_value()) {
+      return Rcpp::toInteger64(retval.value());
+    } else {
+      // We use this sentinel to facilitate a pure-R return value of NA.
+      return Rcpp::toInteger64(-1);
+    }
+}
+
+// [[Rcpp::export]]
+Rcpp::NumericVector maybe_soma_joinid_maxshape(const std::string& uri,
+                          Rcpp::Nullable<Rcpp::CharacterVector> config = R_NilValue) {
+    auto sr = tdbs::SOMADataFrame::open(uri, OpenMode::read, "unnamed", config_vector_to_map(Rcpp::wrap(config)));
+    auto retval = sr->maybe_soma_joinid_maxshape();
+    if (retval.has_value()) {
+      return Rcpp::toInteger64(retval.value());
+    } else {
+      // We use this sentinel to facilitate a pure-R return value of NA.
+      return Rcpp::toInteger64(-1);
+    }
+}
+
+// [[Rcpp::export]]
+Rcpp::LogicalVector has_current_domain(const std::string& uri,
+                          Rcpp::Nullable<Rcpp::CharacterVector> config = R_NilValue) {
+    auto sr = tdbs::SOMAArray::open(OpenMode::read, uri, "unnamed", config_vector_to_map(Rcpp::wrap(config)));
+    return Rcpp::LogicalVector(sr->has_current_domain());
+}
diff --git a/libtiledbsoma/src/soma/soma_array.h b/libtiledbsoma/src/soma/soma_array.h
index 69d0494c4f..1c2fe61a5d 100644
--- a/libtiledbsoma/src/soma/soma_array.h
+++ b/libtiledbsoma/src/soma/soma_array.h
@@ -824,6 +824,18 @@ class SOMAArray : public SOMAObject {
         return _get_current_domain();
     }
 
+    /**
+     * @brief Returns true if the array has a non-empty current domain, else
+     * false.  Note that at the core level it's "current domain" for all arrays;
+     * at the SOMA-API level it's "upgraded_shape" for SOMASparseNDArray and
+     * SOMADenseNDArray, and "upgraded_domain" for SOMADataFrame; here
+     * we use the core language and defer to Python/R to conform to
+     * SOMA-API syntax.
+     */
+    bool has_current_domain() {
+        return !_get_current_domain().is_empty();
+    }
+
    protected:
     // These two are for use nominally by SOMADataFrame. This could be moved in
     // its entirety to SOMADataFrame, but it would entail moving several
diff --git a/libtiledbsoma/src/soma/soma_dataframe.cc b/libtiledbsoma/src/soma/soma_dataframe.cc
index a65773a81e..2900e152a5 100644
--- a/libtiledbsoma/src/soma/soma_dataframe.cc
+++ b/libtiledbsoma/src/soma/soma_dataframe.cc
@@ -68,6 +68,14 @@ std::unique_ptr<SOMADataFrame> SOMADataFrame::open(
         mode, uri, ctx, column_names, result_order, timestamp);
 }
 
+std::unique_ptr<SOMADataFrame> SOMADataFrame::open(
+    std::string_view uri,
+    OpenMode mode,
+    std::string_view name,
+    std::map<std::string, std::string> platform_config) {
+    return std::make_unique<SOMADataFrame>(mode, uri, name, platform_config);
+}
+
 bool SOMADataFrame::exists(
     std::string_view uri, std::shared_ptr<SOMAContext> ctx) {
     try {
diff --git a/libtiledbsoma/src/soma/soma_dataframe.h b/libtiledbsoma/src/soma/soma_dataframe.h
index 231e578baf..0ba3c53e78 100644
--- a/libtiledbsoma/src/soma/soma_dataframe.h
+++ b/libtiledbsoma/src/soma/soma_dataframe.h
@@ -71,6 +71,13 @@ class SOMADataFrame : public SOMAArray {
     /**
      * @brief Open and return a SOMADataFrame object at the given URI.
      *
+     * Note: the first two arguments uri and mode are reversed from
+     * the SOMAArrayConstructor. This is an intentional decision to
+     * avoid ambiguous-overload compiler errors. Even though
+     * SOMADataFrame extends SOMAArray, callers using open
+     * and wishing to obtain a SOMADataFrame rather than a SOMAArray
+     * are advised to place the uri argument before the mode argument.
+     *
      * @param uri URI to create the SOMADataFrame
      * @param mode read or write
      * @param ctx SOMAContext
@@ -91,6 +98,26 @@ class SOMADataFrame : public SOMAArray {
         ResultOrder result_order = ResultOrder::automatic,
         std::optional<TimestampRange> timestamp = std::nullopt);
 
+    /**
+     * @brief Open and return a SOMADataFrame object at the given URI.
+     *
+     * This is nominally for TileDB-SOMA R use, since while we have
+     * TileDB-SOMA-R and TileDB-R co-existing, it's less desirable
+     * to pass ctx from one copy of core to another, and more
+     * desirable to pass a config map.
+     *
+     * @param uri URI to create the SOMADataFrame
+     * @param mode read or write
+     * @param name Name of the array
+     * @param platform_config Config parameter dictionary
+     * @return std::unique_ptr<SOMADataFrame> SOMADataFrame
+     */
+    static std::unique_ptr<SOMADataFrame> open(
+        std::string_view uri,
+        OpenMode mode,
+        std::string_view name,
+        std::map<std::string, std::string> platform_config);
+
     /**
      * @brief Check if the SOMADataFrame exists at the URI.
      *
@@ -132,6 +159,35 @@ class SOMADataFrame : public SOMAArray {
               timestamp) {
     }
 
+    /**
+     * @brief Construct a new SOMADataFrame object.
+     *
+     * This is nominally for TileDB-SOMA R use, since while we have
+     * TileDB-SOMA-R and TileDB-R co-existing, it's less desirable
+     * to pass ctx from one copy of core to another, and more
+     * desirable to pass a config map.
+     *
+     * @param uri URI to create the SOMADataFrame
+     * @param mode read or write
+     * @param name Name of the array
+     * @param platform_config Config parameter dictionary
+     * @return std::unique_ptr<SOMADataFrame> SOMADataFrame
+     */
+    SOMADataFrame(
+        OpenMode mode,
+        std::string_view uri,
+        std::string_view name,
+        std::map<std::string, std::string> platform_config)
+        : SOMAArray(
+              mode,
+              uri,
+              std::make_shared<SOMAContext>(platform_config),
+              name,
+              {},
+              "auto",
+              ResultOrder::automatic) {
+    }
+
     SOMADataFrame(const SOMAArray& other)
         : SOMAArray(other) {
     }


Test patch:
diff --git a/apis/python/tests/test_dataframe.py b/apis/python/tests/test_dataframe.py
index 615a6ba4cd..edb5c6cfc8 100644
--- a/apis/python/tests/test_dataframe.py
+++ b/apis/python/tests/test_dataframe.py
@@ -90,9 +90,16 @@ def test_dataframe(tmp_path, arrow_schema):
         assert sdf.count == 5
         assert len(sdf) == 5
 
+        # More to come on https://github.com/single-cell-data/TileDB-SOMA/issues/2407
+        assert not sdf.has_upgraded_domain
+
         with pytest.raises(AttributeError):
             assert sdf.shape is None
 
+        # soma_joinid is not a dim here
+        assert sdf._maybe_soma_joinid_shape is None
+        assert sdf._maybe_soma_joinid_maxshape is None
+
         # Read all
         table = sdf.read().concat()
         assert table.num_rows == 5
diff --git a/apis/python/tests/test_shape.py b/apis/python/tests/test_shape.py
new file mode 100644
index 0000000000..42c27f2701
--- /dev/null
+++ b/apis/python/tests/test_shape.py
@@ -0,0 +1,211 @@
+from __future__ import annotations
+
+import pyarrow as pa
+import pytest
+
+import tiledbsoma
+
+from tests._util import maybe_raises
+
+
+@pytest.mark.parametrize(
+    "element_dtype",
+    [
+        pa.float64(),
+        pa.float32(),
+        pa.int64(),
+        pa.uint16(),
+    ],
+)
+@pytest.mark.parametrize(
+    "shape_exc",
+    [
+        # Note: non-None exceptions are coming on https://github.com/single-cell-data/TileDB-SOMA/issues/2407
+        [(100,), None],
+        [(100, 200), None],
+        [(100, 200, 300), None],
+    ],
+)
+def test_sparse_nd_array_basics(
+    tmp_path,
+    element_dtype,
+    shape_exc,
+):
+    uri = tmp_path.as_posix()
+    arg_shape, arg_create_exc = shape_exc
+    ndim = len(arg_shape)
+
+    # Create the array
+    with maybe_raises(arg_create_exc):
+        snda = tiledbsoma.SparseNDArray.create(
+            uri,
+            type=element_dtype,
+            shape=arg_shape,
+        )
+    if arg_create_exc is not None:
+        return
+
+    assert tiledbsoma.SparseNDArray.exists(uri)
+
+    # Test the various accessors
+    with tiledbsoma.SparseNDArray.open(uri) as snda:
+
+        assert snda.shape == arg_shape
+
+        # More to come on https://github.com/single-cell-data/TileDB-SOMA/issues/2407
+        assert not snda.has_upgraded_shape
+
+        # Before current-domain support: shape is maxshape.
+        #
+        # With current-domain support: We expect the maxshape to be set to a big
+        # signed int32. (There are details on the exact value of that number,
+        # involving R compatibility, and leaving room for a single tile
+        # capacity, etc ...  we could check for some magic value but it suffices
+        # to check that it's over 2 billion.)
+        assert snda.shape == snda.maxshape
+        # for e in snda.maxshape:
+        #    assert e > 2_000_000_000
+
+        # No data have been written for this test case
+        assert snda.non_empty_domain() == tuple([(0, 0)] * ndim)
+
+    # soma_dim_0: (0,1)
+    # soma_dim_1: (2,3)
+    # soma_dim_2: (4,5)
+    coords = []
+    dim_names = []
+    for i in range(ndim):
+        dim_names.append(f"soma_dim_{i}")
+        coords.append((2 * i, 2 * i + 1))
+    coords = tuple(coords)
+
+    # Write some data
+    with tiledbsoma.SparseNDArray.open(uri, "w") as snda:
+        dikt = {"soma_data": [4, 5]}
+        for i in range(ndim):
+            dikt[dim_names[i]] = coords[i]
+        table = pa.Table.from_pydict(dikt)
+        snda.write(table)
+
+    # Test the various accessors
+    with tiledbsoma.SparseNDArray.open(uri) as snda:
+        assert snda.shape == arg_shape
+        # This will change with current-domain support
+        assert snda.shape == snda.maxshape
+        # for e in snda.maxshape:
+        #    assert e > 2_000_000_000
+        assert snda.non_empty_domain() == coords
+
+    # Test reads out of bounds
+    with tiledbsoma.SparseNDArray.open(uri) as snda:
+        # TODO: make sure this doesn't come through as RuntimeError
+        # https://github.com/single-cell-data/TileDB-SOMA/issues/2407
+        with pytest.raises((RuntimeError, ValueError)):
+            coords = tuple(arg_shape[i] + 10 for i in range(ndim))
+            snda.read(coords).tables().concat()
+
+    # Test writes out of bounds
+    with tiledbsoma.SparseNDArray.open(uri, "w") as snda:
+        with pytest.raises(tiledbsoma.SOMAError):
+            dikt = {"soma_data": [30]}
+            dikt = {name: [shape + 20] for name, shape in zip(dim_names, arg_shape)}
+            table = pa.Table.from_pydict(dikt)
+            snda.write(table)
+
+    with tiledbsoma.SparseNDArray.open(uri) as snda:
+        assert snda.shape == arg_shape
+        assert snda.shape == snda.maxshape
+
+
+## Pending 2.27 timeframe for dense support for current domain, including resize
+## TODO: mark these with a linked GitHub tracking issue
+def test_dense_nd_array_basics(tmp_path):
+    uri = tmp_path.as_posix()
+    shape = (100, 200)
+    tiledbsoma.DenseNDArray.create(uri, type=pa.float64(), shape=shape)
+
+    with tiledbsoma.DenseNDArray.open(uri) as dnda:
+        assert dnda.shape == (100, 200)
+        assert dnda.maxshape == (100, 200)
+
+        assert dnda.non_empty_domain() == ((0, 0), (0, 0))
+
+
+@pytest.mark.parametrize(
+    "soma_joinid_domain",
+    [
+        # TODO: https://github.com/single-cell-data/TileDB-SOMA/issues/2407
+        # None,
+        (0, 1),
+        (0, 3),
+        (0, 100),
+    ],
+)
+@pytest.mark.parametrize(
+    "index_column_names",
+    [
+        ["soma_joinid"],
+        ["soma_joinid", "myint"],
+        ["soma_joinid", "mystring"],
+        ["mystring", "myint"],
+    ],
+)
+def test_dataframe_basics(tmp_path, soma_joinid_domain, index_column_names):
+    uri = tmp_path.as_posix()
+
+    schema = pa.schema(
+        [
+            ("soma_joinid", pa.int64()),
+            ("mystring", pa.string()),
+            ("myint", pa.int16()),
+            ("myfloat", pa.float32()),
+        ]
+    )
+
+    data = pa.Table.from_pydict(
+        {
+            "soma_joinid": [0, 1, 2, 3],
+            "mystring": ["a", "b", "a", "b"],
+            "myint": [20, 30, 40, 50],
+            "myfloat": [1.0, 2.5, 4.0, 5.5],
+        }
+    )
+
+    domain_slots = {
+        "soma_joinid": soma_joinid_domain,
+        "mystring": None,
+        "myint": (-1000, 1000),
+        "myfloat": (-999.5, 999.5),
+    }
+
+    domain = tuple([domain_slots[name] for name in index_column_names])
+
+    soma_joinid_coords = data["soma_joinid"]
+    oob_write = any(
+        e.as_py() < soma_joinid_domain[0] or e.as_py() > soma_joinid_domain[1]
+        for e in soma_joinid_coords
+    )
+    oob_write = oob_write and "soma_joinid" in index_column_names
+
+    with tiledbsoma.DataFrame.create(
+        uri,
+        schema=schema,
+        index_column_names=index_column_names,
+        domain=domain,
+    ) as sdf:
+        if oob_write:
+            with pytest.raises(tiledbsoma.SOMAError):
+                sdf.write(data)
+        else:
+            sdf.write(data)
+
+    with tiledbsoma.DataFrame.open(uri) as sdf:
+        has_sjid_dim = "soma_joinid" in index_column_names
+        if has_sjid_dim:
+            assert sdf._maybe_soma_joinid_shape == 1 + soma_joinid_domain[1]
+            assert sdf._maybe_soma_joinid_maxshape == 1 + soma_joinid_domain[1]
+        else:
+            assert sdf._maybe_soma_joinid_shape is None
+            assert sdf._maybe_soma_joinid_maxshape is None
+
+        assert len(sdf.non_empty_domain()) == len(index_column_names)
diff --git a/apis/python/tests/test_sparse_nd_array.py b/apis/python/tests/test_sparse_nd_array.py
index 1ccc597020..e043e925ba 100644
--- a/apis/python/tests/test_sparse_nd_array.py
+++ b/apis/python/tests/test_sparse_nd_array.py
@@ -51,6 +51,14 @@ def test_sparse_nd_array_create_ok(
     assert a.uri == tmp_path.as_posix()
     assert a.ndim == len(shape)
     assert a.shape == tuple(shape)
+
+    # TODO: more testing with current-domain feature integrated
+    # https://github.com/single-cell-data/TileDB-SOMA/issues/2407
+    assert isinstance(a.maxshape, tuple)
+    assert len(a.maxshape) == len(a.shape)
+    for ms, s in zip(a.maxshape, a.shape):
+        assert ms >= s
+
     assert a.is_sparse is True
 
     assert a.schema is not None
diff --git a/apis/r/tests/testthat/test-SOMASparseNDArray.R b/apis/r/tests/testthat/test-SOMASparseNDArray.R
index a2d2881e74..9474f43ac6 100644
--- a/apis/r/tests/testthat/test-SOMASparseNDArray.R
+++ b/apis/r/tests/testthat/test-SOMASparseNDArray.R
@@ -59,6 +59,17 @@ test_that("SOMASparseNDArray creation", {
   ## shape
   expect_equal(ndarray$shape(), as.integer64(c(10, 10)))
 
+  ## maxshape
+  # TODO: more testing with current-domain feature integrated
+  # https://github.com/single-cell-data/TileDB-SOMA/issues/2407
+  expect_false(ndarray$has_upgraded_shape())
+  shape <- ndarray$shape()
+  maxshape <- ndarray$maxshape()
+  expect_equal(length(shape), length(maxshape))
+  for (i in 1:length(shape)) {
+    expect(maxshape[[i]] >= shape[[i]])
+  }
+
   ## ndim
   expect_equal(ndarray$ndim(), 2L)
 
diff --git a/apis/r/tests/testthat/test-shape.r b/apis/r/tests/testthat/test-shape.r
new file mode 100644
index 0000000000..1dc3626577
--- /dev/null
+++ b/apis/r/tests/testthat/test-shape.r
@@ -0,0 +1,100 @@
+test_that("SOMADataFrame shape", {
+  #uri <- withr::local_tempdir("soma-dataframe-shape")
+  uri <- "/tmp/fooze"
+  asch <- create_arrow_schema()
+
+  index_column_name_choices = list(
+    "soma_joinid",
+    c("soma_joinid", "foo"),
+    c("soma_joinid", "baz"),
+    c("baz", "foo")
+  )
+
+  for (index_column_names in index_column_name_choices) {
+    has_soma_joinid_dim <- "soma_joinid" %in% index_column_names
+
+    if (dir.exists(uri)) unlink(uri, recursive=TRUE)
+
+    sdf <- SOMADataFrameCreate(uri, asch, index_column_names = index_column_names)
+    expect_true(sdf$exists())
+    expect_true(dir.exists(uri))
+
+    tbl0 <- arrow::arrow_table(foo = 1L:4L,
+                               soma_joinid = 1L:4L,
+                               bar = 1.1:4.1,
+                               baz = c("apple", "ball", "cat", "dog"),
+                               schema = asch)
+
+    sdf$write(tbl0)
+    sdf$close()
+
+    sdf <- SOMADataFrameOpen(uri)
+
+    expect_false(sdf$has_upgraded_domain())
+    expect_error(sdf$shape(), class = "notYetImplementedError")
+    expect_error(sdf$maxshape(), class = "notYetImplementedError")
+
+    # Not implemented this way per
+    # https://github.com/single-cell-data/TileDB-SOMA/pull/2953#discussion_r1746125089
+    # sjid_shape <- sdf$.maybe_soma_joinid_shape()
+    # sjid_maxshape <- sdf$.maybe_soma_joinid_maxshape()
+    sjid_shape <- maybe_soma_joinid_shape(sdf$uri, config = as.character(tiledb::config(sdf$tiledbsoma_ctx$context())))
+    sjid_maxshape <- maybe_soma_joinid_maxshape(sdf$uri, config = as.character(tiledb::config(sdf$tiledbsoma_ctx$context())))
+
+    if (has_soma_joinid_dim) {
+      # More testing to come on
+      # https://github.com/single-cell-data/TileDB-SOMA/issues/2407
+      expect_false(sjid_shape == -1)
+      expect_false(sjid_maxshape == -1)
+    } else {
+      expect_true(sjid_shape == -1)
+      expect_true(sjid_maxshape == -1)
+    }
+
+    sdf$close()
+
+    rm(sdf, tbl0)
+
+    gc()
+  }
+})
+
+test_that("SOMASparseNDArray shape", {
+  uri <- withr::local_tempdir("soma-sparse-ndarray-shape")
+  asch <- create_arrow_schema()
+
+  element_type_choices = list(arrow::float32(), arrow::int16())
+  shape_choices = list(
+    c(100),
+    c(100,200),
+    c(100,200,300)
+  )
+  for (element_type in element_type_choices) {
+    for (shape in shape_choices) {
+
+      if (dir.exists(uri)) unlink(uri, recursive=TRUE)
+      ndarray <- SOMASparseNDArrayCreate(uri, element_type, shape = shape)
+
+      expect_equal(ndarray$ndim(), length(shape))
+
+      expect_equal(ndarray$shape(), as.integer64(shape))
+
+      # More generally after current-domain support:
+      readback_shape <- ndarray$shape()
+      readback_maxshape <- ndarray$maxshape()
+      expect_equal(length(readback_shape), length(readback_maxshape))
+      for (i in 1:length(shape)) {
+        s <- as.integer(readback_shape[[i]])
+        ms <- as.integer(readback_maxshape[[i]])
+        expect_true(s <= ms)
+      }
+
+      # Resize tests upcoming on
+      # https://github.com/single-cell-data/TileDB-SOMA/issues/2407
+      ndarray$close()
+
+      rm(ndarray)
+      gc()
+    }
+  }
+})
diff --git a/libtiledbsoma/test/unit_soma_collection.cc b/libtiledbsoma/test/unit_soma_collection.cc
index 903c9710f3..42309e1d7b 100644
--- a/libtiledbsoma/test/unit_soma_collection.cc
+++ b/libtiledbsoma/test/unit_soma_collection.cc
@@ -119,6 +119,7 @@ TEST_CASE("SOMACollection: add SOMADenseNDArray") {
     SOMACollection::create(base_uri, ctx, ts);
     // TODO: add support for current domain in dense arrays once we have that
     // support from core
+    // https://github.com/single-cell-data/TileDB-SOMA/issues/2955
     std::vector<helper::DimInfo> dim_infos(
         {{.name = dim_name,
           .tiledb_datatype = tiledb_datatype,
diff --git a/libtiledbsoma/test/unit_soma_sparse_ndarray.cc b/libtiledbsoma/test/unit_soma_sparse_ndarray.cc
index b1eaa52a93..65e1a94005 100644
--- a/libtiledbsoma/test/unit_soma_sparse_ndarray.cc
+++ b/libtiledbsoma/test/unit_soma_sparse_ndarray.cc
@@ -135,9 +135,11 @@ TEST_CASE("SOMASparseNDArray: basic", "[SOMASparseNDArray]") {
             soma_sparse = SOMASparseNDArray::open(uri, OpenMode::write, ctx);
             // Without current-domain support: this should throw since
             // one cannot resize what has not been sized.
+            REQUIRE(!soma_sparse->has_current_domain());
             REQUIRE_THROWS(soma_sparse->resize(new_shape));
             // Now set the shape
             soma_sparse->upgrade_shape(new_shape);
+            REQUIRE(soma_sparse->has_current_domain());
             // Should not fail since we're setting it to what it already is.
             soma_sparse->resize(new_shape);
             soma_sparse->close();