Repository: single-cell-data/TileDB-SOMA (language: python)

Issue:
[python/r/c++] Revisit `shape` for component arrays
## PRs

**Merged PRs:**

<details>

* #2909 `kerl/schevo-timestamp-methodize`
* #2913 `kerl/name-neaten`
* #2908 `kerl/ut-soma-exc-simplify`
* #2910 `kerl/test-common-parameterize`
* #2918 `kerl/cpp-test-deadstrip`
* #2919 `kerl/minor-unit-test-helper-mod`
* #2936 `kerl/cpp-ut-helper-neaten`
* #2938 `kerl/more-cur-dom-parameterize`
* #2915 `kerl/cpp-strict-int64-shape`
* #2911 `kerl/arrow-util-current-domain-optional`
* #2939 `kerl/step-two-temp`
* #2947 `kerl/cpp-ndarray-resize-testing`
* #2945 `kerl/dataframe-test-fixture`
* #2944 `kerl/cpp-variant-indexed-dataframes`
* #2916 `kerl/sdf-shape`
* #2917 `kerl/cpp-resizes`
* #2948 `kerl/upgrade-shape-int64`
* #2951 `kerl/sdf-test-accessors`
* #2953 `kerl/py-r-accessor-plumbing`
* #2957 `kerl/sdf-domain-accessors`
* #2960 `kerl/dense-link`
* #2963 `kerl/nightly-fix`
* #2970 `kerl/dense-writeable-after-create`
* #2968 `kerl/minor-trim`
* #2969 `kerl/more-py-domain-name-neaten`
* #2972 `kerl/libtiledbsoma-env-logging-level`
* #2962 `kerl/py-r-creation-paths`
* #2950 `kerl/py-r-test-2`
* #2994 `kerl/nanoarrow-helpers`
* #3011 `kerl/polydom3`
* #3017 `kerl/polydom5`
* #3018 `kerl/polydom6`
* #2990 `kerl/variant-nnz-bug`
* #3019 `kerl/index-swap`
* #3020 `kerl/ut-max-shape`
* #3012 `kerl/polydom4`
* #3025 `kerl/fix-3020-merge`
* #3026 `kerl/one-more-rename`
* #3028 `kerl/ff-not`
* #3029 `kerl/ut-vg`
* #3030 `kerl/table-utils-memory`
* #3067 `kerl/improve-sdf-test-field-names`
* #3068 `kerl/ut-generate`
* #3069 `kerl/cpp-sdf-domain-at-create`
* #3027 `kerl/hll-domainish`
* #3088 `kerl/max-domain-int64`
* #3090 `kerl/maybe-resize-soma-joinid-cpp-tweak`
* #3032 `kerl/sdf-domain-at-create` -- fixes #2967
* #3091 `kerl/maybe-resize-soma-joinid-py-r`
* #3095 `kerl/cpp-exp-resize-prep`
* #3089 `kerl/r-dataframe-shapeable`
* https://github.com/single-cell-data/SOMA/pull/233
* #3125 `kerl/cpp-ut-name-shortens`
* #3127 `kerl/helper-rename`
* #3130 `kerl/cpp-can-resizers-names`
* #3132 `kerl/cpp-dataframe-sizing-helpers`
* #3139 `kerl/cpp-dataframe-upgrade-test`
* #3140 `kerl/py-resizer-connects`
* #3151 `kerl/py-can-upgrade-shape`
* #3152 `kerl/registration-shape-acceessors`
* #3156 `kerl/py-exp-shaping`
* #3157 `kerl/py-exp-shaping2`
* #3148 `kerl/py-exp-resize`
* #3191 `kerl/py-domain-at-create-ut-1`
* #3190 `kerl/py-domain-at-create-ut-2`
* #3192 `kerl/py-domain-at-create-ut-3`
* #3193 `kerl/py-domain-at-create-ut-4`
* #3194 `kerl/py-domain-at-create-ut-5`
* #3203 `kerl/min-size-2`
* #3208 `kerl/r-min-sizing`
* #3211 `kerl/cpp-ugr-dom`
* #3232 `kerl/ff-interop`
* #3230 `kerl/ffon`
* #3234 `kerl/docstring-prune`
* #3236 `kerl/prefixing`
* #3241 `kerl/fix-bad-merge`
* #3235 `kerl/py-r-ugr-dom`
* #3237 `kerl/py-r-ugr-dom-2`
* #3238 `kerl/py-r-ugr-dom-3`
* #3253 `kerl/set-coords-rename`
* #3261 `kerl/pybind11-nda-sizing`
* #3265 `kerl/dense-227-a`
* #3263 `kerl/dense-range-trim`
* #3268 `kerl/dim-explosion`
* #3269 `kerl/python-227-dense-ned-read`
* #3270 `kerl/r-227-dense-fixes`
* #3280 `kerl/r-dense-227-more`
* #3286 `kerl/more-fn4m`
* #3283 `kerl/readthedocs-pre-1.15`
* #3288 `kerl/dense-ugrsh`
* #3289 `kerl/notebook-shape-upgrade`
* #3285 `kerl/new-shape-doc-updates`
* #3295 `kerl/notebook-data-refresh`
* #3290 `kerl/notebook-new-shape-refresh`
* #3301 `kerl/ffena`
* #3303 `kerl/r-data-refresh`
* #3300 `kerl/sdf-sjid-lower-zero`
* #3296 `kerl/dense-example-data-refresh`
* #3294 `kerl/new-shape-notebook-and-vignette`
* #3308 `kerl/upgrade-experiment-resources`
* #3309 `kerl/fix-notebook-merge`
* #3321 `kerl/more-use-shape`
* #3358 `kerl/revert-3300`
* #3368 `kerl/227a`
* #3369 `kerl/ucd1`
* #3370 `kerl/ucd2`
* #3371 `kerl/ucd3`
* #3372 `kerl/ucd4`
* #3396 `kerl/domain-at-create-docstrings`
* #3302 `kerl/new-shape-vignette`
* https://github.com/single-cell-data/SOMA/pull/250
* #3399 `kerl/new-shape-more-docstrings`
* #3400 `kerl/check-only-r`
* https://github.com/TileDB-Inc/somacore-feedstock/pull/27

</details>

**Closed/abandoned PRs:**

<details>

* #2785 -- This was only dogfooding for the core 2.25 release -- not to be merged
* #2952 `kerl/feature-flag-temp` -- folded into 2962
* #2995 `kerl/polydom`
* #2964 `kerl/tiledbsoma-io-test`
* #3189 `kerl/min-size`
* #3220 `kerl/cpp-ugr-dom-2`
* #3244 `kerl/dense-227-fixes`
* #3276 `kerl/dataframe-shape`

</details>

**Issues which are related but non-blocking:**

* https://github.com/single-cell-data/SOMA/pull/216
* https://github.com/TileDB-Inc/TileDB/pull/5303
* #2966
* #3081
* Note: R append mode does not exist yet -- see #1630 -- so an experiment-level resizer is not a priority in R
* #3271 
* #3272
* #3273

**See also:** [[sc-51048]](https://app.shortcut.com/tiledb-inc/story/51048).

## Problem to be solved

Users want to know the `shape` of an array, in the SciPy sense:

* Reads and writes are bounds-checked against the shape
* This retains its value regardless of which values of a sparse array are or are not actually occupied
* Users can `resize`.
  * Some users need the ability to grow their datasets later, using either `tiledbsoma.io`'s append mode, or subsequent writes using the `tiledbsoma` API.
  * Note that the cellxgene census doesn't need this: eact week's published census has fixed shape, and any updates will happen in new storage, on a new week.

Using TileDB-SOMA up until the present:

* The TIleDB `domain` is immutable after array creation
  * This does bounds-checking for reads and writes, which is good
  * To leverage this to function as a `shape`, users would need to set the `domain` at array-creation time. However, users lose the ability to grow their datasets later.
* There is a `non_empty_domain` accessor
  * This only indicates min/max coordinates _at which data exists_. Consider an `X` array for 100 cells and 200 genes. If non-zero expression counts exist only for cell join IDs 2-17, then the `non_empty_domain` will indicate `(2,17)` along `soma_dim_0`.
  * Consider an `obms["X_pca"]` within the same experiment. This may be 100 cells by 50 PCA components: we need a placd to store the number 50.
  * Therefore users cannot leverage this to function as a `shape` accessor.
* We have offered a `used_shape` accessor since TileDB-SOMA 1.5.
  * This functions as a `shape` accessor, in the SciPy sense, but it is not multi-writer safe.

New feature for TileDB-SOMA 1.15:

* Arrays will have a  `shape`
* Reads and writes are bounds-checked against the shape
* This retains its value regardless of which values of a sparse array are or are not actually occupied
* Users can `resize`
* The `used_shape` accessor will be deprecated in TileDB-SOMA 1.13, and slated for removal in TileDB-SOMA 1.14.

Compatiblity:

This will now require users to do an explicit `resize` before appending/growing TileDB-SOMA Experiments. Guidance in the form of example notebooks will be provided.

##  Tracking

See also: [[sc-41074]](https://app.shortcut.com/tiledb-inc/story/41074) and [[sc-51048]](https://app.shortcut.com/tiledb-inc/story/51048).

## Scheduling

Support arrives in TileDB Core 2.25. Deprecations for TileDB-SOMA will be released with 1.13. Full support within TileDB-SOMA will be release in 1.14.

## Details

SOMA API mods as we've discussed in a Google doc are as follows.

### `SOMADataFrame`

* `create`: Retain the `domain` argument
  * Issue:
    * Core has a `(lo, hi)` tuple per dim, e.g. `(0,99)` or `(10,19)`
    * SOMA has count per dim, with 0 implicit: e.g. 100 or 20
    * For `SparseNDArray` and `DenseNDArray` core can have `(lo, hi)` and SOMA can have `count`
    * For `DataFrame` there can be multiple dims --- default is a single `soma_joinid`
    * That could be treated either in `(lo, hi)` fashion or `count` fashion
    * However additional dims (e.g. `cell_type`) can be on any type, including strings, floats, etc. where there is no implicit lo=0
    * Therefore we need to keep the current SOMA API wherein `DataFrame` takes a `domain` argument (in `(lo, hi)` fashion) and not a `shape` argument (in `count` fashion)

### `SparseNDArray and DenseNDArray`

* `create`
  * Have an optional shape argument which is of type `Tuple[Int,...]` where each element is the cell count of the corresponding dimension
    * If unsupplied, or if supplied but None in any slot: use the minimum 0 in each slot – nothing larger makes sense since we will not support downsize
  * User guidance should make clear that it will not be possible to create an ‘old’ style array with the ‘new style’ API. (See also the upgrade logic below.)

### All three of `SOMADataFrame`, `SparseNDArray`, `DenseNDArray`

* `write`
  * For new arrays, created with the new shape feature:
    * Core will bounds-check that coordinates provided at `write` time are within the current shape
    * Core will raise `tiledb.cc.TileDBError` to TileDB-SOMA, which will catch and raise `IndexError`, and R-standard behavior on the R side
  * For old arrays created before this feature:
    * Core will not bounds-check that coordinates provided at write time are within the current shape
* Existing `used_shape` accessor
  * TileDB-SOMA will deprecate this over a release cycle.
  * For new arrays: raise `NotImplementedError`
  * For old arrays: return what’s currently returned, with a deprecation warning.
  * Mechanism for determining old vs. new: `array.schema.version` (the core storage version).
* Existing `shape` accessor
  * For new arrays:
    * Have this return the new shape as proposed by core, no longer returning the TileDB domain.
  * For old arrays created before this feature:
    * Return the TileDB domain as now.
* Existing `non_empty_domain` accessor
  * Same behavior for old and new arrays (unaffected by this proposal).
  * Keep this accessor supported, but, with user notes that it’s generally non-useful
  * This should return None (or R equivalent) when there is a schema but no data have been written.
* New `maxshape` accessor
  * Maps the core-level `(lo, hi)` accessor for domain to count-style accessor hi+1. E.g. if the core domain is either `(0,99)` or `(50,99)` then TileDB-SOMA `maxshape` will say 100.
  * Same behavior for old and new arrays.
  * Let users query for what the TileDB domain is, with user notes that it’s the maximum that users can reshape to.
  * Issac suggests: maybe `domain` or `maxshape` (see h5py).
* New `resize` mutator
  * Note: `reshape` means something else in the community (numpy, zarr, h5py), e.g. a 5x20 (total 100 cells) being reinterpreted as 4x25 (still 100 cells). The standard name for changing cell-count is `resize`.
  * For old arrays created before this feature: raise `NotImplementedError`.
  * For new arrays:
    * Will raise `ValueError` if the new shape is smaller on any dim than currently in storage
    * Regardless of whether any data have been written whatsoever
    * Will raise `ValueError` if the new shape exceeds the TileDB domain from create time, which will serve TileDB-SOMA in a role of “max possible shape the user can reshape to”
    * Otherwise, any calls to write from this point will bounds-check writes within this new shape
    * **We don’t expect resize to be multi-writer safe with regard to write ; user notes must be clear on this point**
* New `tiledbsoma_upgrade_shape` method for SparseNDArray and DenseNDArray
  * This will leverage `array.schema.version` to see if an upgrade is needed
  * Leverage core support for storage-version updates
  * This will take a shape argument as in `create`
  * For arrays created with “just-right” size: this will succeed
  * For arrays created with “room-for-growth” / “two billion-ish” size: this will succeed
  * If the user passes a shape which exceeds the current TileDB domain: this will fail
* New `tiledbsoma_upgrade_domain method` for `DataFrame`
  * Same as for `SparseNDArray`/`DenseNDArray` except it will take a domain at the SOMA-API level just as `DataFrame`'s create method

### `tiledbsoma.io`

* The user-facing API has no shape arguments and thus won’t need changing.
* Internally to `tiledbsoma.io`, we’ll still ask the tiledbsoma API for the “big domain” (2 billionish)
* Append mode:
  * Will need a new `resize` method at the `Experiment` level
  * Users will need to:
    * Register as now
    * Call the experiment-level `resize`
      * Could be `exp.resize(...)`, or (better) this could be `tiledbsoma.io.reshape_experiment`
  * In either case: this method will take the new `obs` and `var` counts as inputs:
    * `exp.obs.reshape` to new `obs` count
    * `exp.ms[name].var.reshape` to new `var` count
    * `exp.ms[name].X[name].reshape` to new `obs` count x `var` count
    * `exp.ms[name].obsm[name].reshape` to new `obs` count x same width
    * `exp.ms[name].obsp[name].reshape` to new `obs` count x `obs` count
    * `exp.ms[name].varm[name].reshape` to new `var` count x same width
    * `exp.ms[name].varp[name].reshape` to new `var` count x `var` count
  * Do the individual append-mode writes as now

[python/r/c++] Revisit `shape` for component arrays
## PRs

**Merged PRs:**

<details>

* #2909 `kerl/schevo-timestamp-methodize`
* #2913 `kerl/name-neaten`
* #2908 `kerl/ut-soma-exc-simplify`
* #2910 `kerl/test-common-parameterize`
* #2918 `kerl/cpp-test-deadstrip`
* #2919 `kerl/minor-unit-test-helper-mod`
* #2936 `kerl/cpp-ut-helper-neaten`
* #2938 `kerl/more-cur-dom-parameterize`
* #2915 `kerl/cpp-strict-int64-shape`
* #2911 `kerl/arrow-util-current-domain-optional`
* #2939 `kerl/step-two-temp`
* #2947 `kerl/cpp-ndarray-resize-testing`
* #2945 `kerl/dataframe-test-fixture`
* #2944 `kerl/cpp-variant-indexed-dataframes`
* #2916 `kerl/sdf-shape`
* #2917 `kerl/cpp-resizes`
* #2948 `kerl/upgrade-shape-int64`
* #2951 `kerl/sdf-test-accessors`
* #2953 `kerl/py-r-accessor-plumbing`
* #2957 `kerl/sdf-domain-accessors`
* #2960 `kerl/dense-link`
* #2963 `kerl/nightly-fix`
* #2970 `kerl/dense-writeable-after-create`
* #2968 `kerl/minor-trim`
* #2969 `kerl/more-py-domain-name-neaten`
* #2972 `kerl/libtiledbsoma-env-logging-level`
* #2962 `kerl/py-r-creation-paths`
* #2950 `kerl/py-r-test-2`
* #2994 `kerl/nanoarrow-helpers`
* #3011 `kerl/polydom3`
* #3017 `kerl/polydom5`
* #3018 `kerl/polydom6`
* #2990 `kerl/variant-nnz-bug`
* #3019 `kerl/index-swap`
* #3020 `kerl/ut-max-shape`
* #3012 `kerl/polydom4`
* #3025 `kerl/fix-3020-merge`
* #3026 `kerl/one-more-rename`
* #3028 `kerl/ff-not`
* #3029 `kerl/ut-vg`
* #3030 `kerl/table-utils-memory`
* #3067 `kerl/improve-sdf-test-field-names`
* #3068 `kerl/ut-generate`
* #3069 `kerl/cpp-sdf-domain-at-create`
* #3027 `kerl/hll-domainish`
* #3088 `kerl/max-domain-int64`
* #3090 `kerl/maybe-resize-soma-joinid-cpp-tweak`
* #3032 `kerl/sdf-domain-at-create` -- fixes #2967
* #3091 `kerl/maybe-resize-soma-joinid-py-r`
* #3095 `kerl/cpp-exp-resize-prep`
* #3089 `kerl/r-dataframe-shapeable`
* https://github.com/single-cell-data/SOMA/pull/233
* #3125 `kerl/cpp-ut-name-shortens`
* #3127 `kerl/helper-rename`
* #3130 `kerl/cpp-can-resizers-names`
* #3132 `kerl/cpp-dataframe-sizing-helpers`
* #3139 `kerl/cpp-dataframe-upgrade-test`
* #3140 `kerl/py-resizer-connects`
* #3151 `kerl/py-can-upgrade-shape`
* #3152 `kerl/registration-shape-acceessors`
* #3156 `kerl/py-exp-shaping`
* #3157 `kerl/py-exp-shaping2`
* #3148 `kerl/py-exp-resize`
* #3191 `kerl/py-domain-at-create-ut-1`
* #3190 `kerl/py-domain-at-create-ut-2`
* #3192 `kerl/py-domain-at-create-ut-3`
* #3193 `kerl/py-domain-at-create-ut-4`
* #3194 `kerl/py-domain-at-create-ut-5`
* #3203 `kerl/min-size-2`
* #3208 `kerl/r-min-sizing`
* #3211 `kerl/cpp-ugr-dom`
* #3232 `kerl/ff-interop`
* #3230 `kerl/ffon`
* #3234 `kerl/docstring-prune`
* #3236 `kerl/prefixing`
* #3241 `kerl/fix-bad-merge`
* #3235 `kerl/py-r-ugr-dom`
* #3237 `kerl/py-r-ugr-dom-2`
* #3238 `kerl/py-r-ugr-dom-3`
* #3253 `kerl/set-coords-rename`
* #3261 `kerl/pybind11-nda-sizing`
* #3265 `kerl/dense-227-a`
* #3263 `kerl/dense-range-trim`
* #3268 `kerl/dim-explosion`
* #3269 `kerl/python-227-dense-ned-read`
* #3270 `kerl/r-227-dense-fixes`
* #3280 `kerl/r-dense-227-more`
* #3286 `kerl/more-fn4m`
* #3283 `kerl/readthedocs-pre-1.15`
* #3288 `kerl/dense-ugrsh`
* #3289 `kerl/notebook-shape-upgrade`
* #3285 `kerl/new-shape-doc-updates`
* #3295 `kerl/notebook-data-refresh`
* #3290 `kerl/notebook-new-shape-refresh`
* #3301 `kerl/ffena`
* #3303 `kerl/r-data-refresh`
* #3300 `kerl/sdf-sjid-lower-zero`
* #3296 `kerl/dense-example-data-refresh`
* #3294 `kerl/new-shape-notebook-and-vignette`
* #3308 `kerl/upgrade-experiment-resources`
* #3309 `kerl/fix-notebook-merge`
* #3321 `kerl/more-use-shape`
* #3358 `kerl/revert-3300`
* #3368 `kerl/227a`
* #3369 `kerl/ucd1`
* #3370 `kerl/ucd2`
* #3371 `kerl/ucd3`
* #3372 `kerl/ucd4`
* #3396 `kerl/domain-at-create-docstrings`
* #3302 `kerl/new-shape-vignette`
* https://github.com/single-cell-data/SOMA/pull/250
* #3399 `kerl/new-shape-more-docstrings`
* #3400 `kerl/check-only-r`
* https://github.com/TileDB-Inc/somacore-feedstock/pull/27

</details>

**Closed/abandoned PRs:**

<details>

* #2785 -- This was only dogfooding for the core 2.25 release -- not to be merged
* #2952 `kerl/feature-flag-temp` -- folded into 2962
* #2995 `kerl/polydom`
* #2964 `kerl/tiledbsoma-io-test`
* #3189 `kerl/min-size`
* #3220 `kerl/cpp-ugr-dom-2`
* #3244 `kerl/dense-227-fixes`
* #3276 `kerl/dataframe-shape`

</details>

**Issues which are related but non-blocking:**

* https://github.com/single-cell-data/SOMA/pull/216
* https://github.com/TileDB-Inc/TileDB/pull/5303
* #2966
* #3081
* Note: R append mode does not exist yet -- see #1630 -- so an experiment-level resizer is not a priority in R
* #3271 
* #3272
* #3273

**See also:** [[sc-51048]](https://app.shortcut.com/tiledb-inc/story/51048).

## Problem to be solved

Users want to know the `shape` of an array, in the SciPy sense:

* Reads and writes are bounds-checked against the shape
* This retains its value regardless of which values of a sparse array are or are not actually occupied
* Users can `resize`.
  * Some users need the ability to grow their datasets later, using either `tiledbsoma.io`'s append mode, or subsequent writes using the `tiledbsoma` API.
  * Note that the cellxgene census doesn't need this: eact week's published census has fixed shape, and any updates will happen in new storage, on a new week.

Using TileDB-SOMA up until the present:

* The TIleDB `domain` is immutable after array creation
  * This does bounds-checking for reads and writes, which is good
  * To leverage this to function as a `shape`, users would need to set the `domain` at array-creation time. However, users lose the ability to grow their datasets later.
* There is a `non_empty_domain` accessor
  * This only indicates min/max coordinates _at which data exists_. Consider an `X` array for 100 cells and 200 genes. If non-zero expression counts exist only for cell join IDs 2-17, then the `non_empty_domain` will indicate `(2,17)` along `soma_dim_0`.
  * Consider an `obms["X_pca"]` within the same experiment. This may be 100 cells by 50 PCA components: we need a placd to store the number 50.
  * Therefore users cannot leverage this to function as a `shape` accessor.
* We have offered a `used_shape` accessor since TileDB-SOMA 1.5.
  * This functions as a `shape` accessor, in the SciPy sense, but it is not multi-writer safe.

New feature for TileDB-SOMA 1.15:

* Arrays will have a  `shape`
* Reads and writes are bounds-checked against the shape
* This retains its value regardless of which values of a sparse array are or are not actually occupied
* Users can `resize`
* The `used_shape` accessor will be deprecated in TileDB-SOMA 1.13, and slated for removal in TileDB-SOMA 1.14.

Compatiblity:

This will now require users to do an explicit `resize` before appending/growing TileDB-SOMA Experiments. Guidance in the form of example notebooks will be provided.

##  Tracking

See also: [[sc-41074]](https://app.shortcut.com/tiledb-inc/story/41074) and [[sc-51048]](https://app.shortcut.com/tiledb-inc/story/51048).

## Scheduling

Support arrives in TileDB Core 2.25. Deprecations for TileDB-SOMA will be released with 1.13. Full support within TileDB-SOMA will be release in 1.14.

## Details

SOMA API mods as we've discussed in a Google doc are as follows.

### `SOMADataFrame`

* `create`: Retain the `domain` argument
  * Issue:
    * Core has a `(lo, hi)` tuple per dim, e.g. `(0,99)` or `(10,19)`
    * SOMA has count per dim, with 0 implicit: e.g. 100 or 20
    * For `SparseNDArray` and `DenseNDArray` core can have `(lo, hi)` and SOMA can have `count`
    * For `DataFrame` there can be multiple dims --- default is a single `soma_joinid`
    * That could be treated either in `(lo, hi)` fashion or `count` fashion
    * However additional dims (e.g. `cell_type`) can be on any type, including strings, floats, etc. where there is no implicit lo=0
    * Therefore we need to keep the current SOMA API wherein `DataFrame` takes a `domain` argument (in `(lo, hi)` fashion) and not a `shape` argument (in `count` fashion)

### `SparseNDArray and DenseNDArray`

* `create`
  * Have an optional shape argument which is of type `Tuple[Int,...]` where each element is the cell count of the corresponding dimension
    * If unsupplied, or if supplied but None in any slot: use the minimum 0 in each slot – nothing larger makes sense since we will not support downsize
  * User guidance should make clear that it will not be possible to create an ‘old’ style array with the ‘new style’ API. (See also the upgrade logic below.)

### All three of `SOMADataFrame`, `SparseNDArray`, `DenseNDArray`

* `write`
  * For new arrays, created with the new shape feature:
    * Core will bounds-check that coordinates provided at `write` time are within the current shape
    * Core will raise `tiledb.cc.TileDBError` to TileDB-SOMA, which will catch and raise `IndexError`, and R-standard behavior on the R side
  * For old arrays created before this feature:
    * Core will not bounds-check that coordinates provided at write time are within the current shape
* Existing `used_shape` accessor
  * TileDB-SOMA will deprecate this over a release cycle.
  * For new arrays: raise `NotImplementedError`
  * For old arrays: return what’s currently returned, with a deprecation warning.
  * Mechanism for determining old vs. new: `array.schema.version` (the core storage version).
* Existing `shape` accessor
  * For new arrays:
    * Have this return the new shape as proposed by core, no longer returning the TileDB domain.
  * For old arrays created before this feature:
    * Return the TileDB domain as now.
* Existing `non_empty_domain` accessor
  * Same behavior for old and new arrays (unaffected by this proposal).
  * Keep this accessor supported, but, with user notes that it’s generally non-useful
  * This should return None (or R equivalent) when there is a schema but no data have been written.
* New `maxshape` accessor
  * Maps the core-level `(lo, hi)` accessor for domain to count-style accessor hi+1. E.g. if the core domain is either `(0,99)` or `(50,99)` then TileDB-SOMA `maxshape` will say 100.
  * Same behavior for old and new arrays.
  * Let users query for what the TileDB domain is, with user notes that it’s the maximum that users can reshape to.
  * Issac suggests: maybe `domain` or `maxshape` (see h5py).
* New `resize` mutator
  * Note: `reshape` means something else in the community (numpy, zarr, h5py), e.g. a 5x20 (total 100 cells) being reinterpreted as 4x25 (still 100 cells). The standard name for changing cell-count is `resize`.
  * For old arrays created before this feature: raise `NotImplementedError`.
  * For new arrays:
    * Will raise `ValueError` if the new shape is smaller on any dim than currently in storage
    * Regardless of whether any data have been written whatsoever
    * Will raise `ValueError` if the new shape exceeds the TileDB domain from create time, which will serve TileDB-SOMA in a role of “max possible shape the user can reshape to”
    * Otherwise, any calls to write from this point will bounds-check writes within this new shape
    * **We don’t expect resize to be multi-writer safe with regard to write ; user notes must be clear on this point**
* New `tiledbsoma_upgrade_shape` method for SparseNDArray and DenseNDArray
  * This will leverage `array.schema.version` to see if an upgrade is needed
  * Leverage core support for storage-version updates
  * This will take a shape argument as in `create`
  * For arrays created with “just-right” size: this will succeed
  * For arrays created with “room-for-growth” / “two billion-ish” size: this will succeed
  * If the user passes a shape which exceeds the current TileDB domain: this will fail
* New `tiledbsoma_upgrade_domain method` for `DataFrame`
  * Same as for `SparseNDArray`/`DenseNDArray` except it will take a domain at the SOMA-API level just as `DataFrame`'s create method

### `tiledbsoma.io`

* The user-facing API has no shape arguments and thus won’t need changing.
* Internally to `tiledbsoma.io`, we’ll still ask the tiledbsoma API for the “big domain” (2 billionish)
* Append mode:
  * Will need a new `resize` method at the `Experiment` level
  * Users will need to:
    * Register as now
    * Call the experiment-level `resize`
      * Could be `exp.resize(...)`, or (better) this could be `tiledbsoma.io.reshape_experiment`
  * In either case: this method will take the new `obs` and `var` counts as inputs:
    * `exp.obs.reshape` to new `obs` count
    * `exp.ms[name].var.reshape` to new `var` count
    * `exp.ms[name].X[name].reshape` to new `obs` count x `var` count
    * `exp.ms[name].obsm[name].reshape` to new `obs` count x same width
    * `exp.ms[name].obsp[name].reshape` to new `obs` count x `obs` count
    * `exp.ms[name].varm[name].reshape` to new `var` count x same width
    * `exp.ms[name].varp[name].reshape` to new `var` count x `var` count
  * Do the individual append-mode writes as now

[python] Append mode with resizing
**Issue and/or context:** As tracked on issue #2407 / [[sc-51048]](https://app.shortcut.com/tiledb-inc/story/51048).

Note that the intended Python and R API changes are all agreed on and finalized as described in #2407.

**Changes:**

Support experipment-level `upgrade_shape` and `resize`.

**Notes for Reviewer:**


Discussion / hints:
#2785 is a quick-and-dirty concept-prover -- its sole function is to flush out any API misunderstandings we might have, in prep for 2.25.0 core release.
#2785 is a quick-and-dirty concept-prover -- its sole function is to flush out any API misunderstandings we might have, in prep for 2.25.0 core release.


Fix patch:
diff --git a/apis/python/src/tiledbsoma/io/__init__.py b/apis/python/src/tiledbsoma/io/__init__.py
index f5f5aadfb1..4629c0f89d 100644
--- a/apis/python/src/tiledbsoma/io/__init__.py
+++ b/apis/python/src/tiledbsoma/io/__init__.py
@@ -20,6 +20,11 @@
     to_anndata,
     to_h5ad,
 )
+from .shaping import (
+    resize_experiment,
+    show_experiment_shapes,
+    upgrade_experiment_shapes,
+)
 
 __all__ = (
     "add_matrix_to_collection",
@@ -37,5 +42,8 @@
     "update_matrix",
     "update_obs",
     "update_var",
+    "upgrade_experiment_shapes",
+    "show_experiment_shapes",
+    "resize_experiment",
     "ExperimentAmbientLabelMapping",
 )
diff --git a/apis/python/src/tiledbsoma/io/shaping.py b/apis/python/src/tiledbsoma/io/shaping.py
new file mode 100644
index 0000000000..f82fae2a0d
--- /dev/null
+++ b/apis/python/src/tiledbsoma/io/shaping.py
@@ -0,0 +1,527 @@
+# Copyright (c) 2024 The Chan Zuckerberg Initiative Foundation
+# Copyright (c) 2024 TileDB, Inc.
+#
+# Licensed under the MIT License.
+
+"""
+Provides support for the new shape feature in TileDB-SOMA 1.15, including the
+ability to process all dataframes/arrays contained within a TileDB-SOMA
+Experiment. Please also see
+https://github.com/single-cell-data/TileDB-SOMA/issues/2407.  """
+
+import io
+import sys
+from typing import Any, Dict, Optional, Tuple, TypedDict, Union, cast
+
+import tiledbsoma
+
+Printable = Union[io.TextIOWrapper, io.StringIO]
+
+
+class SizingArgs(TypedDict):
+    """Convenience type-alias for kwargs passed to experiment-level
+    upgrade/resize functions.
+    """
+
+    nobs: Optional[int]
+    nvars: Optional[Dict[str, int]]
+    ms_name: Optional[str]
+    coll_name: Optional[str]
+    verbose: bool
+    check_only: bool
+    context: Optional[tiledbsoma.SOMATileDBContext]
+    output_handle: Printable
+
+
+def show_experiment_shapes(
+    uri: str,
+    *,
+    context: Optional[tiledbsoma.SOMATileDBContext] = None,
+    output_handle: Printable = cast(Printable, sys.stdout),
+) -> bool:
+    """For each dataframe/array contained within the SOMA ``Experiment`` pointed
+    to by the given URI, shows the deprecated ``used_shape`` (for N-D arrays) or
+    the ``count`` (for dataframes), along with the ``shape`` and ``maxshape``
+    (for arrays) or ``domain`` and ``maxdomain`` (for dataframes).
+
+    Args:
+        uri: The URI of a SOMA :class:`Experiment``.
+        context: Optional :class:`SOMATileDBContext``.
+    """
+    args: SizingArgs = dict(
+        nobs=None,
+        nvars=None,
+        ms_name=None,
+        coll_name=None,
+        verbose=True,
+        check_only=True,
+        context=context,
+        output_handle=output_handle,
+    )
+
+    ok = _treewalk(
+        uri,
+        visitor=_leaf_visitor_show_shapes,
+        args=args,
+    )
+    return ok
+
+
+def upgrade_experiment_shapes(
+    uri: str,
+    *,
+    verbose: bool = False,
+    check_only: bool = False,
+    context: Optional[tiledbsoma.SOMATileDBContext] = None,
+    output_handle: Printable = cast(Printable, sys.stdout),
+) -> bool:
+    """For each dataframe contained within the SOMA ``Experiment`` pointed to by
+    the given URI, sets the ``domain`` to match the dataframe's current
+    ``count``.  For each N-D array, sets the ``shape`` to match the array's
+    current ``used_shape``. If ``verbose`` is set to ``True``, an activity log
+    is printed. If ``check_only`` is true, only does a dry run and reports any
+    reasons the upgrade would fail.
+
+    Args:
+        uri: The URI of a SOMA :class:`Experiment``.
+        verbose: If ``True``, produce per-array output as the upgrade runs.
+        check_only: If ``True``,  don't apply the upgrades, but show what would
+            be attempted, and show why each one would fail.
+        context: Optional :class:`SOMATileDBContext``.
+    """
+    args: SizingArgs = dict(
+        nobs=None,
+        nvars=None,
+        ms_name=None,
+        coll_name=None,
+        verbose=verbose,
+        check_only=check_only,
+        context=context,
+        output_handle=output_handle,
+    )
+    ok = _treewalk(
+        uri,
+        visitor=_leaf_visitor_upgrade,
+        args=args,
+    )
+    return ok
+
+
+def resize_experiment(
+    uri: str,
+    *,
+    nobs: int,
+    nvars: Dict[str, int],
+    verbose: bool = False,
+    check_only: bool = False,
+    context: Optional[tiledbsoma.SOMATileDBContext] = None,
+    output_handle: Printable = cast(Printable, sys.stdout),
+) -> bool:
+    """For each dataframe contained within the SOMA ``Experiment`` pointed to by
+    the given URI, resizes the ``domain`` for the ``soma_joinid index column
+    (if it is an indexed column) to match the desired new value.
+
+    The desired new value may be the same size as at present, or bigger, but not
+    exceeding ``maxdomain`` for the ``soma_joinid`` index column.
+
+    For each N-D array contained within the SOMA ``Experiment``, resizes the ``shape``
+    to match the desired values.
+
+    * For ``X`` arrays, the resize is to new ``nobs`` x the measurement's new ``nvar``.
+    * For ``obsm`` arrays, the resize is to new ``nobs`` x the array's existing ``soma_dim_1`` shape.
+    * For ``obsp`` arrays, the resize is to new ``nobs`` x that same ``nobs``.
+    * For ``varm`` arrays, the resize is to new ``nvar`` x the array's existing ``soma_dim_1`` shape.
+    * For ``varp`` arrays, the resize is to new ``nvar`` x that same ``nvar``.
+
+    In all cases, the desired new ``shape`` value may be the same size on the
+    given dimension, or bigger, but not exceeding ``maxshape`` for the given
+    dimension.
+
+    If any array has not been upgraded, then its resize will fail.
+
+    Args:
+        uri: The URI of a SOMA :class:`Experiment``.
+        nobs: The desired new shape of the experiment's ``obs`` dataframe.
+        nvars: The desired new shapes of the experiment's ``var`` dataframes.
+            This should be a dict from measurement name to shape, e.g.
+            ``{"RNA": 10000, "raw": 20000}``.
+        verbose: If ``True``, produce per-array output as the upgrade runs.
+        check_only: If ``True``,  don't apply the upgrades, but show what would
+            be attempted, and show why each one would fail.
+        context: Optional :class:`SOMATileDBContext``.
+    """
+    args: SizingArgs = dict(
+        nobs=nobs,
+        nvars=nvars,
+        ms_name=None,
+        coll_name=None,
+        verbose=verbose,
+        check_only=check_only,
+        context=context,
+        output_handle=output_handle,
+    )
+
+    # Do an early check on the nvars keys vs the experiment's
+    # measurent names. This isn't a can-do status for the experiment;
+    # it's a failure of the user's arguments.
+    with tiledbsoma.Experiment.open(uri) as exp:
+        arg_keys = sorted(nvars.keys())
+        ms_keys = sorted(exp.ms.keys())
+        if arg_keys != ms_keys:
+            raise ValueError(
+                f"resize_experiment: provided nvar keys {arg_keys} do not match experiment keys {ms_keys}"
+            )
+
+    ok = _treewalk(
+        uri,
+        visitor=_leaf_visitor_resize,
+        args=args,
+    )
+    return ok
+
+
+def _treewalk(
+    uri: str,
+    *,
+    node_name: Optional[str] = None,
+    visitor: Any,
+    args: SizingArgs,
+) -> bool:
+    retval = True
+    with tiledbsoma.open(uri) as item:
+
+        if isinstance(item, tiledbsoma.Experiment):
+            if "obs" in item:
+                ok = _treewalk(
+                    item["obs"].uri, node_name="obs", visitor=visitor, args=args
+                )
+                retval = retval and ok
+            if "ms" in item:
+                ok = _treewalk(
+                    item["ms"].uri, node_name="ms", visitor=visitor, args=args
+                )
+                retval = retval and ok
+
+        elif isinstance(item, tiledbsoma.Measurement):
+            if "var" in item:
+                ok = _treewalk(
+                    item["var"].uri, node_name="var", visitor=visitor, args=args
+                )
+                retval = retval and ok
+
+            for coll_name in ["X", "obsm", "obsp", "varm", "varp"]:
+                if coll_name in item:
+
+                    args["coll_name"] = coll_name
+
+                    ok = _treewalk(
+                        item[coll_name].uri,
+                        node_name=coll_name,
+                        visitor=visitor,
+                        args=args,
+                    )
+                    retval = retval and ok
+
+        elif isinstance(item, tiledbsoma.Collection):
+
+            for key in item:
+
+                if node_name == "ms":
+                    args["ms_name"] = key
+
+                ok = _treewalk(item[key].uri, node_name=key, visitor=visitor, args=args)
+                retval = retval and ok
+
+        else:
+            ok = visitor(item, node_name=node_name, args=args)
+            retval = retval and ok
+
+    return retval
+
+
+def _leaf_visitor_show_shapes(
+    item: Any,
+    *,
+    node_name: str,
+    args: SizingArgs,
+) -> bool:
+    retval = True
+    if isinstance(item, tiledbsoma.DataFrame):
+        _print_leaf_node_banner("DataFrame", node_name, item.uri, args)
+        _bannerize(args, "count", item.count)
+        _bannerize(args, "domain", item.domain)
+        _bannerize(args, "maxdomain", item.maxdomain)
+        _bannerize(args, "upgraded", item.tiledbsoma_has_upgraded_domain)
+
+    elif isinstance(item, tiledbsoma.SparseNDArray):
+        _print_leaf_node_banner("SparseNDArray", node_name, item.uri, args)
+        _bannerize(args, "used_shape", item.used_shape())
+        _bannerize(args, "shape", item.shape)
+        _bannerize(args, "maxshape", item.maxshape)
+        _bannerize(args, "upgraded", item.tiledbsoma_has_upgraded_shape)
+
+    elif isinstance(item, tiledbsoma.DenseNDArray):
+        _print_leaf_node_banner("DenseNDArray", node_name, item.uri, args)
+        _bannerize(args, "shape", item.shape)
+        _bannerize(args, "maxshape", item.maxshape)
+        _bannerize(args, "upgraded", item.tiledbsoma_has_upgraded_shape)
+
+    return retval
+
+
+def _leaf_visitor_upgrade(
+    item: Any,
+    *,
+    node_name: str,
+    args: SizingArgs,
+) -> bool:
+    verbose = args["verbose"]
+    check_only = args["check_only"]
+    retval = True
+
+    if isinstance(item, tiledbsoma.DataFrame):
+        count = item.count
+
+        _print_leaf_node_banner("DataFrame", node_name, item.uri, args)
+        if check_only:
+            print(
+                f"  Dry run for: upgrade_soma_joinid_shape({count})",
+                file=args["output_handle"],
+            )
+            ok, msg = item.upgrade_soma_joinid_shape(count, check_only=True)
+            _print_dry_run_result(ok, msg, args)
+            retval = retval and ok
+        elif not item.tiledbsoma_has_upgraded_domain:
+            if verbose:
+                print(
+                    f"  Applying upgrade_soma_joinid_shape({count})",
+                    file=args["output_handle"],
+                )
+            with tiledbsoma.DataFrame.open(item.uri, "w") as writer:
+                writer.upgrade_soma_joinid_shape(count)
+        else:
+            if verbose:
+                print("  Already upgraded", file=args["output_handle"])
+
+    elif isinstance(item, tiledbsoma.SparseNDArray):
+        used_shape = item.used_shape()
+        new_shape = tuple(e[1] + 1 for e in used_shape)
+
+        _print_leaf_node_banner("SparseNDArray", node_name, item.uri, args)
+        if check_only:
+            print(
+                f"  Dry run for: upgrade_shape({new_shape})", file=args["output_handle"]
+            )
+            ok, msg = item.tiledbsoma_upgrade_shape(new_shape, check_only=True)
+            _print_dry_run_result(ok, msg, args)
+            retval = retval and ok
+        elif not item.tiledbsoma_has_upgraded_shape:
+            if verbose:
+                print(
+                    f"  Applying upgrade_shape({new_shape})", file=args["output_handle"]
+                )
+            with tiledbsoma.SparseNDArray.open(item.uri, "w") as writer:
+                writer.tiledbsoma_upgrade_shape(new_shape)
+        else:
+            if verbose:
+                print("  Already upgraded", file=args["output_handle"])
+
+    elif isinstance(item, tiledbsoma.DenseNDArray):
+        _print_leaf_node_banner("DenseNDArray", node_name, item.uri, args)
+        if verbose or check_only:
+            print(
+                "  No action at this time, pending new-shape support for dense arrays in core 2.27",
+                file=args["output_handle"],
+            )
+    return retval
+
+
+def _leaf_visitor_resize(
+    item: Any,
+    *,
+    node_name: str,
+    args: SizingArgs,
+) -> bool:
+    verbose = args["verbose"]
+    check_only = args["check_only"]
+    retval = True
+    if isinstance(item, tiledbsoma.DataFrame):
+
+        if node_name == "obs":
+            new_soma_joinid_shape = args["nobs"]
+            if new_soma_joinid_shape is None:
+                raise tiledbsoma.SOMAError(
+                    "experiment resize: internal error: nobs missing"
+                )
+
+        elif node_name == "var":
+            new_soma_joinid_shape = _get_new_var_shape(args)
+
+        else:
+            raise tiledbsoma.SOMAError(
+                "experiment resize: internal error: dataframe node name '{node_name}'"
+            )
+
+        _print_leaf_node_banner("DataFrame", node_name, item.uri, args)
+
+        if check_only:
+            print(
+                f"  Dry run for: resize_soma_joinid_shape({new_soma_joinid_shape})",
+                file=args["output_handle"],
+            )
+            ok, msg = item.resize_soma_joinid_shape(
+                new_soma_joinid_shape, check_only=True
+            )
+            _print_dry_run_result(ok, msg, args)
+            retval = retval and ok
+        else:
+            if verbose:
+                print(
+                    f"  Applying resize_soma_joinid_shape({new_soma_joinid_shape})",
+                    file=args["output_handle"],
+                )
+            with tiledbsoma.DataFrame.open(item.uri, "w") as writer:
+                writer.resize_soma_joinid_shape(new_soma_joinid_shape)
+
+    elif isinstance(item, tiledbsoma.SparseNDArray):
+
+        _print_leaf_node_banner("SparseNDArray", node_name, item.uri, args)
+
+        new_shape = _get_new_ndarray_shape(args, item.shape)
+
+        if check_only:
+            print(f"  Dry run for: resize({new_shape})", file=args["output_handle"])
+            ok, msg = item.resize(new_shape, check_only=True)
+            _print_dry_run_result(ok, msg, args)
+            retval = retval and ok
+        else:
+            if verbose:
+                print(f"  Applying resize({new_shape})", file=args["output_handle"])
+            with tiledbsoma.SparseNDArray.open(item.uri, "w") as writer:
+                writer.resize(new_shape)
+
+    elif isinstance(item, tiledbsoma.DenseNDArray):
+
+        _print_leaf_node_banner("DenseNDArray", node_name, item.uri, args)
+        if verbose or check_only:
+            print(
+                "  No action at this time, pending new-shape support for dense arrays in core 2.27",
+                file=args["output_handle"],
+            )
+
+    return retval
+
+
+def _print_leaf_node_banner(
+    type_name: str,
+    node_name: str,
+    uri: str,
+    args: SizingArgs,
+) -> None:
+    if args["verbose"] or args["check_only"]:
+        print("", file=args["output_handle"])
+        print(
+            _get_leaf_node_description(type_name, node_name, args),
+            file=args["output_handle"],
+        )
+        print(f"  URI {uri}", file=args["output_handle"])
+
+
+def _get_leaf_node_description(
+    type_name: str,
+    node_name: str,
+    args: SizingArgs,
+) -> str:
+    # Return things like "ms/RNA/X/data"
+    pieces = []
+    if args["ms_name"] is not None:
+        pieces.append("ms")
+        pieces.append(args["ms_name"])
+    if args["coll_name"] is not None:
+        pieces.append(args["coll_name"])
+    pieces.append(node_name)
+    return f"[{type_name}] {'/'.join(pieces)} "
+
+
+def _bannerize(
+    args: SizingArgs,
+    name: str,
+    value: Any,
+) -> None:
+    if args["verbose"] or args["check_only"]:
+        print(f"  {name:<20} {value}", file=args["output_handle"])
+
+
+def _print_dry_run_result(ok: bool, msg: str, args: SizingArgs) -> None:
+    if ok:
+        print("  OK", file=args["output_handle"])
+    else:
+        print(f"  Not OK: {msg}", file=args["output_handle"])
+
+
+def _get_new_var_shape(
+    args: SizingArgs,
+) -> int:
+    """Maps from experiment-level nobs and per-measurement nvar to the correct resizes
+    for var dataframes within a SOMA experiment.
+    """
+    nvars = args["nvars"]
+    if nvars is None:
+        raise tiledbsoma.SOMAError("experiment resize: internal error: nvars missing")
+
+    ms_name = args["ms_name"]
+    if ms_name is None:
+        raise tiledbsoma.SOMAError("experiment resize: internal error: ms_name missing")
+
+    if ms_name not in nvars:
+        raise tiledbsoma.SOMAError(
+            f"experiment resize: missing measurement name '{ms_name}' in provided nvars"
+        )
+    return nvars[ms_name]
+
+
+def _get_new_ndarray_shape(
+    args: SizingArgs,
+    current_shape: Tuple[int, ...],
+) -> Tuple[int, ...]:
+    """Maps from experiment-level nobs and per-measurement nvar to the correct resizes
+    for various kinds of n-d array within a SOMA Experiment.
+    """
+    nobs = args["nobs"]
+    if nobs is None:
+        raise tiledbsoma.SOMAError("experiment resize: internal error: nobs missing")
+
+    nvars = args["nvars"]
+    if nvars is None:
+        raise tiledbsoma.SOMAError("experiment resize: internal error: nvars missing")
+
+    ms_name = args["ms_name"]
+    if ms_name is None:
+        raise tiledbsoma.SOMAError("experiment resize: internal error: ms_name missing")
+
+    if ms_name not in nvars:
+        raise tiledbsoma.SOMAError(
+            f"experiment resize: missing measurement name '{ms_name}' in provided nvars"
+        )
+    nvar = nvars[ms_name]
+
+    coll_name = args["coll_name"]
+    if coll_name is None:
+        raise tiledbsoma.SOMAError(
+            "experiment resize: internal error: coll_name missing"
+        )
+
+    coll_dict = {
+        "X": (nobs, nvar),
+        "obsm": (nobs, current_shape[1]),
+        "obsp": (nobs, nobs),
+        "varm": (nvar, current_shape[1]),
+        "varp": (nvar, nvar),
+    }
+
+    try:
+        return coll_dict[coll_name]
+    except KeyError:
+        raise tiledbsoma.SOMAError(
+            f"experiment resize: internal error: unhandled collection {coll_name}"
+        )


Test patch:
diff --git a/apis/python/tests/test_shape.py b/apis/python/tests/test_shape.py
index 5060fcee81..fd3272a74a 100644
--- a/apis/python/tests/test_shape.py
+++ b/apis/python/tests/test_shape.py
@@ -1,11 +1,13 @@
 from __future__ import annotations
 
+import io
 import tarfile
 
 import pyarrow as pa
 import pytest
 
 import tiledbsoma
+import tiledbsoma.io
 
 from ._util import TESTDATA
 
@@ -369,8 +371,13 @@ def _assert_huge_domainish(d):
         # Exact number depends on tile extent, and is unimportant in any case
         assert d[0][1] > 2**62
 
-    def _check_dataframe(sdf, has_shapes, expected_count):
-        assert sdf.count == expected_count
+    def _check_dataframe(
+        sdf, has_shapes, expected_count, *, count_must_match: bool = True
+    ):
+        if count_must_match:
+            # OK match case: 2000 populated rows and shape is 2000.
+            # OK mismatch case: 2000 populated rows and a reshape to 3000 has been done.
+            assert sdf.count == expected_count
         if not has_shapes:
             _assert_huge_domainish(sdf.domain)
         else:
@@ -396,7 +403,6 @@ def _check_ndarray(ndarray, has_shapes, expected_shape):
         _check_dataframe(exp.obs, has_shapes, 2638)
 
         assert "raw" in exp.ms
-        assert exp.ms["raw"].var.count == 13714
         assert "data" in exp.ms["raw"].X
 
         _check_dataframe(exp.ms["raw"].var, has_shapes, 13714)
@@ -407,3 +413,161 @@ def _check_ndarray(ndarray, has_shapes, expected_shape):
         _check_ndarray(exp.ms["RNA"].obsm["X_pca"], has_shapes, (2638, 50))
         _check_ndarray(exp.ms["RNA"].obsp["connectivities"], has_shapes, (2638, 2638))
         _check_ndarray(exp.ms["RNA"].varm["PCs"], has_shapes, (1838, 50))
+
+    # Check tiledbsoma.io.show_experiment_shapes. This is mainly for interactive
+    # use, so in a unit test, we want to check basics:
+    # * Check that many lines were printed
+    # * Make sure exceptions occurred
+    # * Do some spot-checks on wording
+
+    handle = io.StringIO()
+    tiledbsoma.io.show_experiment_shapes(uri, output_handle=handle)
+    handle.seek(0)
+    lines = handle.readlines()
+    handle.close()
+    # Exact line count doesn't matter: make sure it's a lot.
+    assert len(lines) > 50
+    body = "\n".join(lines)
+    assert "[SparseNDArray] ms/RNA/obsp/distances" in body
+    assert "ms/RNA/obsm/X_draw_graph_fr" in body
+
+    # Check dry run of tiledbsoma.io.upgrade_experiment_shapes
+    handle = io.StringIO()
+    upgradeable = tiledbsoma.io.upgrade_experiment_shapes(
+        uri, check_only=True, output_handle=handle
+    )
+    handle.seek(0)
+    lines = handle.readlines()
+    handle.close()
+    # Exact line count doesn't matter: make sure it's a lot.
+    assert len(lines) > 50
+    body = "\n".join(lines)
+    assert "[SparseNDArray] ms/RNA/obsp/distances" in body
+    assert "ms/RNA/obsm/X_draw_graph_fr" in body
+    assert upgradeable != has_shapes
+
+    # Check dry run of tiledbsoma.io.resize_experiment -- plenty of room
+    handle = io.StringIO()
+    resizeable = tiledbsoma.io.resize_experiment(
+        uri,
+        nobs=100000,
+        nvars={"RNA": 100000, "raw": 200000},
+        check_only=True,
+        output_handle=handle,
+    )
+    handle.seek(0)
+    lines = handle.readlines()
+    handle.close()
+    # Exact line count doesn't matter: make sure it's a lot.
+    assert len(lines) > 50
+    body = "\n".join(lines)
+    assert "[SparseNDArray] ms/RNA/obsp/distances" in body
+    assert "ms/RNA/obsm/X_draw_graph_fr" in body
+    assert resizeable == has_shapes
+
+    # Check dry run of tiledbsoma.io.resize_experiment -- no change
+    handle = io.StringIO()
+    resizeable = tiledbsoma.io.resize_experiment(
+        uri,
+        nobs=2638,
+        nvars={"RNA": 1838, "raw": 13714},
+        check_only=True,
+        output_handle=handle,
+    )
+    assert resizeable == has_shapes
+
+    # Check dry run of tiledbsoma.io.resize_experiment -- downsize
+    handle = io.StringIO()
+    resizeable = tiledbsoma.io.resize_experiment(
+        uri,
+        nobs=2638,
+        nvars={"RNA": 1838, "raw": 13713},
+        check_only=True,
+        output_handle=handle,
+    )
+    assert not resizeable
+    handle.seek(0)
+    lines = handle.readlines()
+    handle.close()
+    body = "\n".join(lines)
+    if not has_shapes:
+        assert (
+            "Not OK: can_resize: array currently has no shape: please upgrade the array"
+            in body
+        )
+    else:
+        assert (
+            "Not OK: can_resize for soma_dim_1: new 13713 < existing shape 13714"
+            in body
+        )
+
+    # Check real run of tiledbsoma.io.upgrade_experiment_shapes
+    handle = io.StringIO()
+    upgraded = tiledbsoma.io.upgrade_experiment_shapes(uri, output_handle=handle)
+    handle.seek(0)
+    lines = handle.readlines()
+    handle.close()
+    # Experiment-level upgrade is idempotent.
+    assert upgraded
+
+    # Check post-upgrade shapes
+    with tiledbsoma.Experiment.open(uri) as exp:
+
+        _check_dataframe(exp.obs, True, 2638)
+
+        assert "raw" in exp.ms
+        assert "data" in exp.ms["raw"].X
+
+        _check_dataframe(exp.ms["raw"].var, True, 13714)
+        _check_ndarray(exp.ms["raw"].X["data"], True, (2638, 13714))
+
+        _check_dataframe(exp.ms["RNA"].var, True, 1838)
+        _check_ndarray(exp.ms["RNA"].X["data"], True, (2638, 1838))
+        _check_ndarray(exp.ms["RNA"].obsm["X_pca"], True, (2638, 50))
+        _check_ndarray(exp.ms["RNA"].obsp["connectivities"], True, (2638, 2638))
+        _check_ndarray(exp.ms["RNA"].varm["PCs"], True, (1838, 50))
+
+    # Check real same-size resize
+    handle = io.StringIO()
+    resized = tiledbsoma.io.resize_experiment(
+        uri,
+        nobs=2638,
+        nvars={"RNA": 1838, "raw": 13714},
+        output_handle=handle,
+    )
+    assert resized
+
+    # Check real down-size resize
+    with pytest.raises(tiledbsoma.SOMAError):
+        tiledbsoma.io.resize_experiment(
+            uri,
+            nobs=2637,
+            nvars={"RNA": 1838, "raw": 13714},
+            output_handle=handle,
+        )
+
+    # Check real up-size resize
+    handle = io.StringIO()
+    resized = tiledbsoma.io.resize_experiment(
+        uri,
+        nobs=2639,
+        nvars={"RNA": 1839, "raw": 13720},
+        output_handle=handle,
+    )
+    assert resized
+
+    # Check new shapes
+    with tiledbsoma.Experiment.open(uri) as exp:
+        _check_dataframe(exp.obs, True, 2639, count_must_match=False)
+
+        assert "raw" in exp.ms
+        assert "data" in exp.ms["raw"].X
+
+        _check_dataframe(exp.ms["raw"].var, True, 13720, count_must_match=False)
+        _check_ndarray(exp.ms["raw"].X["data"], True, (2639, 13720))
+
+        _check_dataframe(exp.ms["RNA"].var, True, 1839, count_must_match=False)
+        _check_ndarray(exp.ms["RNA"].X["data"], True, (2639, 1839))
+        _check_ndarray(exp.ms["RNA"].obsm["X_pca"], True, (2639, 50))
+        _check_ndarray(exp.ms["RNA"].obsp["connectivities"], True, (2639, 2639))
+        _check_ndarray(exp.ms["RNA"].varm["PCs"], True, (1839, 50))