Repository: spacetx/starfish

Issue:
(composite, ragged) experiment and codebook support
# Objective

*As a spaceTX data analyst with an experiment that does not have a matching number of rounds and channels, I want starfish to support this data model, so that  I can create a starfish benchmarking pipeline*

## Acceptance Criteria
* Simone can process his data with starfish
* Brian Long can process his data with starfish

## Notes
Encapsulates work that needs to be done to support data coming in from Ola, Simone, Brian Long, and presumably RNAscope users

Discussion:
codecov-io: # [Codecov](https://codecov.io/gh/spacetx/starfish/pull/1382?src=pr&el=h1) Report
> Merging [#1382](https://codecov.io/gh/spacetx/starfish/pull/1382?src=pr&el=desc) into [master](https://codecov.io/gh/spacetx/starfish/commit/bfc5f8da12061b5c10316f300f80ee695d194c71?src=pr&el=desc) will **decrease** coverage by `22.02%`.
> The diff coverage is `26.53%`.

[![Impacted file tree graph](https://codecov.io/gh/spacetx/starfish/pull/1382/graphs/tree.svg?width=650&token=kliQHejt6T&height=150&src=pr)](https://codecov.io/gh/spacetx/starfish/pull/1382?src=pr&el=tree)

```diff
@@             Coverage Diff             @@
##           master    #1382       +/-   ##
===========================================
- Coverage   90.59%   68.56%   -22.03%     
===========================================
  Files         217      148       -69     
  Lines        8304     5405     -2899     
===========================================
- Hits         7523     3706     -3817     
- Misses        781     1699      +918
```


| [Impacted Files](https://codecov.io/gh/spacetx/starfish/pull/1382?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [starfish/core/experiment/builder/cli.py](https://codecov.io/gh/spacetx/starfish/pull/1382/diff?src=pr&el=tree#diff-c3RhcmZpc2gvY29yZS9leHBlcmltZW50L2J1aWxkZXIvY2xpLnB5) | `70.83% <100%> (-20.48%)` | :arrow_down: |
| [starfish/core/experiment/builder/\_\_init\_\_.py](https://codecov.io/gh/spacetx/starfish/pull/1382/diff?src=pr&el=tree#diff-c3RhcmZpc2gvY29yZS9leHBlcmltZW50L2J1aWxkZXIvX19pbml0X18ucHk=) | `30.58% <23.4%> (-69.42%)` | :arrow_down: |
| [starfish/core/experiment/builder/inplace.py](https://codecov.io/gh/spacetx/starfish/pull/1382/diff?src=pr&el=tree#diff-c3RhcmZpc2gvY29yZS9leHBlcmltZW50L2J1aWxkZXIvaW5wbGFjZS5weQ==) | `0% <0%> (-87.88%)` | :arrow_down: |
| [starfish/test/full\_pipelines/api/test\_dartfish.py](https://codecov.io/gh/spacetx/starfish/pull/1382/diff?src=pr&el=tree#diff-c3RhcmZpc2gvdGVzdC9mdWxsX3BpcGVsaW5lcy9hcGkvdGVzdF9kYXJ0ZmlzaC5weQ==) | `20% <0%> (-80%)` | :arrow_down: |
| [starfish/test/full\_pipelines/api/test\_iss\_api.py](https://codecov.io/gh/spacetx/starfish/pull/1382/diff?src=pr&el=tree#diff-c3RhcmZpc2gvdGVzdC9mdWxsX3BpcGVsaW5lcy9hcGkvdGVzdF9pc3NfYXBpLnB5) | `23.52% <0%> (-76.48%)` | :arrow_down: |
| [starfish/test/full\_pipelines/api/test\_merfish.py](https://codecov.io/gh/spacetx/starfish/pull/1382/diff?src=pr&el=tree#diff-c3RhcmZpc2gvdGVzdC9mdWxsX3BpcGVsaW5lcy9hcGkvdGVzdF9tZXJmaXNoLnB5) | `24.39% <0%> (-75.61%)` | :arrow_down: |
| [...fish/core/imagestack/test/factories/all\_purpose.py](https://codecov.io/gh/spacetx/starfish/pull/1382/diff?src=pr&el=tree#diff-c3RhcmZpc2gvY29yZS9pbWFnZXN0YWNrL3Rlc3QvZmFjdG9yaWVzL2FsbF9wdXJwb3NlLnB5) | `24.44% <0%> (-75.56%)` | :arrow_down: |
| [starfish/core/spacetx\_format/test\_fuzz.py](https://codecov.io/gh/spacetx/starfish/pull/1382/diff?src=pr&el=tree#diff-c3RhcmZpc2gvY29yZS9zcGFjZXR4X2Zvcm1hdC90ZXN0X2Z1enoucHk=) | `31.03% <0%> (-68.97%)` | :arrow_down: |
| [starfish/core/intensity\_table/overlap.py](https://codecov.io/gh/spacetx/starfish/pull/1382/diff?src=pr&el=tree#diff-c3RhcmZpc2gvY29yZS9pbnRlbnNpdHlfdGFibGUvb3ZlcmxhcC5weQ==) | `32.69% <0%> (-63.47%)` | :arrow_down: |
| [.../spots/\_detect\_spots/local\_search\_blob\_detector.py](https://codecov.io/gh/spacetx/starfish/pull/1382/diff?src=pr&el=tree#diff-c3RhcmZpc2gvY29yZS9zcG90cy9fZGV0ZWN0X3Nwb3RzL2xvY2FsX3NlYXJjaF9ibG9iX2RldGVjdG9yLnB5) | `32.2% <0%> (-61.87%)` | :arrow_down: |
| ... and [133 more](https://codecov.io/gh/spacetx/starfish/pull/1382/diff?src=pr&el=tree-more) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/spacetx/starfish/pull/1382?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/spacetx/starfish/pull/1382?src=pr&el=footer). Last update [bfc5f8d...77e52b4](https://codecov.io/gh/spacetx/starfish/pull/1382?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

njmei: Should users be able to import the `TileCoordinate` dataclass and `write_irregular_experiment_json()` from `starfish.experiment.builder`?

It looks like currently the only way to access is to import from `starfish.core.experiment.builder`.
ttung: > Should users be able to import the `TileCoordinate` dataclass and `write_irregular_experiment_json()` from `starfish.experiment.builder`?

> It looks like currently the only way to access is to import from `starfish.core.experiment.builder`.

I pushed a fix for this.
njmei: I think the `TileCoordinate` needs to be importable too from `starfish.experiment.builder`?
ttung: > I think the TileCoordinate needs to be importable too from starfish.experiment.builder?

Fixed.
ttung: Sorry, that was my mistake.  I thought this was the issue, and I also thought this PR was landed. :) 

Patch:
diff --git a/REQUIREMENTS.txt b/REQUIREMENTS.txt
index 814bf02d..009b4e6b 100644
--- a/REQUIREMENTS.txt
+++ b/REQUIREMENTS.txt
@@ -1,4 +1,5 @@
 click
+dataclasses;python_version<"3.7"
 jsonschema
 matplotlib
 numpy != 1.13.0
diff --git a/starfish/core/experiment/builder/__init__.py b/starfish/core/experiment/builder/__init__.py
index 28347624..bb7b6125 100644
--- a/starfish/core/experiment/builder/__init__.py
+++ b/starfish/core/experiment/builder/__init__.py
@@ -1,12 +1,18 @@
+import functools
 import json
 import os
+from dataclasses import astuple, dataclass
 from pathlib import Path
 from typing import (
     Any,
     BinaryIO,
     Callable,
     Dict,
+    Iterable,
     Mapping,
+    MutableMapping,
+    MutableSequence,
+    MutableSet,
     Optional,
     Sequence,
     Union,
@@ -20,6 +26,7 @@ from slicedimage import (
     Writer,
 )
 
+from starfish import FieldOfView
 from starfish.core.codebook.codebook import Codebook
 from starfish.core.experiment.builder.orderediterator import join_axes_labels, ordered_iterator
 from starfish.core.experiment.version import CURRENT_VERSION
@@ -28,13 +35,19 @@ from .defaultproviders import RandomNoiseTile, tile_fetcher_factory
 from .providers import FetchedTile, TileFetcher
 
 
-AUX_IMAGE_NAMES = {
-    'nuclei',
-    'dots',
-}
 DEFAULT_DIMENSION_ORDER = (Axes.ZPLANE, Axes.ROUND, Axes.CH)
 
 
+@dataclass
+class TileIdentifier:
+    """Data class for encapsulating the location of a tile in a 6D tensor (fov, round, ch, zplane,
+    y, and x)."""
+    fov_id: int
+    round_id: int
+    ch_id: int
+    zplane_id: int
+
+
 def _tile_opener(toc_path: Path, tile: Tile, file_ext: str) -> BinaryIO:
     base = toc_path.parent / toc_path.stem
     return open(
@@ -52,55 +65,45 @@ def _fov_path_generator(parent_toc_path: Path, toc_name: str) -> Path:
     return parent_toc_path.parent / "{}-{}.json".format(parent_toc_path.stem, toc_name)
 
 
-def build_image(
-        fovs: Sequence[int],
-        rounds: Sequence[int],
-        chs: Sequence[int],
-        zplanes: Sequence[int],
+def build_irregular_image(
+        tile_coordinates: Iterable[TileIdentifier],
         image_fetcher: TileFetcher,
-        default_shape: Optional[Mapping[Axes, int]]=None,
-        axes_order: Sequence[Axes]=DEFAULT_DIMENSION_ORDER,
+        default_shape: Optional[Mapping[Axes, int]] = None,
 ) -> Collection:
     """
-    Build and returns an image set with the following characteristics:
+    Build and returns an image set that can potentially be irregular (i.e., the cardinality of the
+    dimensions are not always consistent).  It can also build a regular image.
 
     Parameters
     ----------
-    fovs : Sequence[int]
-        Sequence of field of view ids in this image set.
-    rounds : Sequence[int]
-        Sequence of the round numbers in this image set.
-    chs : Sequence[int]
-        Sequence of the ch numbers in this image set.
-    zplanes : Sequence[int]
-        Sequence of the zplane numbers in this image set.
+    tile_coordinates : Iterable[TileIdentifier]
+        Iterable of all the TileCoordinates that are valid in the image.
     image_fetcher : TileFetcher
         Instance of TileFetcher that provides the data for the tile.
     default_shape : Optional[Tuple[int, int]]
         Default shape of the individual tiles in this image set.
-    axes_order : Sequence[Axes]
-        Ordering for which axes vary, in order of the slowest changing axis to the fastest.  For
-        instance, if the order is (ROUND, Z, CH) and each dimension has size 2, then the sequence
-        is:
-          (ROUND=0, CH=0, Z=0)
-          (ROUND=0, CH=1, Z=0)
-          (ROUND=0, CH=0, Z=1)
-          (ROUND=0, CH=1, Z=1)
-          (ROUND=1, CH=0, Z=0)
-          (ROUND=1, CH=1, Z=0)
-          (ROUND=1, CH=0, Z=1)
-          (ROUND=1, CH=1, Z=1)
-        (default = (Axes.Z, Axes.ROUND, Axes.CH))
 
     Returns
     -------
     The slicedimage collection representing the image.
     """
-    axes_sizes = join_axes_labels(
-        axes_order, rounds=rounds, chs=chs, zplanes=zplanes)
+    def reducer_to_sets(
+            accumulated: Sequence[MutableSet[int]], update: TileIdentifier,
+    ) -> Sequence[MutableSet[int]]:
+        """Reduces to a list of sets of tile coordinates, in the order of FOV, round, ch, and
+        zplane."""
+        result: MutableSequence[MutableSet[int]] = list()
+        for accumulated_elem, update_elem in zip(accumulated, astuple(update)):
+            accumulated_elem.add(update_elem)
+            result.append(accumulated_elem)
+        return result
+    initial_value: Sequence[MutableSet[int]] = tuple(set() for _ in range(4))
+
+    fovs, rounds, chs, zplanes = functools.reduce(
+        reducer_to_sets, tile_coordinates, initial_value)
 
     collection = Collection()
-    for fov_id in fovs:
+    for expected_fov in fovs:
         fov_images = TileSet(
             [
                 Coordinates.X,
@@ -117,18 +120,23 @@ def build_image(
             ImageFormat.TIFF,
         )
 
-        for selector in ordered_iterator(axes_sizes):
+        for tile_coordinate in tile_coordinates:
+            current_fov, current_round, current_ch, current_zplane = astuple(tile_coordinate)
+            # filter out the fovs that are not the one we are currently processing
+            if expected_fov != current_fov:
+                continue
             image = image_fetcher.get_tile(
-                fov_id,
-                selector[Axes.ROUND],
-                selector[Axes.CH],
-                selector[Axes.ZPLANE])
+                current_fov,
+                current_round,
+                current_ch,
+                current_zplane
+            )
             tile = Tile(
                 image.coordinates,
                 {
-                    Axes.ZPLANE: (selector[Axes.ZPLANE]),
-                    Axes.ROUND: (selector[Axes.ROUND]),
-                    Axes.CH: (selector[Axes.CH]),
+                    Axes.ZPLANE: current_zplane,
+                    Axes.ROUND: current_round,
+                    Axes.CH: current_ch,
                 },
                 image.shape,
                 extras=image.extras,
@@ -140,22 +148,73 @@ def build_image(
             # the checksum of the original file.
             tile.provider = image
             fov_images.add_tile(tile)
-        collection.add_partition("fov_{:03}".format(fov_id), fov_images)
+        collection.add_partition("fov_{:03}".format(expected_fov), fov_images)
     return collection
 
 
-def write_labeled_experiment_json(
+def build_image(
+        fovs: Sequence[int],
+        rounds: Sequence[int],
+        chs: Sequence[int],
+        zplanes: Sequence[int],
+        image_fetcher: TileFetcher,
+        default_shape: Optional[Mapping[Axes, int]] = None,
+        axes_order: Sequence[Axes] = DEFAULT_DIMENSION_ORDER,
+) -> Collection:
+    """
+    Build and returns an image set with the following characteristics:
+
+    Parameters
+    ----------
+    fovs : Sequence[int]
+        Sequence of field of view ids in this image set.
+    rounds : Sequence[int]
+        Sequence of the round numbers in this image set.
+    chs : Sequence[int]
+        Sequence of the ch numbers in this image set.
+    zplanes : Sequence[int]
+        Sequence of the zplane numbers in this image set.
+    image_fetcher : TileFetcher
+        Instance of TileFetcher that provides the data for the tile.
+    default_shape : Optional[Tuple[int, int]]
+        Default shape of the individual tiles in this image set.
+    axes_order : Sequence[Axes]
+        Ordering for which axes vary, in order of the slowest changing axis to the fastest.  For
+        instance, if the order is (ROUND, Z, CH) and each dimension has size 2, then the sequence
+        is:
+          (ROUND=0, CH=0, Z=0)
+          (ROUND=0, CH=1, Z=0)
+          (ROUND=0, CH=0, Z=1)
+          (ROUND=0, CH=1, Z=1)
+          (ROUND=1, CH=0, Z=0)
+          (ROUND=1, CH=1, Z=0)
+          (ROUND=1, CH=0, Z=1)
+          (ROUND=1, CH=1, Z=1)
+        (default = (Axes.Z, Axes.ROUND, Axes.CH))
+
+    Returns
+    -------
+    The slicedimage collection representing the image.
+    """
+    axes_sizes = join_axes_labels(
+        axes_order, rounds=rounds, chs=chs, zplanes=zplanes)
+    tile_coordinates = [
+        TileIdentifier(fov_id, selector[Axes.ROUND], selector[Axes.CH], selector[Axes.ZPLANE])
+        for fov_id in fovs
+        for selector in ordered_iterator(axes_sizes)
+    ]
+
+    return build_irregular_image(tile_coordinates, image_fetcher, default_shape)
+
+
+def write_irregular_experiment_json(
         path: str,
-        fov_count: int,
         tile_format: ImageFormat,
         *,
-        primary_image_dimension_labels: Mapping[Union[str, Axes], Sequence[int]],
-        aux_name_to_dimension_labels: Mapping[str, Mapping[Union[str, Axes], Sequence[int]]],
-        primary_tile_fetcher: Optional[TileFetcher]=None,
-        aux_tile_fetcher: Optional[Mapping[str, TileFetcher]]=None,
+        image_tile_coordinates: Mapping[str, Iterable[TileIdentifier]],
+        tile_fetchers: Mapping[str, TileFetcher],
         postprocess_func: Optional[Callable[[dict], dict]]=None,
         default_shape: Optional[Mapping[Axes, int]]=None,
-        dimension_order: Sequence[Axes]=(Axes.ZPLANE, Axes.ROUND, Axes.CH),
         fov_path_generator: Callable[[Path, str], Path] = _fov_path_generator,
         tile_opener: Callable[[Path, Tile, str], BinaryIO] = _tile_opener,
 ) -> None:
@@ -166,51 +225,23 @@ def write_labeled_experiment_json(
     ----------
     path : str
         Directory to write the files to.
-    fov_count : int
-        Number of fields of view in this experiment.
     tile_format : ImageFormat
         File format to write the tiles as.
-    primary_image_dimension_labels : Mapping[Union[str, Axes], Sequence[int]]
-        Dictionary mapping dimension name to dimension labels for the primary image.
-    aux_name_to_dimension_labels : Mapping[str, Mapping[Union[str, Axes], Sequence[int]]]
-        Dictionary mapping the auxiliary image type to dictionaries, which map from dimension name
-        to dimension labels.
-    primary_tile_fetcher : Optional[TileFetcher]
-        TileFetcher for primary images.  Set this if you want specific image data to be set for the
-        primary images.  If not provided, the image data is set to random noise via
-        :class:`RandomNoiseTileFetcher`.
-    aux_tile_fetcher : Optional[Mapping[str, TileFetcher]]
-        TileFetchers for auxiliary images.  Set this if you want specific image data to be set for
-        one or more aux image types.  If not provided for any given aux image, the image data is
-        set to random noise via :class:`RandomNoiseTileFetcher`.
+    image_tile_coordinates : Mapping[str, Iterable[TileIdentifier]]
+        Dictionary mapping the image type to an iterable of TileCoordinates.
+    tile_fetchers : Mapping[str, TileFetcher]
+        Dictionary mapping the image type to a TileFetcher.
     postprocess_func : Optional[Callable[[dict], dict]]
         If provided, this is called with the experiment document for any postprocessing.
         An example of this would be to add something to one of the top-level extras field.
         The callable should return what is to be written as the experiment document.
     default_shape : Optional[Tuple[int, int]] (default = None)
         Default shape for the tiles in this experiment.
-    dimension_order : Sequence[Axes]
-        Ordering for which dimensions vary, in order of the slowest changing dimension to the
-        fastest.  For instance, if the order is (ROUND, Z, CH) and each dimension has labels (0, 1),
-        then the sequence is:
-          (ROUND=0, CH=0, Z=0)
-          (ROUND=0, CH=1, Z=0)
-          (ROUND=0, CH=0, Z=1)
-          (ROUND=0, CH=1, Z=1)
-          (ROUND=1, CH=0, Z=0)
-          (ROUND=1, CH=1, Z=0)
-          (ROUND=1, CH=0, Z=1)
-          (ROUND=1, CH=1, Z=1)
-        (default = (Axes.Z, Axes.ROUND, Axes.CH))
     fov_path_generator : Callable[[Path, str], Path]
         Generates the path for a FOV's json file.  If one is not provided, the default generates
         the FOV's json file at the same level as the top-level json file for an image.
     tile_opener : Callable[[Path, Tile, str], BinaryIO]
     """
-    if primary_tile_fetcher is None:
-        primary_tile_fetcher = tile_fetcher_factory(RandomNoiseTile)
-    if aux_tile_fetcher is None:
-        aux_tile_fetcher = {}
     if postprocess_func is None:
         postprocess_func = lambda doc: doc
 
@@ -219,44 +250,20 @@ def write_labeled_experiment_json(
         'images': {},
         'extras': {},
     }
-    primary_image = build_image(
-        range(fov_count),
-        primary_image_dimension_labels[Axes.ROUND],
-        primary_image_dimension_labels[Axes.CH],
-        primary_image_dimension_labels[Axes.ZPLANE],
-        primary_tile_fetcher,
-        axes_order=dimension_order,
-        default_shape=default_shape,
-    )
-    Writer.write_to_path(
-        primary_image,
-        os.path.join(path, "primary_images.json"),
-        pretty=True,
-        partition_path_generator=fov_path_generator,
-        tile_opener=tile_opener,
-        tile_format=tile_format,
-    )
-    experiment_doc['images']['primary'] = "primary_images.json"
-
-    for aux_name, aux_dimension_labels in aux_name_to_dimension_labels.items():
-        auxiliary_image = build_image(
-            range(fov_count),
-            aux_dimension_labels[Axes.ROUND],
-            aux_dimension_labels[Axes.CH],
-            aux_dimension_labels[Axes.ZPLANE],
-            aux_tile_fetcher.get(aux_name, tile_fetcher_factory(RandomNoiseTile)),
-            axes_order=dimension_order,
-            default_shape=default_shape,
-        )
+    for image_type, tile_coordinates in image_tile_coordinates.items():
+        tile_fetcher = tile_fetchers[image_type]
+
+        image = build_irregular_image(tile_coordinates, tile_fetcher, default_shape)
+
         Writer.write_to_path(
-            auxiliary_image,
-            os.path.join(path, "{}.json".format(aux_name)),
+            image,
+            os.path.join(path, f"{image_type}.json"),
             pretty=True,
             partition_path_generator=fov_path_generator,
             tile_opener=tile_opener,
             tile_format=tile_format,
         )
-        experiment_doc['images'][aux_name] = "{}.json".format(aux_name)
+        experiment_doc['images'][image_type] = f"{image_type}.json"
 
     experiment_doc["codebook"] = "codebook.json"
     codebook_array = [
@@ -340,29 +347,41 @@ def write_experiment_json(
         the FOV's json file at the same level as the top-level json file for an image.
     tile_opener : Callable[[Path, Tile, str], BinaryIO]
     """
-    primary_image_dimension_labels: Mapping[Union[str, Axes], Sequence[int]] = {
-        primary_image_dimension_name: range(primary_image_dimension_cardinality)
-        for primary_image_dimension_name, primary_image_dimension_cardinality in
-        primary_image_dimensions.items()
-    }
-    aux_name_to_dimension_labels: Mapping[str, Mapping[Union[str, Axes], Sequence[int]]] = {
-        aux_name: {
-            aux_dimension_name: range(aux_dimension_cardinality)
-            for aux_dimension_name, aux_dimension_cardinality in
-            dimension_name_to_cardinality.items()
-        }
-        for aux_name, dimension_name_to_cardinality in aux_name_to_dimensions.items()
-    }
+    all_tile_fetcher: MutableMapping[str, TileFetcher] = {}
+    if aux_tile_fetcher is not None:
+        all_tile_fetcher.update(aux_tile_fetcher)
+    if primary_tile_fetcher is not None:
+        all_tile_fetcher[FieldOfView.PRIMARY_IMAGES] = primary_tile_fetcher
+
+    image_tile_coordinates: MutableMapping[str, Iterable[TileIdentifier]] = dict()
+    image_tile_fetchers: MutableMapping[str, TileFetcher] = dict()
+
+    dimension_cardinality_of_images = {FieldOfView.PRIMARY_IMAGES: primary_image_dimensions}
+    for image_type, image_dimension in aux_name_to_dimensions.items():
+        dimension_cardinality_of_images[image_type] = image_dimension
+
+    for image_type, dimension_cardinality_of_image in dimension_cardinality_of_images.items():
+        axes_sizes = join_axes_labels(
+            dimension_order,
+            rounds=range(dimension_cardinality_of_image[Axes.ROUND]),
+            chs=range(dimension_cardinality_of_image[Axes.CH]),
+            zplanes=range(dimension_cardinality_of_image[Axes.ZPLANE]),
+        )
+        image_tile_coordinates[image_type] = [
+            TileIdentifier(fov_id, selector[Axes.ROUND], selector[Axes.CH], selector[Axes.ZPLANE])
+            for fov_id in range(fov_count)
+            for selector in ordered_iterator(axes_sizes)
+        ]
+
+        image_tile_fetchers[image_type] = all_tile_fetcher.get(
+            image_type, tile_fetcher_factory(RandomNoiseTile))
 
-    return write_labeled_experiment_json(
-        path, fov_count, tile_format,
-        primary_image_dimension_labels=primary_image_dimension_labels,
-        aux_name_to_dimension_labels=aux_name_to_dimension_labels,
-        primary_tile_fetcher=primary_tile_fetcher,
-        aux_tile_fetcher=aux_tile_fetcher,
+    return write_irregular_experiment_json(
+        path, tile_format,
+        image_tile_coordinates=image_tile_coordinates,
+        tile_fetchers=image_tile_fetchers,
         postprocess_func=postprocess_func,
         default_shape=default_shape,
-        dimension_order=dimension_order,
         fov_path_generator=fov_path_generator,
         tile_opener=tile_opener,
     )
diff --git a/starfish/core/experiment/builder/cli.py b/starfish/core/experiment/builder/cli.py
index b474dc4d..82bf71bf 100644
--- a/starfish/core/experiment/builder/cli.py
+++ b/starfish/core/experiment/builder/cli.py
@@ -4,7 +4,13 @@ from slicedimage import ImageFormat
 
 from starfish.core.types import Axes
 from starfish.core.util import click
-from . import AUX_IMAGE_NAMES, write_experiment_json
+from . import write_experiment_json
+
+
+AUX_IMAGE_NAMES = {
+    'nuclei',
+    'dots',
+}
 
 
 class StarfishIndex(click.ParamType):
diff --git a/starfish/experiment/builder.py b/starfish/experiment/builder.py
index 2165eb1e..a22c687f 100644
--- a/starfish/experiment/builder.py
+++ b/starfish/experiment/builder.py
@@ -1,6 +1,9 @@
 from starfish.core.experiment.builder import (   # noqa: F401
     build_image,
+    build_irregular_image,
     inplace,
+    TileIdentifier,
     write_experiment_json,
+    write_irregular_experiment_json,
 )
 from starfish.core.experiment.builder.providers import FetchedTile, TileFetcher  # noqa: F401


Test patch:
diff --git a/starfish/core/experiment/builder/test/test_build.py b/starfish/core/experiment/builder/test/test_build.py
index b32b9c8a..0d8a56ef 100644
--- a/starfish/core/experiment/builder/test/test_build.py
+++ b/starfish/core/experiment/builder/test/test_build.py
@@ -33,11 +33,11 @@ class TestWithBuildData(unittest.TestCase):
         ],
         [
             "starfish", "validate", "manifest",
-            lambda tempdir, *args, **kwargs: os.sep.join([tempdir, "primary_images.json"])
+            lambda tempdir, *args, **kwargs: os.sep.join([tempdir, "primary.json"])
         ],
         [
             "starfish", "validate", "fov",
-            lambda tempdir, *args, **kwargs: os.sep.join([tempdir, "primary_images-fov_000.json"])
+            lambda tempdir, *args, **kwargs: os.sep.join([tempdir, "primary-fov_000.json"])
         ],
     )