Repository: spacetx/starfish

Issue:
ElementWiseMult replaces ImageStack.xarray with np.ndarray
This line is replacing the xarray with a numpy array. 

https://github.com/spacetx/starfish/blob/master/starfish/image/_filter/element_wise_mult.py#L81

Discussion:
codecov-io: # [Codecov](https://codecov.io/gh/spacetx/starfish/pull/1393?src=pr&el=h1) Report
> Merging [#1393](https://codecov.io/gh/spacetx/starfish/pull/1393?src=pr&el=desc) into [master](https://codecov.io/gh/spacetx/starfish/commit/a210e1560029aaa4009004c6319fb4248b5952f9?src=pr&el=desc) will **decrease** coverage by `20.68%`.
> The diff coverage is `0%`.

[![Impacted file tree graph](https://codecov.io/gh/spacetx/starfish/pull/1393/graphs/tree.svg?width=650&token=kliQHejt6T&height=150&src=pr)](https://codecov.io/gh/spacetx/starfish/pull/1393?src=pr&el=tree)

```diff
@@             Coverage Diff             @@
##           master    #1393       +/-   ##
===========================================
- Coverage   89.47%   68.79%   -20.69%     
===========================================
  Files         152      152               
  Lines        5494     5493        -1     
===========================================
- Hits         4916     3779     -1137     
- Misses        578     1714     +1136
```


| [Impacted Files](https://codecov.io/gh/spacetx/starfish/pull/1393?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [starfish/core/image/\_filter/element\_wise\_mult.py](https://codecov.io/gh/spacetx/starfish/pull/1393/diff?src=pr&el=tree#diff-c3RhcmZpc2gvY29yZS9pbWFnZS9fZmlsdGVyL2VsZW1lbnRfd2lzZV9tdWx0LnB5) | `55.17% <0%> (-34.83%)` | :arrow_down: |
| [starfish/core/experiment/builder/inplace.py](https://codecov.io/gh/spacetx/starfish/pull/1393/diff?src=pr&el=tree#diff-c3RhcmZpc2gvY29yZS9leHBlcmltZW50L2J1aWxkZXIvaW5wbGFjZS5weQ==) | `0% <0%> (-87.88%)` | :arrow_down: |
| [starfish/test/full\_pipelines/api/test\_dartfish.py](https://codecov.io/gh/spacetx/starfish/pull/1393/diff?src=pr&el=tree#diff-c3RhcmZpc2gvdGVzdC9mdWxsX3BpcGVsaW5lcy9hcGkvdGVzdF9kYXJ0ZmlzaC5weQ==) | `20% <0%> (-80%)` | :arrow_down: |
| [starfish/test/full\_pipelines/api/test\_iss\_api.py](https://codecov.io/gh/spacetx/starfish/pull/1393/diff?src=pr&el=tree#diff-c3RhcmZpc2gvdGVzdC9mdWxsX3BpcGVsaW5lcy9hcGkvdGVzdF9pc3NfYXBpLnB5) | `23.52% <0%> (-76.48%)` | :arrow_down: |
| [starfish/test/full\_pipelines/api/test\_merfish.py](https://codecov.io/gh/spacetx/starfish/pull/1393/diff?src=pr&el=tree#diff-c3RhcmZpc2gvdGVzdC9mdWxsX3BpcGVsaW5lcy9hcGkvdGVzdF9tZXJmaXNoLnB5) | `24.39% <0%> (-75.61%)` | :arrow_down: |
| [...fish/core/imagestack/test/factories/all\_purpose.py](https://codecov.io/gh/spacetx/starfish/pull/1393/diff?src=pr&el=tree#diff-c3RhcmZpc2gvY29yZS9pbWFnZXN0YWNrL3Rlc3QvZmFjdG9yaWVzL2FsbF9wdXJwb3NlLnB5) | `24.44% <0%> (-75.56%)` | :arrow_down: |
| [starfish/core/spacetx\_format/test\_fuzz.py](https://codecov.io/gh/spacetx/starfish/pull/1393/diff?src=pr&el=tree#diff-c3RhcmZpc2gvY29yZS9zcGFjZXR4X2Zvcm1hdC90ZXN0X2Z1enoucHk=) | `31.03% <0%> (-68.97%)` | :arrow_down: |
| [starfish/core/experiment/builder/\_\_init\_\_.py](https://codecov.io/gh/spacetx/starfish/pull/1393/diff?src=pr&el=tree#diff-c3RhcmZpc2gvY29yZS9leHBlcmltZW50L2J1aWxkZXIvX19pbml0X18ucHk=) | `31.03% <0%> (-68.97%)` | :arrow_down: |
| [starfish/core/intensity\_table/overlap.py](https://codecov.io/gh/spacetx/starfish/pull/1393/diff?src=pr&el=tree#diff-c3RhcmZpc2gvY29yZS9pbnRlbnNpdHlfdGFibGUvb3ZlcmxhcC5weQ==) | `32.69% <0%> (-63.47%)` | :arrow_down: |
| [.../spots/\_detect\_spots/local\_search\_blob\_detector.py](https://codecov.io/gh/spacetx/starfish/pull/1393/diff?src=pr&el=tree#diff-c3RhcmZpc2gvY29yZS9zcG90cy9fZGV0ZWN0X3Nwb3RzL2xvY2FsX3NlYXJjaF9ibG9iX2RldGVjdG9yLnB5) | `32.2% <0%> (-61.87%)` | :arrow_down: |
| ... and [68 more](https://codecov.io/gh/spacetx/starfish/pull/1393/diff?src=pr&el=tree-more) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/spacetx/starfish/pull/1393?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/spacetx/starfish/pull/1393?src=pr&el=footer). Last update [a210e15...b7524ab](https://codecov.io/gh/spacetx/starfish/pull/1393?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


Patch:
diff --git a/README.rst b/README.rst
index 8957704b..8e61dda8 100644
--- a/README.rst
+++ b/README.rst
@@ -1,8 +1,8 @@
 Starfish
 ========
 
-.. image:: https://travis-ci.org/spacetx/starfish.svg?branch=master
-    :target: https://travis-ci.org/spacetx/starfish
+.. image:: https://travis-ci.com/spacetx/starfish.svg?branch=master
+    :target: https://travis-ci.com/spacetx/starfish
     :width: 20%
 .. image:: https://codecov.io/gh/spacetx/starfish/branch/master/graph/badge.svg
     :target: https://codecov.io/gh/spacetx/starfish
@@ -37,7 +37,7 @@ to see starfish in action on particular assay types. Or, browse our our notebook
  Assay                Loads Data  Single-FoV Pipeline  Multi-FoV Pipeline
 --------------------  ----------  -------------------  ------------------
  MERFISH              [x]         [x] mer_             in process
- ISS                  [x]         [x] iss_             in process
+ ISS                  [x]         [x] iss_             [x]
  osmFISH              [x]         [x] osm_             [ ]
  smFISH               [x]         [x] 3ds_             [x]
  BaristaSeq           [x]         [x] bar_             [ ]
@@ -55,7 +55,7 @@ to see starfish in action on particular assay types. Or, browse our our notebook
 .. _dar: https://github.com/spacetx/starfish/blob/master/notebooks/DARTFISH.ipynb
 .. _str: https://github.com/spacetx/starfish/blob/master/notebooks/STARmap.ipynb
 .. _seq: https://github.com/spacetx/starfish/blob/master/notebooks/SeqFISH.ipynb
-.. _3ds: https://github.com/spacetx/starfish/blob/master/notebooks/3d_smFISH.ipynb
+.. _3ds: https://github.com/spacetx/starfish/blob/master/notebooks/smFISH.ipynb
 
 Contributing
 ------------
diff --git a/REQUIREMENTS.txt b/REQUIREMENTS.txt
index 814bf02d..009b4e6b 100644
--- a/REQUIREMENTS.txt
+++ b/REQUIREMENTS.txt
@@ -1,4 +1,5 @@
 click
+dataclasses;python_version<"3.7"
 jsonschema
 matplotlib
 numpy != 1.13.0
diff --git a/notebooks/BaristaSeq.ipynb b/notebooks/BaristaSeq.ipynb
index 2d2ea0a8..48175405 100644
--- a/notebooks/BaristaSeq.ipynb
+++ b/notebooks/BaristaSeq.ipynb
@@ -454,7 +454,7 @@
     "    *np.unique(pixel_decoded['target'], return_counts=True)[::-1]\n",
     ")\n",
     "\n",
-    "print(pixel_decoded_gene_counts.sort_values(ascending=False).drop(\"nan\")[:20])"
+    "print(pixel_decoded_gene_counts.sort_values(ascending=False)[:20])"
    ]
   }
  ],
@@ -479,4 +479,4 @@
  },
  "nbformat": 4,
  "nbformat_minor": 2
-}
\ No newline at end of file
+}
diff --git a/notebooks/py/BaristaSeq.py b/notebooks/py/BaristaSeq.py
index c8dcc13c..2961ebba 100644
--- a/notebooks/py/BaristaSeq.py
+++ b/notebooks/py/BaristaSeq.py
@@ -329,5 +329,5 @@ pixel_decoded_gene_counts = pd.Series(
     *np.unique(pixel_decoded['target'], return_counts=True)[::-1]
 )
 
-print(pixel_decoded_gene_counts.sort_values(ascending=False).drop("nan")[:20])
+print(pixel_decoded_gene_counts.sort_values(ascending=False)[:20])
 # EPY: END code
diff --git a/starfish/core/experiment/builder/__init__.py b/starfish/core/experiment/builder/__init__.py
index 28347624..bb7b6125 100644
--- a/starfish/core/experiment/builder/__init__.py
+++ b/starfish/core/experiment/builder/__init__.py
@@ -1,12 +1,18 @@
+import functools
 import json
 import os
+from dataclasses import astuple, dataclass
 from pathlib import Path
 from typing import (
     Any,
     BinaryIO,
     Callable,
     Dict,
+    Iterable,
     Mapping,
+    MutableMapping,
+    MutableSequence,
+    MutableSet,
     Optional,
     Sequence,
     Union,
@@ -20,6 +26,7 @@ from slicedimage import (
     Writer,
 )
 
+from starfish import FieldOfView
 from starfish.core.codebook.codebook import Codebook
 from starfish.core.experiment.builder.orderediterator import join_axes_labels, ordered_iterator
 from starfish.core.experiment.version import CURRENT_VERSION
@@ -28,13 +35,19 @@ from .defaultproviders import RandomNoiseTile, tile_fetcher_factory
 from .providers import FetchedTile, TileFetcher
 
 
-AUX_IMAGE_NAMES = {
-    'nuclei',
-    'dots',
-}
 DEFAULT_DIMENSION_ORDER = (Axes.ZPLANE, Axes.ROUND, Axes.CH)
 
 
+@dataclass
+class TileIdentifier:
+    """Data class for encapsulating the location of a tile in a 6D tensor (fov, round, ch, zplane,
+    y, and x)."""
+    fov_id: int
+    round_id: int
+    ch_id: int
+    zplane_id: int
+
+
 def _tile_opener(toc_path: Path, tile: Tile, file_ext: str) -> BinaryIO:
     base = toc_path.parent / toc_path.stem
     return open(
@@ -52,55 +65,45 @@ def _fov_path_generator(parent_toc_path: Path, toc_name: str) -> Path:
     return parent_toc_path.parent / "{}-{}.json".format(parent_toc_path.stem, toc_name)
 
 
-def build_image(
-        fovs: Sequence[int],
-        rounds: Sequence[int],
-        chs: Sequence[int],
-        zplanes: Sequence[int],
+def build_irregular_image(
+        tile_coordinates: Iterable[TileIdentifier],
         image_fetcher: TileFetcher,
-        default_shape: Optional[Mapping[Axes, int]]=None,
-        axes_order: Sequence[Axes]=DEFAULT_DIMENSION_ORDER,
+        default_shape: Optional[Mapping[Axes, int]] = None,
 ) -> Collection:
     """
-    Build and returns an image set with the following characteristics:
+    Build and returns an image set that can potentially be irregular (i.e., the cardinality of the
+    dimensions are not always consistent).  It can also build a regular image.
 
     Parameters
     ----------
-    fovs : Sequence[int]
-        Sequence of field of view ids in this image set.
-    rounds : Sequence[int]
-        Sequence of the round numbers in this image set.
-    chs : Sequence[int]
-        Sequence of the ch numbers in this image set.
-    zplanes : Sequence[int]
-        Sequence of the zplane numbers in this image set.
+    tile_coordinates : Iterable[TileIdentifier]
+        Iterable of all the TileCoordinates that are valid in the image.
     image_fetcher : TileFetcher
         Instance of TileFetcher that provides the data for the tile.
     default_shape : Optional[Tuple[int, int]]
         Default shape of the individual tiles in this image set.
-    axes_order : Sequence[Axes]
-        Ordering for which axes vary, in order of the slowest changing axis to the fastest.  For
-        instance, if the order is (ROUND, Z, CH) and each dimension has size 2, then the sequence
-        is:
-          (ROUND=0, CH=0, Z=0)
-          (ROUND=0, CH=1, Z=0)
-          (ROUND=0, CH=0, Z=1)
-          (ROUND=0, CH=1, Z=1)
-          (ROUND=1, CH=0, Z=0)
-          (ROUND=1, CH=1, Z=0)
-          (ROUND=1, CH=0, Z=1)
-          (ROUND=1, CH=1, Z=1)
-        (default = (Axes.Z, Axes.ROUND, Axes.CH))
 
     Returns
     -------
     The slicedimage collection representing the image.
     """
-    axes_sizes = join_axes_labels(
-        axes_order, rounds=rounds, chs=chs, zplanes=zplanes)
+    def reducer_to_sets(
+            accumulated: Sequence[MutableSet[int]], update: TileIdentifier,
+    ) -> Sequence[MutableSet[int]]:
+        """Reduces to a list of sets of tile coordinates, in the order of FOV, round, ch, and
+        zplane."""
+        result: MutableSequence[MutableSet[int]] = list()
+        for accumulated_elem, update_elem in zip(accumulated, astuple(update)):
+            accumulated_elem.add(update_elem)
+            result.append(accumulated_elem)
+        return result
+    initial_value: Sequence[MutableSet[int]] = tuple(set() for _ in range(4))
+
+    fovs, rounds, chs, zplanes = functools.reduce(
+        reducer_to_sets, tile_coordinates, initial_value)
 
     collection = Collection()
-    for fov_id in fovs:
+    for expected_fov in fovs:
         fov_images = TileSet(
             [
                 Coordinates.X,
@@ -117,18 +120,23 @@ def build_image(
             ImageFormat.TIFF,
         )
 
-        for selector in ordered_iterator(axes_sizes):
+        for tile_coordinate in tile_coordinates:
+            current_fov, current_round, current_ch, current_zplane = astuple(tile_coordinate)
+            # filter out the fovs that are not the one we are currently processing
+            if expected_fov != current_fov:
+                continue
             image = image_fetcher.get_tile(
-                fov_id,
-                selector[Axes.ROUND],
-                selector[Axes.CH],
-                selector[Axes.ZPLANE])
+                current_fov,
+                current_round,
+                current_ch,
+                current_zplane
+            )
             tile = Tile(
                 image.coordinates,
                 {
-                    Axes.ZPLANE: (selector[Axes.ZPLANE]),
-                    Axes.ROUND: (selector[Axes.ROUND]),
-                    Axes.CH: (selector[Axes.CH]),
+                    Axes.ZPLANE: current_zplane,
+                    Axes.ROUND: current_round,
+                    Axes.CH: current_ch,
                 },
                 image.shape,
                 extras=image.extras,
@@ -140,22 +148,73 @@ def build_image(
             # the checksum of the original file.
             tile.provider = image
             fov_images.add_tile(tile)
-        collection.add_partition("fov_{:03}".format(fov_id), fov_images)
+        collection.add_partition("fov_{:03}".format(expected_fov), fov_images)
     return collection
 
 
-def write_labeled_experiment_json(
+def build_image(
+        fovs: Sequence[int],
+        rounds: Sequence[int],
+        chs: Sequence[int],
+        zplanes: Sequence[int],
+        image_fetcher: TileFetcher,
+        default_shape: Optional[Mapping[Axes, int]] = None,
+        axes_order: Sequence[Axes] = DEFAULT_DIMENSION_ORDER,
+) -> Collection:
+    """
+    Build and returns an image set with the following characteristics:
+
+    Parameters
+    ----------
+    fovs : Sequence[int]
+        Sequence of field of view ids in this image set.
+    rounds : Sequence[int]
+        Sequence of the round numbers in this image set.
+    chs : Sequence[int]
+        Sequence of the ch numbers in this image set.
+    zplanes : Sequence[int]
+        Sequence of the zplane numbers in this image set.
+    image_fetcher : TileFetcher
+        Instance of TileFetcher that provides the data for the tile.
+    default_shape : Optional[Tuple[int, int]]
+        Default shape of the individual tiles in this image set.
+    axes_order : Sequence[Axes]
+        Ordering for which axes vary, in order of the slowest changing axis to the fastest.  For
+        instance, if the order is (ROUND, Z, CH) and each dimension has size 2, then the sequence
+        is:
+          (ROUND=0, CH=0, Z=0)
+          (ROUND=0, CH=1, Z=0)
+          (ROUND=0, CH=0, Z=1)
+          (ROUND=0, CH=1, Z=1)
+          (ROUND=1, CH=0, Z=0)
+          (ROUND=1, CH=1, Z=0)
+          (ROUND=1, CH=0, Z=1)
+          (ROUND=1, CH=1, Z=1)
+        (default = (Axes.Z, Axes.ROUND, Axes.CH))
+
+    Returns
+    -------
+    The slicedimage collection representing the image.
+    """
+    axes_sizes = join_axes_labels(
+        axes_order, rounds=rounds, chs=chs, zplanes=zplanes)
+    tile_coordinates = [
+        TileIdentifier(fov_id, selector[Axes.ROUND], selector[Axes.CH], selector[Axes.ZPLANE])
+        for fov_id in fovs
+        for selector in ordered_iterator(axes_sizes)
+    ]
+
+    return build_irregular_image(tile_coordinates, image_fetcher, default_shape)
+
+
+def write_irregular_experiment_json(
         path: str,
-        fov_count: int,
         tile_format: ImageFormat,
         *,
-        primary_image_dimension_labels: Mapping[Union[str, Axes], Sequence[int]],
-        aux_name_to_dimension_labels: Mapping[str, Mapping[Union[str, Axes], Sequence[int]]],
-        primary_tile_fetcher: Optional[TileFetcher]=None,
-        aux_tile_fetcher: Optional[Mapping[str, TileFetcher]]=None,
+        image_tile_coordinates: Mapping[str, Iterable[TileIdentifier]],
+        tile_fetchers: Mapping[str, TileFetcher],
         postprocess_func: Optional[Callable[[dict], dict]]=None,
         default_shape: Optional[Mapping[Axes, int]]=None,
-        dimension_order: Sequence[Axes]=(Axes.ZPLANE, Axes.ROUND, Axes.CH),
         fov_path_generator: Callable[[Path, str], Path] = _fov_path_generator,
         tile_opener: Callable[[Path, Tile, str], BinaryIO] = _tile_opener,
 ) -> None:
@@ -166,51 +225,23 @@ def write_labeled_experiment_json(
     ----------
     path : str
         Directory to write the files to.
-    fov_count : int
-        Number of fields of view in this experiment.
     tile_format : ImageFormat
         File format to write the tiles as.
-    primary_image_dimension_labels : Mapping[Union[str, Axes], Sequence[int]]
-        Dictionary mapping dimension name to dimension labels for the primary image.
-    aux_name_to_dimension_labels : Mapping[str, Mapping[Union[str, Axes], Sequence[int]]]
-        Dictionary mapping the auxiliary image type to dictionaries, which map from dimension name
-        to dimension labels.
-    primary_tile_fetcher : Optional[TileFetcher]
-        TileFetcher for primary images.  Set this if you want specific image data to be set for the
-        primary images.  If not provided, the image data is set to random noise via
-        :class:`RandomNoiseTileFetcher`.
-    aux_tile_fetcher : Optional[Mapping[str, TileFetcher]]
-        TileFetchers for auxiliary images.  Set this if you want specific image data to be set for
-        one or more aux image types.  If not provided for any given aux image, the image data is
-        set to random noise via :class:`RandomNoiseTileFetcher`.
+    image_tile_coordinates : Mapping[str, Iterable[TileIdentifier]]
+        Dictionary mapping the image type to an iterable of TileCoordinates.
+    tile_fetchers : Mapping[str, TileFetcher]
+        Dictionary mapping the image type to a TileFetcher.
     postprocess_func : Optional[Callable[[dict], dict]]
         If provided, this is called with the experiment document for any postprocessing.
         An example of this would be to add something to one of the top-level extras field.
         The callable should return what is to be written as the experiment document.
     default_shape : Optional[Tuple[int, int]] (default = None)
         Default shape for the tiles in this experiment.
-    dimension_order : Sequence[Axes]
-        Ordering for which dimensions vary, in order of the slowest changing dimension to the
-        fastest.  For instance, if the order is (ROUND, Z, CH) and each dimension has labels (0, 1),
-        then the sequence is:
-          (ROUND=0, CH=0, Z=0)
-          (ROUND=0, CH=1, Z=0)
-          (ROUND=0, CH=0, Z=1)
-          (ROUND=0, CH=1, Z=1)
-          (ROUND=1, CH=0, Z=0)
-          (ROUND=1, CH=1, Z=0)
-          (ROUND=1, CH=0, Z=1)
-          (ROUND=1, CH=1, Z=1)
-        (default = (Axes.Z, Axes.ROUND, Axes.CH))
     fov_path_generator : Callable[[Path, str], Path]
         Generates the path for a FOV's json file.  If one is not provided, the default generates
         the FOV's json file at the same level as the top-level json file for an image.
     tile_opener : Callable[[Path, Tile, str], BinaryIO]
     """
-    if primary_tile_fetcher is None:
-        primary_tile_fetcher = tile_fetcher_factory(RandomNoiseTile)
-    if aux_tile_fetcher is None:
-        aux_tile_fetcher = {}
     if postprocess_func is None:
         postprocess_func = lambda doc: doc
 
@@ -219,44 +250,20 @@ def write_labeled_experiment_json(
         'images': {},
         'extras': {},
     }
-    primary_image = build_image(
-        range(fov_count),
-        primary_image_dimension_labels[Axes.ROUND],
-        primary_image_dimension_labels[Axes.CH],
-        primary_image_dimension_labels[Axes.ZPLANE],
-        primary_tile_fetcher,
-        axes_order=dimension_order,
-        default_shape=default_shape,
-    )
-    Writer.write_to_path(
-        primary_image,
-        os.path.join(path, "primary_images.json"),
-        pretty=True,
-        partition_path_generator=fov_path_generator,
-        tile_opener=tile_opener,
-        tile_format=tile_format,
-    )
-    experiment_doc['images']['primary'] = "primary_images.json"
-
-    for aux_name, aux_dimension_labels in aux_name_to_dimension_labels.items():
-        auxiliary_image = build_image(
-            range(fov_count),
-            aux_dimension_labels[Axes.ROUND],
-            aux_dimension_labels[Axes.CH],
-            aux_dimension_labels[Axes.ZPLANE],
-            aux_tile_fetcher.get(aux_name, tile_fetcher_factory(RandomNoiseTile)),
-            axes_order=dimension_order,
-            default_shape=default_shape,
-        )
+    for image_type, tile_coordinates in image_tile_coordinates.items():
+        tile_fetcher = tile_fetchers[image_type]
+
+        image = build_irregular_image(tile_coordinates, tile_fetcher, default_shape)
+
         Writer.write_to_path(
-            auxiliary_image,
-            os.path.join(path, "{}.json".format(aux_name)),
+            image,
+            os.path.join(path, f"{image_type}.json"),
             pretty=True,
             partition_path_generator=fov_path_generator,
             tile_opener=tile_opener,
             tile_format=tile_format,
         )
-        experiment_doc['images'][aux_name] = "{}.json".format(aux_name)
+        experiment_doc['images'][image_type] = f"{image_type}.json"
 
     experiment_doc["codebook"] = "codebook.json"
     codebook_array = [
@@ -340,29 +347,41 @@ def write_experiment_json(
         the FOV's json file at the same level as the top-level json file for an image.
     tile_opener : Callable[[Path, Tile, str], BinaryIO]
     """
-    primary_image_dimension_labels: Mapping[Union[str, Axes], Sequence[int]] = {
-        primary_image_dimension_name: range(primary_image_dimension_cardinality)
-        for primary_image_dimension_name, primary_image_dimension_cardinality in
-        primary_image_dimensions.items()
-    }
-    aux_name_to_dimension_labels: Mapping[str, Mapping[Union[str, Axes], Sequence[int]]] = {
-        aux_name: {
-            aux_dimension_name: range(aux_dimension_cardinality)
-            for aux_dimension_name, aux_dimension_cardinality in
-            dimension_name_to_cardinality.items()
-        }
-        for aux_name, dimension_name_to_cardinality in aux_name_to_dimensions.items()
-    }
+    all_tile_fetcher: MutableMapping[str, TileFetcher] = {}
+    if aux_tile_fetcher is not None:
+        all_tile_fetcher.update(aux_tile_fetcher)
+    if primary_tile_fetcher is not None:
+        all_tile_fetcher[FieldOfView.PRIMARY_IMAGES] = primary_tile_fetcher
+
+    image_tile_coordinates: MutableMapping[str, Iterable[TileIdentifier]] = dict()
+    image_tile_fetchers: MutableMapping[str, TileFetcher] = dict()
+
+    dimension_cardinality_of_images = {FieldOfView.PRIMARY_IMAGES: primary_image_dimensions}
+    for image_type, image_dimension in aux_name_to_dimensions.items():
+        dimension_cardinality_of_images[image_type] = image_dimension
+
+    for image_type, dimension_cardinality_of_image in dimension_cardinality_of_images.items():
+        axes_sizes = join_axes_labels(
+            dimension_order,
+            rounds=range(dimension_cardinality_of_image[Axes.ROUND]),
+            chs=range(dimension_cardinality_of_image[Axes.CH]),
+            zplanes=range(dimension_cardinality_of_image[Axes.ZPLANE]),
+        )
+        image_tile_coordinates[image_type] = [
+            TileIdentifier(fov_id, selector[Axes.ROUND], selector[Axes.CH], selector[Axes.ZPLANE])
+            for fov_id in range(fov_count)
+            for selector in ordered_iterator(axes_sizes)
+        ]
+
+        image_tile_fetchers[image_type] = all_tile_fetcher.get(
+            image_type, tile_fetcher_factory(RandomNoiseTile))
 
-    return write_labeled_experiment_json(
-        path, fov_count, tile_format,
-        primary_image_dimension_labels=primary_image_dimension_labels,
-        aux_name_to_dimension_labels=aux_name_to_dimension_labels,
-        primary_tile_fetcher=primary_tile_fetcher,
-        aux_tile_fetcher=aux_tile_fetcher,
+    return write_irregular_experiment_json(
+        path, tile_format,
+        image_tile_coordinates=image_tile_coordinates,
+        tile_fetchers=image_tile_fetchers,
         postprocess_func=postprocess_func,
         default_shape=default_shape,
-        dimension_order=dimension_order,
         fov_path_generator=fov_path_generator,
         tile_opener=tile_opener,
     )
diff --git a/starfish/core/experiment/builder/cli.py b/starfish/core/experiment/builder/cli.py
index b474dc4d..82bf71bf 100644
--- a/starfish/core/experiment/builder/cli.py
+++ b/starfish/core/experiment/builder/cli.py
@@ -4,7 +4,13 @@ from slicedimage import ImageFormat
 
 from starfish.core.types import Axes
 from starfish.core.util import click
-from . import AUX_IMAGE_NAMES, write_experiment_json
+from . import write_experiment_json
+
+
+AUX_IMAGE_NAMES = {
+    'nuclei',
+    'dots',
+}
 
 
 class StarfishIndex(click.ParamType):
diff --git a/starfish/core/image/_filter/element_wise_mult.py b/starfish/core/image/_filter/element_wise_mult.py
index a27057ec..ff5ba168 100644
--- a/starfish/core/image/_filter/element_wise_mult.py
+++ b/starfish/core/image/_filter/element_wise_mult.py
@@ -29,7 +29,7 @@ class ElementWiseMultiply(FilterAlgorithmBase):
     """
 
     def __init__(
-        self, mult_array: xr.core.dataarray.DataArray, clip_method: Union[str, Clip] = Clip.CLIP
+        self, mult_array: xr.DataArray, clip_method: Union[str, Clip] = Clip.CLIP
     ) -> None:
 
         self.mult_array = mult_array
@@ -78,12 +78,11 @@ class ElementWiseMultiply(FilterAlgorithmBase):
         if not in_place:
             stack = deepcopy(stack)
 
-        # stack._data contains the xarray
-        stack._data *= mult_array_aligned
+        stack._data.data.values *= mult_array_aligned
         if self.clip_method == Clip.CLIP:
-            stack._data = preserve_float_range(stack._data, rescale=False)
+            stack._data.data.values = preserve_float_range(stack._data.data.values, rescale=False)
         else:
-            stack._data = preserve_float_range(stack._data, rescale=True)
+            stack._data.data.values = preserve_float_range(stack._data.data.values, rescale=True)
         return stack
 
     @staticmethod
diff --git a/starfish/core/spots/_detect_pixels/combine_adjacent_features.py b/starfish/core/spots/_detect_pixels/combine_adjacent_features.py
index 7c2a2e2f..6806bc6d 100644
--- a/starfish/core/spots/_detect_pixels/combine_adjacent_features.py
+++ b/starfish/core/spots/_detect_pixels/combine_adjacent_features.py
@@ -267,8 +267,20 @@ class CombineAdjacentFeatures:
                 'x': int(spot_property.centroid[1])
             }
 
-        # we're back to 3d or fake-3d here
-        target_index = decoded_image[spot_attrs['z'], spot_attrs['y'], spot_attrs['x']]
+        # define the target index as the most repeated value in the bounding box of the spot.
+        #  it appears there is no simpler way to do this with a regionprops object
+        bbox = spot_property.bbox
+        if len(bbox) == 6:
+            # 3d bbox
+            target_candidates = np.ndarray.flatten(
+                decoded_image[bbox[0]:bbox[3], bbox[1]:bbox[4], bbox[2]:bbox[5]])
+        else:
+            # flatten and remove zeros
+            target_candidates = np.ndarray.flatten(
+                decoded_image[0, bbox[0]:bbox[2], bbox[1]:bbox[3]])
+        # get the most repeated nonzero value
+        non_zero_target_candidates = target_candidates[target_candidates != 0]
+        target_index = np.argmax(np.bincount(non_zero_target_candidates))
         spot_attrs[Features.TARGET] = target_map.target_as_str(target_index)
         spot_attrs[Features.SPOT_RADIUS] = spot_property.equivalent_diameter / 2
 
diff --git a/starfish/experiment/builder.py b/starfish/experiment/builder.py
index 2165eb1e..a22c687f 100644
--- a/starfish/experiment/builder.py
+++ b/starfish/experiment/builder.py
@@ -1,6 +1,9 @@
 from starfish.core.experiment.builder import (   # noqa: F401
     build_image,
+    build_irregular_image,
     inplace,
+    TileIdentifier,
     write_experiment_json,
+    write_irregular_experiment_json,
 )
 from starfish.core.experiment.builder.providers import FetchedTile, TileFetcher  # noqa: F401


Test patch:
diff --git a/starfish/core/experiment/builder/test/test_build.py b/starfish/core/experiment/builder/test/test_build.py
index b32b9c8a..0d8a56ef 100644
--- a/starfish/core/experiment/builder/test/test_build.py
+++ b/starfish/core/experiment/builder/test/test_build.py
@@ -33,11 +33,11 @@ class TestWithBuildData(unittest.TestCase):
         ],
         [
             "starfish", "validate", "manifest",
-            lambda tempdir, *args, **kwargs: os.sep.join([tempdir, "primary_images.json"])
+            lambda tempdir, *args, **kwargs: os.sep.join([tempdir, "primary.json"])
         ],
         [
             "starfish", "validate", "fov",
-            lambda tempdir, *args, **kwargs: os.sep.join([tempdir, "primary_images-fov_000.json"])
+            lambda tempdir, *args, **kwargs: os.sep.join([tempdir, "primary-fov_000.json"])
         ],
     )
 
diff --git a/starfish/core/image/_filter/test/test_filter.py b/starfish/core/image/_filter/test/test_filter.py
index 0394ec84..90f7f3f1 100644
--- a/starfish/core/image/_filter/test/test_filter.py
+++ b/starfish/core/image/_filter/test/test_filter.py
@@ -2,8 +2,9 @@ from typing import Tuple, Union
 
 import numpy as np
 import pytest
+import xarray as xr
 
-from starfish.core.image._filter import gaussian_high_pass, mean_high_pass
+from starfish.core.image._filter import element_wise_mult, gaussian_high_pass, mean_high_pass
 from starfish.core.imagestack.imagestack import ImageStack
 from starfish.core.types import Clip, Number
 
@@ -40,3 +41,15 @@ def test_mean_high_pass(size: Union[Number, Tuple[Number]], is_volume: bool) ->
     mhp = mean_high_pass.MeanHighPass(size=size, is_volume=is_volume, clip_method=Clip.CLIP)
     result = mhp.run(image_stack)
     assert np.sum(result.xarray) < sum_before
+
+
+def test_element_wise_mult() -> None:
+    image_stack = random_data_image_stack_factory()
+    mult_array = xr.DataArray(
+        np.array([[[[[0.5]]]]]),
+        dims=('r', 'c', 'z', 'y', 'x')
+    )
+    ewm = element_wise_mult.ElementWiseMultiply(mult_array)
+    multiplied = ewm.run(image_stack, in_place=False)
+    assert isinstance(multiplied.xarray, xr.DataArray)
+    assert multiplied.xarray.equals(image_stack.xarray * .5)
diff --git a/starfish/test/full_pipelines/api/test_dartfish.py b/starfish/test/full_pipelines/api/test_dartfish.py
index c4ee5bf7..29aff2a8 100644
--- a/starfish/test/full_pipelines/api/test_dartfish.py
+++ b/starfish/test/full_pipelines/api/test_dartfish.py
@@ -195,4 +195,4 @@ def test_dartfish_pipeline_cropped_data():
     corrcoef = np.corrcoef(x, y)
     corrcoef = corrcoef[0, 1]
 
-    assert np.round(corrcoef, 5) == 0.04422
+    assert np.round(corrcoef, 5) == 0.03028
diff --git a/starfish/test/full_pipelines/api/test_merfish.py b/starfish/test/full_pipelines/api/test_merfish.py
index 71e3a78d..3af1af0e 100644
--- a/starfish/test/full_pipelines/api/test_merfish.py
+++ b/starfish/test/full_pipelines/api/test_merfish.py
@@ -232,8 +232,8 @@ def test_merfish_pipeline_cropped_data():
 
     # assert that number of high-expression detected genes are correct
     expected_counts = pd.Series(
-        [120, 101, 54, 43, 30],
-        index=('nan', 'MALAT1', 'SRRM2', 'FASN', 'PRKDC')
+        [108, 57, 45, 32, 31],
+        index=('MALAT1', 'SRRM2', 'FASN', 'TLN1', 'IGF2R')
     )
     assert np.array_equal(
         expected_counts.values,
@@ -248,4 +248,4 @@ def test_merfish_pipeline_cropped_data():
 
     corrcoef = np.corrcoef(tmp[:, 1], tmp[:, 0])[0, 1]
 
-    assert np.round(corrcoef, 4) == 0.9647
+    assert np.round(corrcoef, 4) == 0.8411
diff --git a/starfish/test/full_pipelines/cli/test_dartfish_cli.py b/starfish/test/full_pipelines/cli/test_dartfish_cli.py
index 60c2255e..bbe1e1c4 100644
--- a/starfish/test/full_pipelines/cli/test_dartfish_cli.py
+++ b/starfish/test/full_pipelines/cli/test_dartfish_cli.py
@@ -104,4 +104,4 @@ class TestWithDartfishData(CLITest, unittest.TestCase):
         corrcoef = np.corrcoef(x, y)
         corrcoef = corrcoef[0, 1]
 
-        assert np.round(corrcoef, 5) == 0.04422
+        assert np.round(corrcoef, 5) == 0.03028