isoview 0.1.0
Multi-view light sheet microscopy image processing pipeline
License: MIT
Requires Python: ==3.12.9

# Isoview Pipeline: MATLAB -> Python

*Conversion/Validation TODO*

- [x] readKLB.m
- [x] clusterPT_RC.m
- [x] clusterPT with zarr benchmarks
- [ ] clusterMF.m

Note: Multiprocessing is not yet implemented, as it is in the MATLAB pipeline.

---

## `readKLB.m` -> `pyklb`

Using our fork of [pyklb](https://github.com/MillerBrainObservatory/pyklb), on [pypi](https://pypi.org/project/pyklb/).

**1. Generate MATLAB Reference**

```bash
matlab -batch \
"run('C://Users//RBO//repos//isoview//excludes//save_matlab_klb.m')"
```

**2. Validate with Python**

```bash
uv run --no-sync python excludes/test_klb_matlab_python.py
```

| File | MATLAB Shape | Python Shape | Max Diff |
|------|--------------|--------------|----------|
| SPM00_TM000000_CM00_CHN00.klb | (2048, 2048, 543) | (543, 2048, 2048) -> (2048, 2048, 543) | 0.0 |
| SPM00_TM000000_CM01_CHN00.klb | (2048, 2048, 543) | (543, 2048, 2048) -> (2048, 2048, 543) | 0.0 |
| SPM00_TM000000_CM02_CHN01.klb | (2048, 2048, 592) | (592, 2048, 2048) -> (2048, 2048, 592) | 0.0 |
| SPM00_TM000000_CM03_CHN01.klb | (2048, 2048, 592) | (592, 2048, 2048) -> (2048, 2048, 592) | 0.0 |

**Total:** 4/4 files exactly equal (100%)

- **MATLAB**: Stores volumes as `(Y, X, Z)` = `(2048, 2048, 543/592)`
- **Python pyklb**: Returns volumes as `(Z, Y, X)` = `(543/592, 2048, 2048)`

---

## clusterPT_RC.m

D:\W2_DATA\foconnell\isoview_development\mosquito-larva_20250930_165806.corrected\SPM00\TM000000

**File structure**

```
project_root\
  SPM00\
    Background_0.tif         (one per camera)
    Background_1.tif
    Background_2.tif
    Background_3.tif
    TM00000\                 (one folder per timepoint)
      ch0.xml              (rename from ch00_spec00.xml)
      ch1.xml              (rename from ch01_spec00.xml)
      ANG000\
        SPC00_TM00000_ANG000_CM0_CHN00_PH0.stack
        SPC00_TM00000_ANG000_CM1_CHN00_PH0.stack
        SPC00_TM00000_ANG000_CM2_CHN01_PH0.stack
        SPC00_TM00000_ANG000_CM3_CHN01_PH0.stack
```

```matlab
inputFolder = 'D:\W2_DATA\foconnell\isoview_development\mosquito-larva_20250930_165806';
inputType = 2;
specimen = 0;

% Run 1: Cameras 0,1 with Channel 0
cameras = [0 1];
channels = 0;

% Run 2: Cameras 2,3 with Channel 1
cameras = [2 3];
channels = 1;
```

1. Background TIFFs: Place in SPM00 root (one per camera)
2. XML files: Rename ch00_spec00.xml to ch0.xml, ch01_spec00.xml to ch1.xml
3. XML files: Copy to each TM folder (TM00000, TM00001, etc.)

### Comparison

See the full notebook [here](https://github.com/MillerBrainObservatory/isoview/blob/master/clusterPT_comparison.ipynb)

Most of the differences happen in the non-dense regions on the edge of the FOV:

<img width="1097" height="508" alt="Image" src="https://github.com/user-attachments/assets/bf06d440-24f9-41d0-a5f7-c2f8737b7281" />

---

## clusterMF.m

Multi-view fusion pipeline. Python implementation in `isoview/fusion.py`.

**MATLAB equivalent**

```matlab
inputFolder = 'D:\W2_DATA\foconnell\isoview_development\mosquito-larva_20250930_165806.corrected_python_zarr';
specimen = 0;
timepoints = 0;
cameras = [0 1 2 3];
channels = [0 1];

% Fusion parameters
fusionType = 1;           % adaptive blending
blendingRange = [20 4];   % [channel, camera]
cameraPairs = [0 1; 2 3];
flipH = true;
flipV = false;
```

**Python equivalent**

```python
from isoview import ProcessingConfig, fuse

config = ProcessingConfig(
    input_dir=Path("D:/W2_DATA/.../mosquito-larva_20250930_165806.corrected_python_zarr"),
    output_dir=Path("D:/W2_DATA/.../mosquito-larva_20250930_165806.corrected_python_zarr"),
    specimen=0,
    timepoints=[0],
    cameras=[0, 1, 2, 3],
    channels=[0, 1],
    fusion_enable=True,
    fusion_type="adaptive_blending",
    fusion_blending_range=(20, 4),
    fusion_camera_pairs=[(0, 1), (2, 3)],
    fusion_flip_h=True,
    fusion_flip_v=False,
)

fuse(config, estimate_params=True, apply_fusion=True)
```

### Feature Flags

All MATLAB clusterMF features are available as config flags (disabled by default):

| Feature | MATLAB | Python Flag |
|---------|--------|-------------|
| Temporal smoothing | rloess | `fusion_temporal_smoothing=True` |
| Smoothing window | smoothingRange | `fusion_smoothing_window=100` |
| Static parameters | staticFlag | `fusion_static=True` |
| Mask fusion mode | maskFusionMode | `fusion_mask_fusion_mode=1` |
| Mask padding | padding | `fusion_mask_padding=2` |
| Small object removal | bwareaopen | `fusion_mask_min_object_size=1e-5` |
| Slab processing | slabSize | `fusion_slab_size_cameras=3` |
| Median filtering | medianFilterRange | `fusion_median_filter_range=100` |
| Gaussian precision | preciseGauss | `fusion_precise_gauss=True` |
| Lookup tables | generateLUT | `fusion_generate_lookup_table=True` |

See [MATLAB_FEATURES.md](MATLAB_FEATURES.md) for complete documentation.

### Diagnostic Output

Pipeline generates diagnostic PNGs when `save_diagnostics=True` (default):

```
output_dir/
  diagnostics/
    TM000000/
      cam0_1/
        registration.png
        intensity_correction.png
        fusion_result.png
        blending_weights.png
        fused_overview.png
```

---

## Zarr I/O Integration and Benchmarks

**Dataset Parameters**

- Data volume: (543, 2048, 2048) uint16
- Uncompressed size: 4344.00 MB (4.242 GB)
- Pixel spacing: [2.0, 0.406, 0.406] um
- Chunk size: (10, 128, 128)

### Quick Compression Performance

Based on a single raw `.stack` from a mosquito recorded on isoview: `mosquito-larva_20250930_165806`

| Method | Compression | Level | Write (s) | Read (s) | Size (MB) | Ratio | Write Speed (GB/s) | Read Speed (GB/s) |
|--------|-------------|-------|-----------|----------|-----------|-------|-------------------|-------------------|
| blosc-zstd-5-sharded | blosc-zstd | 5 | 42.71 | 2.55 | 1780.3 | 2.44x | 0.099 | 1.663 |
| blosc-zstd-9-sharded | blosc-zstd | 9 | 52.96 | 2.52 | 1679.5 | 2.59x | 0.080 | 1.682 |
| blosc-lz4-3-sharded | blosc-lz4 | 3 | 40.12 | 2.50 | 2205.0 | 1.97x | 0.106 | 1.698 |
| blosc-zstd-5-no-shard | blosc-zstd | 5 | 44.99 | 2.25 | 1780.3 | 2.44x | 0.094 | 1.887 |
| blosc-lz4-3-no-shard | blosc-lz4 | 3 | 44.01 | 2.50 | 2205.0 | 1.97x | 0.096 | 1.698 |
| none-sharded | none | 0 | 45.34 | 2.28 | 4344.0 | 1.00x | 0.094 | 1.863 |
| none-no-shard | none | 0 | 52.18 | 1.90 | 4344.0 | 1.00x | 0.081 | 2.237 |

### Pipeline Compression Performance

Now, instead of converting a `.stack`, we run the full `clusterPT` pipeline and compare `.tiff`, `.klb` and `.zarr`.

From the KLB source code (`keller-lab-block-filetype/src/klb_imageIO.cpp`):

```cpp
int BWTblockSize = 9;  // maximum compression
// compress the memory buffer (blocksize=9*100k, verbose=0, worklevel=30)
int ret = BZ2_bzBuffToBuffCompress(bufferOutPtr, &sizeCompressed,
                                    bufferIn, gcount, BWTblockSize, 0, 30);
```

**KLB uses bzip2 with level 9** (block size 9x100k = 900,000 bytes)

```
Testing 11 configurations:

Baselines:
  KLB_bzip2_9          - KLB format with bzip2, level 9 (block size 9x100k)
  TIFF_uncompressed    - Uncompressed TIFF for size reference

Zarr configurations (all with sharding: 10 frames/shard, 1 frame/chunk, full FOV):
  Zarr_bzip2_9         - bzip2 level 9 (match KLB)
  Zarr_zstd_6          - blosc-zstd at level 6
  Zarr_zstd_9          - blosc-zstd at level 9
  Zarr_lz4_6           - blosc-lz4 at level 6
  Zarr_lz4_9           - blosc-lz4 at level 9
  Zarr_lz4hc_6         - blosc-lz4hc at level 6
  Zarr_lz4hc_9         - blosc-lz4hc at level 9
  Zarr_gzip_6          - gzip at level 6
  Zarr_gzip_9          - gzip at level 9
```

<img width="843" height="364" alt="Image" src="https://github.com/user-attachments/assets/0764e115-b731-4075-b1e9-43997a4c11d7" />

## Compression Benchmark Results

Tested on dataset: (543, 2048, 2048) uint16, ~4.24 GB uncompressed

| Format | Compression | Level | Size (MB) | Ratio | vs Uncompressed |
|--------|-------------|-------|-----------|-------|-----------------|
| **KLB_bzip2_9** | bzip2 | 9 | 1107.99 | **3.92x** | **74.5% reduction** |
| Zarr_bzip2_9 | bzip2 | 9 | 1127.56 | 3.85x | 74.0% reduction |
| Zarr_zstd_9 | blosc-zstd | 9 | 1255.15 | 3.46x | 71.1% reduction |
| Zarr_zstd_6 | blosc-zstd | 6 | 1297.95 | 3.35x | 70.1% reduction |
| Zarr_lz4hc_9 | blosc-lz4hc | 9 | 1332.07 | 3.26x | 69.3% reduction |
| Zarr_lz4hc_6 | blosc-lz4hc | 6 | 1337.21 | 3.25x | 69.2% reduction |
| Zarr_gzip_9 | gzip | 9 | 1434.98 | 3.03x | 67.0% reduction |
| Zarr_gzip_6 | gzip | 6 | 1457.96 | 2.98x | 66.4% reduction |
| Zarr_lz4_9 | blosc-lz4 | 9 | 1517.57 | 2.86x | 65.1% reduction |
| Zarr_lz4_6 | blosc-lz4 | 6 | 1519.42 | 2.86x | 65.0% reduction |
| TIFF_uncompressed | none | - | 4344.13 | 1.00x | - |

---

## Module Overview

### Core Pipeline

| Module | Description | Key Functions |
|--------|-------------|---------------|
| `pipeline.py` | Main processing orchestrator | `IsoviewProcessor`, `process_dataset()` |
| `config.py` | All configuration flags | `ProcessingConfig` dataclass |
| `io.py` | File I/O (klb, tiff, zarr, stack) | `read_volume()`, `write_volume()`, `read_xml_metadata()` |
| `array.py` | Lazy loader for processed data | `IsoviewArray` class |

### Image Processing

| Module | Description | Key Functions |
|--------|-------------|---------------|
| `corrections.py` | Dead pixel detection/correction | `correct_dead_pixels()`, `estimate_background()` |
| `segmentation.py` | Foreground segmentation | `segment_foreground()`, `fuse_masks()`, `create_coordinate_masks()` |
| `transforms.py` | Geometric transforms | `rotate_volume()`, `flip_volume()`, `crop_volume()`, `estimate_registration()`, `apply_registration()` |

### Fusion Pipeline

| Module | Description | Key Functions |
|--------|-------------|---------------|
| `fusion.py` | Multi-view fusion | `fuse()`, `blend_views()` |
| `temporal.py` | Temporal parameter processing | `smooth_parameters_rloess()`, `apply_temporal_averaging()`, `create_lookup_table()` |
| `masks.py` | Mask processing for fusion | `combine_masks()`, `pad_mask_to_center()`, `apply_bwareaopen()` |
| `intensity.py` | Intensity correction/filtering | `apply_median_filter()`, `apply_gauss_filter()` |

### Utilities

| Module | Description | Key Functions |
|--------|-------------|---------------|
| `viz.py` | Diagnostic visualization | `plot_projections()`, `plot_registration_result()`, `plot_fusion_result()`, `plot_volume_overview()` |

### Public API

```python
from isoview import (
    ProcessingConfig,    # configuration dataclass
    IsoviewProcessor,    # main processor class
    process_dataset,     # process clusterPT pipeline
    fuse,                # run clusterMF fusion
    blend_views,         # blend two registered views
    temporal,            # temporal processing submodule
    viz,                 # visualization submodule
)
```

### IsoviewArray

Lazy loader for processed data with napari/visualization tool compatibility:

```python
from isoview.array import IsoviewArray

arr = IsoviewArray("path/to/output/TM000000")

# Shape: (Z, Views, Y, X) for single timepoint
# Shape: (T, Z, Views, Y, X) for multi-timepoint
print(arr.shape)       # (543, 4, 2048, 2048)
print(arr.views)       # [(0, 0), (1, 0), (2, 1), (3, 1)]
print(arr.metadata)    # microscope metadata dict

# Lazy indexing
frame = arr[100, 0]    # single Z-slice, first view
volume = arr[:, 0]     # full Z-stack, first view

# Access labels/projections (consolidated structure)
mask = arr.get_labels(timepoint=0, camera=0, label_type='segmentation')
proj = arr.get_projection(timepoint=0, camera=0, proj_type='xy')
```