Package: napari-workflow-optimizer
napari-workflow-optimizerOptimize image processing workflows in napari for segmentation qualityUsageThe starting point for workflow optimization is a workflow and some reference ("ground truth") labels image.
The label image can be a sparse annotation, which means only some objects and also parts of objets are annotated (seehints).
These datasets should be ready. You can reproduce the following procedure by downloading anexamle raw image(derived from thescikit-image cells3d example data set) and a correspondingsparse annotation label image.
For reproducing the following procedure, also follow theinstallation instructionsbelow.
The whole procedure isalso shown in this video, an extended version of the trailer above.Step 0: Loading data and setting up the workflowLoad the "membranes_2d.tif" data set, e.g. by drag&drop on napari and start the Assistant from theTools > Utilities > Assistant (clEsperanto)menu.Click theLabelbutton and select as operation "Seeded watershed using local minima as seeds and an intensity threshold (nsbatwm)".Draw an annotation in a new labels layer or load the example spare annotation "membranes_2d_sparse_labels.tif".In case the image is not displayed as label image, convert it to a label image by right-clicking on the entry in the layers list:Step 1: The Workflow OptimizerStart the Workflow Optimizer from theTools > Utilities > Workflow optimizer (Labels)menu.
Configure the target layer, showing the label image that should be optimized.
Select the manual annotation as reference layer for the optimization.
Consider increasing the number of iterations. This number depends on your segmenation problem.
In the present example, 100 iterations should be enough.The optimizer will plot quality over the number of iterations to show the progress of optimization.
To determine the quality, the optimizer will measure the maximum overlap (Jaccard index)
of any labeled object over the manually annotated objects and calculate the mean of this value over all annotated objects.
After a moment, optimization will finish and update the labeled image.
If your starting point for the optimization was already good, the result may now look better than before.Step 2: Manual parameter space plottingIn case the result is not perfect yet (as the fringed segmentation above suggests), consider manual plotting of the
individual parameters and their relation to segmentation quality to get an idea about the surrounding parameter space.
Therefore, click thePlotbutton next to one of the workflow parameters.
Select the range in which the labeling quality should be determined (green arrows). In our example, the optimizer was setting the parameter to 2.34.
Thus, to demonstrate the procedure we plot the parameter space beween 0 and 10.
The quality plotted over this parameter obviously has a local maxium at 2.34, which was detected by the optimizer.
However, it also has another local maxium at 8 and actually a plateau in the quality plot (orange arrows).For further optimization, we re-configure the algorithm and set a new starting point for optimization of the parameter to 8.
Afterwards, we restart the optimization. It will then optimize the settings again from the new starting point.After another moment, optimization will finish again, potentially leading to an even better result.Step 3: Visualization of resultsMake sure the segmentation has high quality by inspecting the result visually. Use thecontoursetting of the labels layer
and hide/show the outlines of the labeled layer:Optimization HintsThe Workflow Optimizer uses theNelder-Mead simplex methodfor optimizing parameters. This algorithm varies individual parameters and makes steps in the parameter space ideally following a gradient
to a local optimum. Hence, this algorithm may not be capable of determining a global optimum in parameter space.
Parameter optimization is no magic. If it does not immediately work on your data, plot the parameters as introduced in Step 2
and identify parameters with a clear gradient and those with many local maxima.
Consider optimizing the parameters with many local maxima manually and de-selecting their checkboxes for the optimization.
The optimizer will then only optimize the parameters showing the clear gradient.
Repeat these steps a couple of times to get a feeling for your parameter space.Furthermore, parameter optimization works well ifthe initial settings are close to a good segmentation,a small number of parameters (a short workflow) are optimized,the reference annotation is prepared carefully andthe dataset is small. Consider using a small representative crop in case of bigger datasets.Workflow optimization scriptingFor optimizing workflows from within a jupyter notebook, check out ourexample notebook for optimization using spare labels.
The examples are more flexible than the graphical user interface and allow for exampleoptimizing intensity imagesandbinary images.
The membrane segmentation workflow optimization similar to the one shown above is also available asjupyter notebook.Known issuesIf you change the workflow architecture after the optimizer window was opened, please re-open it
to select the parameters that should be optimized. Changing parameters is ok and re-opening is not necessary.Thisnapariplugin was generated withCookiecutterusing@napari'scookiecutter-napari-plugintemplate.InstallationFurthermore, to reproduce the procedure above, please download and installnapari,pyopencl,
thenapari-pyclesperanto-assistantand
thenapari-segment-blobs-and-things-with-membranesplugin. E.g. usingcondaandpip:conda create --name napari-opti python=3.8
conda activate napari-opti

conda install pyopencl napari
pip install napari-pyclesperanto-assistant napari-segment-blobs-and-things-with-membranes
pip install napari-workflow-optimizerContributingContributions are very welcome. Tests can be run withtox, please ensure
the coverage at least stays the same before you submit a pull request.LicenseDistributed under the terms of theBSD-3license,
"napari-workflow-optimizer" is free and open source softwareIssuesIf you encounter any problems, pleasefile an issuealong with a detailed description.