Package: mmv-hitl4trk
We are actively working on the documentationmmv-HITL4TRKA plugin to use with napari to segment and track cells via HumanInTheLoop(HITL)-approach.Thisnapariplugin was generated withCookiecutterusing@napari'scookiecutter-napari-plugintemplate.UsageLoad a zarr-file consisting of Image, Label and Tracks layer.InstallationYou can installmmv_hitl4trkviapip:pip install mmv_hitl4trkBy default, CPU is used for segmentation computing. We have tried to optimize the CPU computing time, but still recommend GPU computing. For more detailed instructions on how to install GPU support lookhere.DocumentationThis plugin was developed to analyze 2D cell migration. It includes the function of segmenting 2D videos usingCellpose(both CPU and GPU implemented) and then tracking them using different automatic tracking algorithms, depending on the use case. For both segmentation and tracking, we have implemented user-friendly options for manual curation after automatic processing. In conjunction with napari's inherent functionalities, our plugin provides the capability to automatically track data and subsequently process the tracks in three different ways based on the reliability of the automated results. Firstly, any potentially existing incorrect tracks can be rectified in a user-friendly manner, thereby maximizing the evaluation of available information. Secondly, unreliable tracks can be selectively deleted, and thirdly, individual tracks can be manually or semi-automatically created for particularly challenging data, ensuring reliable results. In essence, our tool aims to offer a valuable supplement to the existing fully automated tracking tools and a user-friendly means to analyze videos where fully automated tracking has been previously challenging.Common metrics such as speed, cell size, velocity, etc... can then be extracted, plotted and exported from the tracks obtained in this way. Furthermore, the plugin incorporates a functionality to assess the automatic tracking outcomes using aquality score. Since automated tracking may not be consistently 100% accurate, presenting a quality measure alongside scientific discoveries becomes essential. This supplementary metric offers researchers valuable insights into the dependability of the produced tracking results, fostering informed data interpretation and decision-making in the analysis of cell migration.More detailed information and instructions on each topic can be found in the following sections.Get startedTo load raw data, you can simply drag & drop them into napari. Ensure that the 'Image' combobox on the right displays the correct layer afterward. To load custom segmentations, you can do equivalent.The "save as" button can be used to save the existing layers (raw, segmentation, tracks) in a single .zarr file, which can be loaded again later using the "load" button. The "save" button overwrites the loaded .zarr file.The computation mode is used to set how many of the available CPU cores (40% or 80%) are to be used for computing the CPU segmentation and tracking and therefore has a direct impact on performance.SegmentationFor segmentation, we use the state of the art instance segmentation method Cellpose. We provide one (/several??) model that we trained and has proven successful for our data (see more information).(...)Automatic instance segmentation(...)Custom modelsThe plugin supports loading custom Cellpose models. To train your own Cellpose model,thismight be helpful.In future versions, we plan to support fine-tuning of Cellpose models within the plugin.Manual curationBe aware that removing a cell cuts the track the cell is on. Resulting tracks shorter than two timesteps will be removed(...)TrackingThe plugin supports both coordinate-based and overlap-based tracking. Overlap-based tracking requires more computation, but can also be used in particularly complicated data for individual cells.
In our experience, coordinate-based tracking has proven itself in cases with reliable segmentation. Overlap-based tracking serves as a useful complement in cases where the segmentation is not of sufficient quality.(...)Manual curation(...)AnalysisThe plugin supports the calculation of various metrics, which can be divided into two categories: migration-based (such as speed, direction, ...) and cell-based (such as size, eccentricity, ...). Through the use of these metrics, a comprehensive understanding of the available data can be obtained.(...)EvaluationTo be aware of the accuracy of your automatic tracking and segmentation results, we have implemented an evaluation function based onthis manuscript. Evaluation is always carried out against the latest results of automatic segmentation and automatic tracking or previously created results loaded via the plugin's own load function. We may implement the option to evaluate external segmentations in the future, but for now you can use save and load as a workaround.To evaluate results, at least 2 consecutive frames must first be corrected manually. The plugin saves the previously mentioned automatic or loaded results in the background, so no activation via button or similar is necessary before manual correction.(...)Segmentation evaluation(...)Tracking evaluation(...)false positives:
check if cell from eval has a match > .4 IoU. If no, check if cell has second highest match >= .2 IoU. If no, then fpfalse negatives:
check if cell from gt has a match > .4. If no, then fn.
check if matched cell maxIoU is higher than match. If yes, then fn.
check if matched cell top 2 maxIoU are equal. If yes, then half fn (this will apply for both cells)split cell:
check if cell from eval has more than one match, and if second highest match is >= .2 IoU. If yes, then scadded edge:
check if a connection in gt has both cells matched in eval & the matched cells are connected. if no, then aedeleted edge:
check if a connection in eval has both cells matched in gt & the matched cells are connected. if no, then deHotkeysHere's an overview of the hotkeys. All of them can also be found in the corresponding tooltips.E- Load next free segmentation IDS- Overlap-based single cell trackingDevelopment planWe will continue to develop the plugin and implement new features in the future. Some of our plans in arbitrary order:Support of lineagesSupport training custom Cellpose models within the pluginModel optimization to further optimize segmentation computationSupport evaluation of external segmentations...If you have a feature request, pleasefile an issue.ResourcesThe following resources may be of interest:CellposeContributingContributions are very welcome. Tests can be run withtox, please ensure
the coverage at least stays the same before you submit a pull request.LicenseDistributed under the terms of theBSD-3license,
"mmv_hitl4trk" is free and open source softwareIssuesIf you encounter any problems, pleasefile an issuealong with a detailed description.