Package: hesperos
HESPEROS PLUGIN FOR NAPARIA Napari plugin for pre-defined manual segmentation or semi-automatic segmentation with a one-shot learning procedure. The objective was to simplify the interface as much as possible so that the user can concentrate on annotation tasks using a pen on a tablet, or a mouse on a computer.Thisnapariplugin was generated withCookiecutterusing@napari'scookiecutter-napari-plugintemplate.Table of ContentsInstallation and UsageAutomatic installationManual installationUpgrade Hesperos versionHesperos:Manual Segmentation and CorrectionmodeImport and adjust your imageLayer controlsAnnotate your imageSelect slices of interestExport annotationsHesperos:OneShot SegmentationmodeImport and adjust your imageAnnotate your imageRun automatic segmentationExport annotationsInstallation and UsageThe Hesperos plugin is designed to run on Windows (11 or less) and MacOS with Python 3.8 / 3.9 / 3.10.Automatic installationInstallAnacondaand unselectAdd to PATH. Keep in mind the path where you choose to install anaconda.Only download thescript_filesfolder forWindowsorMacos.Add your Anaconda path in these script files:For Windows:
Right click on the .bat files (forinstallationandrunning) and selectModify. ChangePATH_TO_ADDwith your Anaconda path. Then save the changes.for exemple:anaconda_dir=C:\Users\chgodard\anaconda3For Macos:Right click on the .command files (forinstallationandrunning) and selectOpen with TextEdit. ChangePATH_TO_ADDwith your Anaconda path. Then save the changes.for exemple:source ~/opt/anaconda3/etc/profile.d/conda.shIn your terminal, change the permissions to allow the following .command files to be run (changePATHwith the path of your .command files):chmod u+x PATH/install_hesperos_env.command 
chmod u+x PATH/run_hesperos.commandDouble click on theinstall_hesperos_env fileto create a virtual environment in Anaconda with python 3.9 and Napari 0.4.14./!\ The Hesperos plugin is not yet compatible with Napari versions superior to 0.4.14.Double click on therun_hesperos fileto run Napari from your virtual environment.In Napari:Go toPlugins/Install Plugins...Search for "hesperos" (it can take a while to load).Install thehesperosplugin.When the installation is done, close Napari. A restart of Napari is required to finish the plugin installation.Double click on therun_hesperos fileto run Napari.In Napari, use the Hesperos plugin withPlugins/hesperos.Manual installationInstallAnacondaand unselectAdd to PATH.Open your Anaconda prompt command.Create a virtual environment with Python 3.8 / 3.9 / 3.10:conda create -n hesperos_env python=3.9Install the required Python packages in your virtual environment:conda activate hesperos_env
conda install -c conda-forge napari=0.4.14 
conda install -c anaconda pyqt
pip install hesperos/!\ Hesperos plugin is not yet compatible with napari version superior to 0.4.14.Launch Napari:napariUpgrade Hesperos versionDouble click on therun_hesperos fileto run Napari.In Napari:Go toPlugins/Install Plugins...Search for "hesperos" (it can take a while to load).Click onUpdateif a new version of Hesperos has been found. You can check the latest version of Hesperos in theNapari Hub.When the installation is done, close Napari. A restart of Napari is required to finish the plugin installation.Hesperos:Manual Segmentation and CorrectionmodeTheManual Segmentation and Correctionmode of the Hesperos plugin is a simplified and optimized interface to do basic 2D manual segmentation of several structures in a 3D image using a mouse or a stylet with a tablet.Import and adjust your image(use Panel 1)The Hesperos plugin can be used with Digital Imaging and COmmunications in Medicine (DICOM), Neuroimaging Informatics Technology Initiative (NIfTI) or Tagged Image File Format (TIFF) images. To improve performances, use images that are located on your own disk.To import data:use thebutton for(.tiff, .tif, .nii or .nii.gz)image files.use thebutton for a DICOM serie. /!\ Folder with multiple DICOM series is not supported.After the image has loaded, a slider appears that allows to zoom in/out:. Zooming is also possible with thebutton in the layer controls panel.If your data is a DICOM serie, you have the possibility to directly change the contrast of the image (according to the Hounsfield Unit):by choosing one of the two predefined contrasts:CT boneorCT Softin.by creating a custom default contrast with thebutton and selectingCustom Contrast. Settings can be exported as a .json file with thebutton.by loading a saved default contrast with thebutton and selectingCustom Contrast.In the bottom left corner of the application you also have the possibility to:: change the order of the visible axis (for example go to sagittal, axial or coronal planes).: transpose the 3D image on the current axis being displayed.Layer controlsWhen data is loading, two layers are created: theimagelayer and theannotationslayer. Order in the layer list correspond to the overlayed order. By clicking on these layers you will have acces to different layer controls (at the top left corner of the application). All actions can be undone/redone with the Ctrl-Z/Shift-Ctrl-Z keyboard shortcuts. You can also hide a layer by clicking on its eye icon on the layer list.For theimagelayer:opacity: a slider to control the global opacity of the layer.contrast limits: a double slider to manually control the contrast of the image (same as theoption for DICOM data).For theannotationslayer:: erase brush to erase all labels at once (ifpreserve labelsis not selected) or only erase the selected label (ifpreserve labelsis selected).: paint brush with the same color than thelabelrectangle.: fill bucket with the same color than thelabelrectangle.: select to zoom in and out with the mouse wheel (same as the zoom slider at the top right corner in Panel 1).label: a colored rectangle to represent the selected label.opacity: a slider to control the global opacity of the layer.brush size limits: a slider to control size of the paint/erase brush.preserve labels: if selected, all actions are applied only on the selected label (see thelabelrectangle); if not selected, actions are applied on all labels.show selected: if selected, only the selected label will be display on the layer; if not selected, all labels are displayed.Remark: a second option for filling has been addedDrawn the egde of a closed shape with the paint brush mode.Double click to activate the fill bucket.Click inside the closed area to fill it.Double click on the filled area to deactivate the fill bucket and reactivate the paint brush mode.Annotate your image(use Panel 2)Manual annotation and correction on the segmented file is done using the layer controls of theannotationslayer. Click on the layer to display them. /!\ You have to choose a structure to start annotating(see 2.).To modify an existing segmentation, you can directy open the segmented file with thebutton. The file needs to have the same dimensions as the original image./!\ Only .tiff, .tif, .nii and .nii.gz files are supported as segmented files.Choose a structure to annotate in the drop-down menuFetus: to annotate pregnancy image.Shoulder: to annotate bones and muscles for shoulder surgery.Shoulder Bones: to annotate only few bones for shoulder surgery.Feta Challenge: to annotate fetal brain MRI with the same label than the FeTA Challenge (see ADD LIEN WEB).When selecting a structure, a new panel appears with a list of elements to annotate. Each element has its own label and color. Select one element in the list to automatically activate the paint brush mode with the corresponding color (color is updated in thelabelrectangle in the layer controls panel).All actions can be undone with thebutton or Ctrl-Z.If you need to work on a specific slice of your 3D image, but also have to explore the volume to understand some complex structures, you can use the locking option to facilitate the annotation task.To activate the functionality:Go to the slice of interest.Click on thebutton => will change the button toand save the layer index.Scroll in the z-axis to explore the data (with the mouse wheel or the slider under the image).To go back to your slice of interest, click on thebutton.To deactivate the functionality (or change the locked slice index):Go to the locked slice.Click on thebutton  => change the button toand "unlock" the slice.Select slices of interest(use Panel 3 -- only displayed for the Shoulder Bones category)This panel will only be displayed if theShoulder Bonescategory is selected. A maxiumum of 10 slices can be selected in a 3D image and the corresponding z-indexes will be integrated in the metadata during the exportation of the segmentation file./!\ Metadata integration is available only for exported .tiff and .tif files and with theUniquesave option.: to add the currently displayed z-index in the drop-down menu.: to remove the currently displayed z-index from the drop-down menu.: to go to the z-index selected in the drop-down menu. The icon will be checked when the currently displayed z-index matches the selected z-index in the drop-down menu.: a drop-down menu containing the list of selected z-indexes. Select a z-index from the list to work with it more easily.Export annotations(use Panel 3 -- or 4 if the Shoulder Bones category is selected)Annotations can be exported as .tif, .tiff, .nii or .nii.gz file with thebutton in one of the two following saving mode:Unique: segmented data is exported as a unique 3D image with corresponding label ids (1-2-3-...). This file can be re-opened in the application.Several: segmented data is exported as several binary 3D images (0 or 255), one for each label id.: delete annotation data.Automatic segmentation backup: if selected, the segmentation data will be automatically exported as a unique 3D image when the image slice is changed./!\ This process can slow down the display if the image is large.Hesperos:OneShot SegmentationmodeTheOneShot Segmentationmode of the Hesperos plugin is a 2D version of the VoxelLearning method implemented in DIVA (seeour Githuband the latest articleGuérinot, C., Marcon, V., Godard, C., et al. (2022). New Approach to Accelerated Image Annotation by Leveraging Virtual Reality and Cloud Computing.Frontiers in Bioinformatics. doi:10.3389/fbinf.2021.777101).The principle is to accelerate the segmentation without prior information. The procedure consists of:Arapid taggingof few pixels in the image with two labels: one for the structure of interest (named positive tags), and one for the other structures (named negative tags).Atrainingof a simple random forest classifier with these tagged pixels and their features (mean, gaussian, ...).Aninferenceof all the pixels of the image to automatically segment the structure of interest. The output is a probability image (0-255) of belonging to a specific class.Iterative corrections if needed.Import and adjust your image(use Panel 1)Same panel as theManual Segmentation and Correctionmode(seepanel 1 description).Annotate your image(use Panel 2)Annotations and corrections on the segmented file is done using the layer controls of theannotationslayer. Click on the layer to display them. Only two labels are available:Structure of interestandOther.The rapid manual tagging step of the one-shot learning method aims to learn and attribute different features to each label.To achieve that, the user has to:with the labelStructure of interest, tag few pixels of the structure of interest.with the labelOther, tag the greatest diversity of uninteresting structures in the 3D image (avoid tagging too much pixels).see the exemple image withStructure of interestlabel in red andOtherlabel in cyan.To modify an existing segmentation, you can directy open the segmented file with thebutton. The file needs to have the same dimensions as the original image./!\ Only .tiff, .tif, .nii and .nii.gz files are supported as segmented files.All actions can be undone with thebutton or Ctrl-Z.Run automatic segmentation(use Panel 3)From the previously tagged pixels, features are extracted and used to train a basic classifier : the Random Forest Classifier (RFC). When the training of the pixel classifier is done, it is applied to each pixel of the complete volume and outputs a probability to belong to the structure of interest.To run training and inference, click on thebutton:You will be asked to save a .pckl file which corresponds to the model.A new status will appears under thePanel 4:Computing.... You must wait for the message to change to:Readybefore doing anything in the application (otherwise the application may freeze or crash).When the processing is done, two new layers will appear:theprobabilitieslayer which corresponds to the direct probability (between 0 and 1) of a pixel to belong to the structure of interest. This layer is disabled by default, to enable it click on its eye icon in the layer list.thesegmented probabilitieslayer which corresponds to a binary image obtained from the probability image normed and thresholded according to a value manually defined with theProbability thresholdslider:.Remark: If the output is not perfect, you have two possibilities to improve the result:Add some tags with the paint brush to take in consideration unintersting structures or add information in critical areas of your structure of interest (such as in thin sections). Then, run the training and inference process again. /!\ This will overwrite all previous segmentation data.Export your segmentation data and re-open it with theManual Annotation and Correctionmode of Hesperos to manually erase or add annotations.Export annotations(use Panel 4)Segmented probabilites can be exported as .tif, .tiff, .nii or .nii.gz file with thebutton. The image is exported as a unique 3D binary image (value 0 and 255). This file can be re-opened in the application for correction.Probabilities can be exported as .tif, .tiff, .nii or .nii.gz file with thebutton as a unique 3D image. The probabilities image is normed between 0 and 255.: delete annotation data.LicenseDistributed under the terms of theBSD-3license,Hesperosis a free and open source software.