Package: napari-annotatorj
napari-annotatorjThe napari adaptation of the ImageJ/Fiji pluginAnnotatorJfor easy image annotation.Thisnapariplugin was generated withCookiecutterusing with@napari'scookiecutter-napari-plugintemplate.InstallationInstallation is possible withpip,napariorscripts.PipYou can installnapari-annotatorjviapip:pip install napari[all]
pip install napari-annotatorjTo install latest development version :pip install git+https://github.com/spreka/napari-annotatorj.gitOn Linux distributions, the following error may arise upon napari startup after the installation of the plugin:Could not load the Qt platform plugin “xcb” in “” even though it was found. In this case, the manual install oflibxcb-xinerama0for Qt is required:sudo apt install libxcb-xinerama0Bundled napari appThe bundled application version ofnapariallows the pip install of plugins in the .zip distribution. After installation of this release, napari-annotatorj can be installed from thePlugins --> Install/Uninstall plugins...menu by searching for its name and clicking on theInstallbutton next to it.ScriptSingle-file install is supported onWindowsandLinux(currently). It will create a virtual environment namednapariAnnotatorjEnvin the parent folder of the cloned repository, install the package via pip and start napari. It requires a valid Python install.WindowsTo start it, run in the Command promptgit clone https://github.com/spreka/napari-annotatorj.git
cd napari-annotatorj
install.batOr downloadinstall.batand run it from the Command prompt.After install, you can usestartup_napari.batto activate your installed virtual environment and run napari. Run it from the Command prompt with:startup_napari.batLinuxTo start it, run in the Terminalgit clone https://github.com/spreka/napari-annotatorj.git
cd napari-annotatorj
install.shOr downloadinstall.shand run it from the Terminal.After install, you can usestartup_napari.shto activate your installed virtual environment and run napari. Run it from the Terminal with:startup_napari.shIntronapari-annotatorj has several convenient functions to speed up the annotation process, make it easier and more fun. Thesemodescan be activated by their corresponding checkboxes on the left side of the main AnnotatorJ widget.Contour assist modeEdit modeClass modeOverlayFreehand drawing is enabled in the plugin. The "Add polygon" tool is selected by default upon startup. To draw a freehand object (shape) simply hold the mouse and drag it around the object. The contour is visualized when the mouse button is released.See theguidebelow for a quick start or ademo. Seeshortcutsfor easy operation.How to annotateOpen --> opens an image(Optionally)... --> Select annotation type --> Ok --> a default tool is selected from the toolbar that fits the selected annotation typeThe default annotation type is instanceSelected annotation type is saved to a config fileStart annotating objectsinstance: draw contours around objectssemantic: paint the objects' areabounding box: draw rectangles around the objectsSave --> Select class --> saves the annotation to a file in a sub-folder of the original image folder with the name of the selected class(Optionally)Load --> continue a previous annotationOverlay --> display a different annotation as overlay (semi-transparent) on the currently opened imageColours --> select annotation and overlay colours... (coming soon) --> set options for semantic segmentation andContour assistmodecheckboxes --> Various options(default) Add automatically --> adds the most recent annotation to the ROI list automatically when releasing the left mouse buttonSmooth (coming soon) --> smooths the contour (in instance annotation type only)Show contours --> displays all the contours in the ROI listContours assist --> suggests a contour in the region of an initial, lazily drawn contour using the deep learning method U-NetShow overlay --> displays the overlayed annotation if loaded with the Overlay buttonEdit mode --> edits a selected, already saved contour in the ROI list by clicking on it on the imageClass mode --> assigns the selected class to the selected contour in the ROI list by clicking on it on the image and displays its contour in the class's colour (can be set in the Class window); clicking on the object a second time unclassifies it[^] --> quick export in 16-bit multi-labelled .tiff format; if classified, also exports by classesInstance annotationAllows freehand drawing of object contours (shapes) with the mouse as in ImageJ.Shape contour points are tracked automatically when the left mouse button is held and dragged to draw a shape. The shape is closed when the mouse button is released, automatically, and added to the default shapes layer (named "ROI"). In direct selection mode (from the layer controls panel), you can see the saved contour points. The slower you drag the mouse, the more contour points saved, i.e. the more refined your contour will be.Click to watch demo video below.Semantic annotationAllows painting with the brush tool (labels).Useful for semantic (e.g. scene) annotation. Currently saves all labels to binary mask only (foreground-background).Bounding box annotationAllows drawing bounding boxes (shapes, rectangles) around objects with the mouse.Useful for object detection annotation.Contour assist modeAssisted annotation via a pre-trained deep learning model's suggested contour.initialize a contour with mouse drag around an objectthe suggested contour is displayed automaticallymodify the contour:edit with mouse drag orerase holding "Alt" orinvert with pressing "u"finalize itaccept with pressing "q" orreject with pressing "Ctrl" + "Del"if the suggested contour is a merge of multiple objects, you can erase the dividing line around the object you wish to keep, and keep erasing (or splitting with the eraser) until the object you wish to keep is the largest, then press "q" to accept itthis mode requires a Keras model to be present in themodel folderClick to watch demo video belowEdit modeAllows to modify created objects with a brush tool.select an object (shape) to modify by clicking on itan editing layer (labels layer) is created for painting automaticallymodify the contour:edit with mouse drag orerase holding "Alt"finalize itaccept with pressing "q" ordelete with pressing "Ctrl" + "Del" orrevert changes with pressing "Esc" (to the state before editing)if the edited contour is a merge of multiple objects, you can erase the dividing line around the object you wish to keep, and keep erasing (or splitting with the eraser) until the object you wish to keep is the largest, then press "q" to accept itClick to watch demo video belowClass modeAllows to assign class labels to objects by clicking on shapes.select a class from the class list to assignclick on an object (shape) to assign the selected class label to itthe contour colour of the clicked object will be updated to the selected class colour, plus the class label is updated in the text properties of object (turn on "display text" on the layer control panel to see the text properties asobjectID:(classLabel)e.g. 1:(0) for the first object)optionally, you can set a default class for all currently unlabelled objects on the ROI (shapes) layer by selecting a class from the drop-down menu on the right to the text label "Default class"class colours can be changed with the drop-down menu right to the class list; upon selection, all objects whose class label is the currently selected class will have their contour colour updated to the selected colourclicking on an object that has already been assigned a class label will unclassify it: assign the label0to itClick to watch demo video belowExportSee also:Quick exportThe exporter plugin AnnotatorJExport can be invoked from the Plugins menu under the plugin namenapari-annotatorj. It is used for batch export of annotations to various formats directly suitable to train different types of deep learning models. See ademonstrative figurein theAnnotatorJ repositoryand further description in itsREADMEordocumentation.browse original image folder with either the"Browse original..." button ortext input field next to itbrowse annotation folder with either the"Browse annot..." button ortext input field next to itselect the export options you wish to export the annotations to (see tooltips on hover for help)at least one export option must be selected to start export(optional) right click on the checkbox "Coordinates" to switch between the default COCO format and YOLO format; seeexplanationclick on "Export masks" to start the exportthis will open a progress bar in the napari window and close it upon finishThe folder structure required by the exporter is as follows:image_folder
	|--- image1.png
	|--- another_image.png
	|--- something.png
	|--- ...

annotation_folder
	|--- image1_ROIs.zip
	|--- another_image_ROIs.zip
	|--- something_ROIs.zip
	|--- ...Multiple export options can be selected at once, any selected will create a subfolder in the folder where the annotations are saved.Click to watch demo video belowQuick exportClick on the "[^]" button to quickly save annotations and export to mask image. It saves the current annotations (shapes) to an ImageJ-compatible roi.zip file and a generated a 16-bit multi-labelled mask image to the subfolder "masks" under the current original image's folder.Coordinate formatsIn the AnnotatorJExport plugin 2 coordinates formats can be selecting by right clicking on the Coordinates checkbox: COCO or YOLO. The default is COCO.COCO format:[x, y, width, height]based on the top-left corner of the bounding box around the objectcoordinates are not normalizedannotations are saved with header to.csv filetab delimetedYOLO format:[class, x, y, width, height]based on the center point of the bounding box around the objectcoordinates are normalized to the image size as floating point values between 0 and 1annotations are saved with header to.txt filewhitespace delimetedclass is saved as the 1st columnOverlayA separate annotation file can be loaded as overlay for convenience, e.g. to compare annotations.load another annotation file with the "Overlay" button(optional) switch its visibility with the "Show overlay" checkbox(optional) change the contour colour of the overlay shapes with the"Colours" buttonChange coloursClicking on the "Colours" button opens the Colours widget where you can set the annotation and overlay colours.select a colour from the drop-down list either next to the text label "overlay" or "annotation"click the "Ok" button to apply changescontour colour of shapes on the annotation shapes layer (named "ROI") that already have a class label assigned to them willnotbe updated to the new annotation colour, only those not having a class label (the class label can be displayed with the "display text" checkbox on the layer controls panel asobjectID:(classLabel)e.g. 1:(0) for the first object)contour colour of shapes on the overlay shapes layer (named "overlay") will all have the overlay colour set, regardless of any existing class information saved to the annotation file loaded as overlayConfigure model folderThe Contour assist mode imports a pre-trained Keras model from a folder namedmodelsunder exactly the pathnapari_annotatorj. This is automatically created on the first startup in your user folder:C:\Users\Username\.napari_annotatorjon Windows\home\username\.napari_annotatorjon LinuxA pre-trained model for nucleus segmentation is automatically downloaded from the GitHub repository of theImageJ version of AnnotatorJ. The model will be saved to[your user folder]\.napari_annotatorj\models\model_real.h5. This location is printed to the console (command prompt or powershell on Windows, terminal on Linux).(deprecated) When bulding from source the model folder is located atpath\to\napari-annotatorj\src\napari_annotatorj\modelswhereas installing from pypi it is located atpath\to\virtualenv\Lib\site-packages\napari_annotatorj\models.The model must be in either of these file formats:config .json file + weights file:model_real.jsonandmodel_real_weights.h5combined weights file:model_real.hdf5You can also train a new model on your own data in e.g. Python and save it with this code block:# save model as jsonmodel_json=model.to_json()withopen(‘model_real.json’,‘w’)asf:f.write(model_json)# save weights toomodel.save_weights(‘model_real_weights.h5’)You can also train in thetrain widget.This configuration will change in the next release to allow model browse and custom model name in anoptions widget.TrainingTo start training a new model or refine an existing one click theTrainbutton on the right of the napari-annotatorj widget. This will open the training widget where you can set input paths and training options. During training a progress bar will show the epochs passed and plot the loss on a graph. See aguidebelow.The trained model will be saved to themodelfolder under the located training data folder which is namedtrainingby default when preparing data. Each new training will be saved to a new training folder with increased numbering e.g.training_1,training_2etc.When an existing training data folder is browsed with the "Browse train ..." button, themodelfolder will be created under it without an additionatrainingfolder.After training is finished, a message is shown to indicate the newly trained model can be tested by drawing bounding boxes (rectangles) to initiatecontour assistprediction. The presented region on the editing layer (Label layer) can be modified with the paint brush tool (automatically selected) as incontour assist.The trained model can be further refined by selecting the "Retrain latest" checkbox from thetraining parameters(⚙ button on the right).To use this new model for annotation incontour assist mode, you mush set the model path in theOptions widgetor in the configuration file (see how tohere), then restart napari.How to trainOn current annotation"Use current annotation" checkbox --> use this image and its current annotation for trainingPrep data --> prepare data tosuitable format(optional) ⚙ -->set parametersStart --> start trainingAdditional dataSelect images and annotations to use for trainingBrowse original ... --> locate folder of original imagesBrowse annot ... --> locate folder of annotationsPrep data --> prepare data tosuitable formatBrowse train ... --> select already prepared training data(optional) ⚙ -->set parametersStart --> start trainingTraining data formatThe data format expected by the training widget is as follows.images
	|--- image1.png
	|--- another_image.png
	|--- something.png
	|--- ...

unet_masks
	|--- image1.tiff
	|--- another_image.tiff
	|--- something.tiff
	|--- ...Masks are 8-bit binary (black and white) .tiff images that can be exported from theExporter widgetselecting the Semantic (binary) export option. When the "Prep data" button is clicked in the Training widget, these folders are automatically created from the located annotation files and original images.Training parametersThe following configurable parameters can be set after clicking on the ⚙ icon:ParameterDescriptionDefault valueEpochsnumber of epochs to train5Stepsnumber of steps in each epoch1Batch sizenumber of samples in an iteration1Image sizesize of training images256Start from scratchtrain a new model from scratchFalseRetrain latestre-fine latest trainingFalseWrite predwrite test image prediction to fileFalseTest imagepath to test imageNoneNote: by default CPU will be used for training. This can be changed to GPU in theOptions widgetif your computer has a capable CUDA-device.OptionsSettings found in the configuration file can be set in the Options widget opened with the "..." button on the right of the main plugin. For changes to take effect you must save your changes with the "Ok" button at the bottom of the Options widget.The following options can be configured:GroupOptionDescriptionDefaultValid valuesGeneralAnnotation typeseeinstance,bbox,semanticinstanceinstance, bbox, semanticRemember annotation typeuse the same annotation type on next startupTrueTrue,FalseColoursselect annotation and overlay colours; seeherewhitewhite, red, green, blue, cyan, magenta, yellow, orange, blackClassesnames of folders to save annotations when not classified*normal(any string)**Semantic segmentationBrush sizesize of the brush50intAdvanced settingsContour assistMax distancenumber of pixels to extend the initial contour with17intThresholdintensity threshold after prediction- gray0.1floatin [0,1]- R (red)0.2floatin [0,1]- G (green)0.4floatin [0,1]- B (blue)0.2floatin [0,1]Brush sizecorrection brush size5intMethodcontour assist prediction methodU-NetU-Net, Classic***ModelU-Net model to use for Contour assist predictionfolderpath to the model folderuser/.napari_annotatorj/modelsexistingmodelsfolder path.json filename of the model .json filewithoutextensionmodel_realany string**weights filename of the model weights filemodel_real_weights.h5any string**full filename of the combined config+weights filemodel_real.hdf5any string**Devicecomputation device to perform predictioncpucpu,int****Mask/text importAuto mask loadload annotation files automatically when a new image is openedFalseTrue,FalseEnable mask loadload instance annotation mask imageFalseTrue,FalseEnable text loadload object detection bounding box coordinate text fileFalseTrue,FalseMethodload as editable or overlayloadload, overlayOthersSave outlinessave image with annotations outlinedFalseTrue,FalseShow help on startupshow the help window upon every startupFalseTrue,FalseSave annot timessave annotation times to text file*****FalseTrue,False*: right click the last element (other...) to add a new item to the list. When annotations are assigned class labels inclass mode, they will be saved to the foldermasksby default.**: do not use whitespace (' ') if possible***: region growing classical algorithm****: valid id of a GPU device e.g.0or3; if your computer has only one GPU the id is0*****: currently disabled, used for developmentDemoRun a demo of napari-annotatorj with sample data: a small 3-channel RGB image as original image and an ImageJ roi.zip file as annotations loaded.# from the napari-annotatorj folderpythonsrc/napari_annotatorj/load_imagej_roi.pyAlternatively, you can startup the napari-annotatorj plugin by running# from the napari-annotatorj folderpythonsrc/napari_annotatorj/startup_annotatorj.pyShortcutsFunctionShortcutContour assistaClass modecEdit modeShift+eShow contoursShift+vAccept Contour assistqReject Contour assistCtrl+delInvert Contour assistuErase in Edit/Contour assist modeAlt(hold)Revert changes in Edit modeEscSetting device for deep learning model predictionTheContour assistmode uses a pre-trained U-Net model for suggesting contours based on a lazily initialized contour drawn by the user. The default configuration loads and runs the model on the CPU so all users can run it. It is possible to switch to GPU if you have:a CUDA-capable GPU in your computernVidia's CUDA toolkit + cuDNN installedSee installation guide onnVidia's websiteaccording to your system.To switch to GPU utilization, edit_dock_widget.pyand set to the device you would like to use. Valid values are'cpu','0','1','2',.... The default value iscpu. The default GPU device is0if your system has any CUDA-capable GPU. If the device you set cannot be found or utilized by the code, it will fall back tocpu. An informative message is printed to the console upon plugin startup.ContributingContributions are very welcome. Tests can be run withtox, please ensure
the coverage at least stays the same before you submit a pull request.LicenseDistributed under the terms of theBSD-3license,
"napari-annotatorj" is free and open source softwareIssuesIf you encounter any problems, pleasefile an issuealong with a detailed description.