Package: odn
AboutWe provide a set of object detection neural network training and evaluation functions.We currently support Faster-RCNN, SSD and YOLO(v5). An infant funuds image object detection application is provided. This application combined the general-purposed object dection and domain-specific morphological rules. The rule-filtering and ROP zone segmentatin functions are provided in the annotation class of the fundus.py file.NOTE: Faster-RCNN and SSD require tf 1.x. SSD can run under tf 2.x using 'import tensorflow.compat.v1 as tf'. FRCNN may be painful to get running. We recommend using the latest YOLO v5.Cite our paper if you use the odn package or the dataset:"Morphological Rule-Constrained Object Detection of Key Structures in Infant Fundus Image", IEEE/ACM Transactions on Computational Biology and Bioinformatics, 2023CreditThe following 3rd-party packages are used:The Faster-RCNN module is based onhttps://github.com/kentaroy47/frcnn-from-scratch-with-keras. We have updated the code according to the latest keras API change (K..image_dim_ordering () -> K.image_data_format()).Data augmentation package (https://github.com/Paperspace/DataAugmentationForObjectDetection).SSD (Single Shot MultiBox Detector) by tensorflow. We took lots of efforts to integrate tensorflow's objection detection and slim source, mainly revising the relative importing.https://github.com/tensorflow/models/research/object_detectionandhttps://github.com/tensorflow/models/research/slimThe darkflow framework (https://github.com/thtrieu/darkflow)The pytorch_yolov5 project (https://github.com/ultralytics/yolov5). The main changes are in utils/datasets.py (we added filelist.txt support) and utils/plots (we added fundus_zones()).Installpip install torch==1.8.2 torchvision==0.9.2 torchaudio===0.8.2 --extra-index-url https://download.pytorch.org/whl/lts/1.8/cu111 (use LTS version)       
pip install tensorflow-gpu == 1.14.0  
pip install tf-slim 
pip install odnFor tf_ssd, we need to add odn\tf_ssd\protoc-3.4.0-win32\bin to PATH, then run:
cd tf_ssdprotoc object_detection/protos/*.proto --python_out=.How to use - FRCNNOutput images with annotationsfrom odn.fundus import datasetdataset.synthesize_anno(label_file = '../data/fundus/all_labels.csv',
dir_images = '../data/fundus/images/',
dir_output = '../data/fundus/ground_truth/',verbose = True,
display = 5 )The labe file should be in this format:seq,class,cx,cy,filename,height,laterality,width,xmax,xmin,ymax,ymin
 259,Macula,384,293,00569c080bf51c7f182cbe4c76f1823a.jpg,480,L002,720,436,332,345,241
 668,OpticDisk,191,275,00569c080bf51c7f182cbe4c76f1823a.jpg,480,L002,720,224,158,308,242The generated image is like this:Split datasetdataset.split(label_file = '../data/fundus/all_labels.csv',
dir_images = '../data/fundus/images/',
train_output = '../data/fundus/train.txt',
test_output = '../data/fundus/test.txt',
test_size = 0.2, verbose = True)Train a new model by transfer learning uing pre-trained PRN weigthscmd and cd to the the odn folder:python -m odn.train_frcnn -o simple -p ../../data/fundus/train.txt --network vgg16 --rpn ./models/rpn/pretrained_rpn_vgg_model.36-1.42.hdf5 --hf true --num_epochs 20or if using github source,python train_frcnn.py -o simple -p ../../data/fundus/train.txt --network vgg16 --rpn ./models/rpn/pretrained_rpn_vgg_model.36-1.42.hdf5 --hf true --num_epochs 20from odn import utilsutils.plot_training_curves(input_file = '../src/odn/training_log.txt', output_file = '../src/odn/training_curves.png')Testpython -m odn.test_frcnn --network vgg16 -p ../../data/fundus/test_public.txt --load models/vgg16/19e.hdf5 --num_rois 32 --writeor if using github source,python test_frcnn.py --network vgg16 -p ../../data/fundus/test_public.txt --load models/vgg16/19e.hdf5 --num_rois 32 --writeA candidate_rois.txt with all candidate ROIs will be generated.ROI filteringfrom odn.fundus import annotation, datasetannotation.rule_filter_rois(input_file = '../src/odn/candidate_rois.txt',
output_file = '../src/odn/rois.txt',
verbose = True)dataset.synthesize_anno('../src/odn/rois.txt',
dir_images = '../data/fundus/images_public/',
dir_output = '../data/fundus/odn_19e/',
drawzones = True, # set true if you want to draw ROP zonesverbose = True,
display = 5 )or output annotated images in place:dataset.synthesize_anno('../src/odn/rois.txt',
dir_images = '../data/fundus/images_public/',
dir_output = 'inplace',
drawzones = True,
verbose = True,
suffix = '_FRCNN', # inplace image name suffix
display = 5 )Evaluation6.1 Output a single comparison plotfrom odn import metricsmetrics.fundus_compare_metrics(gt = '../data/fundus/all_labels.csv',
pred = '../src/odn/rois.txt',
output_file = './comparison_with_metrics.jpg',
image_dirs = [
'../data/fundus/ground_truth_public',
'../data/fundus/odn_19e_raw','../data/fundus/odn_19e_naive',
'../data/fundus/odn_19e'], verbose = True )6.2 Output image-wise comparison plotfundus_compare_metrics_separate(input_file = '../src/odn/rois.txt',
output_dir = './comparison_separate/',
image_dirs = [
'../data/fundus/ground_truth_public',
'../data/fundus/odn_19e_raw',
'../data/fundus/odn_19e'], verbose = True )6.3 Only output comparison results that are differents between different detection algorithmsfundus_compare_metrics_html(gt = '../data/fundus/all_labels.csv', pred = '../src/odn/rois_naive.txt')fundus_compare_metrics(gt = '../data/fundus/all_labels.csv',
pred = '../src/odn/rois.txt',
output_file = './comparison_with_metrics_diff.jpg',
image_dirs = [
'../data/fundus/ground_truth_public',
'../data/fundus/odn_19e_raw','../data/fundus/odn_19e_naive',
'../data/fundus/odn_19e'], image_subset = image_subset, verbose = True )How to use - TF-SSDFollow our notebook (8.4 Object Detection - Fundus - TF-SSD ) to build the tfrecord.Trainingcmd > cd to '/odn/tf_ssd' > run:python train.py --logtostderr --train_dir=training --pipeline_config_path=ssd_mobilenet_v1_fundus.configTo use a pretrained model, revise ssd_mobilenet_v1_fundus.config:fine_tune_checkpoint: "../tf/export/model.ckpt"If the training runs correctly, should output something like:INFO:tensorflow:global step 1906: loss = 4.3498 (3.848 sec/step)
    I0610 11:49:51.364413 13848 learning.py:512] global step 1906: loss = 4.3498 (3.848 sec/step)
    INFO:tensorflow:global step 1907: loss = 3.9057 (3.903 sec/step)
    I0610 11:49:55.267028 13848 learning.py:512] global step 1907: loss = 3.9057 (3.903 sec/step)
    INFO:tensorflow:global step 1908: loss = 3.4026 (3.856 sec/step)Export model as tf graphpython export_inference_graph.py --input_type image_tensor --pipeline_config_path ssd_mobilenet_v1_fundus.config --trained_checkpoint_prefix training/model.ckpt-1945 --output_directory ./export2002206Predictionfrom odn.fundus import annotation 

    # get image file list to predict
    FILES = utils.get_all_images_in_dir(folder = '../data/fundus/images_public/') 

    # load tf graph model
    detection_graph, category_index = annotation.load_tf_graph(ckpt_path = '../src/odn/tf_ssd/export/frozen_inference_graph.pb',
                    label_path = '../src/odn/tf_ssd/fundus_label_map.pbtxt', 
                    num_classes = 2)

    # object detection
    annotation.tf_batch_object_detection(detection_graph, category_index, FILES, 
                                    '../data/fundus/ssd/', 
                                    '../data/fundus/ssd_202206.txt', 
                                    new_img_width = 300, fontsize = 12)


    Or output annotated images in place:

    annotation.tf_batch_object_detection(detection_graph, category_index, FILES, 
                                  'inplace', 
                                  '../data/fundus/ssd_202206.txt',                        
                                  new_img_width = 900, fontsize = None, suffix = '_SSD')The annotated images will be generated in the ssd folder. A sample image is as follows,We draw ROP Zone I, Posterior Zone II and Zone II.How to use - Darflow YoloPrerequisie: Install darkflow from sourcesrc\odn\darkflow_yolo>pip install -e .
        Obtaining file:///C:/Users/eleve/Documents/codex/py/keras/8.%20Object%20Detection/github/src/odn/darkflow_yolo
        Installing collected packages: darkflow
        Running setup.py develop for darkflow
        Successfully installed darkflowLoad yolo model and make predictionfrom odn import dark_yolo​

    tfnet = dark_yolo.load_tfnet(options = {
            'model': '../src/odn/darkflow_yolo/cfg/yolo.cfg', # 'bin/yolov1.cfg',
            'load': '../src/odn/darkflow_yolo/bin/yolov2.weights', # 'bin/yolov1.weights',
            'config': '../src/odn/darkflow_yolo/cfg/',
            'threshold': 0.3,
            'gpu': 0.7
        })

    yolo.predict(tfnet, '../data/generic/image2.jpg')How to use - pytorch YOLOTrain by transfer learningcd to src\odn\torch_yolo, run (download the yolov5m pretrained model first):python train.py --data data/fundus.yaml --cfg models/yolov5m.yaml --weights models/yolov5m.pt --batch-size 16 --epochs 30 --img 640The training results and models are saved to runs/train/exp/Predict a folderpython detect.py --source ../../../data/fundus/test --weights runs/train/exp/weights/best.ptInplace prediction (i.e., )from odn.fundus import annotation
%matplotlib inlineannotation.torch_batch_object_detection(model_path = '../src/odn/torch_yolo/runs/train/exp15/weights/best.pt',
input_path = 'filelist.txt',
conf_thres=0.3, iou_thres=0.5, max_det=2,
anno_pil = True, colors = [(200,100,100),(55,125,125)],
suffix = '_YOLO5', display = True, verbose = False
)Inside torch_batch_object_detection(), we have extended the original torch yolov5 with these functions to support in-place prediction:datasets.py - LoadImages(input_path) - input_path can now be a filelist.txt of image paths   
 plots.py - Annotation.fundus_zones(self, cx, cy, radius) - draw zone 1, posterior zone 2 and zone 2How to use - Camera Appfrom odn import camera

camera.realtime_object_detection(ckpt_path = '../src/odn/models/ssd_mobilenet_v2_coco_2018_03_29/frozen_inference_graph.pb',
                label_path = '../src/odn/models/mscoco_label_map.pbtxt', 
                num_classes = 90)How to use - tkInter GUIpython -m odn.gui1. 设置目录：设置包含眼底照片的目标目录  
2. 检索图片：默认将按照后缀名规则(*.jpg,*.png)搜素该目录下所有图片（包括子目录）  
3. 列表将列出所有图片。选中某个图片，可以“预览”或“删除”  
4. "FRCNN","SSD","YOLO5"点击其中一个将弹出对话框，点击"Start | 开始"将使用响应的目标检测模型对所有图片进行处理。其中，FRCNN需要额外下载19e.h5模型文件（大小500MB），SSD和YOLO5的模型已经打包在安装文件中。    
5. 处理完毕后，将在图片原位置生成后缀"_FRCNN"、"_SSD"、"_YOLO5"的文件，包含检测到的关键生理结构及眼底分区Jupyter notebooksUnder /notebooks, we provide several examples for both general-purposed and fundus image object detection.Datasetdata/fundus/images : a fundus image set in courtesy of SZEH (Shenzhen Eye Hospital)data/fundus/ground_truth_public : images with ground truth annotationsdata/fundus/frcnn_19e_zones : object detection results by FRCNN (trained 19 epochs)data/fundus/ssd : object detection results by SSD (Single Shot MultiBox Detector)DeploymentAfter training, you will get a keras h5 model file. You can further convert it to tflite format, or tfjs format.Then you can deploy on mobile device or browser-based apps.Functions added since first publicationOne-exam zone segmentation:odn.fundus.annotation.torch_batch_object_detection_for_one_exam()The basic idea is to use images with both optic disc and macula to get the zone 1 radius (left). Later use this radius for images of the same exam only with the optic disc (right).