Use of different object detection methods and classification on medical images
By: Sapeksh Suman
Tags: Yolov3, Object Detection, Medical Image, Image Classification, Yolov5

Step 4

Below are some of the object detection methods we tried —

· Yolov3(https://arxiv.org/abs/1804.02767)

· Faster RCNN(https://arxiv.org/abs/1506.01497)

· Yolov5(https://github.com/ultralytics/yolov5)

· Efficient Net based object detection

Model Training

Snapshot of training stats. ** Yolov3 trained on CPU

Highlights -

· Comparable image size was taken.

· Yolov5 trained on 100 epochs against 30 epochs of Faster-RCNN with similar training duration

· Response time(testing) for Yolov5 was better.

P.S. — We also tried to run Efficient net based object detection model but it was consuming most of the GPU RAM(16Gb) and also was quite slow in training. Therefore, we dropped the idea of training it. But its known that it is a state of the art architecture and may in fact give better results than Yolov5.

Step 5 — Model prediction and result interpretation

During result calculation a score threshold was placed on model prediction and a rule was placed which limited the number of Bounding Box to at max 2. Score threshold was identified basis observation on different iterations run and it was different for different models.

Assessment was done mainly on below 2 metrics i.e. –

1. How accurately models identifies side of joint(Left, Right) especially for images with single joint.

2. How accurately model identifies the area of joint i.e. bounding box accuracy(IoU).

Challenges while calculating results

Before calculating results there were several challenges we faced. Few of them are listed below

· We may lead to conservative accuracy number by taking account of Bounding box side (L/R) Consistency. Hence, we first calculated Intersection over Union(IOU) of every predicted Bbox with each Ground truth Bbox which was used to map them with each other on the basis of highest IoU.

· To account the variation in the size of bounding box (as tagged by multiple people) we calculated Intersection over Ground Truth(IoGT) which would tell us how much area of ground truth bounding box is covered by predicted bounding box.

Results

Observations –

· Model Trained on Dataset B(containing both single and both hip join images) were much superior to Dataset A.

· Yolov5(Dataset B) had the highest coverage of 78.5% i.e. comparing just the number of boxes predicted not taking any account of location of Bbox. Faster-RCNN was a close second with 73.1%.

· When it came to Bbox accuracy Yolov5 performed significantly better in both metrics i.e. IoU and IoGT. Around 90% IoGT as compared to around 60% of other models.

· Class accuracy was also highest for Yolov5 Dataset B i.e. 87.7% for all images whereas 90.5% for images with single joint.

· Bounding box predicted by Yolov3 and Yolov5 were somewhat square in shape like Ground truth Bboxes whereas Faster-RCNN had a rectangular structure(maybe due to presence of anchors).

Step 6 — Building an alternate classification model

Even though Yolov5 gave close to 90% class accuracy for single joint images(more important since for images with both joints, we can get the side by simply comparing the X-coordinate) we wanted to try and build a separate classification model look at its accuracy and performance.

For training dataset we cropped images using Bbox coordinates we annotated and again used these annotations to tag sides. This step ensured removal of noise and only the most important part of image is used for training.

I trained a model with Resnet50 architecture using FastAI library for 7 epochs and saved it. Now for the scoring part we again needed cropped images since the training was done on same. For this part instead using the annotation for test dataset we used Bbox coordinates coming from Yolov5. And, viola we achieved 100% accuracy as opposed to ~90% of Yolov5.

Final Solution

· Passing images through Yolov5(Dataset B) model and cropping images using predicted Bbox coordinates.

· Left and Right hip side prediction using FastAI classification model.

Conclusion

Yolov5 provides very easy implementation steps and is compatible to CVAT output format. Also, it exhibited better performance on most of the performance metrics i.e. training time, testing time, classification, Bbox prediction.

Also, a dedicated classification model performed better on classification task as compared to object detection models trained to predict separate classes.

And lastly the most important point that all the above numbers, results and conclusion points are my personal opinion/observations and in no way we claim or reject anything. There may be a case where better optimization, pre-processing or training the above numbers may completely change. We tried to put similar effort and time in building each model therefore giving me a fair idea of effort needed to reach a desired performance.