RBAD: A Dataset and Benchmark for Retinal Bifurcation Angle Detection

Authors: hao  wang
wenhui  zhu
jiayou  qin
xin  li
oana  dumitrascu
xiwen  chen
peijie  qiu
abolfazl  razi
yalin  wang
Venue: IEEE_org_EMBS_BHI_2024_Conference
Published: 2024-07-05 06:56:06

Abstract: Detecting retinal image analysis, particularly the geometrical features of bifurcation points, plays an essential role in diagnosing eye diseases. However, existing methods used for this purpose often are coarse-level and lack fine-grained analysis for efficient annotation. To mitigate these issues, this paper proposes a novel method for detecting retinal bifurcation angles using a self-configured image processing technique. Additionally, we offer an open-source annotation tool and a benchmark dataset comprising 40 images annotated with retinal bifurcation angles. Our methodology for retinal bifurcation angle detection and calculation is detailed, followed by a benchmark analysis comparing our method with previous approaches. The results indicate that our method is robust under various conditions with high accuracy and efficiency, which offers a valuable instrument for ophthalmic research and clinical applications.

Main paper: 
 i. introduction: 
 Despite the increasing utilization of deep learning (DL)based image processing for retinal disease diagnosis, the lack of interpretability and easy annotations remain two significant challenges of such methods, preventing their widespread adoption by medical professionals [1] - [6] . Consequently, many medical practitioners continue to rely on the quantitative analysis of retinal vessel images, as the most prevalent method in modern medicine. Quantitative analysis is indispensable for the early diagnosis, monitoring, treatment planning, and population studies of various retinal diseases. Variations in vascular morphology and structures often indicate disease progression [7] , [8] . One notable metric in this context is the retinal branching angle, which is instrumental in assessing vascular health and the early detection of systemic diseases such as hypertension and diabetes [9] - [11] . Abnormal branching angles can be indicative of health problems, such as impaired blood flow efficiency due to cardiovascular events. These angles are crucial for maintaining the structural integrity of the retinal vasculature [12] , with deviations potentially signaling vessel damage. The quantitative analysis of branching angles provides standardized and objective metrics for precise disease diagnosis and monitoring. Additionally, this analysis enhances population studies in retinal research, offering insights into underlying disease mechanisms and supporting the development of innovative diagnostic and therapeutic strategies [13] . By providing a robust framework for evaluating vascular health, the study of branching angles can play a key role in extending our understanding and treatment of retinal and systemic diseases.Non-invasive retinal imaging is widely used for retinal vascular analysis due to its ability to capture high-resolution and detailed images of the retinal blood vessels. Advances in retinal imaging and its easy accessibility have incubated a series of tools for branching angle detection that assist medical experts in making decisions and streamlining clinical workflow [14] , [15] . Specifically, the Retina system (VAM-PIRE) [16] and the Singapore 'I' Vessel Assessment program (SIVA) [17] are two popular tools for branching angle analysis. However, these tools are not fully automated and necessitate human experts' intervention (i.e., semi-automated). Recently, there exists an abundant of retinal imaging datasets, exemplified by large-scale datasets like the UK Biobank that consists of more than 100,000 patients [18] . However, these datasets require manual branching angle detection, making them prohibitively impractical to be utilized by medical experts. While the aforementioned semi-automated tools mitigate this issue, conducting branching angle detection for large-scale datasets still remains a time-consuming and labor-intensive task [19] , [20] . In addition, existing semi-automated branching angle detection tools are not open-sourced, which hinders the customization of these tools for specialized and streamlined clinical workflows. Therefore, it is inevitable to develop an automated and open-source tool for branching angle detection. Another key challenge in developing branching angle detection tools is the absence of a benchmark dataset, which complicates the evaluation of these tools. Consequently, there is a press-ing need for to generate a benchmark dataset to assess the performance of branching angle detection tools accurately.In response to the aforementioned needs, we first introduce an open-sourced annotation tool implemented in Python that can efficiently annotate the branching angle from general retinopathy images. This tool is clinically friendly and can enhance retinal vessel visibility using edge detection and high-pass filters, enabling precise manual annotations. Second, we present a benchmark dataset by applying our customized annotation tool to a popular diabetic retinopathy dataset (i.e., DRIVE dataset [21] ) and creating a ready-touse well-annotated dataset. This dataset comprises 40 retinal images with high-quality annotated branching angles. The annotations are initially conducted by three annotators and further corrected by human medical experts. Subsequently, we propose a bifurcation detection algorithm that efficiently calculates branching angles on retinal segmentation maps.The contribution of this paper can be summarized as follows: (i) We introduce a benchmark dataset for retinal branching angle detection, which is well-annotated by experts with our customized annotation tool, and (ii) we proposed an efficient retinal branching angle detection method. We conduct a comprehensive benchmark analysis to demonstrate its superiority by comparing it with recent state-of-the-art algorithms.In the following sections, we will first introduce the proposed dataset and the annotation tool in Section II. Next, we will provide a detailed description of our branching angle detection and calculation methods in Section III. Subsequently, we will summarize previous work and outline our evaluation metrics, followed by an analysis of the annotated dataset in Section IV. Finally, we will discuss the results of our comprehensive benchmark analysis, highlighting the accuracy and efficiency of our approach compared to existing methods.The DRIVE (Digital Retinal Images for Vessel Extraction) dataset [21] B. Annotation Tool 1) Interface: Our annotation tool is designed to promote the fast and accurate annotation of retinal branching angles. The tool is implemented in Python and uses OpenCV library to provide a user-friendly interface. This interface allows users to load retinal images and annotate bifurcation points by simply marking the relevant locations in the image.2) Visual Enhancement: To reduce user fatigue and visual stress during the annotation process, we use different color channels to make retinal vessels and other features more prominent and easier to annotate. Users can dynamically switch between different color channels during annotation, as shown in Figure 1 . Our visual enhancement method mainly including: Green Channel. As shown in Figure 1 (c), the green channel reduces the contrast between the background and the vessels while highlighting the vessels. Particularly, it is commonly used as a preprocessing method in unsupervised vessel segmentation algorithms [22] , [23] . This selective enhancement allows annotators to focus more easily on the vessels without being distracted by the background. Edge Detection. Figure 1 (d) shows the extracted vessel lines through the Laplacian edge detector [24] . Edge detection algorithms are essential for identifying the boundaries of retinal vessels, making them more distinguishable against the background. The Laplacian edge detector works by calculating the second-order derivatives of the image, highlighting regions of rapid intensity change that correspond to edges. In our implementation, we use a kernel size of 5. High-Pass Filter. Figure 1 (e) applies a high-pass filter to an image using kernel convolution. This process enhances the visibility of high-frequency details by emphasizing edges and fine structures within the retinal vessels [25] . The function accepts three parameters: the image to be processed, the kernel size (51 in our case) for convolution, and an optional boolean to choose between a Gaussian kernel and a mean kernel for blurring. In our implementation, we first calculate the average color value of the image:µ = 1 H×W H i=1 W j=1 I norm (i, j), where I norm = I 255 .Then, depending on the boolean parameter, we apply either a Gaussian blur or a mean blur to the image to obtain the blurred version I blurred . The highpass filter then subtracts a blurred version of the image (lowfrequency components) from the original image: I high-pass = I norm -(I blurred -µ), thus highlighting the high-frequency components.Additionally, we implemented useful functions such as mask overlay to separate the annotation from the original image. Users can also edit existing annotations for fast correction.3) Instructions: Users can directly annotate relevant locations within the image display window by selecting three consecutive points, labeled as a, b, and c. Upon selection, these points are automatically connected to form an angle vector, as depicted in Figure ?? (a). This intuitive interface allows users to efficiently identify and mark critical features within the image. The program then employs vector mathematics to After the user confirms the annotations, the data, including the coordinates of the points and the calculated angles, are stored in JSON format. This structured data format ensures compatibility with other analysis tools, ensuring further processing and analysis. Additionally, annotations can be easily retrieved for subsequent review or integration into larger datasets, promising the safety and accessibility of datasets in future development.In our study, we employed three annotators to meticulously annotate the blood vessel segmentation images derived from the DRIVE dataset of retinal images. Prior to commencing the annotation work, we provided standardized training to ensure high-quality annotations. This training included detailed instruction on the accurate selection of vessel branching angles in the segmented images, the precision of the annotation angles, and the distinction between true branching angles and artifacts caused by vessel overlap (illustrated in Fig. 3 ). Moreover, we addressed potential issues arising from sampling instability, such as blurring, residual shadows, and noise introduced by the sampling device itself. To mitigate these issues, we implemented rigorous training sessions aimed at ensuring the validity of the labeled angle information. Each annotator independently annotated all images in the dataset, ensuring thoroughness and accuracy. Following this, we averaged the labeled angles for each retinal image to enhance the stability of the annotations and reduce potential errors. To further ensure that the annotations met medical standards, we submitted the final annotated results for secondary review by a medical professional. Based on the feedback received, the annotators made necessary modifications, which included adding missing annotations and correcting erroneous ones. This rigorous and iterative process ensured the accuracy, standardization, and medical relevance of each annotated branching angle, thereby enhancing the overall reliability of the dataset. Fig. 3 . Some of the problems that tend to occur when labeling are listed, including blurred, overlapping vessels, and false angles due to overlap.As previous key points searching works are designed for general tree-based graphs [26] - [29] , we propose a new bifurcation detection and angle calculation method that is specifically designed for retinal images. Our algorithm not only detects bifurcation nodes but also identifies the nearby nodes and their relationships (e.g., child, parent). A bifurcation map generated using our algorithm clearly illustrates the path of detection and calculation for each angle, as depicted in Figure 4 (e). The first step is converting the retinal segmentation map (Figure 4 (a)) into a skeletonized representation, where the retinal image is binarized and thinned into 1-pixel-width to highlight the blood vessel centerlines. Then we find a random pixel (non-zero) as the initial point. Assume I represent the skeleton mask:EQUATIONThen, the set of non-zero (vessel centerline) pixels is denoted as:EQUATIONThen we denote p ∈ P as a randomly selected initial point. Subsequently, we apply Fast Key Points Detection to the skeleton mask I by starting from the initial point p to generate the initial key points map. Specifically, we detect potential bifurcation nodes by analyzing pixel connectivity. We use a sliding window approach to traverse the skeleton from the initial point p and examine each pixel within a defined neighborhood to count the number of neighbor pixels. The detailed key points searching process is shown in Figure 4 (b) and Algorithm 1. Subsequently, we apply a Gaussian aggregation to every detected bifurcation to create a heatmap [30] , which efficiently identifies the geometric root of the generated tree graph, as shown in Figure 4 (d) . Specifically, a Gaussian distribution mask is applied at each bifurcation node p k : EQUATIONEQUATIONEliminate visited pixels in I:EQUATION)Add 1 on step index c k for node p k :EQUATION)if N (x k , y k ) > T then Record p k as a bifurcation node and add to K and P Generate new windows for each new branch else if N (x k , y k ) ≤ 1 then if c k ≥ nEQUATIONHere, G k (x, y | p k , σ i ) represents the Gaussian distribution centered at key point p k with a standard deviation σ i (typically set to 21 in this study). The resulting image G new is a heatmap indicating the distribution of bifurcations, where the highest pixel intensity area represents the region with the most gathered bifurcations. As shown in Figure 4 (c), the red circle highlights the region where most bifurcation nodes assemble, identifying it as the root r. Thus, we apply our proposed bifurcation detection algorithm again with the new root r ∈ P as the start. For each detected bifurcation node, we calculate the angle formed by the branching vessels. This involves computing the vectors of the vessel segments and determining the angle between them. Specifically, we generate a table to describe each key point, as shown in Table I . The branching angle is determined by analyzing the types of the child nodes. For instance, if one child node is a prune node and the other is an endpoint, they are likely on the same path, indicating an incorrect vector for the branching angle, as shown in Figure 5 (d) . Conversely, if both child nodes are prune nodes, they are more likely to represent the correct vectors for determining the branching angle as they are located on different routes. The node pruning threshold is set to 15 in our experimental section. Assume p k is a bifurcation, and a and c are two adjacent points along the vessel segments. The vectors ⃗ v 1 and ⃗ v 2 are defined as:EQUATION⃗ v 2 = (c x -p x , c y -p y ).The angle θ between these vectors is given by:EQUATION)Thus, we record the angle information while the searching window passes through each bifurcation node to generate an angle map, as shown in Figure 4 (e). The proposed random initialization, Gaussian aggregation, bifurcation detection, and angle calculation methods ensure full automation, making it more accessible compared to previous approaches. IV. BENCHMARKWe summarized and implemented three different methods referenced from previous work that were specifically designed for retinal bifurcation detection and angle calculation [19] , [31] , [32] . These methods are: Line Detection-based method. This method detects lines within retinal vessels and finds intersections of these lines as bifurcation points. The angles between the intersecting lines are then calculated as the branching angles, as illustrated in Figure 5 (a) . ROI-Window-based Detection. This method calculates branching angles by tracing the vessel branches from the bifurcation point within a region of interest (ROI) window. Initially, the root is identified manually, and a 3 × 3 window is used to detect possible bifurcations by checking if the surrounding pixels exceed a certain threshold (e.g., 5-8). For each detected bifurcation, a new window (50 × 50 in our case) is applied to determine the tail of the vectors, and the angle between these vectors is calculated. As depicted in Figure 5 (b), this method provides a structured approach to detecting bifurcation points and calculating angles by focusing on specific regions of interest. Rule-based Detection. This method measures angles by drawing tangents from the bifurcation point along the vessel segments. Acute angles are preferred as branching angles to distinguish actual bifurcations from distorted ones. Pixels with more than three surrounding pixels are identified as bifurcations, and the angles are calculated based on the surrounding sub-pixels. As shown in Figure 5 (c), this method identifies bifurcation points by analyzing pixel connectivity and calculating angles based on the tangents drawn from the bifurcation points.As illustrated in Figure 5 , the Line-Detection-based method (a) demonstrates poor detection performance, as the retinal bifurcation branches are not completely straightforward. In contrast, the Rule-Based method (c) shows efficient angle detection; however, some angles are misdirected because the method does not account for the relationship with other nodes. The ROI-Window-based method (b) produces more robust results compared to (a) and (c), successfully detecting all branching angles with correct directional information. As a comparison, our proposed method (d) surpasses the ROI-Window-based method in angle detection performance. By using the prune node as the child of the bifurcation, our method achieves greater precision than approaches relying on endpoints or other bifurcations as angle vectors.To evaluate the effectiveness of these methods for detecting and calculating retinal branching angles, we illustrate the mathematical equations for each metric calculated for each image:1) Average Angle: The average angle for each image is calculated as follows:EQUATIONwhere θ is the average angle, n is the number of angle annotations in the image, and θ i is the i-th angle in the list of angles for that image.2) Mean Average Angle : The mean average angle across all images is calculated as follows:EQUATIONwhere Mean Avg. Angle is the mean average angle, k is the number of images, and θj is the mean angle for the j-th image.3) Standard Deviation of Angles: The standard deviation of angles for each image is calculated as follows:EQUATION)where σ is the standard deviation of the angles, θ i is the i-th angle in the list of angles, θ is the mean angle, and n is the number of angle annotations in the image.The mean standard deviation across all images is calculated as follows:EQUATIONwhere Mean Std. is the mean standard deviation, k is the number of images, and σ j is the standard deviation for the j-th image.The variance of angles for each image is calculated as follows:EQUATION)where σ 2 is the variance of the angles, θ i is the i-th angle in the list of angles, θ is the mean angle, and n is the number of angle annotations in the image. 6) Mean-Variance: The mean-variance across all images is calculated as follows:EQUATIONwhere Mean Var. is the average variance, k is the number of images, and σ 2 j is the variance for the j-th image. 7) Average Number of Angles: The average number of angles detected per image is calculated as follows:EQUATIONwhere Avg. Num. of Angles is the average number of angles, k is the number of images, and n j is the number of angle annotations in the j-th image. We analyzed 40 annotated retinal images, where an average of 28 branching angles are annotated in each image, as shown in Figure 6 . Our analysis revealed that the Mean Average Angle across all 40 images is 71.31 degrees, and the Mean Standard Deviation of the branching angles is 18.32 degrees, suggesting that most branching angles fall within a range of approximately 53 to 89 degrees (71. 31 ± 18.32) . This value indicates the typical deviation of individual branching angles from the mean, reflecting the variability and dispersion of angles within the dataset. Additionally, the Mean-Variance of the branching angles was calculated to be 348.77, which measures the average squared deviation from the mean angle, provides another indicator of the spread of angle values. A higher variance suggests greater variability in the branching angles, while a lower variance indicates more consistency. To further elucidate the distribution of branching angles across our dataset, we visualized the angles for each image, as depicted in Figure 6 . This visualization aids in illustrating the range and patterns of branching angles within the data, offering a comprehensive overview of their distribution. This result established a benchmark baseline (Human Annotation) for retinal branching angle detection methods. This groundwork is critical for the development and assessment of detection techniques, as the baseline serves as a reference point for evaluating the performance of the implemented detection methods. By providing these standardized metrics, we enable more accurate and reliable comparisons of different detection approaches, thereby advancing the field of retinal branching angle analysis.We evaluated various methods by inputting the retina images into each algorithm and outputting the predicted retinal branching angles. For each image, we mainly compare the predicted Mean Average Angle of the implemented methods with the baseline using Mean Absolute Error (MAE) and Mean Squared Error (MSE). As shown in Table II , the ROI Window-based method previously achieved the best results among earlier works, detecting an average of 134 angles with a mean average angle of 76.71 degrees, an MAE of 6.724, and an MSE of 60.582. However, our proposed method surpasses all prior approaches. Specifically, our method detects an average of 113 angles with a mean average angle of 73.39 degrees, an MAE of 3.519, and an MSE of 17.896. To put these results into context, when the average angle for a retinal image is 71 degrees, our predicted value is 73 degrees. This is significantly more precise compared to the Rule-based method, which had an MAE of 12.479 and an MSE of 185.222, and the Line method, which had an MAE of 9.823 and an MSE of 132.399. While the ROI Window method detected more angles on average (134 compared to 113), our method significantly outperforms it in terms of accuracy. Our proposed method achieves a much lower MAE of 3.519 compared to the ROI Window's MAE of 6.724, demonstrating more precise angle predictions. Additionally, our method maintains a competitive Mean Standard Deviation of 22.017, and Mean-Variance of 487.421, which is very close to the Human Annotation (18.322/348.765), even with approximately four times the number of detected angles. This indicates that our predictions are not only more accurate but also more consistent with the actual distribution of branching angles in the retinal images. Therefore, the performance of our method demonstrates its robustness and reliability in detecting retinal branching angles, providing precise angle predictions from the observed retinal images.V. DISCUSSIONOur study offers several contributions to the detection and annotation of retinal branching angles, yet there are areas where further improvements can enhance the methodology and its applications. Firstly, the implemented ROI-Windowbased method demonstrates significant potential for improvement, given its structural similarity to our proposed method. This method already shows robust performance in detecting branching angles with accurate directional information, which can be further improved by incorporating our pruning method, as our node pruning can reduce outliers by tuning its threshold. Secondly, while our method for finding the root is highly efficient, its robustness can be further enhanced. Some retinal images may exhibit xenomorphic characteristics, presenting challenges for accurate root identification. Future work should focus on improving the algorithm's ability to handle such variations. Thirdly, our annotation tool, which facilitates the accurate marking of branching angles, can be further improved by integrating it with our proposed angle detection method. This integration would enable users to quickly generate an estimated angle map with a single click, significantly speeding up the annotation process. Users could then fine-tune these preliminary annotations, combining the efficiency of automated detection with the precision of manual correction. This enhancement would make the annotation process more efficient and user-friendly, particularly for large datasets. Lastly, in our ongoing efforts to enhance the usability of our annotation tool, we are committed to making it more accessible and clinic-friendly for a broader range of users including those with limited technical expertise to boost the medical research community.VI. CONCLUSIONIn this study, we introduced an annotation tool that enhances retinal vessel visibility through color channels and edge detection, improving annotation accuracy and efficiency. We also created a dataset of 40 high-quality retinal fundus images with branching angle annotations, validated by professional annotators and medical doctors. This dataset fills the gap in high-quality resources for retinal branching angle analysis. Our method for automatic detection of vascular branching angles outperforms previous approaches in terms of MAE and MSE, with highly consistent results. We offer open-source algorithms and datasets to extend retinal analysis and support more accurate diagnoses, contributing to medical imaging and ophthalmology. 
 
 Appendix:  



Reviews: review: title: title: Review of RBAD
paper_summary: paper_summary: This work contains the following main contents: (1)introduce an open-source annotation tool implemented in Python; (2) present a benchmark dataset by applying our customized annotation tool to a popular diabetic retinopathy dataset; (3) propose a bifurcation detection algorithm that efficiently calculates bifurcation angles on retinal segmentation maps
strength_weakness: strength_weakness: 1. There are some clerical errors in the text
2. The explaination of prune node and endpoint may be insufficient. It is better if it can be illustrated by figure.
questions: questions: 1. What is the difference in automation between this paper and other work (semi-automated)? Can be more detailed?
2. Why does the evaluating metirc contains the average number of angles? Do evaluating metrics used in object detection (like sensitivity, specificity, et al) are more suitable?
title: title: Innovative approach and high potential for impact
paper_summary: paper_summary: The paper addresses the challenge of accurately and efficiently detecting retinal bifurcation angles, a crucial metric for diagnosing various eye diseases. To overcome limitations of existing methods, the authors developed an open-source annotation tool and a benchmark dataset consisting of 40 retinal images with annotated bifurcation angles. Additionally, they propose a new method for automatically detecting and calculating these angles.
strength_weakness: strength_weakness: The method demonstrated a highly reliable performance. While its ability to identify variations or outliers is somewhat limited, this is an area for future development. Furthermore, making the method more accessible would be beneficial
questions: questions: None