# Machine Learning Techniques for Quantification of Knee Segmentation from MRI

Authors: Sujeet More; Jimmy Singla; Ahed Abugabah; Ahmad Ali AlZubi
Journal: Complexity
 (2020)

---

## Abstract

Magnetic resonance imaging (MRI) is precise and efficient for interpreting the soft and hard tissues. Moreover, for the detailed diagnosis of varied diseases such as knee rheumatoid arthritis (RA), segmentation of the knee magnetic resonance image is a challenging and complex task that has been explored broadly. However, the accuracy and reproducibility of segmentation approaches may require prior extraction of tissues from MR images. The advances in computational methods for segmentation are reliant on several parameters such as the complexity of the tissue, quality, and acquisition process involved. This review paper focuses and briefly describes the challenges faced by segmentation techniques from magnetic resonance images followed by an overview of diverse categories of segmentation approaches. The review paper also focuses on automatic approaches and semiautomatic approaches which are extensively used with performance metrics and sufficient achievement for clinical trial assistance. Furthermore, the results of different approaches related to MR sequences used to image the knee tissues and future aspects of the segmentation are discussed.


---

## Body

## 1. Introduction

Arthritis is one of the serious, prevalent joint diseases that cause disability and health issues in a large population. This arthritis is categorized with progressive degradation of joint tissues with a variety of abnormalities [1] and is a serious issue in recent years. Nearly sixty to seventy percent of people older than 60 years suffer from arthritis [2, 3]. Osteoarthritis is the damage of joint cartilage and leads to damage of functionality in the knee and hips, and the early signs can be observed with tears in cartilage. But rheumatoid arthritis is an autoimmune disease that largely affects the soft tissues around joints, bones, and cartilage. Therefore, the thickness and volume of the knee joint are the important parameters to evaluate rheumatoid arthritis.Different imaging modalities can be deployed to estimate the quantitative measures of knee arthritis. But the ability of magnetic resonance imaging to provide the quality imaging of bones with soft tissues, cartilage, and tendons as illustrated in Figure1, for the diagnosis and treatment of diverse diseases, is extensively utilized. Magnetic resonance images provide information about damage and inflammation with more sensitivity than other modalities. The magnetic resonance images may contain hundreds of slices depending on the sample rate. The magnetic resonance images are a valuable instrument for the treatment and study of different ailments. The significant cost of labor, hours for analyzing a single scan by the radiologist, and insensitive for detection of progression of arthritis make the treatment more difficult, expensive, and inefficient.Figure 1
Inclusion and exclusion criteria for selecting papers.Regardless of which disease is under study, the processing architecture includes a step known as segmentation for extracting the quantitative measures. The segmentation is a process of selecting the area of interest (AOI) concerning certain characteristics. LetX, Y, and Z be a finite grid of p, q pixels with Y set of intensities and Z objects, respectively.(1)X=x=p,q:p=0,1,…,p−1,q=0,1,…,q−1,Y=0,1,…,y−1,Z=0,1,…,z−1.Letg:X⟶Y and m:X⟶Z denote the image and an area, respectively. The image segmentation divides an image g into Z-connected subimages.This process of segmentation is important in providing the information about knee structure and the progress of the disease to the radiologist for diagnosis. However, this is a perilous and complex task for numerous reasons such as irregular shape, size, and connecting tissues. Therefore, many studies have focused on the progress of different methods to segment the knee magnetic resonance images [4, 5]. Although many studies on automatic and semiautomatic approaches are conducted, this segmentation from the magnetic resonance image is a future research problem [6] for the development of fully automated accurate and precise techniques. Thus, this scientific review provides a comprehensive knowledge of different computational methods utilized for the segmentation of magnetic resonance images. The review also focuses on exploring the challenges during segmentation.Consequently, articles were screened for selection based on title and abstract. Only original papers that are published in journals and conferences are selected. Books, book chapters, reports, and thesis are excluded from selection criteria. Articles that are written in English and focused on rheumatoid arthritis disease are selected for research. Articles written in other than English and not related to other diseases are excluded. Articles using the machine learning approach are included. Articles that entirely use image processing methods are excluded from research.This scientific article is organized as follows: Section2 discusses the challenges faced in the segmentation of magnetic resonance images. Section 3 describes the various approaches followed by automated and semiautomated segmentation methods with their metrics in Section 4. Section 5 includes discussion on existing work reviewed in this article. Section 6 accomplishes the conclusion of the review article.


## 2. Challenges in Segmentation of Magnetic Resonance Image

In the recent past, several efforts have been made for the segmentation of magnetic resonance images with the progression of arthritis. Generally, every algorithm is evaluated with different parameters on the image. These algorithms work well with a few sets of data but do not provide efficient performance measures. This is because of different features associated with knee structure that result in difficulties with the segmentation of magnetic resonance images.


### 2.1. Structure of Tissue

The structure of tissues is measured in terms of thickness. In normal humans, the knee cartilage density is about 2 mm to 4 mm with a curled surface with no blood vessels, and this density decreases at the rate of ∼2.5% to 50% [7, 8] every year in rheumatoid arthritis. These structures become thin and irregular in shape and are a challenging task in all slices. However, the edges between tibia and femur cartilage exhibit complications as represented in Figure 2(b), for providing reliable measures.Figure 2
(a) Irregular cartilage tissue. (b) Diffused edges between the tibia and femur bone.
(a)(b)


### 2.2. Magnetic Field Strength

The signal quantity in the mode of fissile quality is determined in magnetic resonance images. As the frequency of the magnetic field, strength 1.5 T, increases linearly, the signal-to-noise ratio (SNR) is amplified in magnetic resonance imaging [9]. Furthermore, there is proportionality between SNR and scan time. An increase in scan time increases the SNR value and often leads to artifacts in the image. For a high field, i.e., 3 T or 7 T strength, the visual quality and segmentation process of cartilage are improved. However, in 3 T magnetic strength, the T1-weighted images appear to be different than normal, and in 7 T strength, the T2-weighted images interrupt the scanning process and deploy more artifacts in the image leading to difficulty in finding a region of interest and other features [10].


### 2.3. Image Artifacts and Properties

MR imaging images are always liable to dissimilar types of artifacts. These relics are represented in the form of the pulse signal, affected volume, and chemical variations that precede an incorrect diagnosis. These artifacts are instigated by the magnetic field, resulting in uneven tissue representation. The chemical variations result in shady and bright spots on the edges of the tissues. These artifacts result in misinterpretation and may wrongly include the area of interest consisting of artifacts. The decreased quality of the edges due to volume effect and variations in signal intensities makes it difficult to develop computational approaches.The local variation in tissue properties that exhibits several problems within an image is identified as low visibility areas. Figure3 illustrates the variations and decreasing the edge quality of the knee magnetic resonance image.Figure 3
Artifacts in knee magnetic resonance image: (a) susceptibility and (b) artifacts produced by chemical (dashed arrow) and variation artifact (arrow) (reproduced from reference [11]).
(a)(b)As discussed, several issues and challenges need to be considered during the segmentation process for computational efficiency. The above challenges discussed are not only for automatic approaches but also for semiautomatic and manual approaches. The various segmentation techniques are reviewed and examined in the subsequent section.


## 2.1. Structure of Tissue

The structure of tissues is measured in terms of thickness. In normal humans, the knee cartilage density is about 2 mm to 4 mm with a curled surface with no blood vessels, and this density decreases at the rate of ∼2.5% to 50% [7, 8] every year in rheumatoid arthritis. These structures become thin and irregular in shape and are a challenging task in all slices. However, the edges between tibia and femur cartilage exhibit complications as represented in Figure 2(b), for providing reliable measures.Figure 2
(a) Irregular cartilage tissue. (b) Diffused edges between the tibia and femur bone.
(a)(b)


## 2.2. Magnetic Field Strength

The signal quantity in the mode of fissile quality is determined in magnetic resonance images. As the frequency of the magnetic field, strength 1.5 T, increases linearly, the signal-to-noise ratio (SNR) is amplified in magnetic resonance imaging [9]. Furthermore, there is proportionality between SNR and scan time. An increase in scan time increases the SNR value and often leads to artifacts in the image. For a high field, i.e., 3 T or 7 T strength, the visual quality and segmentation process of cartilage are improved. However, in 3 T magnetic strength, the T1-weighted images appear to be different than normal, and in 7 T strength, the T2-weighted images interrupt the scanning process and deploy more artifacts in the image leading to difficulty in finding a region of interest and other features [10].


## 2.3. Image Artifacts and Properties

MR imaging images are always liable to dissimilar types of artifacts. These relics are represented in the form of the pulse signal, affected volume, and chemical variations that precede an incorrect diagnosis. These artifacts are instigated by the magnetic field, resulting in uneven tissue representation. The chemical variations result in shady and bright spots on the edges of the tissues. These artifacts result in misinterpretation and may wrongly include the area of interest consisting of artifacts. The decreased quality of the edges due to volume effect and variations in signal intensities makes it difficult to develop computational approaches.The local variation in tissue properties that exhibits several problems within an image is identified as low visibility areas. Figure3 illustrates the variations and decreasing the edge quality of the knee magnetic resonance image.Figure 3
Artifacts in knee magnetic resonance image: (a) susceptibility and (b) artifacts produced by chemical (dashed arrow) and variation artifact (arrow) (reproduced from reference [11]).
(a)(b)As discussed, several issues and challenges need to be considered during the segmentation process for computational efficiency. The above challenges discussed are not only for automatic approaches but also for semiautomatic and manual approaches. The various segmentation techniques are reviewed and examined in the subsequent section.


## 3. Knee Tissue Segmentation Approaches

The knee bone is the major and largest bone within the human body and is the one that is most affected by rheumatoid arthritis disease. The cartilage and bone segments and features are important for the study of this rheumatoid arthritis disease. Recently, bone shape was proposed for predicting the rheumatoid arthritis progression [12]. This bone segmentation is a very critical assessment as connected cartilage degeneration is possible after the reconstruction of ACL [13]. In the class of approaches, the knee cartilage is physically or semiautomatically segmented, and area of interest (AOI) is estimated. There are numerous approaches for segmentation of bone and cartilage that are reviewed in this section, as illustrated in Figure 4.Figure 4
Taxonomy used in this article.


### 3.1. Manual and Traditional Computational Approaches

It is often seen that the traditional and manual methods used for segmentation result in reliable outcomes. In these traditional and manual methods, the tissues are manually segmented slice by slice from magnetic resonance images. Though the precision, sensitivity, and specificity of the manual methods are considered as the gold standard and high compared to automatic and semiautomatic techniques, it requires time-consuming efforts by the experts and provides inter- or intraobserver inconsistency by different professionals [14]. These manual techniques are not practically utilized for clinical trials, for example, Figure 5 representing femur bone which is manually segmented.Figure 5
Manual segmentation of femur bone.The manual strategy for the identification of segmented regions from the medical image is implemented with high precision. A statistical model [15] is developed to solve the issues related to knee image segmentation. This method used 60 images for identification of knee thickness with an extreme and tiniest density of the cartilage. The method was used on the public dataset with 150 knee images [16], and mostly the data consisted of T1-weighted images with an index of ∼73% and ∼74% of femur and tibia segments, respectively.


#### 3.1.1. Region-Based Technique

A region or area is poised of pixels of the neighbours, and the boundary is estimated by the difference between two regions. In this article, we discuss the most prevalent region-based techniques region growing and thresholding and edge-based method [17]. The initialization and process of segmentation start with some seed points. This process divides the regions based on the intensity of the pixel. The process requires only one seed point and homogenous properties of neighbouring pixels. Figure 6 represents the region growing technique, where two seed points are selected to segment the tissue.Figure 6
(a) Initial image with two seed points. (b) Outcome of region growing for one seed point. (c) Outcome of region growing for two seed points.
(a)(b)(c)


#### 3.1.2. Threshold or Intensity-Based Technique

The simplest and speediest method for segmentation is thresholding. The method assumes the regions with a diverse gray level. The diverse parts of the image are identified with histogram intensity. The intensity is divided into twofold: the first portion known as the foreground is having the pixel intensity higher than or equal to a threshold, second portion known as a background having the pixel intensity less than the threshold value as shown in equation:(2)fx,y=foregroundgx,y≥Pbackgroundgx,y<P,where f (x, y) is the pixel strength at (x, y) location and P is the threshold value. If the threshold value is inappropriate, then it leads to inaccurate segmentation outcomes [18]. This thresholding parameter occurs as a problem in magnetic resonance images and makes an image partition complex.The global thresholding does not provide better results for some types of images, like the images that do not have low contrast and low pixel intensity across the image. For these types of images, the thresholding provides a better outcome in some parts of the image and fails in other parts of the image [19]. In the local thresholding, the image is partitioned into vertical and horizontal lines. The local thresholding needs more time compared to global thresholding.


#### 3.1.3. Edge-Based Technique

The edges or boundaries are relevant information in the image. The incidence of the image is known by the number of changes in pixel strength that is utilized for identifying the different regions of those changes [20]. The automatic classification of boundaries is achieved by some high filters that avoid low-intensity data (uniform data) and preserves high-intensity data (edge data). These edges are extracted for utilization in sophisticated approaches [21]. The tissues appear to be weak or break that are not distinguishable by some methods with edge-based region and distance information.


#### 3.1.4. Atlas or Graph-Based Technique

The gray pixels of the image are utilized to create a graph with gray indices [22]. The atlas and graphs are designed manually by experts who assign labels to the tissue structures. The segmentation process is applied for transferring the image information from labelled data to subjects. Hence, the efficiency of the image segmentation relies on image registering and resemblance between subjects and atlas. The multiatlas method [23, 24] that is a sequence of multiple graphs is used to collect the information in terms of color, structure, and texture of the tissues with gray values. Furthermore, the image is partitioned into several multiple small regions and segmented with foreground and background areas.A three label approach that utilizes the atlas approach [25] for bone and cartilage segmentation is applied and evaluated on knee images. This study extensively described the performance of different classifiers, and the mean Jaccard value is 75.3% and 75.6% for femur and tibia bones. The graph method proposed in [26] offers a subject extraction with dimensionality where the weights of edges are related to boundary properties. The method evaluated the images obtained from the OAI database with a better performance parameter.


### 3.2. Fuzzy-Based Approaches

The fuzzy c-means approach provides a better classification of medical images. The disease is being classified into four levels based on severity which is introduced in [27]. The knee arthritis is categorized as normal, minimal, doubtful, and moderate as per the standard practice of the KL score. The morphometric measures of the knee assigned as osteoarthritis are investigated with image processing methods. The outcomes intensify and reinforce the traditional approaches with the harshness of the application. The tissues such as gray-level and white-level intensified are segmented with bias-filed corrected fuzzy C-means in [28]. A combination of the level-set and fuzzy method clusters the gray and white matter from the magnetic resonance images.The graphical processing units are providing fast computation along with the fuzzy method that is implemented in [29]. A large dataset can be processed by accelerating the fuzzy c-means with GPU to tackle the computational time around 2.24 times faster. The specific region of interest (ROI) is extracted, and the active contour of the cartilage with noninterpolated areas is estimated in [30]. The identical pixels are assigned with some classes by using the membership function to detect the pathological changes in the tissue structure of the knee magnetic resonance image.


### 3.3. Machine Learning Approaches

In the recent past, a great effort has been employed on machine learning practices for solving the segmentation problem [31]. The unsupervised learning approach does not need training or labelling the data. The labelling of the image is accomplished by exploiting shapes in the voxel strength features. The supervised learning approach requires training for the algorithm and learns from the voxel examples. The support vector machines with multiple sets of images [32] are used for the evaluation of knee cartilage with better sensitivity. 36 different dimensional features are selected and experimented on 4 different classifiers with an AUC of 76%. The hidden biomedical information is analyzed clearly for the progression of arthritis.The subcortical segmentation method is used to construct the graph [33]. The reliable and automated segmentation of structures from magnetic resonance images is more important for shape analysis and volumetric study. The random forest classifier is used for assigning a cost for each node in the graph and evaluates the dice index and putamen measures. A method based on the K-NN strategy [34] showed successful segmentation for cartilage. The cartilage is segmented from the background that leads to false-positive values. Furthermore, the study was modified and presented with SVM [35]. The main difficulty of SVM is the distribution and independence of data instances. To incorporate the contextual data such as intensity and structural information, the ML approach builds feature vector [36] with simultaneous use of multiple images. The method utilizes the T1-weighted images for cartilage segmentation problems that are multicontrast and deposited with fat and water gradient. A method based on pattern recognition [32] for gradient data of the knee cartilage segment reports averages dice value to 0.76.


### 3.4. Deep Learning Approaches

A priori knowledge of shape analysis using a convolution neural network [37] is proposed that incorporates the SSM. The method uses 40 validation and 50 subjects for the estimation of knee images. The voxel accuracy of 89% is estimated with manual adjustment, and the correlation between the developed method and the ground truth is investigated. The automated segmentation is based on the CNN approach [38]. This method applies volumetric structure on images and manually segments the magnetic resonance images. The automatic segmentation of images achieved better outcomes with a dice similarity score of 0.95 and a precision of 0.95 on femur tissue.The contextual constrained neural network combined with level set evolution [39] is used for the segmentation of knee magnetic resonance images. The movement of the patella in the knee T2-weighted image should be normal, so tracking this with segmenting the femur and patella is required. The performance of the T1-weighted knee magnetic resonance images in terms of ASD and RMSD is estimated with the convolution network proposed in [40]. The deep learning approach has some promising issues, such as high dependency on the quality and amount of training data that tends to overfit of data [41].The 3D deformable approach introduced in [42] uses CNN with 3D simplex deformable modeling. The method performs pixelwise, multiclass classification which has been tested on the public knee image dataset. This method provides state-of-the-art performance with superior accuracy and segmentation error of VOE. An extended version of the method is proposed in [43] for segmenting the cartilage lesion detection. The T2-weighted fast spin-echo magnetic resonance images of 175 subjects are used with CNN. The ROC and k statistics were used for analyzing the performance and intraobserver detection of cartilage lesions. The volumetric assessment of knee cartilage is introduced in [44]. The dice score and VOE are estimated for different architectures using the CNN approach.


## 3.1. Manual and Traditional Computational Approaches

It is often seen that the traditional and manual methods used for segmentation result in reliable outcomes. In these traditional and manual methods, the tissues are manually segmented slice by slice from magnetic resonance images. Though the precision, sensitivity, and specificity of the manual methods are considered as the gold standard and high compared to automatic and semiautomatic techniques, it requires time-consuming efforts by the experts and provides inter- or intraobserver inconsistency by different professionals [14]. These manual techniques are not practically utilized for clinical trials, for example, Figure 5 representing femur bone which is manually segmented.Figure 5
Manual segmentation of femur bone.The manual strategy for the identification of segmented regions from the medical image is implemented with high precision. A statistical model [15] is developed to solve the issues related to knee image segmentation. This method used 60 images for identification of knee thickness with an extreme and tiniest density of the cartilage. The method was used on the public dataset with 150 knee images [16], and mostly the data consisted of T1-weighted images with an index of ∼73% and ∼74% of femur and tibia segments, respectively.


### 3.1.1. Region-Based Technique

A region or area is poised of pixels of the neighbours, and the boundary is estimated by the difference between two regions. In this article, we discuss the most prevalent region-based techniques region growing and thresholding and edge-based method [17]. The initialization and process of segmentation start with some seed points. This process divides the regions based on the intensity of the pixel. The process requires only one seed point and homogenous properties of neighbouring pixels. Figure 6 represents the region growing technique, where two seed points are selected to segment the tissue.Figure 6
(a) Initial image with two seed points. (b) Outcome of region growing for one seed point. (c) Outcome of region growing for two seed points.
(a)(b)(c)


### 3.1.2. Threshold or Intensity-Based Technique

The simplest and speediest method for segmentation is thresholding. The method assumes the regions with a diverse gray level. The diverse parts of the image are identified with histogram intensity. The intensity is divided into twofold: the first portion known as the foreground is having the pixel intensity higher than or equal to a threshold, second portion known as a background having the pixel intensity less than the threshold value as shown in equation:(2)fx,y=foregroundgx,y≥Pbackgroundgx,y<P,where f (x, y) is the pixel strength at (x, y) location and P is the threshold value. If the threshold value is inappropriate, then it leads to inaccurate segmentation outcomes [18]. This thresholding parameter occurs as a problem in magnetic resonance images and makes an image partition complex.The global thresholding does not provide better results for some types of images, like the images that do not have low contrast and low pixel intensity across the image. For these types of images, the thresholding provides a better outcome in some parts of the image and fails in other parts of the image [19]. In the local thresholding, the image is partitioned into vertical and horizontal lines. The local thresholding needs more time compared to global thresholding.


### 3.1.3. Edge-Based Technique

The edges or boundaries are relevant information in the image. The incidence of the image is known by the number of changes in pixel strength that is utilized for identifying the different regions of those changes [20]. The automatic classification of boundaries is achieved by some high filters that avoid low-intensity data (uniform data) and preserves high-intensity data (edge data). These edges are extracted for utilization in sophisticated approaches [21]. The tissues appear to be weak or break that are not distinguishable by some methods with edge-based region and distance information.


### 3.1.4. Atlas or Graph-Based Technique

The gray pixels of the image are utilized to create a graph with gray indices [22]. The atlas and graphs are designed manually by experts who assign labels to the tissue structures. The segmentation process is applied for transferring the image information from labelled data to subjects. Hence, the efficiency of the image segmentation relies on image registering and resemblance between subjects and atlas. The multiatlas method [23, 24] that is a sequence of multiple graphs is used to collect the information in terms of color, structure, and texture of the tissues with gray values. Furthermore, the image is partitioned into several multiple small regions and segmented with foreground and background areas.A three label approach that utilizes the atlas approach [25] for bone and cartilage segmentation is applied and evaluated on knee images. This study extensively described the performance of different classifiers, and the mean Jaccard value is 75.3% and 75.6% for femur and tibia bones. The graph method proposed in [26] offers a subject extraction with dimensionality where the weights of edges are related to boundary properties. The method evaluated the images obtained from the OAI database with a better performance parameter.


## 3.1.1. Region-Based Technique

A region or area is poised of pixels of the neighbours, and the boundary is estimated by the difference between two regions. In this article, we discuss the most prevalent region-based techniques region growing and thresholding and edge-based method [17]. The initialization and process of segmentation start with some seed points. This process divides the regions based on the intensity of the pixel. The process requires only one seed point and homogenous properties of neighbouring pixels. Figure 6 represents the region growing technique, where two seed points are selected to segment the tissue.Figure 6
(a) Initial image with two seed points. (b) Outcome of region growing for one seed point. (c) Outcome of region growing for two seed points.
(a)(b)(c)


## 3.1.2. Threshold or Intensity-Based Technique

The simplest and speediest method for segmentation is thresholding. The method assumes the regions with a diverse gray level. The diverse parts of the image are identified with histogram intensity. The intensity is divided into twofold: the first portion known as the foreground is having the pixel intensity higher than or equal to a threshold, second portion known as a background having the pixel intensity less than the threshold value as shown in equation:(2)fx,y=foregroundgx,y≥Pbackgroundgx,y<P,where f (x, y) is the pixel strength at (x, y) location and P is the threshold value. If the threshold value is inappropriate, then it leads to inaccurate segmentation outcomes [18]. This thresholding parameter occurs as a problem in magnetic resonance images and makes an image partition complex.The global thresholding does not provide better results for some types of images, like the images that do not have low contrast and low pixel intensity across the image. For these types of images, the thresholding provides a better outcome in some parts of the image and fails in other parts of the image [19]. In the local thresholding, the image is partitioned into vertical and horizontal lines. The local thresholding needs more time compared to global thresholding.


## 3.1.3. Edge-Based Technique

The edges or boundaries are relevant information in the image. The incidence of the image is known by the number of changes in pixel strength that is utilized for identifying the different regions of those changes [20]. The automatic classification of boundaries is achieved by some high filters that avoid low-intensity data (uniform data) and preserves high-intensity data (edge data). These edges are extracted for utilization in sophisticated approaches [21]. The tissues appear to be weak or break that are not distinguishable by some methods with edge-based region and distance information.


## 3.1.4. Atlas or Graph-Based Technique

The gray pixels of the image are utilized to create a graph with gray indices [22]. The atlas and graphs are designed manually by experts who assign labels to the tissue structures. The segmentation process is applied for transferring the image information from labelled data to subjects. Hence, the efficiency of the image segmentation relies on image registering and resemblance between subjects and atlas. The multiatlas method [23, 24] that is a sequence of multiple graphs is used to collect the information in terms of color, structure, and texture of the tissues with gray values. Furthermore, the image is partitioned into several multiple small regions and segmented with foreground and background areas.A three label approach that utilizes the atlas approach [25] for bone and cartilage segmentation is applied and evaluated on knee images. This study extensively described the performance of different classifiers, and the mean Jaccard value is 75.3% and 75.6% for femur and tibia bones. The graph method proposed in [26] offers a subject extraction with dimensionality where the weights of edges are related to boundary properties. The method evaluated the images obtained from the OAI database with a better performance parameter.


## 3.2. Fuzzy-Based Approaches

The fuzzy c-means approach provides a better classification of medical images. The disease is being classified into four levels based on severity which is introduced in [27]. The knee arthritis is categorized as normal, minimal, doubtful, and moderate as per the standard practice of the KL score. The morphometric measures of the knee assigned as osteoarthritis are investigated with image processing methods. The outcomes intensify and reinforce the traditional approaches with the harshness of the application. The tissues such as gray-level and white-level intensified are segmented with bias-filed corrected fuzzy C-means in [28]. A combination of the level-set and fuzzy method clusters the gray and white matter from the magnetic resonance images.The graphical processing units are providing fast computation along with the fuzzy method that is implemented in [29]. A large dataset can be processed by accelerating the fuzzy c-means with GPU to tackle the computational time around 2.24 times faster. The specific region of interest (ROI) is extracted, and the active contour of the cartilage with noninterpolated areas is estimated in [30]. The identical pixels are assigned with some classes by using the membership function to detect the pathological changes in the tissue structure of the knee magnetic resonance image.


## 3.3. Machine Learning Approaches

In the recent past, a great effort has been employed on machine learning practices for solving the segmentation problem [31]. The unsupervised learning approach does not need training or labelling the data. The labelling of the image is accomplished by exploiting shapes in the voxel strength features. The supervised learning approach requires training for the algorithm and learns from the voxel examples. The support vector machines with multiple sets of images [32] are used for the evaluation of knee cartilage with better sensitivity. 36 different dimensional features are selected and experimented on 4 different classifiers with an AUC of 76%. The hidden biomedical information is analyzed clearly for the progression of arthritis.The subcortical segmentation method is used to construct the graph [33]. The reliable and automated segmentation of structures from magnetic resonance images is more important for shape analysis and volumetric study. The random forest classifier is used for assigning a cost for each node in the graph and evaluates the dice index and putamen measures. A method based on the K-NN strategy [34] showed successful segmentation for cartilage. The cartilage is segmented from the background that leads to false-positive values. Furthermore, the study was modified and presented with SVM [35]. The main difficulty of SVM is the distribution and independence of data instances. To incorporate the contextual data such as intensity and structural information, the ML approach builds feature vector [36] with simultaneous use of multiple images. The method utilizes the T1-weighted images for cartilage segmentation problems that are multicontrast and deposited with fat and water gradient. A method based on pattern recognition [32] for gradient data of the knee cartilage segment reports averages dice value to 0.76.


## 3.4. Deep Learning Approaches

A priori knowledge of shape analysis using a convolution neural network [37] is proposed that incorporates the SSM. The method uses 40 validation and 50 subjects for the estimation of knee images. The voxel accuracy of 89% is estimated with manual adjustment, and the correlation between the developed method and the ground truth is investigated. The automated segmentation is based on the CNN approach [38]. This method applies volumetric structure on images and manually segments the magnetic resonance images. The automatic segmentation of images achieved better outcomes with a dice similarity score of 0.95 and a precision of 0.95 on femur tissue.The contextual constrained neural network combined with level set evolution [39] is used for the segmentation of knee magnetic resonance images. The movement of the patella in the knee T2-weighted image should be normal, so tracking this with segmenting the femur and patella is required. The performance of the T1-weighted knee magnetic resonance images in terms of ASD and RMSD is estimated with the convolution network proposed in [40]. The deep learning approach has some promising issues, such as high dependency on the quality and amount of training data that tends to overfit of data [41].The 3D deformable approach introduced in [42] uses CNN with 3D simplex deformable modeling. The method performs pixelwise, multiclass classification which has been tested on the public knee image dataset. This method provides state-of-the-art performance with superior accuracy and segmentation error of VOE. An extended version of the method is proposed in [43] for segmenting the cartilage lesion detection. The T2-weighted fast spin-echo magnetic resonance images of 175 subjects are used with CNN. The ROC and k statistics were used for analyzing the performance and intraobserver detection of cartilage lesions. The volumetric assessment of knee cartilage is introduced in [44]. The dice score and VOE are estimated for different architectures using the CNN approach.


## 4. Performance of Segmentation Approaches

Most of the studies that involve segmentation of the cartilage have been evaluated by estimating the performance parameters against the ground truth. This ground truth is being prepared using manual extraction methods by an expert. To estimate the efficiency of progressive methods for estimating the cartilage boundaries, different metrics such as sensitivity, specificity, reliability, dice similarity coefficient (DSC), accuracy, and efficiency are considered from [6, 45, 46]. Based on the ground truth and developed methods, the sensitivity, specificity, DSC, and accuracy are measured using equations (3)–(6), respectively:(3)sensitivity=TPTP+FN,(4)specificity=TNTN+FP,(5)DSC=2TPTP+FP+FN+TP,(6)accuracy=TP+TNTP+TN+FP+FN,where TP is true positive, i.e., area correctly labelled as cartilage area, TN is true negative, i.e., area correctly labelled as noncartilage area, FP is false positive, i.e., area incorrectly labelled as cartilage area, and FN is false negative, i.e., area incorrectly labelled as noncartilage area. These parameters are estimated with base values over the segmented areas.Sensitivity is the parameter for a ratio of true positives and measures approximate to 100% for accurate classification of cartilage areas of ground truth and developed methods. In this parameter, the total quantity of small false negatives represents high sensitivity. The specificity parameter provides the ratio of true negatives that are appropriately identified through comparative analysis. A combined effect of both sensitivity and specificity reflects the best efficiency of the developed approach. Accurateness is a measure for representing the ground truth agreeing with the developed methods. It shows how many correctly classified cartilage areas are included and noncartilage areas are excluded. Other performance parameters are also considered by many researchers.


### 4.1. Automatic Segmentation Approaches

Automatic segmentation from magnetic resonance image is being achieved for widespread studies in rheumatoid arthritis, which makes the process that requires to be accomplished fast and consistent. Research provides more sophisticated methods to segment the images and moves towards an entirely automatic process. Once the tissues are segmented, the quantitative or measurable analysis is carried out by estimating the volume, density, and structure of the tissues. Most of the automated techniques that are grounded on voxel arrangement require a large number of training dataset. The most used machine learning approach is K-nearest neighbour applied with a multivariant classifier.Several studies have discussed the utilization of voxel classification to segment the tissues. Although these voxel methods can segment the magnetic resonance images, the data required is the primary concern. The main limitation of this kind of method is the generation of a new dataset. The automatic methods used by different researchers with different metrics are demonstrated in Table1. All methods discussed in the table are utilizing intensity or pixel for the computation. Generally, image segmentation can be achieved by using numerous parameters such as edges, regions, clusters, and image intensity. The threshold-based segmentation approach is limited by tissue properties such as (a) varying intensities of cartilage region, (b) low signal intensity, and (c) low visibility of tissues. The development of these techniques is still under research, and a lot yet has to be discovered. Current developments in the area of segmentation have been aimed at multiple pulse sequence of magnetic resonance images.Table 1
Automated segmentation approaches.
StudyMR sampleNumber of subjectsMethods usedEfficiency measures[47]3D DESS33 subjectsRegion growing and seed selectionDSC 82.8%, 83.1%, and 72.6% for femur, tibia, and patella, respectively[15]T1-weighted 3D DESS20 subjectsSide selection and coupled cartilage segmentationDSC 0.80Sensitivity 90.0Specificity 99.8[48]3D DESS20 subjectsSpatial fuzzy C-meanDSC of femur, tibia, and patella are 87.1, 81.1, and 84.8, respectively[37]DESS50 subjectsStatistical shape modelAccuracy 74 ± 5DSC 86.1%[49]2D scan100 subjectsRandom walker algorithmDSC 0.8758[50]DESS8 subjectsDRLSE algorithmDSC 90.28Sensitivity 91.14Specificity 99.12[35]3D DESS10 subjectsK-nearest neighborSensitivity 84.2Specificity 99.9Dice volume 0.81[34]3D image139 subjectsVoxel classificationDSC 0.80 ± 0.04[51]DESS13 subjectsHierarchical classifier and random forestSurface positioning errors femur 0.03 ± 0.19 tibia 0.10 ± 0.17[52]T1-weighted 3D SPGR and T2-weighted GRE9 subjectsThree-label segmentationMedian dice coefficient femur 0.935 and tibia 0.938[53]3D SPGR155 subjectsLongitudinal three label segmentationFemur DSC 75%Sensitivity 78%Specificity 99.9%[54]Sagittal DESS88 subjectsConvolution neural networkDSC 83.8%[55]DESS40 subjectsActive appearance model (statistical model)RMSE for femur 1.49, tibia 1.21[56]PDW TSE14 subjectsDistance-weighted directional gradientIntensity level, 2∼4[57]T1-weighted30 subjectsMultiatlas segmentation constrained graphFemur average surface distance 0.36 mm[58]2D ASM140 subjectsRigid multiatlas registrationDice volume tibia 0.8 and femur 0.87[59]T1-weighted SPGR20 subjectsStatistical methodDSC of femur 0.871, tibia 0.852, and patella 0.645. Sensitivity of femur 0.947, tibia 0.949, and patella 0.909. Specificity of femur 0.988, tibia 0.993, and patella 0.996.[60]T1-weighted FS14 subjectsValidation of statistical methodMeniscus DSC 0.75, sensitivity 0.72, and specificity 1.00[61]T1-weighted100 subjectsCoarse to fine method with min-cutDSC of femur 0.947, and tibia 0.968To automatically segment the tissues from the magnetic resonance image, the successive approach is performed with automated segmentation by extracting probabilities using the k-NN classifier. The statistical shape and models are more precise segmentation methods. However, the utilization of these methods has some limitations due to the initialization of sensitivity [62].


### 4.2. Semiautomatic Segmentation Approaches

This approach requires minimal user intervention during the segmentation process. These approaches are considered for reducing the efforts required for manual segmentation. These approaches with the need for human interaction complete some tasks of pattern recognition and image processing. The semiautomated approach uses some of the methods such as snake contours, gradient vectors, watershed, graph search, and region growing for segmentation. As depicted in Table2, the methods used for semiautomatic segmentation have resulted in great success in extracting the soft and hard tissue features. A repetitive averaging filter is used for replicating Gaussian function in the deblurring process. To dehaze the images by segmentation using the super pixels technique [72], intensities are summed and the transmission map is estimated. This method preserves the features such as texture and edges of real-time images.Table 2
Semiautomated segmentation approaches.
StudyMR sampleNumber of subjectsMethods usedEfficiency measures[63]PDW SPAIR12 subjectsRandom forestFemur DSC 94.9%Tibia DSC 92.5%[27]T1-weighted axial103 subjectsSupport vector machineAccuracy 72%[58]CCBR159 subjectsPLDS methodDice volume 0.82[64]3D SGPR4 subjectsRegion growingError −6.53%[65]T1-weighted image5 subjectsActive shape modelMean error −0.57[66]Flash 3D15 subjectsB-spline with manual adjustmentInterobserver 3.3 to 13.6[45]3DMR20 subjectsActive contourThickness 0.996 and 0.998 for femur and tibia[4]SPGR7 subjectsWatershed transformationDSC 89.5%Sensitivity 90%Specificity 99.9%[23]3D DESS320 slicesGraph-cut algorithmDSC 94.3%[67]Flash GRE50 subjectsK-means with manual adjustmentDSC 0.77 and 0.80Sensitivity 83.1 and 85.3Specificity 99.9 and 099.9 for femur and tibia bones, respectively[68]DESS17 subjectsSupport vector machineDSC patella 0.82, tibia 0.83, and femur 0.86[23]3T MR images10 subjectsGraph-cut methodDSC 0.943[13]3D DESS12 subjectsActive contour modelRoot mean square 0.8% to 1.3%[69]3D DESS10 subjectsMesh morphing approachMean S.D of femur 0.87, tibia 0.40 and, patella 0.53[70]T1-weighted15 subjectsWatershed methodCartilage volume tibia 3.3 mmThe semiautomated approach requires less computational time and provides an accurate outcome in the studies when compared with manual techniques, but includes the human interaction, viewer inconsistency that needs to be performed. Figure7 represents the femur and tibia tissue segmentation using the atlas-based approach for sagittal and axial images. The method discussed in [14] provides a multicontrast image that has a rich set of the feature given for classifier using support vector machine technique. Furthermore, besides the local features, the global features are also included in the anatomical direction of femur and tibia bones. SVM is extended, and new SVM-DRF includes the independent estimation of pixels to incorporate the spatial dependency between neighbouring pixels.Figure 7
First row: segmentation of knee bone (left: original image, middle: automated segmentation, and right: segmented bone). Second row (left: original image, middle: automated segmentation, and right: cartilage segmented) (reproduced from reference [25]).


## 4.1. Automatic Segmentation Approaches

Automatic segmentation from magnetic resonance image is being achieved for widespread studies in rheumatoid arthritis, which makes the process that requires to be accomplished fast and consistent. Research provides more sophisticated methods to segment the images and moves towards an entirely automatic process. Once the tissues are segmented, the quantitative or measurable analysis is carried out by estimating the volume, density, and structure of the tissues. Most of the automated techniques that are grounded on voxel arrangement require a large number of training dataset. The most used machine learning approach is K-nearest neighbour applied with a multivariant classifier.Several studies have discussed the utilization of voxel classification to segment the tissues. Although these voxel methods can segment the magnetic resonance images, the data required is the primary concern. The main limitation of this kind of method is the generation of a new dataset. The automatic methods used by different researchers with different metrics are demonstrated in Table1. All methods discussed in the table are utilizing intensity or pixel for the computation. Generally, image segmentation can be achieved by using numerous parameters such as edges, regions, clusters, and image intensity. The threshold-based segmentation approach is limited by tissue properties such as (a) varying intensities of cartilage region, (b) low signal intensity, and (c) low visibility of tissues. The development of these techniques is still under research, and a lot yet has to be discovered. Current developments in the area of segmentation have been aimed at multiple pulse sequence of magnetic resonance images.Table 1
Automated segmentation approaches.
StudyMR sampleNumber of subjectsMethods usedEfficiency measures[47]3D DESS33 subjectsRegion growing and seed selectionDSC 82.8%, 83.1%, and 72.6% for femur, tibia, and patella, respectively[15]T1-weighted 3D DESS20 subjectsSide selection and coupled cartilage segmentationDSC 0.80Sensitivity 90.0Specificity 99.8[48]3D DESS20 subjectsSpatial fuzzy C-meanDSC of femur, tibia, and patella are 87.1, 81.1, and 84.8, respectively[37]DESS50 subjectsStatistical shape modelAccuracy 74 ± 5DSC 86.1%[49]2D scan100 subjectsRandom walker algorithmDSC 0.8758[50]DESS8 subjectsDRLSE algorithmDSC 90.28Sensitivity 91.14Specificity 99.12[35]3D DESS10 subjectsK-nearest neighborSensitivity 84.2Specificity 99.9Dice volume 0.81[34]3D image139 subjectsVoxel classificationDSC 0.80 ± 0.04[51]DESS13 subjectsHierarchical classifier and random forestSurface positioning errors femur 0.03 ± 0.19 tibia 0.10 ± 0.17[52]T1-weighted 3D SPGR and T2-weighted GRE9 subjectsThree-label segmentationMedian dice coefficient femur 0.935 and tibia 0.938[53]3D SPGR155 subjectsLongitudinal three label segmentationFemur DSC 75%Sensitivity 78%Specificity 99.9%[54]Sagittal DESS88 subjectsConvolution neural networkDSC 83.8%[55]DESS40 subjectsActive appearance model (statistical model)RMSE for femur 1.49, tibia 1.21[56]PDW TSE14 subjectsDistance-weighted directional gradientIntensity level, 2∼4[57]T1-weighted30 subjectsMultiatlas segmentation constrained graphFemur average surface distance 0.36 mm[58]2D ASM140 subjectsRigid multiatlas registrationDice volume tibia 0.8 and femur 0.87[59]T1-weighted SPGR20 subjectsStatistical methodDSC of femur 0.871, tibia 0.852, and patella 0.645. Sensitivity of femur 0.947, tibia 0.949, and patella 0.909. Specificity of femur 0.988, tibia 0.993, and patella 0.996.[60]T1-weighted FS14 subjectsValidation of statistical methodMeniscus DSC 0.75, sensitivity 0.72, and specificity 1.00[61]T1-weighted100 subjectsCoarse to fine method with min-cutDSC of femur 0.947, and tibia 0.968To automatically segment the tissues from the magnetic resonance image, the successive approach is performed with automated segmentation by extracting probabilities using the k-NN classifier. The statistical shape and models are more precise segmentation methods. However, the utilization of these methods has some limitations due to the initialization of sensitivity [62].


## 4.2. Semiautomatic Segmentation Approaches

This approach requires minimal user intervention during the segmentation process. These approaches are considered for reducing the efforts required for manual segmentation. These approaches with the need for human interaction complete some tasks of pattern recognition and image processing. The semiautomated approach uses some of the methods such as snake contours, gradient vectors, watershed, graph search, and region growing for segmentation. As depicted in Table2, the methods used for semiautomatic segmentation have resulted in great success in extracting the soft and hard tissue features. A repetitive averaging filter is used for replicating Gaussian function in the deblurring process. To dehaze the images by segmentation using the super pixels technique [72], intensities are summed and the transmission map is estimated. This method preserves the features such as texture and edges of real-time images.Table 2
Semiautomated segmentation approaches.
StudyMR sampleNumber of subjectsMethods usedEfficiency measures[63]PDW SPAIR12 subjectsRandom forestFemur DSC 94.9%Tibia DSC 92.5%[27]T1-weighted axial103 subjectsSupport vector machineAccuracy 72%[58]CCBR159 subjectsPLDS methodDice volume 0.82[64]3D SGPR4 subjectsRegion growingError −6.53%[65]T1-weighted image5 subjectsActive shape modelMean error −0.57[66]Flash 3D15 subjectsB-spline with manual adjustmentInterobserver 3.3 to 13.6[45]3DMR20 subjectsActive contourThickness 0.996 and 0.998 for femur and tibia[4]SPGR7 subjectsWatershed transformationDSC 89.5%Sensitivity 90%Specificity 99.9%[23]3D DESS320 slicesGraph-cut algorithmDSC 94.3%[67]Flash GRE50 subjectsK-means with manual adjustmentDSC 0.77 and 0.80Sensitivity 83.1 and 85.3Specificity 99.9 and 099.9 for femur and tibia bones, respectively[68]DESS17 subjectsSupport vector machineDSC patella 0.82, tibia 0.83, and femur 0.86[23]3T MR images10 subjectsGraph-cut methodDSC 0.943[13]3D DESS12 subjectsActive contour modelRoot mean square 0.8% to 1.3%[69]3D DESS10 subjectsMesh morphing approachMean S.D of femur 0.87, tibia 0.40 and, patella 0.53[70]T1-weighted15 subjectsWatershed methodCartilage volume tibia 3.3 mmThe semiautomated approach requires less computational time and provides an accurate outcome in the studies when compared with manual techniques, but includes the human interaction, viewer inconsistency that needs to be performed. Figure7 represents the femur and tibia tissue segmentation using the atlas-based approach for sagittal and axial images. The method discussed in [14] provides a multicontrast image that has a rich set of the feature given for classifier using support vector machine technique. Furthermore, besides the local features, the global features are also included in the anatomical direction of femur and tibia bones. SVM is extended, and new SVM-DRF includes the independent estimation of pixels to incorporate the spatial dependency between neighbouring pixels.Figure 7
First row: segmentation of knee bone (left: original image, middle: automated segmentation, and right: segmented bone). Second row (left: original image, middle: automated segmentation, and right: cartilage segmented) (reproduced from reference [25]).


## 5. Discussion

The assessment of tissues is the crucial stage in assessing the progression of rheumatoid arthritis. The segmentation process is not only needed for treatment but also used for quantitative parameter estimation. This has become a need due to complex variations related to different tissues in the knee magnetic resonance image. The segmentation techniques discussed above are broadly classified as traditional or manual, machine learning approach, deep learning approach, automatic approach, and semiautomatic approach. Manual segmentation is a tedious task and has result variability amongst professionals within the same image dataset. Semiautomatic segmentation is a less complex task compared to manual segmentation but is the same in terms of variations of manual segmentation. The intraobserver reproducibility for the density is measured in several areas of cartilage [73]. The discussed techniques are included such as region growing, active contour, and graph-cut. All these techniques require less human intervention and provide better sensitivity, specificity, and dice similarity scores. To overcome the drawback of inter- or intraobserver, many researchers came up with automated segmentation approaches that are discussed and reviewed in Section 4.Table3, provides a comparative analysis of other articles and this article. This paper shows an indetail insight selecting all the features related to rheumatoid arthritis. In Table 3, this review work selects synovial fluid and meniscus that is the main source for rheumatoid arthritis. The use of magnetic resonance images and machine learning approaches is the key parameter for selecting the review articles. This review gives an overview of different techniques utilized for the segmentation of tissues with resolving major challenges in the magnetic resonance image. The techniques available require more time for computation for segmentation of the entire knee magnetic resonance image. Some of the automated and semiautomated approaches discussed above are highlighted with the main strengths and drawbacks. However, the growing rate of data and increasing rate of computational power make the machine learning and deep learning approaches most promising for future applications concerning IoT applications [79–83].Table 3
Comparative analysis with other published review articles.
ArticleDatatype consideredMedical disease consideredClassification and summarization of existing workMagnetic resonance imagingMachine learningCartilageMeniscusLigaments and lesionsPatella, tibia, and femur boneSynovial fluid[74]√√XX√XX[75]√√√XXXX[76]√√X√XXX[77]X√XXXX√[78]√√XXX√XThis article√√√√√√√


## 6. Conclusion

This systematic review approximates the magnetic resonance imaging key parameters reminiscent of rheumatoid arthritis. Our nature of the study selects sensitivity, specificity, and dice similarity index for an approximation of imaging results. However, from the results of our study, more frequent parameters for rheumatoid arthritis disease such as synovial fluid volume, meniscus volume, the dissimilar structure of tibia, femur, and tears of ligaments. Different segmentation techniques are considered for review with pros and cons of themselves that may increase the productivity of a novel hybrid approach.Our future work includes selecting more parameters that are directly related to rheumatoid arthritis. The synovial fluid and cystic lesions are the less considered features than the cartilage and other bone tissues. These considered features may be used in the future for targeted treatment, in those most required for rheumatoid arthritis diagnosis.



---

*Source: 6613191-2020-12-07.xml*