Gradient‐based kernel selection technique for tumour detection and extraction of medical images using graph cut

IET Image ProcessingVolume 14, Issue 1 p. 84-93 Research ArticleFree Access Gradient-based kernel selection technique for tumour detection and extraction of medical images using graph cut Jyotsna Dogra, Corresponding Author Jyotsna Dogra jyotsnadogra1989@gmail.com orcid.org/0000-0001-8135-7535 Department of Electronics and Communication, Jaypee University of Information Technology, Wakhnaghatn, Solan, IndiaSearch for more papers by this authorShruti Jain, Shruti Jain Department of Electronics and Communication, Jaypee University of Information Technology, Wakhnaghatn, Solan, IndiaSearch for more papers by this authorMeenakshi Sood, Meenakshi Sood Department of Electronics and Communication, Jaypee University of Information Technology, Wakhnaghatn, Solan, India Current affiliation: National Institute of Technical Teachers Training & Research, Chandigarh, MHRD, IndiaSearch for more papers by this author Jyotsna Dogra, Corresponding Author Jyotsna Dogra jyotsnadogra1989@gmail.com orcid.org/0000-0001-8135-7535 Department of Electronics and Communication, Jaypee University of Information Technology, Wakhnaghatn, Solan, IndiaSearch for more papers by this authorShruti Jain, Shruti Jain Department of Electronics and Communication, Jaypee University of Information Technology, Wakhnaghatn, Solan, IndiaSearch for more papers by this authorMeenakshi Sood, Meenakshi Sood Department of Electronics and Communication, Jaypee University of Information Technology, Wakhnaghatn, Solan, India Current affiliation: National Institute of Technical Teachers Training & Research, Chandigarh, MHRD, IndiaSearch for more papers by this author First published: 01 January 2020 https://doi.org/10.1049/iet-ipr.2018.6615Citations: 2 AboutSectionsPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onFacebookTwitterLinkedInRedditWechat Abstract Magnetic resonance imaging is a powerful, ubiquitous imaging technique that provides detailed high-contrast images differentiating soft tissues. The low radio-frequency bias field creates intensity inhomogeneity generating low contrast that often creates difficulty for quantitative and qualitative analyses. Segmentation aids in analysis of changes occurring in brain, where bias effect severely affects performance. The graph-cut (GC) segmentation provides supervised computer-assisted diagnosis and treatment. GC's interactive nature requires manual selection of kernels for initialisation. The shrinkage behaviour of GC creates inaccurate and fallacious extraction. On the basis of these problems, this study proposes gradient-based kernel selection GC method that simultaneously removes shrinkage problem and locates tumour in image, eliminating human interaction with accurate segmentation for even bias field images. The proposed method addresses these problems by emphasising on directive inclination of intensity scales of symmetrical halves of images. The proposed method is evaluated for high-grade glioma and low-grade glioma images with and without bias field. The average performance metrics evaluated for these images depict remarkable improvement in comparison with existing techniques. The proposed technique is validated by applying on real-time dataset of tumour images obtained from State Government Hospital, Shimla, India. 1 Introduction Magnetic resonance imaging (MRI) is the standard technique used for brain tumour diagnosis. This technique is widely used in clinics as it provides a different type of tissue contrast by varying the excitation and repetition times [1]. The analysis and study of brain tumour MRI images are considered to be complex and challenging. This is due to the tumour's heterogeneous structure that generates difficulty in locating the boundary. Also, tumour growth and formation are integrally unpredictable. Consequently, the shape and different textural properties do not fall into a standard set of shape or size. Besides these difficulties, a bias field effect corrupts MR images by creating the intensity inhomogeneity [2]. Bias field is a low-frequency undesirable signal that has varying intensity distribution with multiplicative nature. These invariabilities are biggest hurdle in medical analysis. The intensity inhomogeneity is particularly severe in MRI at ultra-high field strengths (e.g. 3 and 7 T) and sometimes makes it difficult even for expert human observers to view the images. To provide an efficient and impartial evaluation medical image analysis is gaining attention. With the huge on-going research in this field, many methods reported are closer to the routine clinical application. One of the methods in image analysis is segmentation technique that allows the extraction of the abnormal tissue from the normal. This extraction is helpful for the feature and texture analyses of the tumour. There has been an enormous work in the field of image segmentation. Among various segmentation techniques, the automatic image segmentation is an active area of research work, which has attracted significant attention. To fully automate the method, user interaction must be completely terminated. The accurate knowledge of prior information is critical and depends on the preferences of the segmented region for the ultimate goal of segmentation. Owing to the intensity dependence of the selection of these a priori information from the images with bias effect becomes much difficult. One of the popularly used segmentation technique in image analysis is the graph-cut (GC) method [3]. The energy function used in this method consists spatial coherency term and data term. The former term evaluates the abrupt changes in the boundary. The latter is the likelihood energy that defines the Gaussian intensity distribution of the MR brain image. This distribution is formed if the kernel value obtained from the selected markers, indicating the highest resemblance with the object and the background region, are known (which are the prior information). The kernels are key values that are used to localise the tumour in the brain MR image and initialise the subsequent steps of the segmentation process. The domain priors can be empirically or randomly selected [4] with the help of radiologist for the breast tumour. Researchers have also initialised the algorithm with random pixel values and detected the texture feature and spatial characteristics of the object region for automating the segmentation method [5, 6]. 1.1 Our contribution With the variant application-driven advantages provided by the GC method, on the one hand, it partakes the limitation of small cuts or shrinkage behaviour [7, 8]. There is an occurrence of the erroneous pixel regions, which mainly occur due to more than one tumour settlements comprising high affinity with the tumour region. In this work, the focus is on estimating the kernels that serve important purposes of segmentation for all images, with and without bias field. The proposed gradient-based kernel selection GC (GBKS GC) technique focuses on the abrupt changes caused due to the glioma-affected areas. These abrupt changes are evaluated, and pixel location forms the initial markers providing the essential kernel values. It eliminates user interaction and creates a fully automatic algorithm. The selection of accurate kernels aids in obtaining single-connected region and removal of shrinkage problem. On applying the proposed method on MR images affected by the bias field, accurate segmentation of the tumour region achieved. Moreover, the proposed technique gave accurate segmentation results when applied on the real-time images of the brain obtained from Government Hospital, Indra Gandhi Medical Hospital (IGMC), Shimla, Himachal Pradesh (India). The brief description of the achieved objectives is discussed below: Automatic marker selection for the GC algorithm: The GC algorithm depends on the intensity values of the brain image and requires markers for its initialisation. In this paper, we have provided the kernel values by using the pixel intensity and exploiting the symmetrical nature of brain structure. Also, the selection of markers is done without any human intervention achieving automatic initialisation. Removing shrinkage problem: To detect a complete tumour region excluding any extra or erroneous region, the kernel values are extracted with their site of the existence to eliminate these regions. By including these sites during segmentation, an effective extraction of the homogeneous single-connected region is achieved. This results in the removal of the shrinkage problem caused in the conventional GC method. Removing effect of bias field: The bias field creates abrupt intensity changes that may affect the marker selection and consequently the segmentation. The segmented tumour region contains abrupt boundaries and the inherent intensity inhomogeneity causes due to the bias field inside the object region. This abruptness and non-uniformity in the segmented tumour are removed by applying the proposed GBKS GC technique. 1.2 Related work Among the various interactive methods, GC segmentation employs Gaussian mixture models (GMMs) for intensity distribution, and its energy equation formulates the cut for partitioning the foreground [9]. The GrabCut uses an iterative energy minimisation by utilising two GMMs providing intensity distribution for both the object and foreground regions [10]. Owing to its interactive nature, a bounding box is required, drawn by the user that encloses the object. The intelligent scissor technique [11] uses the user-defined points and crops the nearest object region boundary. The extraction is done via graph search methods for its automatic execution. In this research work, automatic GC segmentation is employed by removing this interactive nature and providing the most accurate markers to perform good object segmentation. Even if the interactive nature of this technique is removed still some unwanted region is segmented along the object region due to the shrinkage bias. These are formed for the region with the shorter boundary as there are intensity discontinuities at the boundary of the segmented region. This problem presents a difficulty for segmenting a single-connected object region. The normal brain MR image region is disguised as the infected tumour region. To address this, issues of some methods are reported to remove this problem. Jermyn and Ishikawa [12], Sinop and Grady [13], Kimmel and Bruckstein [14] and Kolmogorov and Boykov [15] demonstrated a flux of a vector field to overcome the shrinkage problem. Mostly this technique is applied on thin objects, for example, blood vessel grey-scale image of the human body. In these images, the object and background regions are represented by intensity values varying from white till grey. The vector fields are created by observing the image gradients. However, these methods lack two major aspects: first, this approach does not work on the colour image, and second, it faces the issue of choosing the initial vector field. In GC technique, Boykov et al. [16] demonstrated an approximate solution for multiple labelling. The terminal and non-terminal links obtained from the normalised cuts described by Shi and Malik [17] partitions the graph forming the disconnected and connected avoids the problem. Hao et al. [18], Gao et al. [19], Alemán-Flores et al. [20] and Liu et al. [21] applied this approach for segmenting lesions in the breast ultrasound images. A necessary requirement of the initial markers is critical for the initialisation of an automatic algorithm. These initial markers are given by Hanaoka et al. [3] by proposing a Riemannian metric for the three-dimensional (3D) image by enhancing the GC segmentation and consequently solving the shrinking problem. This Riemannian metric provides prior knowledge of the predefined shape or initial contour. However, this method remains limited while segmenting the thin-elongated object. Besides, the different approaches discussed above for segmentation, a major issue also lies in partitioning the tumour from a bias field affected image. To reduce this effect many researchers have reported literature on the removal of the bias effect on MR images [22, 23]. Sled et al. [22] proposed an iterative and automatic non-parametric non-uniform intensity normalisation (N3) approach, in which no a priori information was used. Wells et al. [23] presented an adaptive estimation method by employing expectation maximisation. This method is able to segment each brain tissue, but accurate knowledge of a priori information is required. Li et al. [24] used the coherent local intensity clustering for estimating the bias field and segmenting the object region. Image denoising is done by Bilenia et al. [25] for removing the bias field from the T1 modality of the brain MRI images with skull stripping at the initial phase. Many researchers introduce many semi-automatic and automatic approaches regarding GC technique are in the past few years. In semi-automatic approach, some of the clustering techniques are developed by researchers [26-28]. The region growing method is combined with two clustering method: k-mean and fuzzy c-mean (FCM) to provide a multi-stage approach [26]. This technique aided in segmented following regions: peritumoural region and enhancing tumour region. Also, a multidimensional feature set is constructed that comprise texture characteristics of different MRI modalities (T1, T1ce and T2) [27]. The k-mean clustering method is used on these feature sets and a Dirichlet a priori segmentation is easily performed as normal brain region intensity clusters are completely omitted from the feature set framed. Another hybrid technique for multi-stage segmentation Kanas et al. [28] used random walks and FCM approach. Jiang et al. [29] proposed a semi-automatic approach that included training through learning with the help of some feature sets. This method is applied on all the modalities (i.e. T1, T1ce, T2 and flair). In this approach, classifier is trained through learning by providing the population feature sets. The segmentation is performed using a GC technique considering the probability maps formed by the classifiers. Zhao et al. [30] used Markov random field for extraction with good accuracy. This proposed method is applied to the complete set of modalities, except for T1. The technique presented by Ilunga-Mbuyamba et al. [31] employs a hybrid form of active contour model and k-mean. This technique of using active contour model enhances the speed for the process of segmentation. Mostly, the fully automatic approaches use the neural network as it extracts high-level information from the dataset to provide the learning. These methods are highly accurate and fully automatic requiring a large amount of data for the learning process. The neural network approaches developed by the researchers [32-35] collect the key prior information required for the segmentation. Havaei et al. [32] presented a deep neural network that uses global and local contextual features. This network comprises training at a two-phase level to remove any imbalance that occurs in the tumour labels. A toolkit named ilastik is also used for extraction [33] that does not employ any human interaction. Zikic et al. [34] applied convolutional neural network with all likelihood to directly apply segmentation to the brain tumour tissues. The labelling to all the surrounding tissue is provided with respect to the information generated by the network. A classification within the tumour region is also done by Havaei et al. [35] by employing a neural network. Li et al. [36] have provided a likelihood estimation using a machine learning method that gave the best estimation and good accuracy value for the segmentation using GCs. Wu and Yizhou [37] proposed an automatic method for fine quality extraction of objects from photographs. The foreground pixels are extracted to form texture feature using classifiers and convolution features are generated using a deep neural network. These feature aids in the extraction of the objects. This paper is organised in the following sections: Section 2 contains the problem formulation. Section 3 comprises the materials and methodology. In Section 4, the experimental results obtained are tabulated and the output images obtained are depicted. Also, a comparison of our result with the existing method is tabulated in this section. Finally, in Section 5, the conclusion and future work are formulated. 2 Problem formulation The general structure of graph consists of nodes which represents the pixels and edges formed by the neighbouring pixels. For bi-object segmentation, the source s and sink terminals t are required for partitioning the graph in the object and the background region. The connectedness of these terminals with the graph is shown by an arc known as 't-link' (terminal link). The second type of arc is the 'n-link' (non-terminal link), which connects the neighbouring pixels. The s/t cut in the graph is a partitioning of into two disjoint sets of s and t. A cut with minimal cost is the best cut applied. This cut is obtained by minimising the energy function. The energy equation comprises of the summation of data and spatial coherency term. The standard energy function for GC-based image segmentation approaches [9, 10] in the equation below: (1)Our goal is to obtain kernels for the initialisation of the algorithm without any user interaction and remove the shrinkage problem in the GC method. These goals are obtained under certain connectivity constraints. These constrained assume that an undirected graph with defined connectivity between the nodes is provided. In our experiment, a 2D grid graph with energy equation given in (1) is considered. To perform segmentation, complete image must be partitioned as: , where and are the sets of pixels belonging to the object and background region, respectively. These sets associate high resemblance with the selected kernel (s and t) that even aids in the initialisation of the GC methodology. The path formed between s and t must consist of connected nodes . One way for the marker selection is to provide a user interface by clicking on image. These markers help in creating the kernels. Unfortunately, this user interaction employs multiple clicks and it becomes a non-deterministic polynomial hard problem. Once the segmentation is performed, it is difficult to obtain a single-connected component due to problem of shrinkage. As the object pixel set is labelled as 1, and the small area formed due to small cuts, the extra regions are obtained. In an interactive environment, it is possible that inaccurate prior information is considered and structures are incorrectly segmented. Even the bias field MR images that portray non-uniformity of the intensity scale creates even bigger challenge for selecting the markers. This intensity inhomogeneity in the MR images can be observed from Fig. 1. Fig. 1Open in figure viewerPowerPoint Bias field MR images (a) HGG, (b) LGG 3 Material and methodology The dataset for conducting the experiment for this research work are taken from the Brain Tumour Segmentation Challenge, a part of MICCAI challenges [38-40]. This dataset consists of different types of glioma images comprising high-grade glioma (HGG) and low-grade glioma (LGG). There are four modalities of size 240 × 240 which are co-registered and low-intensity tumour images [41], namely T1, T1ce, T2 and flair. For our research work, 70 and 70 LGG flair images without any bias field effect and with bias field effect are utilised and the evaluated results are reported. To validate the experimental results, the ground truth (GT) provided by the expert radiologists is used. The real-time dataset comprising meningioma and glioblastoma multiforme (GBM) MR images are obtained from the radiology department of State Government Hospital, IGMC, Shimla, Himachal Pradesh (India). The proposed method consists of five main steps: intensity normalisation, marker selection, GC segmentation, post-processing and parameter evaluation. The proposed technique using GBKS GC is depicted in Fig. 2. Fig. 2Open in figure viewerPowerPoint Proposed GBKS GC segmentation technique The steps involved in the proposed algorithm are detailed in the following section. 3.1 Intensity normalisation The brain MR images have intensity values in the range 0–255 × 255, leading to bad estimates of mean/variance and increases the memory space. The obtained dataset provides MR image of size 240 × 240 containing pixel value ranging from 0 to 65,025. By applying intensity normalisation, the size of image is reduced from 16 to 8 bit and the intensity range is converted to a scale of 0–255. Fig. 3 illustrates the effect of intensity normalisation on one of the flair image. Fig. 3Open in figure viewerPowerPoint Effect of intensity normalisation on one of the flair image (a) Original MR flair image, (b) Intensity normalised image 3.2 Proposed GBKS The pictorial illustration of the proposed GBKS methodology is depicted in Fig. 4. The process is entirely intensity dependent, less iterative with low mathematical complexity. It principally depends on the gradient calculation and inclination of these gradients. Both the positive and negative inclinations represent the abrupt changes in two halves of the images. Fig. 4Open in figure viewerPowerPoint Workflow of the proposed GBKS technique Fig. 4 depicts a two-level division of the input MR image, and the extraction of the markers with their location after converting the grey image into a binary form. The workflow of the proposed GBKS technique for marker selection is provided in Fig. 5. The pixel values in the main image and both horizontal half ( and ) of the image are used to calculate the gradient. Among these images, the inverse represents tumour region. Initially, random weights are assigned to each image and the gradients are calculated. These gradients help in obtaining the majorly affected tumour region of the brain. In a normal brain image, symmetrical intensity distribution is present and the gradients do not show any inclination (negative or positive) (2) (3)At both the levels, gradients and the amount of negative slope are calculated (4)where NS is the negative inclination. Fig. 5Open in figure viewerPowerPoint Flowchart of the proposed GBKS technique Large amount of NS implies the presence of the tumour region in one of the horizontal half of the MR image, which is generated for the second level. The gradients and their inclination are again calculated at this level (5)where image generated for the second level of the algorithm. No further division is done after image size . After the second-level division, final image with size is converted into binary form and location of tumour and markers are calculated. These two important key points aid in the automatic initialisation and in obtaining single-connected region by removing the shrinkage problem. The kernels that are calculated using the detected markers and their location provide an accurate tumour segmentation. The GBKS terminates any manual or interactive kernel selection, removing the inaccuracy and false segmentation. Even the problem of shrinkage is removed. The proposed method not only provides the markers, but also their location hence providing a single-connected object region. Different techniques are employed before for the selection of these kernels. Rosenfled and Pfaltz [42] segmented the image using distance transform operator that is viable for only single local maxima in target regions. Distance transform along with global thresholding is able to distinguish various regions. Park et al. [43] provided ultimate erosion for convex sets that is an iterative morphological algorithm required to extract the seed points from overlapping regions limited to the nanoparticles and their morphological analysis. 3.3 GC segmentation The object and background kernels are evaluated from the markers obtained from the proposed GBKS method. These kernel values execute the segmentation of the tumour region by an algorithm developed by authors Dogra et al. [44]. Even after segmentation, the obtained image may contain some erroneous regions that occur due to the limitation of the GC method known as shrinkage problem. The location obtained from the flowchart given in Fig. 5 create a single-connected tumour region. These location are further used for removing the shrinkage problem by the following stepwise procedure explained in Algorithm 1 (Fig. 6). All the pixels showing high similarity with these location form a single-connected tumour region resulting in the extraction of the segmented output. Fig. 6Open in figure viewerPowerPoint Algorithm 1: algorithm for extraction of single-connected tumour region using the proposed GBKS GC technique 3.4 Post-processing: bias field correction Bias field is a multiplicative noise causing intensity inhomogeneity (6)where is the pixel intensity at the ith position of the 2D image and is the intensity variation of the bias field. This bias field has Gaussian distribution (7)where p, and are the image intensity, standard deviation and mean value, respectively. The performance of segmentation method on bias field images gets degraded due to intensity inhomogeneity. Therefore, removal of this bias field from the segmented images is achieved by applying post-processing technique using non-linear median filter. Fig. 7 depicts the removal of the bias field while preserving the edges of extracted region. Fig. 7Open in figure viewerPowerPoint Removal of the bias field (a), (b) Segmented bias field HGG image by the proposed algorithm and post-processing, (c), (d) Segmented bias field LGG image by the proposed algorithm and post-processing 3.5 Parameter evaluation The complete evaluation for the proposed GBKS GC is done on the basis of various parameters as reported in the literature review. Parameters calculated for the accuracy of the selected kernel are: mean square error (MSE) and correlation coefficient (CC). The MSE value quantifies the difference between the mask obtained and the GT. The strength between the variables and relationships is calculated by the CC parameter. For the validation of mask area, perimeter, major axis length (MAL) and minor axis length (MiAL) parameters of the obtained tumour region with the GT are measured. These properties give the confirmation of the shape of the obtained tumour region. To evaluate the performance of the proposed GBKS method, Jaccard index (JI), dice similarity coefficient (DSC), positive predictive value (PPV), sensitivity and specificity are calculated. The performance metrics are calculated using the parameters: true positive (TP), false positive, true negative and false negative (FN). DSC: It measures the overlap or the amount of similarity in the pixel between the GT and the affected region obtained after automatic segmentation (8)where ET is the extracted tumour and GT is the ground truth available. Sensitivity: This parameter measures the number of TP and FN detections, which means it calculates the fraction of positives that are correctly detected by the experiment. JI: It is the measured amount of similarity between the two samples obtained from the GT and the segmented result (9)Specificity: This parameter calculates the fraction of negatives that are correctly detected. It covers all those pixels that were not be detected and the experiments does not detect them. PPV: It predicts the amount of segmented region that are oversegmented or the pixel region that gets segmented even though it is the part of the normal brain tissue region. 4 Results and discussion This research work addresses three major issues of automatic extraction of glioma-affected portion from MR images. The first section evaluates the effectiveness of the selected kernels in removing the shrinkage problem [7, 8]. The results are quantitatively and qualitatively validated by extracting the portion affected with bias field. 4.1 Automated segmentation The proposed GBKS GC method focuses on the issues of kernel selection for the initialisation of the algorithm and removal of the shrinkage problem [7, 8]. Owing to intensity-based nature of the GC method, it is difficult to obtain a single extracted region. Small cuts and leakages are formed due to similar intensity regions in the normal part of the brain. These extra regions get extracted by the GC method, which is a limitation of this method. Figs. 8b and e show the results obtained from conventional GC for HGG and LGG brain tumour MRIs clearly indicating the fallacious segmentation regions. Figs. 9b and e show the similar results for bias field MR images. For Figs. 8c and f, it is observed that by employing the proposed GBKS GC method, the shrinkage problem is completely removed as erroneous pixels are not obtained. Also, a single-connected component is obtained at the output implying accuracy in the selected kernels that initialise the algorithm. Similarly, accurate outputs are also obtained for bias field images as shown in Figs. 9c and f. Fig. 8Open in figure viewerPowerPoint Automated segmentation (a), (d) Original HGG and LGG MR images, (b), (e) Segmentation