# Iris Recognition Development Techniques: A Comprehensive Review

Authors: Jasem Rahman Malgheet; Noridayu Bt Manshor; Lilly Suriani Affendey
Journal: Complexity
 (2021)

---

## Abstract

Recently, iris recognition techniques have achieved great performance in identification. Among authentication techniques, iris recognition systems have received attention very much due to their rich iris texture which gives robust standards for identifying individuals. Notwithstanding this, there are several challenges in unrestricted recognition environments. In this article, the researchers present the techniques used in different phases of the recognition system of the iris image. The researchers also reviewed the methods associated with each phase. The recognition system is divided into seven phases, namely, the acquisition phase in which the iris images are acquired, the preprocessing phase in which the quality of the iris image is improved, the segmentation phase in which the iris region is separated from the background of the image, the normalization phase in which the segmented iris region is shaped into a rectangle, the feature extraction phase in which the features of the iris region are extracted, the feature selection phase in which the unique features of the iris are selected using feature selection techniques, and finally the classification phase in which the iris images are classified. This article also explains the two approaches of iris recognition which are the traditional approach and the deep learning approach. In addition, the researchers discuss the advantages and disadvantages of previous techniques as well as the limitations and benefits of both the traditional and deep learning approaches of iris recognition. This study can be considered as an initial step towards a large-scale study about iris recognition.


---

## Body

## 1. Introduction

Computer vision is a significant research field, which provides efficient solutions to many problems. Pattern recognition is mainly used to automatically recognize different entities from an image. The security sector has given computer vision much attention, particularly for identification. Every human has unique, particular properties such as shape and size. Modern security sciences use these unique features to control access to restricted places, which, in the field of security, is a fundamental problem. The increasing demand for efficient authentication systems in the security field has sparked the development of authentication systems that are more secure and efficient. Traditional approaches to identification such as the use of a key or password are unsatisfactory in several application areas as these methods can easily be forgotten, stolen, or cracked. To overcome these weaknesses, modern science is interested in automating identification systems using biometric techniques [1].The need for reliable and secure systems has led to the emergence of the physiological and behavioral models in biometric systems. Both these models work effectively for security measures [2]. Physiological biometrics include iris recognition [3], fingerprint recognition [4], face recognition [5], retina recognition, and hand geometry recognition [6]. Biometric techniques in behavioral models comprise signature recognition, voice recognition, and gait recognition [7, 8]. Among all biometric recognition systems, the iris recognition system (IRS) is the system with higher efficiency and is the more reliable system for checking authenticity [9, 10]. This is due to the stability of the human iris, its invariance over time (i.e., it remains stable in spite of the aging process), and its uniqueness for every person, even between siblings or twins [11]. It is covered via a structure that, if changed, could affect a person’s health and can be accessed using a non-invasive device [12]. Thus, several leading companies, particularly in the sector of security, are looking forward to the future of the IRS due to different applications and potential of this technology.Furthermore, the iris region is also deemed as one of the very stable biometric features which hardly change over time [13]. Hence, the structure of the iris region can effectively be utilized for recognition or identification [14]. With large-scale-range national identification programs being increasingly deployed all over the world, the demand for correct and reliable biometric recognition systems has increased. National identification programs are increasingly utilizing the IRS for better accuracy and reliability to register citizens (in addition to the utilization of different biometric recognition techniques such as two-dimensional face and fingerprint recognition system).The IRS is a high-accuracy verification technology [2] and has a high ability for personal identification. The IRS is increasingly being applied in automated systems (i.e., without human operator supervision) [11] and is used especially in the security field. Thus, many countries use the IRS to improve security such as at the gates of smart airports and borders, in mobile devices, and at government buildings like hospitals [15].The iris images are typically captured by using operational systems in the near-infrared (NIR) spectrum, where the iris region appears clearer than it does in the visible wavelength spectrum, even with the iris region’s dark color [13, 16]. According to Drozdowski [16, 17], more than half of the adult population around the world, such as in East Asia [17, 18], are wearing eyeglasses mainly due to the pervasiveness of myopia (short-sightedness) for the purpose of vision correction [19–23]. Eyeglasses are also worn for eye protection [24] and fashion purposes [25, 26]. Some studies have shown that under specific conditions, wearing eyeglasses may affect the performance of recognition systems, such as iris recognition systems [19, 27], periocular recognition, and face recognition systems [28] negatively.Although research on iris recognition systems (IRSs) is robust in constrained (ideal) environments, there is a lack of research on iris recognition in unconstrained (non-ideal) environments. In unconstrained recognition environments, there are several challenges such as off-angle, small-sized iris, low resolution, rotational, distortion, cropping, scaling, occlusion by an eyelash, blurry image or “noise,” interference from the eyeglass frame, and reflection from eyeglass lens [17, 29–35]. Also, there are some other limitations of the IRS which lead to an increase in false rejection due to pupil stretching, sagging of eyelids due to age, wearing of cosmetic and fashion lenses, and reflective environments. The existing studies of iris recognition using deep learning approaches for eyeglasses in such environments do not outperform the performance of traditional approaches.Eyeglasses are making IRS more challenging, and in general, the eyeglasses can have a negative impact on the quality of the sample because of specular reflections, optical distortions, dirt, and shadows [32]. Consequently, biometric performance of iris recognition scheme is worsened [26, 36–39]. The experimental results of some previous studies showed that the error rate on the subset of iris images with eyeglasses is twice the error rate of iris images without eyeglasses [17]. There are two main differences between iris images with eyeglasses and those without eyeglasses, namely, (1) the iris images with eyeglasses have more robust edges and (2) the iris images with eyeglasses cause sharp specular reflections.In the iris segmentation phase, the false-negative error occurs due to reflection noise caused by eyeglasses [40, 41]. Also, explicit failures of segmentation algorithms that were due to the reflections or the frame of the eyeglasses could be misconstrued as pupillary or limbic boundaries [17]. Thus, the segmentation phase of the IRS for eyeglasses is still challenging in unconstrained environments. As such, researchers should work to improve segmentation techniques to develop recognition systems because improving this technique will enhance the biometric performance level significantly. The deep learning approaches are more appropriate than the traditional approaches in improving segmentation techniques and detecting fake iris images [31].Additionally, as a result of blurriness, specular reflections, scratches, and other factors, the eyeglasses can reduce the iris images’ quality and subsequently the accuracy of performance of the iris recognition systems. Various studies have identified the effects of eyeglasses on facial recognition systems; however, few works were related to the subject of eyeglasses in the scientific studies of iris recognition. One study that investigated a small-scale quantification of the eyeglasses effects on the preprocessing phase is [42]. Other studies which identified eyeglasses as one of the influential noise factors are [17, 32–39].In spite of many suggested methods to reduce deficiencies in iris image recognition systems, it is challenging for new researchers to find common research problems [31]. Therefore, this paper aims to pave the way in providing applicable solutions to the aforementioned challenges encountered in iris image recognition systems.This article provides the following contributions:(1)
It discusses and analyzes the significance of the IRS which has been introduced in the literature.(2)
It reviews the existing iris recognition systems (IRSs) and applications comprehensively.(3)
It discusses current challenges and directions of future studies for iris recognition systems.The remaining sections are structured as follows. Section2 presents a comprehensive view of the recognition system for iris image. Section 3 presents examples of image acquisition and databases used in the experiments of researchers. Section 4 discusses the preprocessing techniques for removing influences used in previous works. Section 5 describes the different segmentation techniques used based on literature for segregating the iris from the rest of the parts of the iris images. Section 6 describes different normalization techniques which turn segmented circular iris regions into rectangular patterns. Section 7 presents various feature extraction techniques which describe the iris features. Section 8 discusses the feature selection methods which are operations to select the best features to reduce computational complexities. Section 9 discusses the different types of classification techniques related to iris recognition. Section 10 presents the challenges related to iris recognition and areas for future research. Finally, Section 11 concludes this review paper.


## 2. Iris Recognition System Structure

Figure1 illustrates the standard structure of the IRS. In general, the structure of the IRS in both traditional and deep learning approaches consists of seven principal phases in the following order: iris image acquisition phase, preprocessing phase, iris image segmentation phase, iris normalization phase, feature extraction phase, feature selection phase, and finally iris classification or matching phase.Figure 1
A fundamental structure of the iris recognition system (IRS).A review of the scientific literature on iris recognition shows that there is a lack of studies quantifying the impacts of eyeglasses and other types of noise on the performance accuracy of IRS. Table1 illustrates the influence of different noise types on various phases of the IRS.Table 1
Influence of different noise types on phases of the iris recognition system.
PhasesNoise from eyeglassesLensOff-angleEyelashes and eyelidsLightingMakeupDiseasesDistanceSegmentation∗∗∗∗∗∗∗∗∗∗∗∗∗∗∗∗∗∗∗∗∗∗∗∗∗∗∗∗∗∗∗Feature extraction∗∗∗∗∗∗∗∗∗∗∗∗∗∗∗∗∗∗∗∗∗∗∗Classification∗∗∗∗∗∗∗∗∗∗∗∗∗∗∗∗∗∗∗∗∗∗∗∗∗∗∗∗∗∗The influence level of these different challenges on the IRS is shown in Table1. “∗” represents the level of challenge in a particular phase of the recognition system of the iris image. When the number of “∗” increases, so does the influence level of these challenges on the algorithms in the corresponding phase. As shown in Table 1, the degree of difficulty, associated with the noise specific in phases, is evaluated in terms of the performance of the algorithms using metrics such as accuracy rate, information loss rate, error rate, or the failure of the algorithms to deal with that noise. This means that future research should focus on enhancing the techniques of any of the challenges in a preprocessing phase in order to improve the performance accuracy of the IRS by eliminating the negative impact of different noise types.


## 3. Acquisition of Iris Image and Standard Databases

An iris image is typically taken using the visible wavelength spectrum or the near-infrared (NIR) spectrum. The four units for iris image acquisition are lighting, lens, sensor, and console. Images acquired using the NIR spectrum tend to transact with the complex texture of the iris region, instead of its pigmentation [14]. Also, the iris images captured using the NIR are less prone to different noise types than the iris images taken using the visible wavelength spectrum. This allows the iris texture to be captured even with dark colors, thereby enhancing the performance efficiency of iris recognition. Figure 2 shows some samples of the images taken in visible wavelength spectrum and near-infrared (NIR) wavelength spectrum with and without eyeglasses. Researchers recommended one of these three publicly available databases: CASIA-IrisV4, MICHE, or UBIRIS, because these databases contain several types of challenges that can be exploited to improve the performance of the IRS.Figure 2
Some samples from iris images databases: with and without eyeglasses. (a) CASIA 4-Iris 1000 database. (b) UBIRIS datasets. (c) BERC mobile-iris database [27]. (d) UBIPr periocular database [43].
(a)(b)(c)(d)Apart from iris capture devices, there are a number of databases of iris images worldwide to assist researchers to test their techniques or algorithms and validate their performance accuracy. Most of the databases of the iris images are acquired using the NIR spectrum, and some other databases are captured using the visible wavelength (VIS) spectrum. In the databases, the iris images are stored in compressed and uncompressed formats. Some of the iris images were acquired under a constrained (ideal) environment, while other images were acquired under an unconstrained (non-ideal) environment. Table2 provides the URLs of iris image databases for testing algorithms that are available to be downloaded directly. Table 3 lists some previous works and the corresponding iris image databases that were used for testing.Table 2
URL of iris image databases.
DatabasesWeb links to downloadIIT Delhi Iris.V1 (IITD)https://www4.comp.polyu.edu.hk/∼csajaykr/IITD/Database_Iris.htmCASIAhttps://biometrics.idealtest.org/findTotalDbByMode.do?mode=IrisMICHEhttps://biplab.unisa.it/MICHE/MobBIOhttps://paginas.fe.up.pt/∼mobbio2013/UBIRIShttps://iris.di.ubi.pt/ubiris2.htmlNICE-IIhttps://nice2.di.ubi.pt/registration.htmND-IRIS-0405https://cvrl.nd.edu/projects/data/#nd-iris-0405-data-setUTIRIShttps://utiris.wordpress.com/UBIPr periocularhttps://socia-lab.di.ubi.pt/∼ubipr/VISOBhttps://sce.umkc.edu/research-sites/cibit/visob_v1.htmlVSSIRIShttps://www.nislab.no/biometrics_lab/vssir%20is_db_2VSIAhttps://www.nislab.no/nbl_old/vsia_dbMMUhttps://pesonna.mmu.edu.my/ccteo/Biosec-baselinehttps://atvs.ii.uam.es/databases.jspPolyU bi-spectral irishttps://www4.comp.polyu.edu.hk/∼csajaykr/polyuiris.htmBath Irishttps://www.smartsensors.co.uk/products/iris-database/Cross-Eyed-2016https://sites.google.com/site/crossspectrumcompetition/homeUPOLhttps://phoenix.inf.upol.cz/iris/JLU-4.0 Irishttps://www.jlucomputer.com/index/irislibrary/irislibrary.htmND-Cross Sensor-Iris-2013https://cvrl.nd.edu/projects/data/#nd-crosssensor-iris%202013-data-setCSIPhttps://csip.di.ubi.pt/ND-Iris Template-Aginghttps://cvrl.nd.edu/projects/data/#nd-iris-template-aging-2008-2010Table 3
Summary of iris image databases from some previous works (spectrum: visible (VIS); near-infrared (NIR)).
SourceDatabaseNo. of subjectsNo. of samplesSpectrumResolutionFormatDrozdowski et al. [17]CASIA-10001,00020,000NIR640 × 480.jpegDe Marsico et al. [44]CASIA.V1108756NIR320 × 280.bmpZhang et al. [45]CASIA.V3161422,548NIR640 × 480.jpegArsalan et al. [41]CASIA.V4328432,537NIR640 × 480.jpegProenca et al. [19]UBIRIS.V226111,102VIS800 × 600.tiffKim et al. [27]BERC mobile753,011NIR960 × 1280—Proença and Alexandre [46]UBIRIS.V12411,249VIS800 × 600.jpegDobes and Machala [47]UPOL64384VIS786 × 576.pngDe Marsico et al. [48]MICHE-I30 by IPhone530 by GalaxyS4240240VIS960 × 12801080 × 1920.jpgRattani and Derakhshani [43]UBIPr periocular3445,126VIS—.bmpKhan et al. [49]ND-IRIS-040535664,980NIR640 × 480.jpgRattani et al. [50]VISOB550158,136VISiPhone: 720pSamsung and Oppo: 1080p.pngWang and Kumar [51]PolyU bi-spectral20912,540NIR, VIS640 × 480—Cheng et al. [52]JLU-4.0 Iris8826,400NIR640 × 480.bmpZanlorensi et al. [53]VSSIRIS28560VISiPhone 5S: 3264 × 2448Nokia Lumia: 7712 × 5360—Rakshit [54]Bath80080031997NIR1280 × 960. j2cOmelina et al. [55]UTIRIS791,540NIR, VISNIR (1000 × 776)VIS (2048 × 1360).bmp -.jpgUmer et al. [56]IITD2242,240NIR320 × 240.bmpNguyen et al. [57]https://cvrl.nd.edu/projects/data/676111,564NIR640 × 480—Sequeira et al. [58]MobBIO1051,680VIS300 × 200.jpgArsalan et al. [59]NICE-II671000VIS400 × 300.tiffChen and Zhang [60]VSIA55550VIS——Chirchi et al. [61]MMU100995NIR320 × 240.bmpFierrez et al. [62]BioSec-baseline2003,200NIR——Wang and Kumar [51]Cross-Eyed-20161203,840NIR, VIS400 × 300.jpgFenker and Bowyer [63]ND-Iris Template-Aging 2008–201020011,776NIR—.tiffLucio et al. [64]CSIP502,004VISVarious—


## 4. Preprocessing Techniques

A preprocessing method is recommended to be implemented in the first phase to carry out any of the suggested techniques or algorithms of the recognition system. These techniques can help eliminate the different noise types that happened during the process of obtaining the iris image. Typically, iris images obtained suffer from different types of noise such as closing caused by eyelids or eyelashes, lighting or illumination, noise due to eyeglasses such as blurring, specular reflections, scratches, and other factors, distortion, and off-angle iris. Removing these types of noise through the preprocessing phase will enhance the performance accuracy of the IRS.Table4 shows different types of preprocessing methods used to remove the different types of noise. According to the literature, a review showed several traditional methods of image preprocessing which can be categorized into four approaches: Hough transform method, histogram and filtering method, morphological operations, and fusion method.Table 4
Types of noise and their corresponding preprocessing methods.
SourceType of noiseProposed methodNixon and Aguado [65]NoiseMedian filter (MF)Huang et al. [66]BlurrinessKernel estimation of the blurLee et al. [67]ReflectionsLine intensity profile (LIP)Santos and Hoyle [68]Low lighting (low illumination)CLAHE methodDehkordi and Abu-Bakar [69]Noise such as eyelids, eyelash, and light reflections and pupil pixelsMultiple thresholding methodLiu et al. [70]Poor performance accuracy of low-resolution (LR) iris recognitionA heterogeneous metric learning algorithmRaffei et al. [71]Low contrastAdaptive histogram equalizationBakshi et al. [72]Occlusions due to eyelashes and eyelidsGaussian filters and the Hough line detectorKumar et al. [73]The difference between brighter and darker pixelsTop-hat and bottom-hat filtersBaqar et al. [74]Specular highlight due to reflection2D linear interpolationDjoumessi [75]Occlusions due to eyelidsHough line detectorGangwar et al. [76]Specular reflectionThresholdRadman et al. [77]ReflectionsMorphological-retinex methodRibeiro et al. [78]Low resolution and quality of iris imagesStacked autoencoder (SAE) technique and convolutional neural network (CNN) techniqueArsenovic et al. [79]Shadows on the eye imagesHistogram normalizationGad et al. [80]Specular reflectionMorphological operationsSusitha and Subban [81]Low quality due to poor contrastCLAHE methodDas and Derakhshani [82]Poor contrast and brightnessBPDFHE methodDonida et al. [83]Specular reflections and noiseInpainting algorithm and Gaussian-based bilateral filter


### 4.1. Hough Transform Method

During the preprocessing phase of some previous works by using the Hough transform (HT) method, Raffei et al. used the linear Hough transform method to remove the occlusions due to eyelids [84]. Despite that, there were limitations, for example, the iris images obtained in visible light had more limitations such as reflection noise and occlusions due to eyelashes and eyelids and also the shape of the iris was non-ideal [85], so Chai et al. introduced a method to locate eyelid localization as well as to identify and to eliminate reflection [85]. In eyelid localization, it was assumed that the position of the iris center can be estimated by using a circular Hough transform (CHT) [86] or Haar cascade detector [87]. This localized method was proposed to decrease the noise impact, which generates two NxN areas (windows) of interest at both the upper and lower parts of the iris region. The average intensity of the pixel of the iris region for each window is calculated to discover the eyelid pixels. In reflection identification and elimination, they proposed to first identify the regions impacted by different reflections types, applying a thresholding method [88]. Then, by using a dilation morphological operation, pixels are further added close to the borders of the important iris region to enlarge the selected regions. It is noted that the occlusion of the iris region due to eyelashes and eyelids was still prevalent, leading Yahiaoui et al. to further develop the method by applying a CHT and localizing the pupil region using the Canny edge map [89]. In order to preserve the robust edges and to overcome the sensitivity of the edge detection algorithm, they applied an anisotropic filter followed by an adaptive threshold. With these limitations such as reflections due to glasses, eyelashes, eyebrows, and corrective or fashion lens with the obtained image, all these methods found it difficult to localize the iris region [31, 90].


### 4.2. Histogram and Filtering Method

To increase the iris recognition rate and segmentation accuracy of the iris region from the remaining background of the image, Santos and Hoyle used a histogram equalization (HE) method to enhance an iris image acquired at low lighting to identify the boundaries of the pupil region. These techniques are used to redistribute pixel intensities [68]. For the improvement of the iris recognition performance under variable conditions of image quality and also the contrast of the acquired iris images using low contrast or low illumination, Alvarez-Betancourt and Garcia-Silvente proposed the contrast-limited adaptive histogram equalization (CLAHE) method [30], a modified version of the adaptive histogram equalization (AHE) method which was introduced by Zuiderveld [91]. This method improves the contrast of images with grayscale and can also be used to prevent any noise that might be present on the image. Maheshan et al. also worked on the same limitations and improved the image by applying two techniques: HE and CLAHE. The objective of HE is to determine the frequency of dark colors that usually extends in the range of 0–50 pixels. The CLAHE, on the other hand, sets a contrast limit that gives proportionate white balance for the selected image [92].Kumar et al. introduced a method for contrast improvement by utilizing top and bottom hat filters to improve the difference between pixels in the brighter region and pixels in the darker region. The proposed technique has four steps [73]. The first is HE [65] which was applied to enhance contrast. The second step is the median filter which was applied by Mark [65] to remove noise. The third step is gamma intensity correction (GIC), which according to Saeedi et al. is used for illumination variation [93]. In the fourth step, the iris image is exposed to a sequence of top hat and bottom hat filters, as suggested by Bai [94].Zhang et al. [95] in contrast applied the homomorphic filtering method as was suggested by Zhang and Shen [96] to make the iris image have a robust contrast and to overcome the occlusion impact [96]. However, if the iris images were generated by computer, the specular highlights from the eye images need to be eliminated [95]. This can be done by a technique that explores discrepancies of the eye region as proposed by Carvalho et al. [97]. The first stage of this method is to discover the eye region by using the algorithm that was suggested by Viola and Jones [87].Then, this is followed by an approach suggested by Chang et al. [98, 99] who applied the preprocessing method on the input eye image in three phases to remove specular highlights of the eye image. In the first step, the Gaussian filter technique was applied with a sigma value which was equal to 0.9. The second step was to transform the eye images from gray to binary, using a threshold value which was equal to 0.18. The last step was to apply the Gaussian filter again on the binary eye image with a sigma value which was equal to 2, after which the median filter was used to smooth the eye image [98, 99].Likewise, Hofbauer et al. [100] also applied the median filtering to smooth the eye image. The median blurring filter allows the image to be more regular and also eliminates the impact of eyelashes so as not to reduce the sharpness of the iris region. Despite that, there were other challenges, and the images obtained often showed significant differences in the background and low contrast in some regions [100]. One of the main reasons for this variation is the highlight in the pupil region. To overcome these limitations, Khan et al. [49] also used the Gaussian filter technique but with a standard deviation σ. This is aimed to improve the non-suitable background as well as to create eye images that are more appropriate for thresholding. The chosen value of σ was based on the volume of the pupil region and the iris region. It must be smaller than pupil radius but larger than the width of eyelashes. For the eye images in Khan et al. [49], the value of σ = 5 was more suitable than other values of σ for appreciating the local background.More recently, Liu et al. [101] applied the haze removal technique to enhance an iris image as was suggested by He et al. [102] who initially applied it to decrease the light effect. This technique considered the dark pixels as one of the RGB parts with very bad intensity. The fog property from the iris image was eliminated through eliminating the dark region which had dark pixels [101]. However, iris images that had a reflection on glasses and that occluded due to eyelids and eyelashes were still prevalent, leading Liu et al. to apply a fuzzy average filter, fuzzy median filter, and Gaussian filter for preprocessing the iris images to fortify the out of bounds and to enhance noise ratios as well as the edges of iris images. Despite that, the success rate has still been unsatisfactory, which led to the use of alternative preprocessing techniques such as morphological operations [35].


### 4.3. Morphological Operations

To remove small components, Umer et al. [56] proposed a preprocessing method. This method begins with analyzing the original eye image histogram by using thresholding which converted the original input image into a binary image. Then, the binary iris image was enhanced by eliminating noises using a morphological operation to obtain an enhanced iris image. However, performance accuracy was still affected by noise factors in unconstrained (non-ideal) environments such as reflection, eyelids, and eyelashes [56]. According to Djoumessi [75], morphological techniques could be used to remove the reflection noise from iris images and then thresholding could be applied to remove eyelashes from the image. However, other limitations such as eyeglasses, deviation due to sharp gaze, and occlusion due to hair and eyelids arose. Hence, Radman et al. proposed a morphological-retinex method to overcome these limitations. They applied morphological operations to improve the contrast and to remove the reflection noise from iris images within unconstrained (non-ideal) environments. Then, root mean square (RMS) was applied to decide if the enhancement of the iris image contrast was required or not. Finally, they applied an extended retinex algorithm to improve the contrast of iris images [77]. Likewise, Gad et al. [80] also worked on the same limitations and applied a series of morphological operations, also aiming to get the accurate and quick performance of the corneal reflection. Then, an adaptive local threshold (ALT) method based on the mean filter was applied to filter the output iris image from the bright region pixels. Finally, by using the masking technique (MT) [103], the value of all pixels above and below the pupil diameter was converted to zero to remove the eyelashes and eyelids [80].


### 4.4. Fusion Method

Li and Ma [104] proposed an algorithm to localize the boundaries of the non-circular iris which was based on the random sample consensus (RANSAC) algorithm. In this study, their concerns were the problems of the recognition system of the iris images that were acquired in unconstrained imaging environments [104]. In these conditions, the performance efficiency of recognition systems of the iris image becomes more difficult due to different noise types such as the off-angle imaging, blurring, occlusion caused by eyelids and eyelashes, and specular reflection from illumination [104]. Raffei et al. [105] proposed a fusion technique which consists of three substages. The stages are identifying reflection noise, classifying reflections, and filling in reflections. In identifying reflection noise of the iris image, they used the line intensity profile (LIP) technique which was applied by Lee et al. [67] who had used the LIP technique to remove reflections on color images of teeth. In classifying reflection, both reflections and nonreflections were classified by applying a support vector machine (SVM) technique. Finally, in filling in reflection, we use morphological operations to remove the reflections from the RGB colors of iris images [39, 105].The preprocessing techniques have been used in the enhancement of the eye image in order to develop a pterygium detection system. The iris segmentation method is required to improve the technique to detect the pterygium of the anterior segment photographed images (ASPIs). Abdani et al. [106] introduced the preprocessing phase to enhance an eye image based on three subphases. First, they converted the RGB channel to HSV channel as suggested by Shuhua and Gaizhi [107]. Then, the contrast stretching method was applied to modify the brightness and lighting of the iris image without modifying any information of the iris image colors for the S and V channels. Finally, the sigmoid function, as suggested in Imtiaz and Wahid [108], is used to apply an adaptive contrast manipulation method [109].Zhao and Ajay [110] used the single-scale retinex (SSR) technique to normalize iris image lighting. The SSR technique improves colors cohesion within the sharp lighting discrepancy. Once the image lighting was improved, a median filter was used on the iris image to remove noisy pixels that were isolated [110]. Also, in an attempt to overcome the same limitations such as occlusion due to eyelids and eyelashes and reflection, Al-Waisy et al. [111] used the preprocessing method based on three phases to improve the performance. The first stage is to discover all the specular reflections in the iris image. The second stage is to determine the reflection mask to remove the reflection. The last stage is to detect the boundaries of pupil and iris based on the Gaussian filter technique as well as the histogram equalization method to improve the contrast between the region of the iris and sclera and to smooth the eye image. Then, the CHT technique is applied to obtain the radius and center of both the iris and pupil regions. Lastly, the boundaries of the eyelids were detected using an eyelid detection algorithm [111]. However, limitations such as the weak light, intense light, and reflection were still prevalent, affecting performance accuracy. This led Zhao et al. [112] to apply the preprocessing phase in four subphases. First, they located and segmented the iris part of the 640 × 480 original images. Next, the region of interest (ROI) was separated from the image. Then, they normalized the ROI image to 256 × 32 and enhanced the ROI of the iris region. Finally, they unified the measurement of the input iris image by using the nearest neighbor technique in order to resize the normalized iris image to 197 × 197 as the input image [112].Recently, Das and Derakhshani [82] proposed a preprocessing method to enhance image contrast, along with transfer learning that can be used to enhance the efficiency in classifying ocular images of pretrained neural networks. Das and Derakhshani [82] used the brightness preserving dynamic fuzzy histogram equalization (BPDFHE) method to enhance the images [113]. The BPDFHE method is a modification of the brightness preserving dynamic histogram equalization (BPDHE) method through the application of fuzzy statistics to process as well as represent digital images [114]. The limitations of this method are noise from the motion blur as well as failure to identify novel ocular samples without retraining all previous samples [82].To remove noise that reduces the segmentation accuracy of the iris region in an ocular image, Donida et al. [83] proposed a two-phased method that removes reflections as well as reduces noise. The first phase involves removing reflections by first calculating a binarized map of the reflection areas through analyzing the iris image response to Gabor filter utilizing an empirically estimated frequency. Then, the reflections are eliminated by utilizing the inpainting algorithm [115]. This is followed by the next phase which involves reducing the noise by applying a bilateral filter based on a Gaussian filter [116]. The bilateral filter is a non-linear method that allows an eye image to be blurred while maintaining robust boundaries [83].


## 4.1. Hough Transform Method

During the preprocessing phase of some previous works by using the Hough transform (HT) method, Raffei et al. used the linear Hough transform method to remove the occlusions due to eyelids [84]. Despite that, there were limitations, for example, the iris images obtained in visible light had more limitations such as reflection noise and occlusions due to eyelashes and eyelids and also the shape of the iris was non-ideal [85], so Chai et al. introduced a method to locate eyelid localization as well as to identify and to eliminate reflection [85]. In eyelid localization, it was assumed that the position of the iris center can be estimated by using a circular Hough transform (CHT) [86] or Haar cascade detector [87]. This localized method was proposed to decrease the noise impact, which generates two NxN areas (windows) of interest at both the upper and lower parts of the iris region. The average intensity of the pixel of the iris region for each window is calculated to discover the eyelid pixels. In reflection identification and elimination, they proposed to first identify the regions impacted by different reflections types, applying a thresholding method [88]. Then, by using a dilation morphological operation, pixels are further added close to the borders of the important iris region to enlarge the selected regions. It is noted that the occlusion of the iris region due to eyelashes and eyelids was still prevalent, leading Yahiaoui et al. to further develop the method by applying a CHT and localizing the pupil region using the Canny edge map [89]. In order to preserve the robust edges and to overcome the sensitivity of the edge detection algorithm, they applied an anisotropic filter followed by an adaptive threshold. With these limitations such as reflections due to glasses, eyelashes, eyebrows, and corrective or fashion lens with the obtained image, all these methods found it difficult to localize the iris region [31, 90].


## 4.2. Histogram and Filtering Method

To increase the iris recognition rate and segmentation accuracy of the iris region from the remaining background of the image, Santos and Hoyle used a histogram equalization (HE) method to enhance an iris image acquired at low lighting to identify the boundaries of the pupil region. These techniques are used to redistribute pixel intensities [68]. For the improvement of the iris recognition performance under variable conditions of image quality and also the contrast of the acquired iris images using low contrast or low illumination, Alvarez-Betancourt and Garcia-Silvente proposed the contrast-limited adaptive histogram equalization (CLAHE) method [30], a modified version of the adaptive histogram equalization (AHE) method which was introduced by Zuiderveld [91]. This method improves the contrast of images with grayscale and can also be used to prevent any noise that might be present on the image. Maheshan et al. also worked on the same limitations and improved the image by applying two techniques: HE and CLAHE. The objective of HE is to determine the frequency of dark colors that usually extends in the range of 0–50 pixels. The CLAHE, on the other hand, sets a contrast limit that gives proportionate white balance for the selected image [92].Kumar et al. introduced a method for contrast improvement by utilizing top and bottom hat filters to improve the difference between pixels in the brighter region and pixels in the darker region. The proposed technique has four steps [73]. The first is HE [65] which was applied to enhance contrast. The second step is the median filter which was applied by Mark [65] to remove noise. The third step is gamma intensity correction (GIC), which according to Saeedi et al. is used for illumination variation [93]. In the fourth step, the iris image is exposed to a sequence of top hat and bottom hat filters, as suggested by Bai [94].Zhang et al. [95] in contrast applied the homomorphic filtering method as was suggested by Zhang and Shen [96] to make the iris image have a robust contrast and to overcome the occlusion impact [96]. However, if the iris images were generated by computer, the specular highlights from the eye images need to be eliminated [95]. This can be done by a technique that explores discrepancies of the eye region as proposed by Carvalho et al. [97]. The first stage of this method is to discover the eye region by using the algorithm that was suggested by Viola and Jones [87].Then, this is followed by an approach suggested by Chang et al. [98, 99] who applied the preprocessing method on the input eye image in three phases to remove specular highlights of the eye image. In the first step, the Gaussian filter technique was applied with a sigma value which was equal to 0.9. The second step was to transform the eye images from gray to binary, using a threshold value which was equal to 0.18. The last step was to apply the Gaussian filter again on the binary eye image with a sigma value which was equal to 2, after which the median filter was used to smooth the eye image [98, 99].Likewise, Hofbauer et al. [100] also applied the median filtering to smooth the eye image. The median blurring filter allows the image to be more regular and also eliminates the impact of eyelashes so as not to reduce the sharpness of the iris region. Despite that, there were other challenges, and the images obtained often showed significant differences in the background and low contrast in some regions [100]. One of the main reasons for this variation is the highlight in the pupil region. To overcome these limitations, Khan et al. [49] also used the Gaussian filter technique but with a standard deviation σ. This is aimed to improve the non-suitable background as well as to create eye images that are more appropriate for thresholding. The chosen value of σ was based on the volume of the pupil region and the iris region. It must be smaller than pupil radius but larger than the width of eyelashes. For the eye images in Khan et al. [49], the value of σ = 5 was more suitable than other values of σ for appreciating the local background.More recently, Liu et al. [101] applied the haze removal technique to enhance an iris image as was suggested by He et al. [102] who initially applied it to decrease the light effect. This technique considered the dark pixels as one of the RGB parts with very bad intensity. The fog property from the iris image was eliminated through eliminating the dark region which had dark pixels [101]. However, iris images that had a reflection on glasses and that occluded due to eyelids and eyelashes were still prevalent, leading Liu et al. to apply a fuzzy average filter, fuzzy median filter, and Gaussian filter for preprocessing the iris images to fortify the out of bounds and to enhance noise ratios as well as the edges of iris images. Despite that, the success rate has still been unsatisfactory, which led to the use of alternative preprocessing techniques such as morphological operations [35].


## 4.3. Morphological Operations

To remove small components, Umer et al. [56] proposed a preprocessing method. This method begins with analyzing the original eye image histogram by using thresholding which converted the original input image into a binary image. Then, the binary iris image was enhanced by eliminating noises using a morphological operation to obtain an enhanced iris image. However, performance accuracy was still affected by noise factors in unconstrained (non-ideal) environments such as reflection, eyelids, and eyelashes [56]. According to Djoumessi [75], morphological techniques could be used to remove the reflection noise from iris images and then thresholding could be applied to remove eyelashes from the image. However, other limitations such as eyeglasses, deviation due to sharp gaze, and occlusion due to hair and eyelids arose. Hence, Radman et al. proposed a morphological-retinex method to overcome these limitations. They applied morphological operations to improve the contrast and to remove the reflection noise from iris images within unconstrained (non-ideal) environments. Then, root mean square (RMS) was applied to decide if the enhancement of the iris image contrast was required or not. Finally, they applied an extended retinex algorithm to improve the contrast of iris images [77]. Likewise, Gad et al. [80] also worked on the same limitations and applied a series of morphological operations, also aiming to get the accurate and quick performance of the corneal reflection. Then, an adaptive local threshold (ALT) method based on the mean filter was applied to filter the output iris image from the bright region pixels. Finally, by using the masking technique (MT) [103], the value of all pixels above and below the pupil diameter was converted to zero to remove the eyelashes and eyelids [80].


## 4.4. Fusion Method

Li and Ma [104] proposed an algorithm to localize the boundaries of the non-circular iris which was based on the random sample consensus (RANSAC) algorithm. In this study, their concerns were the problems of the recognition system of the iris images that were acquired in unconstrained imaging environments [104]. In these conditions, the performance efficiency of recognition systems of the iris image becomes more difficult due to different noise types such as the off-angle imaging, blurring, occlusion caused by eyelids and eyelashes, and specular reflection from illumination [104]. Raffei et al. [105] proposed a fusion technique which consists of three substages. The stages are identifying reflection noise, classifying reflections, and filling in reflections. In identifying reflection noise of the iris image, they used the line intensity profile (LIP) technique which was applied by Lee et al. [67] who had used the LIP technique to remove reflections on color images of teeth. In classifying reflection, both reflections and nonreflections were classified by applying a support vector machine (SVM) technique. Finally, in filling in reflection, we use morphological operations to remove the reflections from the RGB colors of iris images [39, 105].The preprocessing techniques have been used in the enhancement of the eye image in order to develop a pterygium detection system. The iris segmentation method is required to improve the technique to detect the pterygium of the anterior segment photographed images (ASPIs). Abdani et al. [106] introduced the preprocessing phase to enhance an eye image based on three subphases. First, they converted the RGB channel to HSV channel as suggested by Shuhua and Gaizhi [107]. Then, the contrast stretching method was applied to modify the brightness and lighting of the iris image without modifying any information of the iris image colors for the S and V channels. Finally, the sigmoid function, as suggested in Imtiaz and Wahid [108], is used to apply an adaptive contrast manipulation method [109].Zhao and Ajay [110] used the single-scale retinex (SSR) technique to normalize iris image lighting. The SSR technique improves colors cohesion within the sharp lighting discrepancy. Once the image lighting was improved, a median filter was used on the iris image to remove noisy pixels that were isolated [110]. Also, in an attempt to overcome the same limitations such as occlusion due to eyelids and eyelashes and reflection, Al-Waisy et al. [111] used the preprocessing method based on three phases to improve the performance. The first stage is to discover all the specular reflections in the iris image. The second stage is to determine the reflection mask to remove the reflection. The last stage is to detect the boundaries of pupil and iris based on the Gaussian filter technique as well as the histogram equalization method to improve the contrast between the region of the iris and sclera and to smooth the eye image. Then, the CHT technique is applied to obtain the radius and center of both the iris and pupil regions. Lastly, the boundaries of the eyelids were detected using an eyelid detection algorithm [111]. However, limitations such as the weak light, intense light, and reflection were still prevalent, affecting performance accuracy. This led Zhao et al. [112] to apply the preprocessing phase in four subphases. First, they located and segmented the iris part of the 640 × 480 original images. Next, the region of interest (ROI) was separated from the image. Then, they normalized the ROI image to 256 × 32 and enhanced the ROI of the iris region. Finally, they unified the measurement of the input iris image by using the nearest neighbor technique in order to resize the normalized iris image to 197 × 197 as the input image [112].Recently, Das and Derakhshani [82] proposed a preprocessing method to enhance image contrast, along with transfer learning that can be used to enhance the efficiency in classifying ocular images of pretrained neural networks. Das and Derakhshani [82] used the brightness preserving dynamic fuzzy histogram equalization (BPDFHE) method to enhance the images [113]. The BPDFHE method is a modification of the brightness preserving dynamic histogram equalization (BPDHE) method through the application of fuzzy statistics to process as well as represent digital images [114]. The limitations of this method are noise from the motion blur as well as failure to identify novel ocular samples without retraining all previous samples [82].To remove noise that reduces the segmentation accuracy of the iris region in an ocular image, Donida et al. [83] proposed a two-phased method that removes reflections as well as reduces noise. The first phase involves removing reflections by first calculating a binarized map of the reflection areas through analyzing the iris image response to Gabor filter utilizing an empirically estimated frequency. Then, the reflections are eliminated by utilizing the inpainting algorithm [115]. This is followed by the next phase which involves reducing the noise by applying a bilateral filter based on a Gaussian filter [116]. The bilateral filter is a non-linear method that allows an eye image to be blurred while maintaining robust boundaries [83].


## 5. Iris Segmentation Techniques

The region of interest (ROI) is extracted immediately after the preprocessing phase [31]. The extraction of the region of interest is called segmentation or localization [117].The segmentation phase primarily aims to detach the iris part from the non-useful residual parts, namely, the parts around the iris region (sclera, eyelids, and skin) and the pupil part [118]. The iris recognition techniques depend on the quality of the features of the region that has been segmented. Thus, the performance accuracy of the recognition system is mainly due to the accuracy of the segmentation phase [117].The main reason for most failures in the recognition systems is because of inaccurate segmentation [119]. In the following sections, the iris segmentation phase will be described in the following sequence: iris segmentation using traditional techniques followed by iris segmentation using deep learning techniques.


### 5.1. Iris Segmentation Using Traditional Techniques

Most of these techniques execute segmentation through the application of traditional image processing methods, like Daugman’s method, clustering, and semantic segmentation algorithm. In this section, some segmentation techniques which were used in some previous works to segment the iris region are explained.Table5 illustrates the performance accuracy rate (ACC) or error rate (EER) of different iris segmentation methods using traditional techniques. Although all these techniques show high-performance accuracy for segmentation techniques of the iris image, their performances are still determined by different environmental factors, such as long standoff distance, off-angle, blurring, reflections, occlusion, the small size of the iris, and ghost effect in unconstrained (non-ideal) conditions. Hence, the recent improvement of the techniques based on deep learning approach has led to deep learning features being introduced for iris segmentation. The iris segmentation using traditional techniques can be split into three subsections based on the architecture used which are Hough transform-based segmentation techniques, histogram- and contour-based segmentation techniques, and other segmentation techniques.Table 5
Accuracy rate (ACC) or error rate (EER) for some iris segmentation methods using traditional techniques.
SourceDatabasesMethodsIssues focused onACC/EER (%)Haindl and Krupička [120]UBIRIS.v2Used CHT for iris segmentationThe detection of the reflection and eyelid—Abdani et al. [109]Database [121]They applied digital image processing (DIP) algorithms, [122] frame differencing, and morphological operationsPterygium disease87.05Roy and Soni [123]CASIAUsed Daugman’s methodEyelid and eyelashes—Man et al. [124]Miles and “lu_wei_feb_2006”Geodesic active contour (GAC) methodPigment spots on the iris surface7789Gangwar and Joshi [125]CASIA.v4 and FOCSUsed a coarse-to-fine system based on adaptive filters and thresholding methodOcclusion due to eyelids93.41Abdullah et al. [126]CASIA.v4UBIRIS.v1MMU2They applied the fusion of shrinking active contour and expanding for the iris segmentationThe iris images captured with visible light and eyelids95.194.493.7Osorio-Roig et al. [127]UBIRIS.v2Used a semantic segmentation algorithm named HMRF-PyrSeg.Eyelids, glasses, eyelashes, skin, hair, and reflection85.81Susitha and Subban [81]UBIRIS.v2CASIA.v2CASIA.v1CASIA.v3Applied histogram, morphological operations, and super-pixel segmentation (SPS)The upper and lower eyelids97.697.398.898.3Khan et al. [49]CASIA.v3ND-0405Morphological operatorOcclusion by eyelashes and eyelids98.0798.55Donida et al. [83]DB Q-FIREDB WVUDBUBIRIS.v2DB CASIA.v4DB IITDDBNotredameThey introduced an iris segmentation technique, namely, polar spline RANSAC, which is designed for implementation on non-ideal ocular imagesPoor contrast or illumination, gaze deviations, occlusions, reflections, and blur due to motion or poor focusEER = 5.98EER = 4.34EER > 30EER = 3.33EER = 4.02EER = 7.98


#### 5.1.1. Hough Transform-Based Segmentation Techniques

Iris segmentation using traditional techniques began with the Hough transform (HT) method. Bakshi et al. [72] and Raffei et al. [105] applied an iris segmentation phase which consists of three subphases, namely, localization of the iris region, occlusion caused by the eyelid, and occlusion caused by an eyelash. In the localization of the iris region, the circular Hough transform (CHT) method is implemented to get the boundaries of the iris region [128]. Because the pupil is within the iris part, this operation starts with the external boundary (the boundary between iris and sclera) rather than the entire eye image. They applied a linear Hough transform (HT) method to remove the eyelid occlusions. Then, they used the thresholding method to remove the occlusions of the eyelash [39, 72, 105]. Despite that, there were limitations due to at-a-distance and less-constrained environments such as reflection noise and occlusion due to eyelids, eyelashes, and shadow. Thus, Zhao and Ajay [110] proposed the application of the RTV-L2 method which is an improved relative total variation (RTV) model which was suggested by Xu et al. [129] to first localize the principal eye structures such as the pupil, sclera, and eyelid boundaries. Next, iris localization using the CHT method that was introduced in Davies [130] was used to detect the lower half circles to determine the circle center and prevent interference from eyelashes or eyebrows. Then, the lower half region of the iris pixels was identified through local gray level analysis using thresholding that is based on histogram to determine noisy pixels, which in turn helped to determine the thresholds of the upper half region of the iris pixels for segmentation accuracy. Also, the error rate of the iris segmentation process is very high because it is very time consuming and the quality of the iris images is poor [110].To reduce the search area as well as the computation time of the CHT, Umer et al. [131] introduced a restricted circular Hough transform (RCHT) technique to locate both outer (iris-sclera) and inner (pupil-iris) edges of the iris region. First, they used a morphological filter to eliminate small noises to enhance the image. Then, the RCHT was used on the edge points of the binary eye image to obtain the center and the radius of the circular inner edge (between iris and pupil) of the iris region. To obtain the circular outer edge (between iris and sclera) of the iris region, the high-intensity difference between iris pixels for both vertical and horizontal direction from the center of the inner edge was detected. Here, this system is not proposed to remove reflection, eyelids, eyelashes, and noise [131].Salve and Narote [132] applied a fusion method based on the Canny edge detector as well as the HT method [133] to segment the iris part. The performance of iris segmentation is low due to low image quality in some databases such as CASIA-V4-Iris Twins [132]. To increase the performance accuracy of the segmentation of the iris region with some limitations such as eyelid and reflection, Ahuja et al. [134] used the segmentation algorithm which was based on the generalized Hough transform (GHT) method used by Ballard [135], which was further developed using a modified integro-differential Daugman algorithm [16] to segregate the pixel of the iris region in the eye image as described in Haindl and Krupička [120]. This algorithm provided the segmented and normalized pixels of the iris region along with the binary mask to eliminate the noise of the iris image [134].Thus, Schlett et al. [136] introduced a thorough multi-spectral analysis to enhance iris segmentation efficiency in a visible environment which was done through obtaining spectral elements in the form of RGB color channels. In this method, three segmentation mechanisms of iris region were used, namely, (1) contrast-adjusted Hough transform (CAHT), (2) iterative Fourier push-pull (IFPP), and (3) weighted adaptive Hough and ellipsopolar transform (WAHET) [137] by University of Salzburg Iris Toolkit (USIT) [136].


#### 5.1.2. Histogram- and Contour-Based Segmentation Techniques

Furthermore, in order to do the segmentation, Othman et al. [138] presented a description of three versions of the open-source iris recognition system (OSIRIS) which were OSIRISV2 (https://svnext.it-sudparis.eu/svnview2-eph/ref_syst/Iris_Osiris/), OSIRISV4 (https://svnext.it-sudparis.eu/svnview2-eph/ref_syst/Iris_Osiris_v4.1/), and OSIRISV4.1. OSIRISV2 is the first open-source version since 2009. The segmentation of OSIRISV2 consists of two steps: (1) iris localization using the circular Hough transform (CHT) method and (2) iris contour initialization using the approach suggested by Xu and Prince [139] to initialize an active contour and then initialize a window mask generation which references only the texture of the iris region by exploiting the refined contours. In practice, the evaluation of OSIRIS.V2 versions [139] has shown the weaknesses of the segmentation phase [138]. These weaknesses motivated researchers to increase performance accuracy of the segmentation technique in all the OSIRIS versions [138]. Both OSIRIS.V4 and OSIRIS.V4.1 versions used the Viterbi algorithm to overcome the limitations of iris segmentation in OSIRIS.V2 [140]. However, low-quality images generally have blurriness, low contrast, illumination changes, sharp specular reflections, and occlusions due to eyelids and eyelashes, and reflections due to eyeglasses and may consequently pose a challenge when segmenting by using the usual algorithms based on contour fitting [138].To develop the performance of iris segmentation of the OSIRISV4 open-source model for these challenges, Yahiaoui et al. [89] introduced a method by extending the Viterbi algorithm [140] by adding statistical techniques for iris segmentation based on unsupervised approaches, and they focused especially on the hidden Markov chain method as suggested in [89, 141, 142]. To increase the performance rate of the segmentation phase of iris images, some limitations were still prevalent such as reflection from glasses and different scanners as well as reflection occluded by eyelids and eyelashes, leading He et al. [143] and Liu et al. [35] to further develop the Hough transform (HT) method based on the edge detection method which was applied by Canny [144] to segment the iris region. The Hough transform method was used to detect boundaries by exploiting the duplicity between points on the boundaries and the parameters which were the coordinates of the center and radius of those boundaries [135], and these parameters were determined based on the weight of the matrix after the threshold of the radius was given [35, 143].Other researchers like Banerjee and Mery [145] proposed an unsupervised algorithm for iris segmentation automatically based on geodesic active contours (GAC) [146] and GrabCut [147]. In this method, four segmentation steps of the iris region were used, namely, (1) to detect near a central point inside the pupil using intensity-based profiling of the eye image; (2) to apply that point as the center, in which the boundary of the iris and the pupil was estimated by using GAC (an iterative power minimization technique based on the gradient of intensities); (3) to segment the region of the iris using these two estimations by applying a version of GrabCut (an energy minimization technique of the graph cut family that describes the iris image as a Markov random field); and (4) to improve the final output by using the GrabCut-based ellipse-fitting scheme. Despite applying these four steps, there were still limitations such as off-focus, specular reflections, lighting, hair, eyeglasses, occlusion due to eyelashes, and the speed of the process for running the morphological GAC operation [145].Segmentation of the iris region that is captured in an unrestricted environment remains a challenging task. Thus, Chai et al. [85] introduced a localized active contour method to segment the iris that was suggested by Lankton and Tannenbaum [148]. A local Chan–Vese (LCV) region-based active contour method was employed to segment the iris images within a non-ideal visible wavelength [85].They applied B-spline, the explicit method, that was suggested by Barbosa et al. [149] to overcome the limitations of the original method by enhancing its computational performance. In the geodesic active contour method, many iterations are required for accuracy as this consumes a lot of processing time [85]. Thus, it is necessary to adopt a better technology that considers more local information to avoid occlusion due to eyelashes and eyelids to increase the accuracy of the proposed approach. High-resolution segmentation of the iris region is requisite for the future of IRS, particularly the unconstrained environment, as most of the images of the iris taken under visible wavelength spectrum are expected to be non-ideal and noisy, to have reflections due to eyeglasses, and to have occlusions. Radman et al. [77] introduced an iris segmentation technique that focuses on the challenges in the IRS by using images in unconstrained environments. This technique localizes the iris accurately based on the histogram of oriented gradients (HOG) [150] and SVM technique [151, 152], namely, HOG-SVM technique suggested by Zaklouta and Stanciulescu [153] and Mao et al. [154], to reduce the results of false fragmentation by defining the structure of the iris region. Based on this method of localization, the texture of the iris region is extracted automatically through means of cellular automata methods that were developed for the GrowCut method [155] with the labelling of only a few of its pixels. Also, preprocessing and postprocessing are implemented in order to enhance the accuracy of segmentation performance [77].


#### 5.1.3. Other Segmentation Techniques

Osorio-Roig et al. [127] implemented a semantic segmentation technique called HMRF-PyrSeg [156] which was basically designed to segment public images. The proposed technique was used to identify the iris region in the eye images captured within the visible wavelength spectrum. Also, Osorio-Roig et al. [127] provided semantic information about various regions within the eye image, such as eyebrows, iris, sclera, and pupil to segment the region of the iris. To improve performance accuracy of this technique, future research may want to apply hybrid techniques that support combining regions of various segmentation levels and also to determine the most suitable segmentation level for each image automatically [127].Djoumessi [75] proposed an improvement in the techniques performed as suggested by Masek [157] for iris segmentation. The proposed method used integro-differential Daugman operator to determine boundaries of both pupil and iris regions. Then, they used the linear Hough transform (LHT) method to remove eyelids, and the threshold was used to remove other occlusions such as eyelashes from the segmented iris [75]. To increase performance accuracy of the iris segmentation method in removing the eyelashes and eyelids, Nguyen et al. [158] applied the integro-differential operator based on Gaussian blurring filter to detect the circular edges that were the internal and external edges of the iris region via iteratively exploring the highest replies of a contour determined through the parameters. When there is occlusion of the iris due to the eyelids and eyelashes, the best candidate is selected by utilizing the iterative approach of the thresholding technique localizing the eyelids along with contour integration. Applying the integro-differential operator changes the iris to an arc rather than a circle. The iris pixels are differentiated from the non-iris pixels (such as eyelashes and eyelids) in an input image through the noise mask method. The noise masks, which correspond to each input image, are created through the segmentation phase and applied in the proceeding phases of iris recognition [158].Gangwar et al. [76] introduced a segmentation method designed particularly for iris images taken in unconstrained (non-ideal) environments. The first step in the segmentation scheme would be applying pupil detection techniques. According to [159–161], pupil detection techniques are based on a pupil candidate bank (PCB) which was generated by utilizing the iterative approach of the thresholding technique followed by a selection of the best candidate by exploiting multiple local features. This was done by applying an adaptive filter method to roughly estimate the limbic boundary in polar space. Next, the limbic boundary would be further refined in the Cartesian space using the dual circles [159–161]. Following this, adaptive and similarity-based filters were used to detect eyelid boundaries. Then, the morphological operation method with arched kernels was used for edge refinements, followed by the estimation of the final boundaries based on parabola fitting suggested by Wildes [162] as well as Daugman’s [163] experimental study which had been limited to iris images captured in NIR environment only. Therefore, they recommended that future work should emphasise the adaptation of the segmentation algorithms to visual spectrum images. The performance accuracy of the suggested method may degrade significantly in visual spectrum images because the visual spectrum images are more challenging than NIR images [76].To enhance the speed and performance of different challenges such as reflections and occlusions, Gad et al. [80] applied the modified masking technique (MMT) for iris segmentation. The execution of this algorithm involves three steps, namely, (1) mask generation to remove the eyelashes, (2) iris detection by mask convolution operations to detect the iris region border and eyelid detection by using adaptive local threshold (ALT) filter, and (3) convolution with a unit vector (M) to extract iris region by applying three points derived from iris mask with three-distance points. These points are not equal due to boundary detection factors and occlusion and challenges such as the use of lenses or eyeglasses, distance, poor contrast, specular reflections, prevalent eyelids and eyelashes, and off-angle or off-axis [80].This has led Khan et al. [49] to introduce the iris segmentation method based on a field programmable gate array (FPGA), which is considered most appropriate for real-time methods. The proposed method used the image taken after subtracting the background, along with morphological processes to localize the pupil. The suggested non-iterative method was executed on a field programmable gate array (FPGA) to achieve high performance and speed with smaller memory [49]. However, the suggested method failed to locate the pupillary edge in the presence of thick eyelashes and eyebrows, and it will not work accurately on visible spectrum images due to the lower contrast between both the iris and pupil regions in the visible spectrum.Reddy et al. [164] proposed an iris segmentation algorithm on mobile-captured images which relies on the integration of Daugman’s integro-differential algorithms and K-means clustering. The suggested algorithm can be divided into three main steps. It begins with the removal of specular reflection, if any, in addition to enhancing the contrast of eye images by applying CLAHE [165]. It is then followed by implementing K-means clustering to extract the iris region. Finally, Daugman’s integro-differential algorithm is applied to detect the radius and center of the pupil and iris regions. To evaluate the accuracy of the performance of the algorithm proposed in this study, a publicly available VISOB dataset is applied to prove the efficiency of the introduced method [164]. To improve the accuracy of the iris segmentation performance to several challenges in non-ideal ocular images, like deviation of eye, poor contrast or illumination, occlusions, specular reflections, and blurring due to motion or poor focus, Donida et al. [83] introduced an iris segmentation technique based on random sample consensus (RANSAC), namely, polar spline RANSAC technique. The introduced PS-RANSAC method benefits from the performance of the RANSAC technique [166] to deal with noise and the ability of splines to perform functions. However, other iris segmentation methods based on the RANSAC technique in previous works considered the shape of the iris edge as conic as opposed to circular or other shapes, but this has reduced the accuracy performance when dealing with ocular images in non-ideal environments [167, 168]. Future studies should focus on new lighting compensation methods designed to enhance feature extraction of ocular images. Special interest should be on techniques designed for the iris images collected with noise due to eyeglasses and a big distance between the camera and the subject [83].


### 5.2. Iris Segmentation Using Deep Learning Techniques

This section describes the major contributions using deep learning techniques in the area of iris segmentation over the last five years. This shows the increasing interest of the scientific research community in this topic. Table6 illustrates the performance efficiency rate for the segmentation phase utilizing deep learning techniques. In addition, this section also explains some segmentation techniques used in previous works that were based on the concept of deep learning to extract the iris region. The iris segmentation using deep learning techniques can be split into three subsections based on the architecture used which are U-Net architecture-based models, VGG and R-CNN architecture-based models, and other CNN architecture-based models.Table 6
Accuracy rate (ACC) or error rate (EER) of some iris segmentation methods using deep learning techniques.
SourceDatabasesMethodsIssues focused onACC/EER (%)He et al. [169]MEIZU4DCNNVisible spectral iris images97.5Lakra et al. [170]IIITDCataract surgeryThe proposed algorithm, namely, SegDenseNet, is a deep learning algorithm based on DenseNet [171]Cataract and postcataract surgery95.4–96.7Varkarakis et al. [172]CASIA-1000Bath 800Used a fully convolutional neural network (FCNN)Off-axis iris images99.3499.12Ahmad and Fuller [173]UBIRIS-v2MR-CNN algorithms based on a CNN techniqueEyelid, eyelashes, eyeglasses, and reflection94.8Yang et al. [174]CASIA-v4UBIRIS-v2IITD DelhiFully convolutional network (FCN) with dilated convolutionsOcclusion caused by hair and eyelash98.695.798.4Bazrafkan et al. [8]CASIA-1000Bath800UBIRISMobBioFully convolutional deep neural network (FCDNN)Shadows, socket, resolution, contrast, and blurring99.7198.5597.8296.12Lian et al. [175]UBIRIS.v2CASIA. V4They used an ATT-UNet structure that has a structure similar to the VGG16 without FC layersPoor illumination, blurring, and reflection96.3296.81Lozej et al. [176]CASIA-v1They used a U-Net for iris segmentationEyelids and eyelashes96.90Arsalan et al. [40]CASIA-v4IITDelhiv1IrisDenseNet: densely connected convolutional network (DenseNet) with SegNet based on encoder-decoder convolutional networkOff-angle, glasses, rotated eyes, and specular reflection97.5897.56Arsalan et al. [41]CASIA-v4IITD-v1UBIRIS-v2Fully residual encoder-decoder network (FRED-Net)Eyelid, eyelash, and glint detections97.8397.6196.30Li et al. [177]CASIAFaster region-based convolutional neural network (R-CNN) with only six layersSpecular reflections, eyelashes, and eyelids95.49Wu and Zhao [178]CASIA-V4-iris-interval.The proposed method relied on Dense U-Net, which combines densen with the U-NetOcclusion from eyelash, shooting angle, and light reflection98.36Varkarakis et al. [179]CASIA-1000Bath800UBIRIS-v2They suggested a deep neural network to segment the iris regionThe off-axis iris images99.4099.1398.83Wang et al. [180]CASIA. DistanceUBIRIS-v2MICHE-IThey proposed a deep learning-based iris segmentation method, referred to as IrisParseNetDifferent-sized iris, dark iris, eyeglasses or eyelids, illumination, and specular reflectionsEER = 2.71EER = 11.3EER = 5.70


#### 5.2.1. U-Net Architecture-Based Models

Lian et al. [175] proposed attention U-Net (ATT-UNet) to the segmentation task of the iris image. The proposed method was based on primary U-Net [181] in addition to an attention mask which guided the method for learning more distinctive features for the classification of pixels of the iris region and pixels of the non-iris region. ATT-UNet first chooses a bounding box of the possible iris part whereby an attention mask is generated. After that, attention masks are applied as a weighted function to merge with the method’s discriminatory feature maps. The structure of the ATT-UNet method is similar to the structure of VGG16 as described by Krizhevsky et al. [182], but it does not contain fully connected (FC) layers, and the weights are initialized applying the pretrained ImageNet template. Despite that, there were limitations such as occlusion due to hair, eyelashes, and eyeglasses, robust reflection, poor illumination, and blurring [175]. As such, Zhang et al. [183] proposed four network schemes that combined the dilated convolution method [184] and the U-Net [181]. The dilated convolution method can extract more information about the iris image and enhance both performance and the efficiency of the iris segmentation method. The first three schemes of this network are based on combining part of the dilated convolution with the U-Net, namely, PD-UNet, and the fourth scheme of the proposed network is based on combining a fully dilated convolution method with the U-Net network, namely, FD-UNet, which has the best performance [183]. The FD-UNet method employs dilated convolution to extract features rather than the original convolution, so that image details can be better processed. Zhang et al. recommended that the improvement of the performance of network segmentation accuracy in future work can be achieved by modifying the parameters of the network that is combined with a high performing network. Through these improvements, the performance of the network should be more accurate [183].To enhance the performance of iris segmentation in the non-ideal (unconstrained) conditions and cross-device applications, Wu and Zhao [178] proposed an iris segmentation technique referred to as Dense U-Net. The proposed Dense U-Net combines dense connectivity with U-Net’s contraction and expansion paths, in which enhanced parameters and information are important to decrease the training problems of the deep learning network. Future research should consider developing the proposed method and integrating it with the Internet of Things (IoT) to achieve full automation and improve the performance of iris segmentation in non-ideal conditions [178]. To further enhance the segmentation performance for non-ideal iris images such as different-sized iris, dark iris, occlusions due to eyeglasses or eyelids, illumination, non-cooperative samples, and specular reflections, Wang et al. [180] proposed a high-efficiency segmentation technique for iris image that relies on a deep learning approach, named IrisParseNet. The proposed network is a multi-tasking attention network for joint learning of the outer edge of the iris, iris mask, and pupil mask. In this method, the two main tasks to segment the iris are jointly implemented, i.e., the segmentation of the iris mask, as well as the localizing of the parameterized of the inner (iris-pupil) and outer (sclera-iris) iris edges. It is recommended that future studies explore further efficient approaches to directly use the spatial relation between the iris mask and boundaries of the iris region to enhance the segmentation performance of the iris region. Furthermore, lightweight iris segmentation networks are being developed for practical research and the application of mobile phone devices [180].


#### 5.2.2. VGG and R-CNN Architecture-Based Models

Patil et al. [185] proposed two approaches of a CNN based on architectures to the region of interest extraction of an iris, namely, IPSegNet1 and IPSegNet2, to obtain a circular region through iris and pupil segmentation together [185]. These two approaches were inspired by the modern detection networks which are single-shot multi-box detector (SSD) [186] and R-CNN [187]. Despite that, there were limitations in segmenting non-ideal iris images such as off-angle images and occlusion due to the eyelashes, in which the iris is skewed [185], so Rot et al. [188] introduced a method for iris segmentation based on SegNet architecture suggested by Badrinarayanan et al. [189] which uses a convolutional encoder-decoder (CED) network approach. This was previously used for the problem with sclera segmentation [190]. SegNet is a special FCN for a semantic segmentation function, which uses a 13-layer encoder for the underlying network corresponding to VGG16 without fully connected layers. This method is used to divide the eye images into six regions, i.e., the pupil region, the medial canthus region, the iris region, the sclera region, eyelashes, and the periocular region [188].Similarly, to address more challenging conditions of the iris images that allow the use of iris recognition in a wide range of cases, Korobkin et al. [191] introduced an iris segmentation method based on CNN. The proposed method was designed to combine the FCN [192] and SegNet architectures [189]. This solved some earlier issues but not all. Occlusion issues of the iris due to eyelids or eyelashes, reflections, and strong off-angle were still prevalent, leading Hofbauer et al. [100] to introduce a parameterization method to process the segmentation task on the basis of CNN. The parameterization process creates candidate segmentations by utilizing a circular Hough transform (CHT) method [193]. Hofbauer et al. [100] applied the RefineNet as suggested by Lin et al. [194]. RefineNet is a multi-track learning network for semantic segmentation, which uses a sequential architecture with four RefineNet parts, where every one of them directly relates to the result of the residual net [100].In contrast, Arsalan et al. [59] proposed a two-phase iris segmentation method based on CNN. In an unconstrained environment, this method is able to accurately detect the edges of the iris region inside eye images. In the first phase, a rough iris edge is obtained by using a modified circular Hough transform (CHT) method, which defines the region of interest (ROI) through the slight increase in the iris radius. In the second stage, CNN is used by adjusting the VGG interface to the data obtained from the ROI. From the CNN output layer are two output features, which are used to classify noniris pixels to find the edge of the iris region. The challenges are the lack of huge public databases to explore and train deep learning techniques as a huge number of 21 × 21 images are required for CNN training [59]. Although great results have been achieved using this method, segmentation of the iris region is still very difficult due to low-quality iris images in unconstrained environments, such as low resolution, side views, occlusion, eyeglasses, blurriness, off-angle, rotated eyes, and specular reflection. To increase performance accuracy of iris segmentation in unconstrained conditions, Arsalan et al. [40] introduced a densely connected fully convolutional network (IrisDenseNet) based on a deep learning approach. The proposed technique is a combination of two main techniques which are densely connected convolutional network (DenseNet) techniques [171] and SegNet techniques as applied by Badrinarayanan et al. [189] using deep convolutional encoder-decoder network. The challenges of this method are to keep the mini-batch size low owing to more time required for training due to dense connectivity and both false-positive and false-negative errors caused by eyelash pixels or noise reflection pixels or pupil pixels, which have a value similar to that of the iris pixel value and reflection noise due to a dark iris area or eyeglasses, which is still prevalent [40]. To improve iris segmentation performance, Arsalan et al. [41] proposed a fully residual encoder-decoder network (FRED-Net) based on a deep learning approach, which is an end-to-end semantic segmentation network. The proposed network is based on SegNet [189]. The proposed FRED-Net method used only 8 convolutional layers instead of 13 convolutional layers for the residual skip connections. Some limitations of this method are that the performance accuracy of semantic segmentation method depends on the number of training images and that there are false-positive errors and false-negative errors [41]. These limitations motivated Li et al. [177] to introduce an iris segmentation method that is based on the reconstructed architecture of CNN network called Faster R-CNN for iris segmentation based on the original CNN model [195] and the VGG16 model [196] that was used to detect and locate the pupillary region, followed by the Gaussian mixture model (GMM) which used expectation maximization (EM) [197, 198]. Then, to detect and enhance the limbus edge points, the limbus edge localization method [199] was applied. Finally, the iris part is located through knowing the pupillary and limbus edges. To further improve this technique, heterogeneous methods combining the strength of CNN with the speed of traditional methods were recommended. Also, another way to improve performance accuracy would be by using the semantic segmentation technique and combining it with the suggested method [177]. In order to improve the accuracy of segmentation and recognition of the iris, Zhao and Kumar [200] proposed a framework based on deep learning, called UniNet.v2, which can provide generalizability to the focused challenges. In the proposed framework, they used the Mask R-CNN [201], which implements the detection and segmentation of the iris region, identifying iris/non-iris pixels. The Mask R-CNN [201] consists of subtechniques, which include an FCN [41] and the Faster R-CNN [202] in order to split instance masks at the same time within the suggested regions. Higher accuracy in the detection of iris and segmentation of the iris region was achieved by the proposed Mask R-CNN compared to the previous works in which UniNet was used [110, 203]. In future studies, the researchers should focus on improving further efficient techniques for the simultaneous optimization for feature training processes and iris segmentation by the deep networks [200].


#### 5.2.3. Other CNN Architecture-Based Models

Liu et al. [204] introduced two networks to process the iris segmentation problem, namely, multi-scale fully convolutional networks (MFCNs) and hierarchical convolutional neural networks (HCNNs), to be used for iris images collected in noisy environments such as at-a-distance and on-the-motion environments. A comparison and analysis of these two networks show that the MFCN has better performance and effectiveness. Despite that, there were limitations, for example, the non-iris pixels similar to the pixels of the iris region can be incorrectly detected as iris pixels [204]. To address that, He et al. [169] introduced an iris segmentation method based on a deep CNN to extract the eye features and segment the iris, pupil, and sclera. The structure of CNN is a modified version of the DeepLab model [205]. This structure changed the fully connected (FC) layers to convolution layers that are similar to the structure of FCN [192]. To improve some limitations of the iris segmentation such as noise level, image contrast, image blurring, eye resolution, and shadows in the image and in order to solve the issue of the segmentation task of the iris image in handheld devices, Bazrafkan and Corcoran used a deep U-shaped network based on deep learning architecture [206]. Prevalent issues include eyeglasses, gaze direction, and eyelashes which led Lozej et al. [207] to propose a new iris segmentation method based on deep learning approach to segment heterogeneous iris images. The proposed method used DeepLabV3 as was suggested by Chen et al. [208] along with MobileNet as was introduced in Howard et al. [209] as the backbone for segmentation. The mask corresponding to the iris region is extracted, i.e., the region of interest (ROI). Then, the mask is multiplied by the input eye image to hide the residual image parts which are non-iris (i.e., multiplying input image with zero). In this method, it was found that applying unwrapped iris images had no advantage because small details such as thin eyelashes on the eye image were not removed. Furthermore, the results were much worse when standard segmentation methods were used and when the irises were normalized [207]. To improve the performance accuracy with the low complexity of iris segmentation with challenging off-axis eye patches, Varkarakis et al. [179] proposed an iris segmentation technique based on FCN [8], which contains 10 layers. The proposed network with low complexity and has been trained to produce the segmentation of the iris region for low-quality off-axis eye samples, when captured by user-facing cameras on wearable AR/VR headsets. The main challenge of this algorithm is the off-axis recognition. Future research is recommended to focus on reforms in the network architectures and augmentation/training strategies to enhance performance on particular AR/VR headsets. Further research should involve improving an optimized CNN architecture based on the semi-parallel deep neural network (SPDNN) technique [8] with a smaller number of parameters that can offer high segmentation accuracy. Additionally, further research should look into studying disease-related iris and the design of the CNN segmentation technique that is capable of handling such samples [179].


## 5.1. Iris Segmentation Using Traditional Techniques

Most of these techniques execute segmentation through the application of traditional image processing methods, like Daugman’s method, clustering, and semantic segmentation algorithm. In this section, some segmentation techniques which were used in some previous works to segment the iris region are explained.Table5 illustrates the performance accuracy rate (ACC) or error rate (EER) of different iris segmentation methods using traditional techniques. Although all these techniques show high-performance accuracy for segmentation techniques of the iris image, their performances are still determined by different environmental factors, such as long standoff distance, off-angle, blurring, reflections, occlusion, the small size of the iris, and ghost effect in unconstrained (non-ideal) conditions. Hence, the recent improvement of the techniques based on deep learning approach has led to deep learning features being introduced for iris segmentation. The iris segmentation using traditional techniques can be split into three subsections based on the architecture used which are Hough transform-based segmentation techniques, histogram- and contour-based segmentation techniques, and other segmentation techniques.Table 5
Accuracy rate (ACC) or error rate (EER) for some iris segmentation methods using traditional techniques.
SourceDatabasesMethodsIssues focused onACC/EER (%)Haindl and Krupička [120]UBIRIS.v2Used CHT for iris segmentationThe detection of the reflection and eyelid—Abdani et al. [109]Database [121]They applied digital image processing (DIP) algorithms, [122] frame differencing, and morphological operationsPterygium disease87.05Roy and Soni [123]CASIAUsed Daugman’s methodEyelid and eyelashes—Man et al. [124]Miles and “lu_wei_feb_2006”Geodesic active contour (GAC) methodPigment spots on the iris surface7789Gangwar and Joshi [125]CASIA.v4 and FOCSUsed a coarse-to-fine system based on adaptive filters and thresholding methodOcclusion due to eyelids93.41Abdullah et al. [126]CASIA.v4UBIRIS.v1MMU2They applied the fusion of shrinking active contour and expanding for the iris segmentationThe iris images captured with visible light and eyelids95.194.493.7Osorio-Roig et al. [127]UBIRIS.v2Used a semantic segmentation algorithm named HMRF-PyrSeg.Eyelids, glasses, eyelashes, skin, hair, and reflection85.81Susitha and Subban [81]UBIRIS.v2CASIA.v2CASIA.v1CASIA.v3Applied histogram, morphological operations, and super-pixel segmentation (SPS)The upper and lower eyelids97.697.398.898.3Khan et al. [49]CASIA.v3ND-0405Morphological operatorOcclusion by eyelashes and eyelids98.0798.55Donida et al. [83]DB Q-FIREDB WVUDBUBIRIS.v2DB CASIA.v4DB IITDDBNotredameThey introduced an iris segmentation technique, namely, polar spline RANSAC, which is designed for implementation on non-ideal ocular imagesPoor contrast or illumination, gaze deviations, occlusions, reflections, and blur due to motion or poor focusEER = 5.98EER = 4.34EER > 30EER = 3.33EER = 4.02EER = 7.98


### 5.1.1. Hough Transform-Based Segmentation Techniques

Iris segmentation using traditional techniques began with the Hough transform (HT) method. Bakshi et al. [72] and Raffei et al. [105] applied an iris segmentation phase which consists of three subphases, namely, localization of the iris region, occlusion caused by the eyelid, and occlusion caused by an eyelash. In the localization of the iris region, the circular Hough transform (CHT) method is implemented to get the boundaries of the iris region [128]. Because the pupil is within the iris part, this operation starts with the external boundary (the boundary between iris and sclera) rather than the entire eye image. They applied a linear Hough transform (HT) method to remove the eyelid occlusions. Then, they used the thresholding method to remove the occlusions of the eyelash [39, 72, 105]. Despite that, there were limitations due to at-a-distance and less-constrained environments such as reflection noise and occlusion due to eyelids, eyelashes, and shadow. Thus, Zhao and Ajay [110] proposed the application of the RTV-L2 method which is an improved relative total variation (RTV) model which was suggested by Xu et al. [129] to first localize the principal eye structures such as the pupil, sclera, and eyelid boundaries. Next, iris localization using the CHT method that was introduced in Davies [130] was used to detect the lower half circles to determine the circle center and prevent interference from eyelashes or eyebrows. Then, the lower half region of the iris pixels was identified through local gray level analysis using thresholding that is based on histogram to determine noisy pixels, which in turn helped to determine the thresholds of the upper half region of the iris pixels for segmentation accuracy. Also, the error rate of the iris segmentation process is very high because it is very time consuming and the quality of the iris images is poor [110].To reduce the search area as well as the computation time of the CHT, Umer et al. [131] introduced a restricted circular Hough transform (RCHT) technique to locate both outer (iris-sclera) and inner (pupil-iris) edges of the iris region. First, they used a morphological filter to eliminate small noises to enhance the image. Then, the RCHT was used on the edge points of the binary eye image to obtain the center and the radius of the circular inner edge (between iris and pupil) of the iris region. To obtain the circular outer edge (between iris and sclera) of the iris region, the high-intensity difference between iris pixels for both vertical and horizontal direction from the center of the inner edge was detected. Here, this system is not proposed to remove reflection, eyelids, eyelashes, and noise [131].Salve and Narote [132] applied a fusion method based on the Canny edge detector as well as the HT method [133] to segment the iris part. The performance of iris segmentation is low due to low image quality in some databases such as CASIA-V4-Iris Twins [132]. To increase the performance accuracy of the segmentation of the iris region with some limitations such as eyelid and reflection, Ahuja et al. [134] used the segmentation algorithm which was based on the generalized Hough transform (GHT) method used by Ballard [135], which was further developed using a modified integro-differential Daugman algorithm [16] to segregate the pixel of the iris region in the eye image as described in Haindl and Krupička [120]. This algorithm provided the segmented and normalized pixels of the iris region along with the binary mask to eliminate the noise of the iris image [134].Thus, Schlett et al. [136] introduced a thorough multi-spectral analysis to enhance iris segmentation efficiency in a visible environment which was done through obtaining spectral elements in the form of RGB color channels. In this method, three segmentation mechanisms of iris region were used, namely, (1) contrast-adjusted Hough transform (CAHT), (2) iterative Fourier push-pull (IFPP), and (3) weighted adaptive Hough and ellipsopolar transform (WAHET) [137] by University of Salzburg Iris Toolkit (USIT) [136].


### 5.1.2. Histogram- and Contour-Based Segmentation Techniques

Furthermore, in order to do the segmentation, Othman et al. [138] presented a description of three versions of the open-source iris recognition system (OSIRIS) which were OSIRISV2 (https://svnext.it-sudparis.eu/svnview2-eph/ref_syst/Iris_Osiris/), OSIRISV4 (https://svnext.it-sudparis.eu/svnview2-eph/ref_syst/Iris_Osiris_v4.1/), and OSIRISV4.1. OSIRISV2 is the first open-source version since 2009. The segmentation of OSIRISV2 consists of two steps: (1) iris localization using the circular Hough transform (CHT) method and (2) iris contour initialization using the approach suggested by Xu and Prince [139] to initialize an active contour and then initialize a window mask generation which references only the texture of the iris region by exploiting the refined contours. In practice, the evaluation of OSIRIS.V2 versions [139] has shown the weaknesses of the segmentation phase [138]. These weaknesses motivated researchers to increase performance accuracy of the segmentation technique in all the OSIRIS versions [138]. Both OSIRIS.V4 and OSIRIS.V4.1 versions used the Viterbi algorithm to overcome the limitations of iris segmentation in OSIRIS.V2 [140]. However, low-quality images generally have blurriness, low contrast, illumination changes, sharp specular reflections, and occlusions due to eyelids and eyelashes, and reflections due to eyeglasses and may consequently pose a challenge when segmenting by using the usual algorithms based on contour fitting [138].To develop the performance of iris segmentation of the OSIRISV4 open-source model for these challenges, Yahiaoui et al. [89] introduced a method by extending the Viterbi algorithm [140] by adding statistical techniques for iris segmentation based on unsupervised approaches, and they focused especially on the hidden Markov chain method as suggested in [89, 141, 142]. To increase the performance rate of the segmentation phase of iris images, some limitations were still prevalent such as reflection from glasses and different scanners as well as reflection occluded by eyelids and eyelashes, leading He et al. [143] and Liu et al. [35] to further develop the Hough transform (HT) method based on the edge detection method which was applied by Canny [144] to segment the iris region. The Hough transform method was used to detect boundaries by exploiting the duplicity between points on the boundaries and the parameters which were the coordinates of the center and radius of those boundaries [135], and these parameters were determined based on the weight of the matrix after the threshold of the radius was given [35, 143].Other researchers like Banerjee and Mery [145] proposed an unsupervised algorithm for iris segmentation automatically based on geodesic active contours (GAC) [146] and GrabCut [147]. In this method, four segmentation steps of the iris region were used, namely, (1) to detect near a central point inside the pupil using intensity-based profiling of the eye image; (2) to apply that point as the center, in which the boundary of the iris and the pupil was estimated by using GAC (an iterative power minimization technique based on the gradient of intensities); (3) to segment the region of the iris using these two estimations by applying a version of GrabCut (an energy minimization technique of the graph cut family that describes the iris image as a Markov random field); and (4) to improve the final output by using the GrabCut-based ellipse-fitting scheme. Despite applying these four steps, there were still limitations such as off-focus, specular reflections, lighting, hair, eyeglasses, occlusion due to eyelashes, and the speed of the process for running the morphological GAC operation [145].Segmentation of the iris region that is captured in an unrestricted environment remains a challenging task. Thus, Chai et al. [85] introduced a localized active contour method to segment the iris that was suggested by Lankton and Tannenbaum [148]. A local Chan–Vese (LCV) region-based active contour method was employed to segment the iris images within a non-ideal visible wavelength [85].They applied B-spline, the explicit method, that was suggested by Barbosa et al. [149] to overcome the limitations of the original method by enhancing its computational performance. In the geodesic active contour method, many iterations are required for accuracy as this consumes a lot of processing time [85]. Thus, it is necessary to adopt a better technology that considers more local information to avoid occlusion due to eyelashes and eyelids to increase the accuracy of the proposed approach. High-resolution segmentation of the iris region is requisite for the future of IRS, particularly the unconstrained environment, as most of the images of the iris taken under visible wavelength spectrum are expected to be non-ideal and noisy, to have reflections due to eyeglasses, and to have occlusions. Radman et al. [77] introduced an iris segmentation technique that focuses on the challenges in the IRS by using images in unconstrained environments. This technique localizes the iris accurately based on the histogram of oriented gradients (HOG) [150] and SVM technique [151, 152], namely, HOG-SVM technique suggested by Zaklouta and Stanciulescu [153] and Mao et al. [154], to reduce the results of false fragmentation by defining the structure of the iris region. Based on this method of localization, the texture of the iris region is extracted automatically through means of cellular automata methods that were developed for the GrowCut method [155] with the labelling of only a few of its pixels. Also, preprocessing and postprocessing are implemented in order to enhance the accuracy of segmentation performance [77].


### 5.1.3. Other Segmentation Techniques

Osorio-Roig et al. [127] implemented a semantic segmentation technique called HMRF-PyrSeg [156] which was basically designed to segment public images. The proposed technique was used to identify the iris region in the eye images captured within the visible wavelength spectrum. Also, Osorio-Roig et al. [127] provided semantic information about various regions within the eye image, such as eyebrows, iris, sclera, and pupil to segment the region of the iris. To improve performance accuracy of this technique, future research may want to apply hybrid techniques that support combining regions of various segmentation levels and also to determine the most suitable segmentation level for each image automatically [127].Djoumessi [75] proposed an improvement in the techniques performed as suggested by Masek [157] for iris segmentation. The proposed method used integro-differential Daugman operator to determine boundaries of both pupil and iris regions. Then, they used the linear Hough transform (LHT) method to remove eyelids, and the threshold was used to remove other occlusions such as eyelashes from the segmented iris [75]. To increase performance accuracy of the iris segmentation method in removing the eyelashes and eyelids, Nguyen et al. [158] applied the integro-differential operator based on Gaussian blurring filter to detect the circular edges that were the internal and external edges of the iris region via iteratively exploring the highest replies of a contour determined through the parameters. When there is occlusion of the iris due to the eyelids and eyelashes, the best candidate is selected by utilizing the iterative approach of the thresholding technique localizing the eyelids along with contour integration. Applying the integro-differential operator changes the iris to an arc rather than a circle. The iris pixels are differentiated from the non-iris pixels (such as eyelashes and eyelids) in an input image through the noise mask method. The noise masks, which correspond to each input image, are created through the segmentation phase and applied in the proceeding phases of iris recognition [158].Gangwar et al. [76] introduced a segmentation method designed particularly for iris images taken in unconstrained (non-ideal) environments. The first step in the segmentation scheme would be applying pupil detection techniques. According to [159–161], pupil detection techniques are based on a pupil candidate bank (PCB) which was generated by utilizing the iterative approach of the thresholding technique followed by a selection of the best candidate by exploiting multiple local features. This was done by applying an adaptive filter method to roughly estimate the limbic boundary in polar space. Next, the limbic boundary would be further refined in the Cartesian space using the dual circles [159–161]. Following this, adaptive and similarity-based filters were used to detect eyelid boundaries. Then, the morphological operation method with arched kernels was used for edge refinements, followed by the estimation of the final boundaries based on parabola fitting suggested by Wildes [162] as well as Daugman’s [163] experimental study which had been limited to iris images captured in NIR environment only. Therefore, they recommended that future work should emphasise the adaptation of the segmentation algorithms to visual spectrum images. The performance accuracy of the suggested method may degrade significantly in visual spectrum images because the visual spectrum images are more challenging than NIR images [76].To enhance the speed and performance of different challenges such as reflections and occlusions, Gad et al. [80] applied the modified masking technique (MMT) for iris segmentation. The execution of this algorithm involves three steps, namely, (1) mask generation to remove the eyelashes, (2) iris detection by mask convolution operations to detect the iris region border and eyelid detection by using adaptive local threshold (ALT) filter, and (3) convolution with a unit vector (M) to extract iris region by applying three points derived from iris mask with three-distance points. These points are not equal due to boundary detection factors and occlusion and challenges such as the use of lenses or eyeglasses, distance, poor contrast, specular reflections, prevalent eyelids and eyelashes, and off-angle or off-axis [80].This has led Khan et al. [49] to introduce the iris segmentation method based on a field programmable gate array (FPGA), which is considered most appropriate for real-time methods. The proposed method used the image taken after subtracting the background, along with morphological processes to localize the pupil. The suggested non-iterative method was executed on a field programmable gate array (FPGA) to achieve high performance and speed with smaller memory [49]. However, the suggested method failed to locate the pupillary edge in the presence of thick eyelashes and eyebrows, and it will not work accurately on visible spectrum images due to the lower contrast between both the iris and pupil regions in the visible spectrum.Reddy et al. [164] proposed an iris segmentation algorithm on mobile-captured images which relies on the integration of Daugman’s integro-differential algorithms and K-means clustering. The suggested algorithm can be divided into three main steps. It begins with the removal of specular reflection, if any, in addition to enhancing the contrast of eye images by applying CLAHE [165]. It is then followed by implementing K-means clustering to extract the iris region. Finally, Daugman’s integro-differential algorithm is applied to detect the radius and center of the pupil and iris regions. To evaluate the accuracy of the performance of the algorithm proposed in this study, a publicly available VISOB dataset is applied to prove the efficiency of the introduced method [164]. To improve the accuracy of the iris segmentation performance to several challenges in non-ideal ocular images, like deviation of eye, poor contrast or illumination, occlusions, specular reflections, and blurring due to motion or poor focus, Donida et al. [83] introduced an iris segmentation technique based on random sample consensus (RANSAC), namely, polar spline RANSAC technique. The introduced PS-RANSAC method benefits from the performance of the RANSAC technique [166] to deal with noise and the ability of splines to perform functions. However, other iris segmentation methods based on the RANSAC technique in previous works considered the shape of the iris edge as conic as opposed to circular or other shapes, but this has reduced the accuracy performance when dealing with ocular images in non-ideal environments [167, 168]. Future studies should focus on new lighting compensation methods designed to enhance feature extraction of ocular images. Special interest should be on techniques designed for the iris images collected with noise due to eyeglasses and a big distance between the camera and the subject [83].


## 5.1.1. Hough Transform-Based Segmentation Techniques

Iris segmentation using traditional techniques began with the Hough transform (HT) method. Bakshi et al. [72] and Raffei et al. [105] applied an iris segmentation phase which consists of three subphases, namely, localization of the iris region, occlusion caused by the eyelid, and occlusion caused by an eyelash. In the localization of the iris region, the circular Hough transform (CHT) method is implemented to get the boundaries of the iris region [128]. Because the pupil is within the iris part, this operation starts with the external boundary (the boundary between iris and sclera) rather than the entire eye image. They applied a linear Hough transform (HT) method to remove the eyelid occlusions. Then, they used the thresholding method to remove the occlusions of the eyelash [39, 72, 105]. Despite that, there were limitations due to at-a-distance and less-constrained environments such as reflection noise and occlusion due to eyelids, eyelashes, and shadow. Thus, Zhao and Ajay [110] proposed the application of the RTV-L2 method which is an improved relative total variation (RTV) model which was suggested by Xu et al. [129] to first localize the principal eye structures such as the pupil, sclera, and eyelid boundaries. Next, iris localization using the CHT method that was introduced in Davies [130] was used to detect the lower half circles to determine the circle center and prevent interference from eyelashes or eyebrows. Then, the lower half region of the iris pixels was identified through local gray level analysis using thresholding that is based on histogram to determine noisy pixels, which in turn helped to determine the thresholds of the upper half region of the iris pixels for segmentation accuracy. Also, the error rate of the iris segmentation process is very high because it is very time consuming and the quality of the iris images is poor [110].To reduce the search area as well as the computation time of the CHT, Umer et al. [131] introduced a restricted circular Hough transform (RCHT) technique to locate both outer (iris-sclera) and inner (pupil-iris) edges of the iris region. First, they used a morphological filter to eliminate small noises to enhance the image. Then, the RCHT was used on the edge points of the binary eye image to obtain the center and the radius of the circular inner edge (between iris and pupil) of the iris region. To obtain the circular outer edge (between iris and sclera) of the iris region, the high-intensity difference between iris pixels for both vertical and horizontal direction from the center of the inner edge was detected. Here, this system is not proposed to remove reflection, eyelids, eyelashes, and noise [131].Salve and Narote [132] applied a fusion method based on the Canny edge detector as well as the HT method [133] to segment the iris part. The performance of iris segmentation is low due to low image quality in some databases such as CASIA-V4-Iris Twins [132]. To increase the performance accuracy of the segmentation of the iris region with some limitations such as eyelid and reflection, Ahuja et al. [134] used the segmentation algorithm which was based on the generalized Hough transform (GHT) method used by Ballard [135], which was further developed using a modified integro-differential Daugman algorithm [16] to segregate the pixel of the iris region in the eye image as described in Haindl and Krupička [120]. This algorithm provided the segmented and normalized pixels of the iris region along with the binary mask to eliminate the noise of the iris image [134].Thus, Schlett et al. [136] introduced a thorough multi-spectral analysis to enhance iris segmentation efficiency in a visible environment which was done through obtaining spectral elements in the form of RGB color channels. In this method, three segmentation mechanisms of iris region were used, namely, (1) contrast-adjusted Hough transform (CAHT), (2) iterative Fourier push-pull (IFPP), and (3) weighted adaptive Hough and ellipsopolar transform (WAHET) [137] by University of Salzburg Iris Toolkit (USIT) [136].


## 5.1.2. Histogram- and Contour-Based Segmentation Techniques

Furthermore, in order to do the segmentation, Othman et al. [138] presented a description of three versions of the open-source iris recognition system (OSIRIS) which were OSIRISV2 (https://svnext.it-sudparis.eu/svnview2-eph/ref_syst/Iris_Osiris/), OSIRISV4 (https://svnext.it-sudparis.eu/svnview2-eph/ref_syst/Iris_Osiris_v4.1/), and OSIRISV4.1. OSIRISV2 is the first open-source version since 2009. The segmentation of OSIRISV2 consists of two steps: (1) iris localization using the circular Hough transform (CHT) method and (2) iris contour initialization using the approach suggested by Xu and Prince [139] to initialize an active contour and then initialize a window mask generation which references only the texture of the iris region by exploiting the refined contours. In practice, the evaluation of OSIRIS.V2 versions [139] has shown the weaknesses of the segmentation phase [138]. These weaknesses motivated researchers to increase performance accuracy of the segmentation technique in all the OSIRIS versions [138]. Both OSIRIS.V4 and OSIRIS.V4.1 versions used the Viterbi algorithm to overcome the limitations of iris segmentation in OSIRIS.V2 [140]. However, low-quality images generally have blurriness, low contrast, illumination changes, sharp specular reflections, and occlusions due to eyelids and eyelashes, and reflections due to eyeglasses and may consequently pose a challenge when segmenting by using the usual algorithms based on contour fitting [138].To develop the performance of iris segmentation of the OSIRISV4 open-source model for these challenges, Yahiaoui et al. [89] introduced a method by extending the Viterbi algorithm [140] by adding statistical techniques for iris segmentation based on unsupervised approaches, and they focused especially on the hidden Markov chain method as suggested in [89, 141, 142]. To increase the performance rate of the segmentation phase of iris images, some limitations were still prevalent such as reflection from glasses and different scanners as well as reflection occluded by eyelids and eyelashes, leading He et al. [143] and Liu et al. [35] to further develop the Hough transform (HT) method based on the edge detection method which was applied by Canny [144] to segment the iris region. The Hough transform method was used to detect boundaries by exploiting the duplicity between points on the boundaries and the parameters which were the coordinates of the center and radius of those boundaries [135], and these parameters were determined based on the weight of the matrix after the threshold of the radius was given [35, 143].Other researchers like Banerjee and Mery [145] proposed an unsupervised algorithm for iris segmentation automatically based on geodesic active contours (GAC) [146] and GrabCut [147]. In this method, four segmentation steps of the iris region were used, namely, (1) to detect near a central point inside the pupil using intensity-based profiling of the eye image; (2) to apply that point as the center, in which the boundary of the iris and the pupil was estimated by using GAC (an iterative power minimization technique based on the gradient of intensities); (3) to segment the region of the iris using these two estimations by applying a version of GrabCut (an energy minimization technique of the graph cut family that describes the iris image as a Markov random field); and (4) to improve the final output by using the GrabCut-based ellipse-fitting scheme. Despite applying these four steps, there were still limitations such as off-focus, specular reflections, lighting, hair, eyeglasses, occlusion due to eyelashes, and the speed of the process for running the morphological GAC operation [145].Segmentation of the iris region that is captured in an unrestricted environment remains a challenging task. Thus, Chai et al. [85] introduced a localized active contour method to segment the iris that was suggested by Lankton and Tannenbaum [148]. A local Chan–Vese (LCV) region-based active contour method was employed to segment the iris images within a non-ideal visible wavelength [85].They applied B-spline, the explicit method, that was suggested by Barbosa et al. [149] to overcome the limitations of the original method by enhancing its computational performance. In the geodesic active contour method, many iterations are required for accuracy as this consumes a lot of processing time [85]. Thus, it is necessary to adopt a better technology that considers more local information to avoid occlusion due to eyelashes and eyelids to increase the accuracy of the proposed approach. High-resolution segmentation of the iris region is requisite for the future of IRS, particularly the unconstrained environment, as most of the images of the iris taken under visible wavelength spectrum are expected to be non-ideal and noisy, to have reflections due to eyeglasses, and to have occlusions. Radman et al. [77] introduced an iris segmentation technique that focuses on the challenges in the IRS by using images in unconstrained environments. This technique localizes the iris accurately based on the histogram of oriented gradients (HOG) [150] and SVM technique [151, 152], namely, HOG-SVM technique suggested by Zaklouta and Stanciulescu [153] and Mao et al. [154], to reduce the results of false fragmentation by defining the structure of the iris region. Based on this method of localization, the texture of the iris region is extracted automatically through means of cellular automata methods that were developed for the GrowCut method [155] with the labelling of only a few of its pixels. Also, preprocessing and postprocessing are implemented in order to enhance the accuracy of segmentation performance [77].


## 5.1.3. Other Segmentation Techniques

Osorio-Roig et al. [127] implemented a semantic segmentation technique called HMRF-PyrSeg [156] which was basically designed to segment public images. The proposed technique was used to identify the iris region in the eye images captured within the visible wavelength spectrum. Also, Osorio-Roig et al. [127] provided semantic information about various regions within the eye image, such as eyebrows, iris, sclera, and pupil to segment the region of the iris. To improve performance accuracy of this technique, future research may want to apply hybrid techniques that support combining regions of various segmentation levels and also to determine the most suitable segmentation level for each image automatically [127].Djoumessi [75] proposed an improvement in the techniques performed as suggested by Masek [157] for iris segmentation. The proposed method used integro-differential Daugman operator to determine boundaries of both pupil and iris regions. Then, they used the linear Hough transform (LHT) method to remove eyelids, and the threshold was used to remove other occlusions such as eyelashes from the segmented iris [75]. To increase performance accuracy of the iris segmentation method in removing the eyelashes and eyelids, Nguyen et al. [158] applied the integro-differential operator based on Gaussian blurring filter to detect the circular edges that were the internal and external edges of the iris region via iteratively exploring the highest replies of a contour determined through the parameters. When there is occlusion of the iris due to the eyelids and eyelashes, the best candidate is selected by utilizing the iterative approach of the thresholding technique localizing the eyelids along with contour integration. Applying the integro-differential operator changes the iris to an arc rather than a circle. The iris pixels are differentiated from the non-iris pixels (such as eyelashes and eyelids) in an input image through the noise mask method. The noise masks, which correspond to each input image, are created through the segmentation phase and applied in the proceeding phases of iris recognition [158].Gangwar et al. [76] introduced a segmentation method designed particularly for iris images taken in unconstrained (non-ideal) environments. The first step in the segmentation scheme would be applying pupil detection techniques. According to [159–161], pupil detection techniques are based on a pupil candidate bank (PCB) which was generated by utilizing the iterative approach of the thresholding technique followed by a selection of the best candidate by exploiting multiple local features. This was done by applying an adaptive filter method to roughly estimate the limbic boundary in polar space. Next, the limbic boundary would be further refined in the Cartesian space using the dual circles [159–161]. Following this, adaptive and similarity-based filters were used to detect eyelid boundaries. Then, the morphological operation method with arched kernels was used for edge refinements, followed by the estimation of the final boundaries based on parabola fitting suggested by Wildes [162] as well as Daugman’s [163] experimental study which had been limited to iris images captured in NIR environment only. Therefore, they recommended that future work should emphasise the adaptation of the segmentation algorithms to visual spectrum images. The performance accuracy of the suggested method may degrade significantly in visual spectrum images because the visual spectrum images are more challenging than NIR images [76].To enhance the speed and performance of different challenges such as reflections and occlusions, Gad et al. [80] applied the modified masking technique (MMT) for iris segmentation. The execution of this algorithm involves three steps, namely, (1) mask generation to remove the eyelashes, (2) iris detection by mask convolution operations to detect the iris region border and eyelid detection by using adaptive local threshold (ALT) filter, and (3) convolution with a unit vector (M) to extract iris region by applying three points derived from iris mask with three-distance points. These points are not equal due to boundary detection factors and occlusion and challenges such as the use of lenses or eyeglasses, distance, poor contrast, specular reflections, prevalent eyelids and eyelashes, and off-angle or off-axis [80].This has led Khan et al. [49] to introduce the iris segmentation method based on a field programmable gate array (FPGA), which is considered most appropriate for real-time methods. The proposed method used the image taken after subtracting the background, along with morphological processes to localize the pupil. The suggested non-iterative method was executed on a field programmable gate array (FPGA) to achieve high performance and speed with smaller memory [49]. However, the suggested method failed to locate the pupillary edge in the presence of thick eyelashes and eyebrows, and it will not work accurately on visible spectrum images due to the lower contrast between both the iris and pupil regions in the visible spectrum.Reddy et al. [164] proposed an iris segmentation algorithm on mobile-captured images which relies on the integration of Daugman’s integro-differential algorithms and K-means clustering. The suggested algorithm can be divided into three main steps. It begins with the removal of specular reflection, if any, in addition to enhancing the contrast of eye images by applying CLAHE [165]. It is then followed by implementing K-means clustering to extract the iris region. Finally, Daugman’s integro-differential algorithm is applied to detect the radius and center of the pupil and iris regions. To evaluate the accuracy of the performance of the algorithm proposed in this study, a publicly available VISOB dataset is applied to prove the efficiency of the introduced method [164]. To improve the accuracy of the iris segmentation performance to several challenges in non-ideal ocular images, like deviation of eye, poor contrast or illumination, occlusions, specular reflections, and blurring due to motion or poor focus, Donida et al. [83] introduced an iris segmentation technique based on random sample consensus (RANSAC), namely, polar spline RANSAC technique. The introduced PS-RANSAC method benefits from the performance of the RANSAC technique [166] to deal with noise and the ability of splines to perform functions. However, other iris segmentation methods based on the RANSAC technique in previous works considered the shape of the iris edge as conic as opposed to circular or other shapes, but this has reduced the accuracy performance when dealing with ocular images in non-ideal environments [167, 168]. Future studies should focus on new lighting compensation methods designed to enhance feature extraction of ocular images. Special interest should be on techniques designed for the iris images collected with noise due to eyeglasses and a big distance between the camera and the subject [83].


## 5.2. Iris Segmentation Using Deep Learning Techniques

This section describes the major contributions using deep learning techniques in the area of iris segmentation over the last five years. This shows the increasing interest of the scientific research community in this topic. Table6 illustrates the performance efficiency rate for the segmentation phase utilizing deep learning techniques. In addition, this section also explains some segmentation techniques used in previous works that were based on the concept of deep learning to extract the iris region. The iris segmentation using deep learning techniques can be split into three subsections based on the architecture used which are U-Net architecture-based models, VGG and R-CNN architecture-based models, and other CNN architecture-based models.Table 6
Accuracy rate (ACC) or error rate (EER) of some iris segmentation methods using deep learning techniques.
SourceDatabasesMethodsIssues focused onACC/EER (%)He et al. [169]MEIZU4DCNNVisible spectral iris images97.5Lakra et al. [170]IIITDCataract surgeryThe proposed algorithm, namely, SegDenseNet, is a deep learning algorithm based on DenseNet [171]Cataract and postcataract surgery95.4–96.7Varkarakis et al. [172]CASIA-1000Bath 800Used a fully convolutional neural network (FCNN)Off-axis iris images99.3499.12Ahmad and Fuller [173]UBIRIS-v2MR-CNN algorithms based on a CNN techniqueEyelid, eyelashes, eyeglasses, and reflection94.8Yang et al. [174]CASIA-v4UBIRIS-v2IITD DelhiFully convolutional network (FCN) with dilated convolutionsOcclusion caused by hair and eyelash98.695.798.4Bazrafkan et al. [8]CASIA-1000Bath800UBIRISMobBioFully convolutional deep neural network (FCDNN)Shadows, socket, resolution, contrast, and blurring99.7198.5597.8296.12Lian et al. [175]UBIRIS.v2CASIA. V4They used an ATT-UNet structure that has a structure similar to the VGG16 without FC layersPoor illumination, blurring, and reflection96.3296.81Lozej et al. [176]CASIA-v1They used a U-Net for iris segmentationEyelids and eyelashes96.90Arsalan et al. [40]CASIA-v4IITDelhiv1IrisDenseNet: densely connected convolutional network (DenseNet) with SegNet based on encoder-decoder convolutional networkOff-angle, glasses, rotated eyes, and specular reflection97.5897.56Arsalan et al. [41]CASIA-v4IITD-v1UBIRIS-v2Fully residual encoder-decoder network (FRED-Net)Eyelid, eyelash, and glint detections97.8397.6196.30Li et al. [177]CASIAFaster region-based convolutional neural network (R-CNN) with only six layersSpecular reflections, eyelashes, and eyelids95.49Wu and Zhao [178]CASIA-V4-iris-interval.The proposed method relied on Dense U-Net, which combines densen with the U-NetOcclusion from eyelash, shooting angle, and light reflection98.36Varkarakis et al. [179]CASIA-1000Bath800UBIRIS-v2They suggested a deep neural network to segment the iris regionThe off-axis iris images99.4099.1398.83Wang et al. [180]CASIA. DistanceUBIRIS-v2MICHE-IThey proposed a deep learning-based iris segmentation method, referred to as IrisParseNetDifferent-sized iris, dark iris, eyeglasses or eyelids, illumination, and specular reflectionsEER = 2.71EER = 11.3EER = 5.70


### 5.2.1. U-Net Architecture-Based Models

Lian et al. [175] proposed attention U-Net (ATT-UNet) to the segmentation task of the iris image. The proposed method was based on primary U-Net [181] in addition to an attention mask which guided the method for learning more distinctive features for the classification of pixels of the iris region and pixels of the non-iris region. ATT-UNet first chooses a bounding box of the possible iris part whereby an attention mask is generated. After that, attention masks are applied as a weighted function to merge with the method’s discriminatory feature maps. The structure of the ATT-UNet method is similar to the structure of VGG16 as described by Krizhevsky et al. [182], but it does not contain fully connected (FC) layers, and the weights are initialized applying the pretrained ImageNet template. Despite that, there were limitations such as occlusion due to hair, eyelashes, and eyeglasses, robust reflection, poor illumination, and blurring [175]. As such, Zhang et al. [183] proposed four network schemes that combined the dilated convolution method [184] and the U-Net [181]. The dilated convolution method can extract more information about the iris image and enhance both performance and the efficiency of the iris segmentation method. The first three schemes of this network are based on combining part of the dilated convolution with the U-Net, namely, PD-UNet, and the fourth scheme of the proposed network is based on combining a fully dilated convolution method with the U-Net network, namely, FD-UNet, which has the best performance [183]. The FD-UNet method employs dilated convolution to extract features rather than the original convolution, so that image details can be better processed. Zhang et al. recommended that the improvement of the performance of network segmentation accuracy in future work can be achieved by modifying the parameters of the network that is combined with a high performing network. Through these improvements, the performance of the network should be more accurate [183].To enhance the performance of iris segmentation in the non-ideal (unconstrained) conditions and cross-device applications, Wu and Zhao [178] proposed an iris segmentation technique referred to as Dense U-Net. The proposed Dense U-Net combines dense connectivity with U-Net’s contraction and expansion paths, in which enhanced parameters and information are important to decrease the training problems of the deep learning network. Future research should consider developing the proposed method and integrating it with the Internet of Things (IoT) to achieve full automation and improve the performance of iris segmentation in non-ideal conditions [178]. To further enhance the segmentation performance for non-ideal iris images such as different-sized iris, dark iris, occlusions due to eyeglasses or eyelids, illumination, non-cooperative samples, and specular reflections, Wang et al. [180] proposed a high-efficiency segmentation technique for iris image that relies on a deep learning approach, named IrisParseNet. The proposed network is a multi-tasking attention network for joint learning of the outer edge of the iris, iris mask, and pupil mask. In this method, the two main tasks to segment the iris are jointly implemented, i.e., the segmentation of the iris mask, as well as the localizing of the parameterized of the inner (iris-pupil) and outer (sclera-iris) iris edges. It is recommended that future studies explore further efficient approaches to directly use the spatial relation between the iris mask and boundaries of the iris region to enhance the segmentation performance of the iris region. Furthermore, lightweight iris segmentation networks are being developed for practical research and the application of mobile phone devices [180].


### 5.2.2. VGG and R-CNN Architecture-Based Models

Patil et al. [185] proposed two approaches of a CNN based on architectures to the region of interest extraction of an iris, namely, IPSegNet1 and IPSegNet2, to obtain a circular region through iris and pupil segmentation together [185]. These two approaches were inspired by the modern detection networks which are single-shot multi-box detector (SSD) [186] and R-CNN [187]. Despite that, there were limitations in segmenting non-ideal iris images such as off-angle images and occlusion due to the eyelashes, in which the iris is skewed [185], so Rot et al. [188] introduced a method for iris segmentation based on SegNet architecture suggested by Badrinarayanan et al. [189] which uses a convolutional encoder-decoder (CED) network approach. This was previously used for the problem with sclera segmentation [190]. SegNet is a special FCN for a semantic segmentation function, which uses a 13-layer encoder for the underlying network corresponding to VGG16 without fully connected layers. This method is used to divide the eye images into six regions, i.e., the pupil region, the medial canthus region, the iris region, the sclera region, eyelashes, and the periocular region [188].Similarly, to address more challenging conditions of the iris images that allow the use of iris recognition in a wide range of cases, Korobkin et al. [191] introduced an iris segmentation method based on CNN. The proposed method was designed to combine the FCN [192] and SegNet architectures [189]. This solved some earlier issues but not all. Occlusion issues of the iris due to eyelids or eyelashes, reflections, and strong off-angle were still prevalent, leading Hofbauer et al. [100] to introduce a parameterization method to process the segmentation task on the basis of CNN. The parameterization process creates candidate segmentations by utilizing a circular Hough transform (CHT) method [193]. Hofbauer et al. [100] applied the RefineNet as suggested by Lin et al. [194]. RefineNet is a multi-track learning network for semantic segmentation, which uses a sequential architecture with four RefineNet parts, where every one of them directly relates to the result of the residual net [100].In contrast, Arsalan et al. [59] proposed a two-phase iris segmentation method based on CNN. In an unconstrained environment, this method is able to accurately detect the edges of the iris region inside eye images. In the first phase, a rough iris edge is obtained by using a modified circular Hough transform (CHT) method, which defines the region of interest (ROI) through the slight increase in the iris radius. In the second stage, CNN is used by adjusting the VGG interface to the data obtained from the ROI. From the CNN output layer are two output features, which are used to classify noniris pixels to find the edge of the iris region. The challenges are the lack of huge public databases to explore and train deep learning techniques as a huge number of 21 × 21 images are required for CNN training [59]. Although great results have been achieved using this method, segmentation of the iris region is still very difficult due to low-quality iris images in unconstrained environments, such as low resolution, side views, occlusion, eyeglasses, blurriness, off-angle, rotated eyes, and specular reflection. To increase performance accuracy of iris segmentation in unconstrained conditions, Arsalan et al. [40] introduced a densely connected fully convolutional network (IrisDenseNet) based on a deep learning approach. The proposed technique is a combination of two main techniques which are densely connected convolutional network (DenseNet) techniques [171] and SegNet techniques as applied by Badrinarayanan et al. [189] using deep convolutional encoder-decoder network. The challenges of this method are to keep the mini-batch size low owing to more time required for training due to dense connectivity and both false-positive and false-negative errors caused by eyelash pixels or noise reflection pixels or pupil pixels, which have a value similar to that of the iris pixel value and reflection noise due to a dark iris area or eyeglasses, which is still prevalent [40]. To improve iris segmentation performance, Arsalan et al. [41] proposed a fully residual encoder-decoder network (FRED-Net) based on a deep learning approach, which is an end-to-end semantic segmentation network. The proposed network is based on SegNet [189]. The proposed FRED-Net method used only 8 convolutional layers instead of 13 convolutional layers for the residual skip connections. Some limitations of this method are that the performance accuracy of semantic segmentation method depends on the number of training images and that there are false-positive errors and false-negative errors [41]. These limitations motivated Li et al. [177] to introduce an iris segmentation method that is based on the reconstructed architecture of CNN network called Faster R-CNN for iris segmentation based on the original CNN model [195] and the VGG16 model [196] that was used to detect and locate the pupillary region, followed by the Gaussian mixture model (GMM) which used expectation maximization (EM) [197, 198]. Then, to detect and enhance the limbus edge points, the limbus edge localization method [199] was applied. Finally, the iris part is located through knowing the pupillary and limbus edges. To further improve this technique, heterogeneous methods combining the strength of CNN with the speed of traditional methods were recommended. Also, another way to improve performance accuracy would be by using the semantic segmentation technique and combining it with the suggested method [177]. In order to improve the accuracy of segmentation and recognition of the iris, Zhao and Kumar [200] proposed a framework based on deep learning, called UniNet.v2, which can provide generalizability to the focused challenges. In the proposed framework, they used the Mask R-CNN [201], which implements the detection and segmentation of the iris region, identifying iris/non-iris pixels. The Mask R-CNN [201] consists of subtechniques, which include an FCN [41] and the Faster R-CNN [202] in order to split instance masks at the same time within the suggested regions. Higher accuracy in the detection of iris and segmentation of the iris region was achieved by the proposed Mask R-CNN compared to the previous works in which UniNet was used [110, 203]. In future studies, the researchers should focus on improving further efficient techniques for the simultaneous optimization for feature training processes and iris segmentation by the deep networks [200].


### 5.2.3. Other CNN Architecture-Based Models

Liu et al. [204] introduced two networks to process the iris segmentation problem, namely, multi-scale fully convolutional networks (MFCNs) and hierarchical convolutional neural networks (HCNNs), to be used for iris images collected in noisy environments such as at-a-distance and on-the-motion environments. A comparison and analysis of these two networks show that the MFCN has better performance and effectiveness. Despite that, there were limitations, for example, the non-iris pixels similar to the pixels of the iris region can be incorrectly detected as iris pixels [204]. To address that, He et al. [169] introduced an iris segmentation method based on a deep CNN to extract the eye features and segment the iris, pupil, and sclera. The structure of CNN is a modified version of the DeepLab model [205]. This structure changed the fully connected (FC) layers to convolution layers that are similar to the structure of FCN [192]. To improve some limitations of the iris segmentation such as noise level, image contrast, image blurring, eye resolution, and shadows in the image and in order to solve the issue of the segmentation task of the iris image in handheld devices, Bazrafkan and Corcoran used a deep U-shaped network based on deep learning architecture [206]. Prevalent issues include eyeglasses, gaze direction, and eyelashes which led Lozej et al. [207] to propose a new iris segmentation method based on deep learning approach to segment heterogeneous iris images. The proposed method used DeepLabV3 as was suggested by Chen et al. [208] along with MobileNet as was introduced in Howard et al. [209] as the backbone for segmentation. The mask corresponding to the iris region is extracted, i.e., the region of interest (ROI). Then, the mask is multiplied by the input eye image to hide the residual image parts which are non-iris (i.e., multiplying input image with zero). In this method, it was found that applying unwrapped iris images had no advantage because small details such as thin eyelashes on the eye image were not removed. Furthermore, the results were much worse when standard segmentation methods were used and when the irises were normalized [207]. To improve the performance accuracy with the low complexity of iris segmentation with challenging off-axis eye patches, Varkarakis et al. [179] proposed an iris segmentation technique based on FCN [8], which contains 10 layers. The proposed network with low complexity and has been trained to produce the segmentation of the iris region for low-quality off-axis eye samples, when captured by user-facing cameras on wearable AR/VR headsets. The main challenge of this algorithm is the off-axis recognition. Future research is recommended to focus on reforms in the network architectures and augmentation/training strategies to enhance performance on particular AR/VR headsets. Further research should involve improving an optimized CNN architecture based on the semi-parallel deep neural network (SPDNN) technique [8] with a smaller number of parameters that can offer high segmentation accuracy. Additionally, further research should look into studying disease-related iris and the design of the CNN segmentation technique that is capable of handling such samples [179].


## 5.2.1. U-Net Architecture-Based Models

Lian et al. [175] proposed attention U-Net (ATT-UNet) to the segmentation task of the iris image. The proposed method was based on primary U-Net [181] in addition to an attention mask which guided the method for learning more distinctive features for the classification of pixels of the iris region and pixels of the non-iris region. ATT-UNet first chooses a bounding box of the possible iris part whereby an attention mask is generated. After that, attention masks are applied as a weighted function to merge with the method’s discriminatory feature maps. The structure of the ATT-UNet method is similar to the structure of VGG16 as described by Krizhevsky et al. [182], but it does not contain fully connected (FC) layers, and the weights are initialized applying the pretrained ImageNet template. Despite that, there were limitations such as occlusion due to hair, eyelashes, and eyeglasses, robust reflection, poor illumination, and blurring [175]. As such, Zhang et al. [183] proposed four network schemes that combined the dilated convolution method [184] and the U-Net [181]. The dilated convolution method can extract more information about the iris image and enhance both performance and the efficiency of the iris segmentation method. The first three schemes of this network are based on combining part of the dilated convolution with the U-Net, namely, PD-UNet, and the fourth scheme of the proposed network is based on combining a fully dilated convolution method with the U-Net network, namely, FD-UNet, which has the best performance [183]. The FD-UNet method employs dilated convolution to extract features rather than the original convolution, so that image details can be better processed. Zhang et al. recommended that the improvement of the performance of network segmentation accuracy in future work can be achieved by modifying the parameters of the network that is combined with a high performing network. Through these improvements, the performance of the network should be more accurate [183].To enhance the performance of iris segmentation in the non-ideal (unconstrained) conditions and cross-device applications, Wu and Zhao [178] proposed an iris segmentation technique referred to as Dense U-Net. The proposed Dense U-Net combines dense connectivity with U-Net’s contraction and expansion paths, in which enhanced parameters and information are important to decrease the training problems of the deep learning network. Future research should consider developing the proposed method and integrating it with the Internet of Things (IoT) to achieve full automation and improve the performance of iris segmentation in non-ideal conditions [178]. To further enhance the segmentation performance for non-ideal iris images such as different-sized iris, dark iris, occlusions due to eyeglasses or eyelids, illumination, non-cooperative samples, and specular reflections, Wang et al. [180] proposed a high-efficiency segmentation technique for iris image that relies on a deep learning approach, named IrisParseNet. The proposed network is a multi-tasking attention network for joint learning of the outer edge of the iris, iris mask, and pupil mask. In this method, the two main tasks to segment the iris are jointly implemented, i.e., the segmentation of the iris mask, as well as the localizing of the parameterized of the inner (iris-pupil) and outer (sclera-iris) iris edges. It is recommended that future studies explore further efficient approaches to directly use the spatial relation between the iris mask and boundaries of the iris region to enhance the segmentation performance of the iris region. Furthermore, lightweight iris segmentation networks are being developed for practical research and the application of mobile phone devices [180].


## 5.2.2. VGG and R-CNN Architecture-Based Models

Patil et al. [185] proposed two approaches of a CNN based on architectures to the region of interest extraction of an iris, namely, IPSegNet1 and IPSegNet2, to obtain a circular region through iris and pupil segmentation together [185]. These two approaches were inspired by the modern detection networks which are single-shot multi-box detector (SSD) [186] and R-CNN [187]. Despite that, there were limitations in segmenting non-ideal iris images such as off-angle images and occlusion due to the eyelashes, in which the iris is skewed [185], so Rot et al. [188] introduced a method for iris segmentation based on SegNet architecture suggested by Badrinarayanan et al. [189] which uses a convolutional encoder-decoder (CED) network approach. This was previously used for the problem with sclera segmentation [190]. SegNet is a special FCN for a semantic segmentation function, which uses a 13-layer encoder for the underlying network corresponding to VGG16 without fully connected layers. This method is used to divide the eye images into six regions, i.e., the pupil region, the medial canthus region, the iris region, the sclera region, eyelashes, and the periocular region [188].Similarly, to address more challenging conditions of the iris images that allow the use of iris recognition in a wide range of cases, Korobkin et al. [191] introduced an iris segmentation method based on CNN. The proposed method was designed to combine the FCN [192] and SegNet architectures [189]. This solved some earlier issues but not all. Occlusion issues of the iris due to eyelids or eyelashes, reflections, and strong off-angle were still prevalent, leading Hofbauer et al. [100] to introduce a parameterization method to process the segmentation task on the basis of CNN. The parameterization process creates candidate segmentations by utilizing a circular Hough transform (CHT) method [193]. Hofbauer et al. [100] applied the RefineNet as suggested by Lin et al. [194]. RefineNet is a multi-track learning network for semantic segmentation, which uses a sequential architecture with four RefineNet parts, where every one of them directly relates to the result of the residual net [100].In contrast, Arsalan et al. [59] proposed a two-phase iris segmentation method based on CNN. In an unconstrained environment, this method is able to accurately detect the edges of the iris region inside eye images. In the first phase, a rough iris edge is obtained by using a modified circular Hough transform (CHT) method, which defines the region of interest (ROI) through the slight increase in the iris radius. In the second stage, CNN is used by adjusting the VGG interface to the data obtained from the ROI. From the CNN output layer are two output features, which are used to classify noniris pixels to find the edge of the iris region. The challenges are the lack of huge public databases to explore and train deep learning techniques as a huge number of 21 × 21 images are required for CNN training [59]. Although great results have been achieved using this method, segmentation of the iris region is still very difficult due to low-quality iris images in unconstrained environments, such as low resolution, side views, occlusion, eyeglasses, blurriness, off-angle, rotated eyes, and specular reflection. To increase performance accuracy of iris segmentation in unconstrained conditions, Arsalan et al. [40] introduced a densely connected fully convolutional network (IrisDenseNet) based on a deep learning approach. The proposed technique is a combination of two main techniques which are densely connected convolutional network (DenseNet) techniques [171] and SegNet techniques as applied by Badrinarayanan et al. [189] using deep convolutional encoder-decoder network. The challenges of this method are to keep the mini-batch size low owing to more time required for training due to dense connectivity and both false-positive and false-negative errors caused by eyelash pixels or noise reflection pixels or pupil pixels, which have a value similar to that of the iris pixel value and reflection noise due to a dark iris area or eyeglasses, which is still prevalent [40]. To improve iris segmentation performance, Arsalan et al. [41] proposed a fully residual encoder-decoder network (FRED-Net) based on a deep learning approach, which is an end-to-end semantic segmentation network. The proposed network is based on SegNet [189]. The proposed FRED-Net method used only 8 convolutional layers instead of 13 convolutional layers for the residual skip connections. Some limitations of this method are that the performance accuracy of semantic segmentation method depends on the number of training images and that there are false-positive errors and false-negative errors [41]. These limitations motivated Li et al. [177] to introduce an iris segmentation method that is based on the reconstructed architecture of CNN network called Faster R-CNN for iris segmentation based on the original CNN model [195] and the VGG16 model [196] that was used to detect and locate the pupillary region, followed by the Gaussian mixture model (GMM) which used expectation maximization (EM) [197, 198]. Then, to detect and enhance the limbus edge points, the limbus edge localization method [199] was applied. Finally, the iris part is located through knowing the pupillary and limbus edges. To further improve this technique, heterogeneous methods combining the strength of CNN with the speed of traditional methods were recommended. Also, another way to improve performance accuracy would be by using the semantic segmentation technique and combining it with the suggested method [177]. In order to improve the accuracy of segmentation and recognition of the iris, Zhao and Kumar [200] proposed a framework based on deep learning, called UniNet.v2, which can provide generalizability to the focused challenges. In the proposed framework, they used the Mask R-CNN [201], which implements the detection and segmentation of the iris region, identifying iris/non-iris pixels. The Mask R-CNN [201] consists of subtechniques, which include an FCN [41] and the Faster R-CNN [202] in order to split instance masks at the same time within the suggested regions. Higher accuracy in the detection of iris and segmentation of the iris region was achieved by the proposed Mask R-CNN compared to the previous works in which UniNet was used [110, 203]. In future studies, the researchers should focus on improving further efficient techniques for the simultaneous optimization for feature training processes and iris segmentation by the deep networks [200].


## 5.2.3. Other CNN Architecture-Based Models

Liu et al. [204] introduced two networks to process the iris segmentation problem, namely, multi-scale fully convolutional networks (MFCNs) and hierarchical convolutional neural networks (HCNNs), to be used for iris images collected in noisy environments such as at-a-distance and on-the-motion environments. A comparison and analysis of these two networks show that the MFCN has better performance and effectiveness. Despite that, there were limitations, for example, the non-iris pixels similar to the pixels of the iris region can be incorrectly detected as iris pixels [204]. To address that, He et al. [169] introduced an iris segmentation method based on a deep CNN to extract the eye features and segment the iris, pupil, and sclera. The structure of CNN is a modified version of the DeepLab model [205]. This structure changed the fully connected (FC) layers to convolution layers that are similar to the structure of FCN [192]. To improve some limitations of the iris segmentation such as noise level, image contrast, image blurring, eye resolution, and shadows in the image and in order to solve the issue of the segmentation task of the iris image in handheld devices, Bazrafkan and Corcoran used a deep U-shaped network based on deep learning architecture [206]. Prevalent issues include eyeglasses, gaze direction, and eyelashes which led Lozej et al. [207] to propose a new iris segmentation method based on deep learning approach to segment heterogeneous iris images. The proposed method used DeepLabV3 as was suggested by Chen et al. [208] along with MobileNet as was introduced in Howard et al. [209] as the backbone for segmentation. The mask corresponding to the iris region is extracted, i.e., the region of interest (ROI). Then, the mask is multiplied by the input eye image to hide the residual image parts which are non-iris (i.e., multiplying input image with zero). In this method, it was found that applying unwrapped iris images had no advantage because small details such as thin eyelashes on the eye image were not removed. Furthermore, the results were much worse when standard segmentation methods were used and when the irises were normalized [207]. To improve the performance accuracy with the low complexity of iris segmentation with challenging off-axis eye patches, Varkarakis et al. [179] proposed an iris segmentation technique based on FCN [8], which contains 10 layers. The proposed network with low complexity and has been trained to produce the segmentation of the iris region for low-quality off-axis eye samples, when captured by user-facing cameras on wearable AR/VR headsets. The main challenge of this algorithm is the off-axis recognition. Future research is recommended to focus on reforms in the network architectures and augmentation/training strategies to enhance performance on particular AR/VR headsets. Further research should involve improving an optimized CNN architecture based on the semi-parallel deep neural network (SPDNN) technique [8] with a smaller number of parameters that can offer high segmentation accuracy. Additionally, further research should look into studying disease-related iris and the design of the CNN segmentation technique that is capable of handling such samples [179].


## 6. Normalization Techniques

Normalization refers to the operation of converting the circular pattern of the segmented iris region into a rectangular pattern. The normalization phase is applied after the segmentation phase to help in the feature extraction.To normalize the iris region, Umer et al. [56], Yahiaoui et al. [89], and Al-Waisy et al. [111] used a rubber-sheet model [16] to change the segmented iris from a circle shape into a rectangular shape. The rubber-sheet model transforms the linear pattern of the iris area from Cartesian to polar coordinates. Daugman [163] improved the rubber-sheet model for normalizing the segmented iris part which aids to reduce any imaging inconsistencies while taking the iris image. Researchers like Raffei et al. applied the rubber-sheet model as suggested by Daugman [163] to convert dimensions of the segmented iris region from the circular shape into a rectangle shape, with dimensions of 20 × 240 pixels [39, 105].Likewise, the rubber-sheet model was utilized by Bakshi et al. [72], Djoumessi [75], Gad et al. [80], Salve and Narote [132], Nguyen et al. [158], and Kaur et al. [210] to normalize the iris region. This was accomplished by changing the segmented iris region into a fixed rectangle. Therefore, a popular choice would be the rubber-sheet method which can be used repeatedly. However, the challenge of this method is that it is more sensitive to rotation. Due to the difference in the radius of both the pupil and iris, some of the pixels are repeated through the normalization process to fill the empty space of the iris image. Several past and present researchers have attempted to continuously improve normalization methods in order to overcome this limitation such as Shamsi and Rasouli, who introduced a trapezoidal approach [211]. In this approach, the iris region is divided into a number of concentric circles and then unwrapped. There is no need for iteration because it is a one-to-one mapping. Thus, this technique reduces the false acceptance. The weakness in this approach is that it has a fixed structure [211]. As such, Hilal et al. [212] introduced a flexible method for non-circular images of the pupil region. This method used flexible disks rather than a fixed circular pattern to trace the curved nature of the iris region. The challenge of this approach is to detect the center of the pupil region. Extracting the feature would be hard as any error made in locating the center may impact the elastic contours.


## 7. Feature Extraction Technique

After obtaining the boundaries of the iris and mapping this region, feature extraction methods should be utilized to obtain the special features of the texture of the iris region which differs from individual to individual. The feature extraction phase is a very important process to achieve high accuracy in identifying a person because each person’s unique features must be extracted. The feature extraction techniques can be split into two sections which are feature extraction using traditional techniques and feature extraction using deep learning techniques.


### 7.1. Feature Extraction Using Traditional Techniques

This section explains the application of traditional techniques for feature extraction. One such technique to extract the features of normalized iris images would be the 1D log-Gabor filters. Gong et al. [213], Raffei et al. [39, 71, 105], Bakshi et al. [72], and Salve and Narote [132] applied 1D log-Gabor filters for feature extraction of normalized iris images. The1D log-Gabor filters are an extension of the Gabor filters which were not suitable to deal with spectral information with maximal spatial localization. The disadvantage of the Gabor filter is that whenever the bandwidth exceeds one octave even the symmetric filter will have a direct current (DC) component, while the log-Gabor filters have no DC components for any bandwidth [132]. By having extended ends, the log-Gabor filters are more efficient to encode original images than the traditional Gabor filters, which would overrepresent the low-frequency segments and underrepresent the high-frequency segments in any encoding. The challenges of Gabor filters are addressed with the application of the log-Gabor filters.Other researchers such as Yahiaoui et al. [89], Othman et al. [138], Jan et al. [214, 215], Jan and Usman [216], Raja et al. [217], and, more recently, Nazmdeh et al. [6] proposed the use of 2D-Gabor filters, which are very popular and have been successfully implemented in practical applications. 2D-Gabor filters are typically used to obtain the global tissue information in iris regions for the feature extraction method.Ahmadi and Akbarizadeh [218] have proposed a hybrid robust iris recognition approach using iris image preprocessing, two-dimensional Gabor kernel (2-DGK) features, and multi-layer perceptron neural network-particle swarm optimization (MLPNN-PSO). The challenge of the IRS with 2-DGK feature extractor method and MLPNN-PSO classifier was that its efficiency rate was not at an acceptable level. To improve the performance of feature extraction of the iris texture, Ahmadi et al. [219] introduced a technique relying on an intelligent hybrid system for iris recognition. The proposed technique is based on a combination of three techniques which are 2-DGK [218], polynomial filtering (PF) [220, 221], and step filtering (SF) to extract features of the iris image. For future studies, focus should be on improving IRS by fusing advanced techniques such as deep learning techniques in order to train the algorithms with other evolutionary-based algorithms to reduce the required storage space and computational complexity.Several key point-based techniques for iris recognition have been introduced in the past by Santos and Hoyle [68] and Sun et al. [222], and to improve the performance of previous research, Alvarez-Betancourt and Garcia-Silvente [30] suggested a method to extract features based on the key points from the iris region. The suggested method utilized three filters to identify key points such as Fast Hessian which was applied by Bay et al. [223], the Harris–Laplace method which merged the Harris detector method and the scale selection method that were used by Mikolajczyk and Schmid [224], and lastly the Hessian–Laplace detector that was presented by Tuytelaars and Mikolajczyk [225]. The suggested method is based on the merging of three information exporters of scale-invariant feature transform (SIFT) which was applied by Lowe [226]. This method worked well enough in some cases of iris image noise, but it is still needed to be developed in terms of time and accuracy even in research that is considered recent [30].Kumar et al. [73] applied the combination of discrete wavelet transform (DWT) that was introduced in Jang et al. [227] and triangle-based discrete cosine transform (DCT) from Kekre et al. [228] and Nagi et al. [229] to extract features of the iris region. The proposed method still has limitations such as implementation in unrestricted environments and real-time applications [73]. Thus, Barpanda et al. [230] proposed using the wavelet derived from the Cohen–Daubechies–Feauveau 9/7 filter bank to extract iris features in three steps, namely, (1) computing the two-level wavelet from the iris template, (2) computing the power spectrum of the wavelet transformed image, and (3) computing the discrete cosine transform to get the spectrum coefficients [230]. Recently, Barpanda et al. [231] proposed applying wavelet mel-cepstrum in order to extract the iris feature. The suggested technique is based on the wavelet derived from the public orthogonal Cohen–Daubechies–Feauveau (CDF) 9/7 filter bank. In the proposed method, the mel-frequency cepstrum coefficients (MFCCs) were applied to distinguish the iris tissues. The proposed technique is used to extract iris features in three steps. The first step is to calculate the two-level detail coefficients of the iris tissue. After that, the logarithmic method is applied to divide these detailed coefficients into irregular squares. This helps in reducing the dimensions of wavelet coefficients followed by designated irregular weights to various frequency segments. Then, the separate cosine transform method is used to calculate the same features, through which the features are extracted [231].The weakness of the suggested method is that with decreased feature size, there is a subtle rise in inaccuracy. Through the analysis of the results, it is obvious that the cepstrum features relied on the wavelet to perform on par with the CDF 9/7 filter bank. Thus, it is believed that the focus of future research should be on developing an iris recognition system with higher accuracy in a time-efficient manner for collected images under NIR and visible spectrum utilizing colors information.Other researchers such as Gad et al. [80] suggested a Cognitive Internet of Things (CIoT) method by using a combined vision between the single-modal and multi-algorithm of the recognition system of the iris image for the purpose of investigation. They applied delta mean (DM) and multi-algorithm mean (MAM) to extract the features [80].The multi-algorithm methods were carried out in a parallel processing method. Other researchers like Liu et al. introduced a collaborative cooperative method to extract features of iris texture [101].


### 7.2. Feature Extraction Using Deep Learning Techniques

Minaee et al. [232] applied architecture of VGG16-Net as suggested by Krizhevsky et al. [182] and Simonyan and Zisserman [196] to obtain deep feature extraction from iris images and draw deep feature map by executing a sequence of convolution. Then, the principal component analysis (PCA) method was applied by Abdi and Williams [233] to reduce features’ dimensionality. The result of this method could be more enhanced by training a deep network specifically for recognition of iris images [232]. To discover eye images generated by the computer by exploring the differences in the eye region, Carvalho et al. [97] introduced VGG architecture with 19 layers (VGG19) instead of the VGG architecture with 16 layers (VGG16) that was initially suggested by Simonyan and Zisserman [196] based on deep neural networks (DNNs) as the basis of the ImageNet. In the proposed method, the convolutional blocks functioned as feature extraction layers. Future research directions should delve deeper into improving the features extraction techniques by introducing the fusion of bottleneck features extracted from different deep CNN models as the fusion system is said to achieve higher performance [97, 158].To improve the performance with less computational complexity and required memory space as well as full automation in the deep network design of the IRS for both automatic feature design and network structure design at the same time, Nguyen et al. [57] introduced a method to restructure the design of the iris recognition network that accounted for constraints of computation and memory. The proposed method was used to investigate the traditional handcrafted IrisCode performance to compare with deep network techniques. It also tried to further enhance the current deep networks of iris recognition to realize similar or higher efficiency with the same level of memory and computational complexity. More importantly, the proposed technique fully automates the process of network design to search for the best network architecture of iris recognition confined to the compactness of model and computation constraints. According to this proposed algorithm, it can be concluded that the performance of deep networks is affected when the computation is robustly limited while this enhances the performance of traditional IrisCode techniques in both computation and accuracy of iris recognition [57].Ahuja et al. [134] proposed two methods based on CNN to verify the images of the periocular of the eye, which comprises the iris region. In the first method, they proposed a CNN-based unsupervised technique, fusing the result of external feature vectors and the RootSIFT baseline technique as suggested by Arandjelović and Zisserman [234]. In the second method, they proposed the CNN-based supervised technique and used a cosine similarity as suggested by Nguyen and Bai [235] to get the similarity between two images. Both the methods are based on deep CNN, and both the methods used a stacked-layer architecture [134]. In order to improve the performance accuracy of ocular biometric recognition, Reddy et al. [236] introduced OcularNet based on a CNN architecture for mobile ocular biometrics. In the proposed OcularNet model, they extracted six enrolled patches from the regions of periocular and ocular as well as trained a small CNN for every patch called PatchCNN to extract and generate feature descriptors that are differentiable between the individuals. The recommendation was for future studies to increase the analysis of different mobile-optimized architectures like MobileNet as well as MobileNet-v2 to further decrease the number of parameters [236].To obtain iris features with a clearer distinction as well as to solve the determination problems of classification thresholds for iris recognition tasks by increasing the iris samples, Chen et al. [237] proposed tight center (T-center) loss function to extract features of the iris region with more prominent distinction. The proposed T-center loss function was utilized to resolve the insufficient distinction problem with the conventional softmax loss function. The discriminative ability of deep features based on the CNN model can be enhanced by the linear fusion of softmax and T-center loss functions [237].Other researchers like Zhang et al. [95] and Al-Waisy et al. [111] introduced a technique on the basis of a CNN approach to extract the features of the iris region. In the suggested technique, the features of the iris region were extracted automatically by applying convolution with pooling layers. In practical uses, taking iris images is a challenge when there is reflection due to glasses, different lighting environments, etc [95, 111]. Also, further research is still needed to improve the accuracy of randomized performance by randomly reading the iris image. Liu et al. [35] introduced the F-CNN and F-Capsule to increase the robustness of deep learning training as opposed to CNN and capsule. The suggested F-CNN and F-Capsule contribute to the training process of the R-CNN by applying a selective search for image feature extraction to enhance the speed and performance of the training process [35]. Zhao et al. [112] introduced a deep learning technique to extract features based on the architecture of a capsule network for the IRS. In this method, the roles of the convolutional part are to extract the features of low-intensity pixels of the iris images and to form the main features used for the main capsule layer. Based on the dynamic routing approach between two layers of the capsule network, Zhao et al. [112] implemented a modified routing method to make the suggested algorithm adapt to a recognition system of iris images. This migration learning method allows the use of deep learning technology when the number of iris images is limited. Accordingly, three state-of-the-art pretrained designs, VGG16, InceptionV3, and ResNet50, are introduced and split into a sequence of subnetwork structures in accordance with the number of their primary constituent blocks [112]. Future research should further develop this method by applying capsule (vector) feature learning network to solve iris recognition challenges in unconstrained environments and improve research network processing capabilities and processing techniques when the number of classes becomes very huge.Nguyen et al. [158] presented five off-the-shelf CNN to extract features of the iris region in eye images. They are AlexNet that was used by Krizhevsky et al. [182], VGG network from Simonyan and Zisserman [196], Google Inception network that was introduced by Szegedy [238], ResNet that was applied by He et al. [239], and DenseNet that was suggested by Huang et al. [171]. To increase the performance off-the-shelf CNN for extracting features of iris images, classical CNNs can be combined with other methods such as recurrent neural network (RNN), and stacked autoencoder (SAE), and deep belief network (DBN) [158]. To increase the performance accuracy of the feature extraction, Marra et al. [240] suggested a technique based on CNN techniques to adapt and to decrease the number of parameters as well as computational complexity (both in training and testing stages) of the AlexNet that was suggested by Krizhevsky et al. [182]. The proposed AlexNet technique consists of three convolutional layers to extract the feature. In these three layers, a rectified linear unit (ReLu) is implemented as a non-linear task following convolution, which is followed by max-pooling layers [240]. Zhao and Kumar [200] proposed a framework based on deep learning, named UniNet.v2, for iris detection, segmentation, and recognition. In the proposed framework, they used FeatNet, based on FCN, which was originally advanced for semantic segmentation [192] to extract iris features of the iris region. A particularly designed extended triplet loss (ETL) [241] function was performed to combine the non-iris masking and bit-shifting, which were found important for learning significant and distinctive spatial features of the iris region. Future studies should focus on improving extraction techniques to exploit the spatially corresponding iris features more efficiently.Based on the deep learning approach for the IRS, He et al. [143] introduced an adaptive Gabor filter technique and a deep belief network (DBN) for iris extraction. First of all, they used Gabor filters technique to identify information on iris texture. To discover the appropriate Gabor kernels for providing the most informative bands of identified iris features, they introduced a data-driven Gabor filter optimization technique based on binary particle swarm optimization (BPSO). Next, they introduced the DBN stacked by several RBMs in a semi-unsupervised method to extract more deep features of the iris from Gabor codes to find out the different types of noise. There are many challenges encountered including eyelashes, eyelids, and glasses which may occlude the ROI of the iris for feature extraction [143]. To improve the performance accuracy of the IRS based on a deep learning approach, Wang and Kumar [242] proposed an architecture for feature extraction based on a deep learning approach. The suggested architecture involved the dilated convolutional neural networks which were introduced by Yu and Koltun [184] and residual learning kernels that were suggested in He et al. [239]. Also, Wang and Kumar’s work used a MaskNet which was separately trained. This approach focuses on the development of the architecture of a dilated residual feature net (DRFNet), which was applied by Zhao and Kumar [203] to produce binary feature maps with higher accuracy. For future research, improvement can be made to end-to-end architecture which can together disregard masked bits or combine training of end-to-end MaskNet [242].Lu and Pan [243] introduced a technique based on the enhanced residual network for the classification of the iris image. They immediately utilized the original iris images as the input images to avoid insufficiency in extracting features. In the introduced network, they proposed the convolution channel dividing module, which is applying convolution kernels in order to extract features from the iris images, and the features were extracted at various scales. They applied the residual module to enhance the learning ability of the suggested network [243]. In many studies, deep learning techniques require a huge number of samples for training. Accordingly, researchers will consider expanding the number of samples of non-ideal iris images within the database from more individuals, and iris image segmentation techniques can also be applied, so as to achieve faster and accurate classification.Lee et al. [244] introduced an ocular recognition system that defines the rough edge of the iris without implementing the segmentation of the iris region. To address the reduced efficiency problem for iris recognition that relied on the rough determination, an ocular region slightly bigger than the iris region was utilized to implement iris recognition, and a deep residual network (ResNet) was applied to solve the issue of a reduced recognition rate due to a misalignment between the registered and verification images [244].There were still limitations to the proposed methods, such as severe rotation, thick eyelashes, off-angle, motion blurring, and eyeglass frame in the iris image. Other limitations include the iris images with bad resolution such as when the image was taken of uncooperative subjects and when there was a long distance between the camera and the subject. To resolve these limitations, the researchers should investigate ways to enhance the performance of the recognition system by using the recognition information of both eyes and by collecting the dataset of iris images such as of twins due to their similarity. Furthermore, the focus should also be on studying techniques that can increase the receptive domain of the CNN architecture and extract features to resolve the challenge of reduced performance of iris recognition due to the misalignment between the registered and the recognition iris images caused by the rough detection from the ocular region.


## 7.1. Feature Extraction Using Traditional Techniques

This section explains the application of traditional techniques for feature extraction. One such technique to extract the features of normalized iris images would be the 1D log-Gabor filters. Gong et al. [213], Raffei et al. [39, 71, 105], Bakshi et al. [72], and Salve and Narote [132] applied 1D log-Gabor filters for feature extraction of normalized iris images. The1D log-Gabor filters are an extension of the Gabor filters which were not suitable to deal with spectral information with maximal spatial localization. The disadvantage of the Gabor filter is that whenever the bandwidth exceeds one octave even the symmetric filter will have a direct current (DC) component, while the log-Gabor filters have no DC components for any bandwidth [132]. By having extended ends, the log-Gabor filters are more efficient to encode original images than the traditional Gabor filters, which would overrepresent the low-frequency segments and underrepresent the high-frequency segments in any encoding. The challenges of Gabor filters are addressed with the application of the log-Gabor filters.Other researchers such as Yahiaoui et al. [89], Othman et al. [138], Jan et al. [214, 215], Jan and Usman [216], Raja et al. [217], and, more recently, Nazmdeh et al. [6] proposed the use of 2D-Gabor filters, which are very popular and have been successfully implemented in practical applications. 2D-Gabor filters are typically used to obtain the global tissue information in iris regions for the feature extraction method.Ahmadi and Akbarizadeh [218] have proposed a hybrid robust iris recognition approach using iris image preprocessing, two-dimensional Gabor kernel (2-DGK) features, and multi-layer perceptron neural network-particle swarm optimization (MLPNN-PSO). The challenge of the IRS with 2-DGK feature extractor method and MLPNN-PSO classifier was that its efficiency rate was not at an acceptable level. To improve the performance of feature extraction of the iris texture, Ahmadi et al. [219] introduced a technique relying on an intelligent hybrid system for iris recognition. The proposed technique is based on a combination of three techniques which are 2-DGK [218], polynomial filtering (PF) [220, 221], and step filtering (SF) to extract features of the iris image. For future studies, focus should be on improving IRS by fusing advanced techniques such as deep learning techniques in order to train the algorithms with other evolutionary-based algorithms to reduce the required storage space and computational complexity.Several key point-based techniques for iris recognition have been introduced in the past by Santos and Hoyle [68] and Sun et al. [222], and to improve the performance of previous research, Alvarez-Betancourt and Garcia-Silvente [30] suggested a method to extract features based on the key points from the iris region. The suggested method utilized three filters to identify key points such as Fast Hessian which was applied by Bay et al. [223], the Harris–Laplace method which merged the Harris detector method and the scale selection method that were used by Mikolajczyk and Schmid [224], and lastly the Hessian–Laplace detector that was presented by Tuytelaars and Mikolajczyk [225]. The suggested method is based on the merging of three information exporters of scale-invariant feature transform (SIFT) which was applied by Lowe [226]. This method worked well enough in some cases of iris image noise, but it is still needed to be developed in terms of time and accuracy even in research that is considered recent [30].Kumar et al. [73] applied the combination of discrete wavelet transform (DWT) that was introduced in Jang et al. [227] and triangle-based discrete cosine transform (DCT) from Kekre et al. [228] and Nagi et al. [229] to extract features of the iris region. The proposed method still has limitations such as implementation in unrestricted environments and real-time applications [73]. Thus, Barpanda et al. [230] proposed using the wavelet derived from the Cohen–Daubechies–Feauveau 9/7 filter bank to extract iris features in three steps, namely, (1) computing the two-level wavelet from the iris template, (2) computing the power spectrum of the wavelet transformed image, and (3) computing the discrete cosine transform to get the spectrum coefficients [230]. Recently, Barpanda et al. [231] proposed applying wavelet mel-cepstrum in order to extract the iris feature. The suggested technique is based on the wavelet derived from the public orthogonal Cohen–Daubechies–Feauveau (CDF) 9/7 filter bank. In the proposed method, the mel-frequency cepstrum coefficients (MFCCs) were applied to distinguish the iris tissues. The proposed technique is used to extract iris features in three steps. The first step is to calculate the two-level detail coefficients of the iris tissue. After that, the logarithmic method is applied to divide these detailed coefficients into irregular squares. This helps in reducing the dimensions of wavelet coefficients followed by designated irregular weights to various frequency segments. Then, the separate cosine transform method is used to calculate the same features, through which the features are extracted [231].The weakness of the suggested method is that with decreased feature size, there is a subtle rise in inaccuracy. Through the analysis of the results, it is obvious that the cepstrum features relied on the wavelet to perform on par with the CDF 9/7 filter bank. Thus, it is believed that the focus of future research should be on developing an iris recognition system with higher accuracy in a time-efficient manner for collected images under NIR and visible spectrum utilizing colors information.Other researchers such as Gad et al. [80] suggested a Cognitive Internet of Things (CIoT) method by using a combined vision between the single-modal and multi-algorithm of the recognition system of the iris image for the purpose of investigation. They applied delta mean (DM) and multi-algorithm mean (MAM) to extract the features [80].The multi-algorithm methods were carried out in a parallel processing method. Other researchers like Liu et al. introduced a collaborative cooperative method to extract features of iris texture [101].


## 7.2. Feature Extraction Using Deep Learning Techniques

Minaee et al. [232] applied architecture of VGG16-Net as suggested by Krizhevsky et al. [182] and Simonyan and Zisserman [196] to obtain deep feature extraction from iris images and draw deep feature map by executing a sequence of convolution. Then, the principal component analysis (PCA) method was applied by Abdi and Williams [233] to reduce features’ dimensionality. The result of this method could be more enhanced by training a deep network specifically for recognition of iris images [232]. To discover eye images generated by the computer by exploring the differences in the eye region, Carvalho et al. [97] introduced VGG architecture with 19 layers (VGG19) instead of the VGG architecture with 16 layers (VGG16) that was initially suggested by Simonyan and Zisserman [196] based on deep neural networks (DNNs) as the basis of the ImageNet. In the proposed method, the convolutional blocks functioned as feature extraction layers. Future research directions should delve deeper into improving the features extraction techniques by introducing the fusion of bottleneck features extracted from different deep CNN models as the fusion system is said to achieve higher performance [97, 158].To improve the performance with less computational complexity and required memory space as well as full automation in the deep network design of the IRS for both automatic feature design and network structure design at the same time, Nguyen et al. [57] introduced a method to restructure the design of the iris recognition network that accounted for constraints of computation and memory. The proposed method was used to investigate the traditional handcrafted IrisCode performance to compare with deep network techniques. It also tried to further enhance the current deep networks of iris recognition to realize similar or higher efficiency with the same level of memory and computational complexity. More importantly, the proposed technique fully automates the process of network design to search for the best network architecture of iris recognition confined to the compactness of model and computation constraints. According to this proposed algorithm, it can be concluded that the performance of deep networks is affected when the computation is robustly limited while this enhances the performance of traditional IrisCode techniques in both computation and accuracy of iris recognition [57].Ahuja et al. [134] proposed two methods based on CNN to verify the images of the periocular of the eye, which comprises the iris region. In the first method, they proposed a CNN-based unsupervised technique, fusing the result of external feature vectors and the RootSIFT baseline technique as suggested by Arandjelović and Zisserman [234]. In the second method, they proposed the CNN-based supervised technique and used a cosine similarity as suggested by Nguyen and Bai [235] to get the similarity between two images. Both the methods are based on deep CNN, and both the methods used a stacked-layer architecture [134]. In order to improve the performance accuracy of ocular biometric recognition, Reddy et al. [236] introduced OcularNet based on a CNN architecture for mobile ocular biometrics. In the proposed OcularNet model, they extracted six enrolled patches from the regions of periocular and ocular as well as trained a small CNN for every patch called PatchCNN to extract and generate feature descriptors that are differentiable between the individuals. The recommendation was for future studies to increase the analysis of different mobile-optimized architectures like MobileNet as well as MobileNet-v2 to further decrease the number of parameters [236].To obtain iris features with a clearer distinction as well as to solve the determination problems of classification thresholds for iris recognition tasks by increasing the iris samples, Chen et al. [237] proposed tight center (T-center) loss function to extract features of the iris region with more prominent distinction. The proposed T-center loss function was utilized to resolve the insufficient distinction problem with the conventional softmax loss function. The discriminative ability of deep features based on the CNN model can be enhanced by the linear fusion of softmax and T-center loss functions [237].Other researchers like Zhang et al. [95] and Al-Waisy et al. [111] introduced a technique on the basis of a CNN approach to extract the features of the iris region. In the suggested technique, the features of the iris region were extracted automatically by applying convolution with pooling layers. In practical uses, taking iris images is a challenge when there is reflection due to glasses, different lighting environments, etc [95, 111]. Also, further research is still needed to improve the accuracy of randomized performance by randomly reading the iris image. Liu et al. [35] introduced the F-CNN and F-Capsule to increase the robustness of deep learning training as opposed to CNN and capsule. The suggested F-CNN and F-Capsule contribute to the training process of the R-CNN by applying a selective search for image feature extraction to enhance the speed and performance of the training process [35]. Zhao et al. [112] introduced a deep learning technique to extract features based on the architecture of a capsule network for the IRS. In this method, the roles of the convolutional part are to extract the features of low-intensity pixels of the iris images and to form the main features used for the main capsule layer. Based on the dynamic routing approach between two layers of the capsule network, Zhao et al. [112] implemented a modified routing method to make the suggested algorithm adapt to a recognition system of iris images. This migration learning method allows the use of deep learning technology when the number of iris images is limited. Accordingly, three state-of-the-art pretrained designs, VGG16, InceptionV3, and ResNet50, are introduced and split into a sequence of subnetwork structures in accordance with the number of their primary constituent blocks [112]. Future research should further develop this method by applying capsule (vector) feature learning network to solve iris recognition challenges in unconstrained environments and improve research network processing capabilities and processing techniques when the number of classes becomes very huge.Nguyen et al. [158] presented five off-the-shelf CNN to extract features of the iris region in eye images. They are AlexNet that was used by Krizhevsky et al. [182], VGG network from Simonyan and Zisserman [196], Google Inception network that was introduced by Szegedy [238], ResNet that was applied by He et al. [239], and DenseNet that was suggested by Huang et al. [171]. To increase the performance off-the-shelf CNN for extracting features of iris images, classical CNNs can be combined with other methods such as recurrent neural network (RNN), and stacked autoencoder (SAE), and deep belief network (DBN) [158]. To increase the performance accuracy of the feature extraction, Marra et al. [240] suggested a technique based on CNN techniques to adapt and to decrease the number of parameters as well as computational complexity (both in training and testing stages) of the AlexNet that was suggested by Krizhevsky et al. [182]. The proposed AlexNet technique consists of three convolutional layers to extract the feature. In these three layers, a rectified linear unit (ReLu) is implemented as a non-linear task following convolution, which is followed by max-pooling layers [240]. Zhao and Kumar [200] proposed a framework based on deep learning, named UniNet.v2, for iris detection, segmentation, and recognition. In the proposed framework, they used FeatNet, based on FCN, which was originally advanced for semantic segmentation [192] to extract iris features of the iris region. A particularly designed extended triplet loss (ETL) [241] function was performed to combine the non-iris masking and bit-shifting, which were found important for learning significant and distinctive spatial features of the iris region. Future studies should focus on improving extraction techniques to exploit the spatially corresponding iris features more efficiently.Based on the deep learning approach for the IRS, He et al. [143] introduced an adaptive Gabor filter technique and a deep belief network (DBN) for iris extraction. First of all, they used Gabor filters technique to identify information on iris texture. To discover the appropriate Gabor kernels for providing the most informative bands of identified iris features, they introduced a data-driven Gabor filter optimization technique based on binary particle swarm optimization (BPSO). Next, they introduced the DBN stacked by several RBMs in a semi-unsupervised method to extract more deep features of the iris from Gabor codes to find out the different types of noise. There are many challenges encountered including eyelashes, eyelids, and glasses which may occlude the ROI of the iris for feature extraction [143]. To improve the performance accuracy of the IRS based on a deep learning approach, Wang and Kumar [242] proposed an architecture for feature extraction based on a deep learning approach. The suggested architecture involved the dilated convolutional neural networks which were introduced by Yu and Koltun [184] and residual learning kernels that were suggested in He et al. [239]. Also, Wang and Kumar’s work used a MaskNet which was separately trained. This approach focuses on the development of the architecture of a dilated residual feature net (DRFNet), which was applied by Zhao and Kumar [203] to produce binary feature maps with higher accuracy. For future research, improvement can be made to end-to-end architecture which can together disregard masked bits or combine training of end-to-end MaskNet [242].Lu and Pan [243] introduced a technique based on the enhanced residual network for the classification of the iris image. They immediately utilized the original iris images as the input images to avoid insufficiency in extracting features. In the introduced network, they proposed the convolution channel dividing module, which is applying convolution kernels in order to extract features from the iris images, and the features were extracted at various scales. They applied the residual module to enhance the learning ability of the suggested network [243]. In many studies, deep learning techniques require a huge number of samples for training. Accordingly, researchers will consider expanding the number of samples of non-ideal iris images within the database from more individuals, and iris image segmentation techniques can also be applied, so as to achieve faster and accurate classification.Lee et al. [244] introduced an ocular recognition system that defines the rough edge of the iris without implementing the segmentation of the iris region. To address the reduced efficiency problem for iris recognition that relied on the rough determination, an ocular region slightly bigger than the iris region was utilized to implement iris recognition, and a deep residual network (ResNet) was applied to solve the issue of a reduced recognition rate due to a misalignment between the registered and verification images [244].There were still limitations to the proposed methods, such as severe rotation, thick eyelashes, off-angle, motion blurring, and eyeglass frame in the iris image. Other limitations include the iris images with bad resolution such as when the image was taken of uncooperative subjects and when there was a long distance between the camera and the subject. To resolve these limitations, the researchers should investigate ways to enhance the performance of the recognition system by using the recognition information of both eyes and by collecting the dataset of iris images such as of twins due to their similarity. Furthermore, the focus should also be on studying techniques that can increase the receptive domain of the CNN architecture and extract features to resolve the challenge of reduced performance of iris recognition due to the misalignment between the registered and the recognition iris images caused by the rough detection from the ocular region.


## 8. Feature Selection Techniques

Feature selection is an operation to discover ideal features that can decrease computational complexity but can still achieve an accurate performance, consequently reducing the number of features to be saved in the feature subset. A decrease in the complexity of the computation is expected to increase the speed of the IRS [245].The primary objectives of feature selection are to reduce computation time as well as the space needed to operate algorithms, to enhance the classifier by eliminating noisy or unnecessary features, and to select the features that might be associated with a particular problem as mentioned in Guyon and Elisseeff [246]. This section describes some techniques of feature selection from previous works on iris recognition.Roy and Bhattacharya [247] introduced a feature selection technique based on the multi-objective genetic algorithm (MOGA) to enhance the performance accuracy of iris recognition. The MOGA subfeature selection technique enhances the performance accuracy of IRS and reduces the extracted features. However, MOGA incurs an additional cost for finding the optimal subset of features through several iterations [247]. In order to enhance the performance efficiency of the IRS, Roy et al. used a genetic algorithm (GA) to select the important subsets of features extracted from the texture of the iris region. The introduced method successfully decreased the feature set size from 2048 to 105. Although it assists in identifying the related features, this method has not been compensated with high accuracy [248].A genetic algorithm optimization (GAO) of neural networks for the IRS was introduced in Raja and Rajagopalan [249]. The introduced method applied the genetic algorithms for optimization of the neural network technique to improve performance accuracy as well as reduce learning time. GA was performed 10 times to get the standard deviation values, the average for a number of neurons, and the number of layers and methods. Although the results were promising, the network in the initial stages started stealing results rather than learning and forecasting [249].Wang et al. [250] used the AdaBoost technique to choose the most discriminative features and calculate the similarity. Based on the errors of the iris segmentation, this technique selects the best features calculated from Gabor filters. This technique gives good results in noisy iris images [250]. Wang et al. [251] introduced a feature selection method which can learn an integrated and effective ordinal set of iris recognition features based on linear programming. In this method, the principle of large margin is used to learn a robust technique, and discriminatory information is obtained for each feature to make the technique more robust to noise. The method can be efficiently solved by the simplex algorithm [251]. To improve the performance of biometric systems in real-time, Sahu et al. introduced a feature reduction method using a clustering algorithm based on density named phase intensive local patterns (PILP). It was used to discover key points and then cluster them using density-based spatial clustering of applications with noise (DBSCAN) [245].In order to select optimal features, Kumar et al. [73] used binary particle swarm optimization (BPSO) as suggested in Kennedy and Eberhart [252], which is based on stochastic optimization. BPSO is implemented after extracting features to select the required features. BPSO is a separate version of the particle swarm optimization (PSO) as suggested by Brownlee and Kennedy [253] which modifies particle speeds based on the possibility that a particle coordinate will change to either 0 or 1 [73]. To decrease the number of extracted features and select important subset features, Rao et al. introduced a feature selection stochastic technique, namely, dynamic binary particle swarm optimization (DBPSO), for enhancing the performance of the IRS [254]. Subban et al. implemented the PSO method to select useful features of Haralick measures of the iris region texture. The PSO is much improved due to fuzzy selection techniques. This technique achieves a faster and reliable recognition system [255].


## 9. Classification Techniques

The classification phase is the last phase of the recognition system (RS). The purpose of classification is to measure the level of resemblance between the test samples and samples in the databases of the iris images. Often, the full matching of these samples is not possible. Therefore, the approximate rate of each sample is used to help the recognition system identify persons. In this paper, classification techniques can be classified into two sections: classification using traditional techniques and classification using deep learning techniques.


### 9.1. Classification Using Traditional Techniques

The efficiency of training SVM is still a bottleneck, especially for large-scale learning problems. To improve a fast training algorithm for SVM to solve various problems, Gu et al. proposed a technique to improve SVM to classify features which have been extracted from the iris region. To make the SVM more efficient in embedded applications, they presented some optimizations to increase the performance accuracy of SVM for IRS and keep its high performance of classification. The suggested method transforms a given problem from multi-class to two-class classification [256].Rai and Yadav [257] in contrast proposed a cascaded classifier method. In this method, Haar wavelets (HWs) were applied to train SVM methods for each class. In general, while this cascaded classifier method has great performance properties, it takes a longer time because it is a sequential classifier [257]. Salve and Narote introduced a qualitative analysis on two different methods, namely, SVM and artificial neural network (ANN), to classify features that have been extracted from the iris region texture with the highest accuracy. It is found that the SVM works better in classification [132]. As such, Carvalho et al. [97] and Lee et al. [258] used the SVM-based classifier that was introduced by Jang et al. [227]. The SVM technique is based on the theory of structural risk reduction, which delimits the maximum distance between two categories [97]. Also, based on the same approach, Minaee et al. [232] and Nguyen et al. [158] used multi-class SVM which was applied in Weston and Watkins [259] as well as in Schölkopf et al. [260] to execute the classification task [158, 232].To match or classify two iris templates, Raffei et al. [39, 105], Kim et al. [27], Yahiaoui et al. [89], Othman et al. [138], Gangwar and Joshi [125], Zhao and Kumar [200, 203], Zhou et al. [261], and Nazmdeh et al. [6] applied the Hamming distance (HD) technique which was used by Masek [157], Álvarez Mariño et al. [262], and Daugman [163]. To classify the features of the iris image, the recorded sample was matched with the sample saved in the iris images database.Kumar et al. [73], Gad et al. [80], Ahuja et al. [134], Gangwar and Joshi [125], Lee et al. [244], Reddy et al. [236], and Dhage et al. [263] applied the Euclidean distance (ED) to classify features that have been extracted from the iris region texture in the feature extraction phase.Another method that was used to classify two-class models for each subimage is the k-nearest neighbor (k-NN). The k-NN method introduced by Elgamal and Al-Biqami [264] was applied by Badejo et al. [265] and Xia et al. [266] as a classifier due to ease of implementation, low computational complexity, and the absence of tuning parameters. Also, the k-NN classifier operates well with huge databases. Liu et al. introduced a collaborative cooperative method to classify the iris texture [101].However, many studies have criticized the accuracy of cross-sensor iris matching. It is said that the classifying performance of eye images captured by two various sensors or various resolutions [267] has deteriorated seriously. The challenge of the cross-spectral iris recognition system (CS-IRS) is that the iris matching process requires matching the iris images collected in the VIS spectrum with the iris images collected traditionally in the NIR spectrum. This challenge is generally considered more difficult as well and has attracted the attention of many researchers to address the low performance in cross-sensor iris matching. To improve the accuracy matching of iris images collected under different fields (VIS and NIR) with pixel-to-pixel correspondences, Ramaiah and Kumar [268] proposed a bi-spectral system of iris recognition. The classification method is applied based on domain adaptation of naive Bayes nearest neighbor (NBNN) to enhance the performance of the iris matching for the AS-IRS. The NBNN classifier as applied in [269–271] used distance learning from image-to-class (I2C). Virtually all the domain adaptation techniques available at present utilize image-to-image (I2I) learning while I2C has only been tried under the NBNN technique to domain adaptation (DA-NBNN) as suggested by Tommasi and Caputo [272]. More studies are needed to develop the AS-IRS with emerging applications in different fields such as human surveillance. Emphasis should also be given to restore discriminant features from the periocular area that can assist to increase matching performance for the CS-IRS.To further enhance the accuracy of matching for CS-IRS, Wang and Kumar [51] introduced two approaches for the classification of the cross-spectral iris features generated from three CNN architectures which used an architecture similar to AlexNet [182], namely, CNN with softmax loss and Siamese and triplet networks. The first approach for the classification relies on the joint-Bayesian inference [273] while the second approach includes supervised discrete hashing (SDH) [274] which produces match scores applying the Hamming distance (HD). To improve the matching techniques, Ahmadi et al. [219] proposed an intelligent hybrid radial basis function neural network (RBFNN) with a genetic algorithm (GA), called RBFNN-GA classifier, which is applied for classification problems. Thus, to improve performance, future research should focus on studying various classification methods and working on different hybrid models to classify the iris images with more challenges such as uncooperative subjects, blurring due to motion, the different sizes of the iris, reflection due to eyeglasses, and long distance between the camera and the subjects.


### 9.2. Classification Using Deep Learning Techniques

The deep learning approach has gained much attention among researchers to provide excellent solutions for iris recognition. In order to optimize the parameter of classification, Du et al. applied an approach based on CNN with unified architecture to enhance the performance accuracy of classification with a different set of iris image databases [275]. Zhao and Kumar [203] introduced a unified network known as UniNet, which consists of two networks: FeatNet and MaskNet. The proposed network is based on FCN [192] which is basically improved for semantic segmentation. The popular CNN is different from the FCN in that it does not contain a fully connected layer. Also, the proposed method contains the function of the extended triplet loss (ETL) which is implemented to combine the bit-shifting operation and non-iris pixels masking. For matching, they use the Hamming distance from the extended masks and binarized feature maps [203].A deep neural method was proposed for iris recognition by Gaxiola et al. [276]. In the proposed method, three network modules were used, each module working with a deep neural network. Finally, the gating network integrator does the recognition [276].For the classification of iris pattern, Baqar et al. [74] applied a technique which is dependent on a deep belief network (DBN) through restricted Boltzmann machine (RBM) with modified backpropagation algorithm-based feed-forward neural network (RVLR-NN) [74]. He et al. introduced a method which used a deep belief network (DBN) to implement the iris classification [143]. In addition, Marra et al. [240] introduced a network based on CNN. The suggested network consists of three fully connected layers of AlexNet as applied by Krizhevsky et al. [182] for the classification of the feature which was extracted through the feature extraction phase [240].In order to perform the classification task, Zhang et al. [95] and Al-Waisy et al. [111] used the top layers of a proposed CNN technique that are two FC layers. Next, the output of the last FC layer is fed into the softmax classifier [95, 111]. In order to classify gender using an iris image, Tapia and Aravena [277] proposed two different CNN structures. This method performed well even with a smaller dataset and a simple model. However, enhancement is needed to increase the efficiency of the classifier [277]. To increase the performance accuracy of the IRS for some challenges of the iris images, Wang et al. [34] introduced the MiCoRe-Net method which is a mixed convolutional neural and residual network for the iris recognition task. The proposed MiCoRe-Net method used the SoftMax layer as a classifier. The proposed method showed various data augmentation strategies that affected the performance efficiency of the recognition system and showed that iris recognition technology depended on an appropriate augmentation strategy to work appropriately [34]. Further more, Liu et al. [35], Zhao et al. [112], and Lozej et al. [207] used the softmax layer of a CNN to classify the iris pattern [35, 112, 207]. For classifications of Asian and non-Asian ethnic iris images, Lu and Pan [243] proposed a classification algorithm of iris images using the enhanced residual network that relies on a deep learning approach. In order to extract further features of the iris image, they divided the convolution channel as well as extracted features under various convolution kernel dimensions. In the proposed algorithm, a hollow convolution was applied to increase the receptive field, which, when compared with the conventional network, improved the ability of classification and decreased parameters [243].To improve classification accuracy, Cheng et al. [52] introduced a multi-classification framework based on the CNN and the Hadamard error correction output code (Hadamard-ECOC). Novel classes of the iris images were added to the iris recognition framework without the need to retrain all the parameters. The proposed framework combines the ability to correct errors with the ability to learn the features of the iris images. Compared with other methods of multi-classification, it not only used the excellent image processing performance of the CNN but also combined the Hadamard-ECOC features of simple architecture as well as the ability to adapt to arbitrary categories [52]. The disadvantage of the proposed framework is that the number of classes that can be added to the proposed framework is limited. Thus, recommendation for researchers would be to concentrate on using other encoding techniques to deal with the limitations of the number of classes that can be added in future studies.


## 9.1. Classification Using Traditional Techniques

The efficiency of training SVM is still a bottleneck, especially for large-scale learning problems. To improve a fast training algorithm for SVM to solve various problems, Gu et al. proposed a technique to improve SVM to classify features which have been extracted from the iris region. To make the SVM more efficient in embedded applications, they presented some optimizations to increase the performance accuracy of SVM for IRS and keep its high performance of classification. The suggested method transforms a given problem from multi-class to two-class classification [256].Rai and Yadav [257] in contrast proposed a cascaded classifier method. In this method, Haar wavelets (HWs) were applied to train SVM methods for each class. In general, while this cascaded classifier method has great performance properties, it takes a longer time because it is a sequential classifier [257]. Salve and Narote introduced a qualitative analysis on two different methods, namely, SVM and artificial neural network (ANN), to classify features that have been extracted from the iris region texture with the highest accuracy. It is found that the SVM works better in classification [132]. As such, Carvalho et al. [97] and Lee et al. [258] used the SVM-based classifier that was introduced by Jang et al. [227]. The SVM technique is based on the theory of structural risk reduction, which delimits the maximum distance between two categories [97]. Also, based on the same approach, Minaee et al. [232] and Nguyen et al. [158] used multi-class SVM which was applied in Weston and Watkins [259] as well as in Schölkopf et al. [260] to execute the classification task [158, 232].To match or classify two iris templates, Raffei et al. [39, 105], Kim et al. [27], Yahiaoui et al. [89], Othman et al. [138], Gangwar and Joshi [125], Zhao and Kumar [200, 203], Zhou et al. [261], and Nazmdeh et al. [6] applied the Hamming distance (HD) technique which was used by Masek [157], Álvarez Mariño et al. [262], and Daugman [163]. To classify the features of the iris image, the recorded sample was matched with the sample saved in the iris images database.Kumar et al. [73], Gad et al. [80], Ahuja et al. [134], Gangwar and Joshi [125], Lee et al. [244], Reddy et al. [236], and Dhage et al. [263] applied the Euclidean distance (ED) to classify features that have been extracted from the iris region texture in the feature extraction phase.Another method that was used to classify two-class models for each subimage is the k-nearest neighbor (k-NN). The k-NN method introduced by Elgamal and Al-Biqami [264] was applied by Badejo et al. [265] and Xia et al. [266] as a classifier due to ease of implementation, low computational complexity, and the absence of tuning parameters. Also, the k-NN classifier operates well with huge databases. Liu et al. introduced a collaborative cooperative method to classify the iris texture [101].However, many studies have criticized the accuracy of cross-sensor iris matching. It is said that the classifying performance of eye images captured by two various sensors or various resolutions [267] has deteriorated seriously. The challenge of the cross-spectral iris recognition system (CS-IRS) is that the iris matching process requires matching the iris images collected in the VIS spectrum with the iris images collected traditionally in the NIR spectrum. This challenge is generally considered more difficult as well and has attracted the attention of many researchers to address the low performance in cross-sensor iris matching. To improve the accuracy matching of iris images collected under different fields (VIS and NIR) with pixel-to-pixel correspondences, Ramaiah and Kumar [268] proposed a bi-spectral system of iris recognition. The classification method is applied based on domain adaptation of naive Bayes nearest neighbor (NBNN) to enhance the performance of the iris matching for the AS-IRS. The NBNN classifier as applied in [269–271] used distance learning from image-to-class (I2C). Virtually all the domain adaptation techniques available at present utilize image-to-image (I2I) learning while I2C has only been tried under the NBNN technique to domain adaptation (DA-NBNN) as suggested by Tommasi and Caputo [272]. More studies are needed to develop the AS-IRS with emerging applications in different fields such as human surveillance. Emphasis should also be given to restore discriminant features from the periocular area that can assist to increase matching performance for the CS-IRS.To further enhance the accuracy of matching for CS-IRS, Wang and Kumar [51] introduced two approaches for the classification of the cross-spectral iris features generated from three CNN architectures which used an architecture similar to AlexNet [182], namely, CNN with softmax loss and Siamese and triplet networks. The first approach for the classification relies on the joint-Bayesian inference [273] while the second approach includes supervised discrete hashing (SDH) [274] which produces match scores applying the Hamming distance (HD). To improve the matching techniques, Ahmadi et al. [219] proposed an intelligent hybrid radial basis function neural network (RBFNN) with a genetic algorithm (GA), called RBFNN-GA classifier, which is applied for classification problems. Thus, to improve performance, future research should focus on studying various classification methods and working on different hybrid models to classify the iris images with more challenges such as uncooperative subjects, blurring due to motion, the different sizes of the iris, reflection due to eyeglasses, and long distance between the camera and the subjects.


## 9.2. Classification Using Deep Learning Techniques

The deep learning approach has gained much attention among researchers to provide excellent solutions for iris recognition. In order to optimize the parameter of classification, Du et al. applied an approach based on CNN with unified architecture to enhance the performance accuracy of classification with a different set of iris image databases [275]. Zhao and Kumar [203] introduced a unified network known as UniNet, which consists of two networks: FeatNet and MaskNet. The proposed network is based on FCN [192] which is basically improved for semantic segmentation. The popular CNN is different from the FCN in that it does not contain a fully connected layer. Also, the proposed method contains the function of the extended triplet loss (ETL) which is implemented to combine the bit-shifting operation and non-iris pixels masking. For matching, they use the Hamming distance from the extended masks and binarized feature maps [203].A deep neural method was proposed for iris recognition by Gaxiola et al. [276]. In the proposed method, three network modules were used, each module working with a deep neural network. Finally, the gating network integrator does the recognition [276].For the classification of iris pattern, Baqar et al. [74] applied a technique which is dependent on a deep belief network (DBN) through restricted Boltzmann machine (RBM) with modified backpropagation algorithm-based feed-forward neural network (RVLR-NN) [74]. He et al. introduced a method which used a deep belief network (DBN) to implement the iris classification [143]. In addition, Marra et al. [240] introduced a network based on CNN. The suggested network consists of three fully connected layers of AlexNet as applied by Krizhevsky et al. [182] for the classification of the feature which was extracted through the feature extraction phase [240].In order to perform the classification task, Zhang et al. [95] and Al-Waisy et al. [111] used the top layers of a proposed CNN technique that are two FC layers. Next, the output of the last FC layer is fed into the softmax classifier [95, 111]. In order to classify gender using an iris image, Tapia and Aravena [277] proposed two different CNN structures. This method performed well even with a smaller dataset and a simple model. However, enhancement is needed to increase the efficiency of the classifier [277]. To increase the performance accuracy of the IRS for some challenges of the iris images, Wang et al. [34] introduced the MiCoRe-Net method which is a mixed convolutional neural and residual network for the iris recognition task. The proposed MiCoRe-Net method used the SoftMax layer as a classifier. The proposed method showed various data augmentation strategies that affected the performance efficiency of the recognition system and showed that iris recognition technology depended on an appropriate augmentation strategy to work appropriately [34]. Further more, Liu et al. [35], Zhao et al. [112], and Lozej et al. [207] used the softmax layer of a CNN to classify the iris pattern [35, 112, 207]. For classifications of Asian and non-Asian ethnic iris images, Lu and Pan [243] proposed a classification algorithm of iris images using the enhanced residual network that relies on a deep learning approach. In order to extract further features of the iris image, they divided the convolution channel as well as extracted features under various convolution kernel dimensions. In the proposed algorithm, a hollow convolution was applied to increase the receptive field, which, when compared with the conventional network, improved the ability of classification and decreased parameters [243].To improve classification accuracy, Cheng et al. [52] introduced a multi-classification framework based on the CNN and the Hadamard error correction output code (Hadamard-ECOC). Novel classes of the iris images were added to the iris recognition framework without the need to retrain all the parameters. The proposed framework combines the ability to correct errors with the ability to learn the features of the iris images. Compared with other methods of multi-classification, it not only used the excellent image processing performance of the CNN but also combined the Hadamard-ECOC features of simple architecture as well as the ability to adapt to arbitrary categories [52]. The disadvantage of the proposed framework is that the number of classes that can be added to the proposed framework is limited. Thus, recommendation for researchers would be to concentrate on using other encoding techniques to deal with the limitations of the number of classes that can be added in future studies.


## 10. Discussion

Vast improvements have been achieved in the IRS in the last 10 years. These achievements resulted from various studies in various fields regarding iris recognition. Without these outstanding contributions of these researchers and inventors, we would have never realized the advancement of technologies and their benefits to us. Table7 shows the performance efficiency rate of both traditional and deep learning techniques in some previous works of the IRS.Table 7
Accuracy rate (ACC) or error rate (EER) of some previous works on iris recognition system.
SourceDatabaseFeature extraction methodsClassification methodsACC/EER (%)Liu et al. [267]CASIA, Q-FIREPairwise filter bankThey used CNNEER = 0.31Kumar et al. [73]IITD, MMUExtracting featureb using DWT and DCTThey used ED94.59Salve and Narote [132]CASIA-IrisV41D log-Gabor waveletANN and SVM92.5–95.9Du et al. [275]NIST-ICEFlipped imagesThey used the CNN techniqueAutomatic classification based on the CNN technique98.9-98.597.5–98.6Alvarez-Betancour and Garcia-Silvente [30]CASIAUBIRISMMUFusion of three information sources of SIFT featuresThe nearest neighbor classifier99.9099.8999.05Al-Waisy et al. [111]SDUMLA-HMTCASIA-V3IITDThey used CNNSoftmax classifier + fusion10099.9299.88Sahu et al. [245]CASIA-V3BATHPILP and DBSCANClustering approach97.6896.23Gad et al [80]SDUMLA-HMTCASIA v. 1UBIRIS v. 1CASIA v. 4Delta mean (DM) and multi-algorithm mean (MAM)Euclidean distance (ED)99.4999.4699.4699.49Barpanda et al. [230]CASIA, UBIRISIITDTunable filter bankCanberra distance91.65Liu et al. [101]MICHE-I: Galaxy S4iPhone 5They proposed a multilayer analogous convolutional architecture inspired by CNNCollaborative representation scheme9798Wang and Kumar [242]ND-IRIS-0405CASIA-distanceWVU non-idealResidual learning and dilated kernel utilizing deep CNNThe Hamming distances97.787.596.1Barpanda et al. [231]CASIA, IITDUBIRISThis approach is based on the wavelet derived from the popular biorthogonal Cohen–Daubechies–Feauveau 9/7 filter bankCanberra distance91.65Zhao and Kumar [203]ND-IRIS-0405-IrisCASIA-distanceWVU non-idealThey used an FCN-based framework, which produces spatially corresponding descriptors of iris featuresThey applied the Hamming distance (HD) from the extended masks and binarized feature maps97.184.194.3Wang and Kumar [51]PolyU bi-spectralCross-eyed-cross-spectral irisThe CNN applied is similar to AlexNet architectureThey used the SDH for compression as well as classification90.7187.18Reddy et al. [236]VISOBUBIRIS-IUBIRIS-IICross-eyedThey proposed OcularNet based on a CNN architecture and trained a small CNN for every patch called PatchCNN to extract the iris feature descriptorsThey applied the ED technique between every patch’s embedded iris featureEER = 1.89EER = 9.86EER = 9.77EER = 14.9Zhao and Kuamar [200]ND-IRIS-0405CASIA-distanceIITD iris WVU non-idealThey designed the FeatNet model for extracting features of the iris region based on FCN, which is primarily advanced for semantic segmentationThe fractional Hamming distance (HD) for extended masks and the binarized feature mapsEER = 1.12EER = 4.07EER = 0.76EER = 2.20Lee et al. [244]CASIA-DistanceCASIA-LampCASIA-1000They used deep ResNet-based recognitionThey used ED to classify the output of CNN’s fully connected layerEER = 2.16EER = 1.59EER = 1.33Ahmadi et al. [219]CASIA-V3UBIRIS-V1UCI datasets.They used hybrid 2-DGK/SF/PF techniques to extract the features of the iris textureThey used intelligent hybrid RBFNN-GA classifier99.991499.988999.98Lee et al. [258]NICE.IIMICHECASIA-distanceThey used three CNN models to implement feature extractionThey applied SVM as a classifierEER = 8.58EER = 17.39EER = 2.96Implementing the iris recognition system on a mobile phone device has raised some novel problems associated with performance declination due to the user’s gazing point, illumination, uncooperative subjects, and the low central processing unit (CPU) ability in a mobile phone. Furthermore, the IRS in a mobile phone is principally applied for data protection or unlocking the mobile phone for a single user. Generally, iris recognition is performed several times a day when the user operates the mobile phone. Therefore, both ease of use and computation time are just as paramount as the recognition rate. For these reasons, researchers should strive to improve the speed of the techniques by designing fused models that combine the strength of CNN techniques with the speed of traditional techniques of computer vision. Another way is to try to utilize the semantic segmentation approach and combine it with the state-of-the-art iris segmentation techniques relying on neural networks like IrisDenseNet [40] or Faster R-CNN technique as proposed by Girshick [278] or the techniques suggested by He et al. [169] and Li et al. [177]. The semantic segmentation techniques have a high sensibility of foretelling the reflection pixels in the iris region, which enhances the overall efficiency of the technique. Researchers should extend the application of these techniques to be used on mobile phone devices, by utilizing further concise deep learning approaches, such as XNOR-NET [279]. The main objective is to implement a fast and accurate IRS on mobile phone devices.Future studies should aim at improving the ability of the IRS to maintain accuracy even when implemented on databases with a huge number of samples, classes, and subjects. Some databases of iris images captured in constrained conditions at the NIR wavelength spectrum (IITD, CASIA-IrisV4-1000, ND-Cross Sensor-Iris-2013, Bath800) and other iris images databases that have been collected in unconstrained conditions with the VIS wavelength spectrum (VISOB, UBIRIS.V2., MICHE-I, VSSIRIS, and CSIP) should be used to investigate the use of iris images in different conditions and sensors in ocular biometrics systems. The images in unconstrained environments databases are taken by the volunteer himself or herself, and these images contain issues related to lighting, shadows, defocus, pose, resolution, distance, and image quality (normally affected by the conditions illumination), among other challenges. Due to these challenges, the IRS using such eye images may not be reliable; therefore, some methods using features of the periocular region have been introduced [134, 280–284] which usually occurs in unconstrained conditions at VIS wavelength. It is important to research novel techniques as well as novel databases with a huge number of eye images, classes, and subjects in order to evaluate the scalability of methods in the literature. These novel techniques are the fusion of different biometric features existing in the images, e.g., the entire face, periocular, and iris. In this method, there is still scope for enhancement in the detection/segmentation of biometric features existing in the face areas and in techniques for fusing features extracted from these regions into different levels, like extract features and matching rate [53, 285] as well as through introducing fusing techniques that combine classic IrisCode techniques and deep learning networks [57].This review paper presents the various phases related to iris recognition technologies. It also clearly describes the performance efficiency of each method through every IRS phase and then highlights the contributions and research directions to improve each technique. A large number of studies are still needed to develop the performance efficiency of the IRS. For instance, there is a strong need to improve accuracy in current techniques in both deep learning and traditional approaches to increase the performance efficiency of the recognition system for the iris images. Notwithstanding, some other challenges were found in unconstrained (non-ideal) environments which contain many types of noise such as motion, blurriness, off-angle, low resolution, rotation, distortion, cropping, scaling, reflection and scratches from eyeglasses, closing due to the eyelashes, noise, and interference from the eyeglass frames. Furthermore, in many of the techniques that existed in previous works regarding the subject of iris recognition, the level of performance has not appeared to be the same over the databases. Accurate analysis should be performed to identify the cause(s) of these differences in terms of performance. Furthermore, the available databases require large datasets with different noise types to support researchers working on improving the performance by using a deep learning approach.All of the noise types from the obtained iris images are removed by using preprocessing methods. The challenges of the IRS have resulted in an increase in false-positive and false-negative errors due to dark area, pupil expansion, the pixels being similar to the iris pixels, sagging of eyelids due to age, the wearing of lenses, and reflection due to eyeglasses. Because they obscure texture features of the iris region, these challenges require more studies to be conducted. Thus, as the number of classes becomes extremely large, more research would be required to study network processing scope and processing techniques.The region of the iris texture is separated from the surrounding texture in the segmentation phase. Deep learning techniques assist to produce accurate and fast segmentation. The iris region texture descriptors are then represented by using the proper techniques based on deep learning of feature extraction. Multiple features provide enough information about the features of the iris image. Also, the CNN techniques mostly require large and complex computation. Therefore, the most suitable approaches in some methods are successful due to appropriate selection of the most beneficial features. Swarm intelligence approaches and genetic algorithms can be applied to select optimal features of the iris region. Classification techniques can assist the system to make the decision in order to obtain matching/recognition. In general, the performance of the SVM classifier is better than that of the artificial neural network classifier. However, the performance efficiency of artificial neural network techniques can be further enhanced through optimal adjustment of parameters. Also, other hybrid methods like classifiers based on fusion methods can enhance the performance of the classification techniques of recognition systems.On the whole, this review paper has described the progress that has been achieved to increase the performance accuracy of various phases of the iris recognition systems (IRSs). Although some iris recognition systems in the deep learning approach as mentioned by Winston and Hemanth [31] do not use the techniques of segmentation and normalization, some have used the techniques of segmentation and normalization to improve the performance efficiency of iris recognition. As such, this paper has also highlighted the use of hybrid techniques to enhance the performance efficiency of the iris recognition system by focusing on increasing accuracy and decreasing the computational complexity. Also, to improve the biometric performance of the IRS, deep learning approaches are suitable to use through other notable techniques like RNN and unsupervised deep belief network (DBN). These techniques have their special advantages and can be utilized both independently or in combination with standard CNNs to enhance the ability to extract features of iris images. Thus, deep learning approaches are regarded as the novel boundary of machine learning, with the potential to process the above challenges. It will be interesting to compare the accuracy of deep learning performance with existing traditional approaches in future studies.Generally, due to millions of parameters used in the network, deep learning methods are equipped with a high degree of computational complexity during the training process that requires powerful GPUs and big data. This is in contrast with traditional techniques, especially Daugman’s Gabor features, which can extract and compare thousands of IrisCodes in a second on a general-purpose CPU. To remove layers and redundant neurons and decrease the network size, compressing and pruning may be employed. Deep learning achieves higher accuracy in tasks such as iris segmentation, feature extraction, and iris recognition, compared to traditional techniques. Since techniques based on the deep learning approach are trained instead of programmed, applications associated with deep learning often require less expert analysis and fine-tuning; besides, they exploit a tremendous amount of data available in the current systems. Deep learning techniques provide greater versatility because CNN models and architectures can be retrained to utilize a custom database to any use case, in contrast to traditional techniques, which tend to be more specific in the domain.


## 11. Conclusion

IRS has gained much attention around the world as an authoritative biometric system due to the uniqueness of iris features. Its accuracy has led to its adoption for screening and detecting criminals, terrorists, fugitives, and refugees. For the security of the country and safety of its people, a state-of-the-art technology is no longer a mere requirement but a vital necessity. Therefore, researchers must diligently pursue new techniques to develop more sophisticated recognition systems. In terms of accuracy, IRS is better than other kinds of biometric systems such as face recognition, periocular recognition, and fingerprint recognition due to its high sensitivity to fake, the stability of the human iris, and its uniqueness for every person [286].However, iris recognition systems become imperceptibly weak when it comes to images which are captured from videos or images that are of artificial iris images, images in which contact lenses are worn, and images generated by the computer [287]. Studies also highlighted other challenges such as occlusion due to eyelashes and eyelids, rotations or off-angle, reflection due to wearing eyeglasses, deformity due to diseases, and high spectrum illumination [31].A variety of approaches of iris image recognition system according to different phases based on two approaches which are traditional and deep learning have been presented in this paper along with the contributions of researchers as well as the strengths and weaknesses of some techniques of the IRS. To reiterate, the IRS has seven main phases: the acquisition phase of the iris image, the preprocessing phase to enhance the image, the iris segmentation phase to separate the iris region from the eye image, the normalization phase to turn the segmented iris region into a rectangular pattern, the feature extraction phase to extract the iris features, the feature selection phase to select the unique features, and the classification/matching phase to find the similarity between the iris image and images in databases. This review paper shows that the future of the IRS is bright and encourages researchers to further conduct studies to resolve the challenges mentioned above and provide proper solutions for them.



---

*Source: 6641247-2021-08-23.xml*