Ear Biometrics — Machine Learning a little further... | by Hitesh Valecha | Towards Data Science

Like other biometric using face, iris, and finger, the ear as a biometric contains a large amount of specific and unique features that allow for human identification. The ear morphology changes slightly after the age of 10 years and medical studies have shown that significant changes in the shape of the ear happen only before the age of 8 years and after the age of 70 years. It does grow symmetrically in size and begins to bulge downwards as the person ages, but that is a measurable effect. Studies suggest that ear changes only 1.22 mm per year. Also, the color distribution of ear, unlike face, is almost uniform. The position of the ear is almost in the middle of the profile face. Ear data can be captured even without the awareness of the subject from a distance. Ear biometrics can stand as an excellent example for passive biometrics and does not need much cooperation from the subject, which meets the demand of the secrecy of the authentication system present in the environment.
A digital camera takes the profile face images of the subjects in the environment from different angles, from which the section of the ear is segmented, preprocessed. The feature vectors are then analyzed in different test cases which consist of the rotation of the face in the same plane, different plane, different light conditions, etc. will be given to the machine learning model as input which would be trained to recognize a person in the environment. The process, though complicated, would develop a system which would provide an authenticated ear-based biometric identification system.
Controlled Environment
The purpose of CAMERA A is to get a clear shot of the profile face without expecting much tilt or rotation of the ear. Meanwhile, CAMERA B and CAMERA C are used for capturing more profile photos for multiple images of the same person to form the feature matrices.
Segmentation of ear from profile face —
Image (a) is the profile face captured from camera. In image (b) topographical labeling is done. Topographical labeling is used to find curved surfaces and edges. Image (c)is the difference of erosion & dilation i.e. applying morphological operations on the profile face to sharpen the features. Image (d) is the simple dot product of (b) & (c). Image (e) is obtained by blurring (d) 10 times so that connected component analysis can be applied. The complexity of each connected component is found and shaded with differentcolors in image (f). As the highest complexity region is found out to be ear, it is segmented and converted into grayscale in image (g) for faster computation.
Dataset and Preprocessing
Dataset preparation is the first step towards any machine learning model. In this system, a deep neural network is used to obtain the features and characteristics of the subjects in the environment. A large dataset is required to feed as an input to the Neural Network. At least 500 images per subject to train the network properly because all the classes are the images of ears machine can be easily confused with the features and characteristics, the more samples per subject, the easier it is to find the distinct features. The dataset should cover a wide range of variety of images e.g images with different light conditions, different angles, Gaussian filter, blur, etc.
There are small occlusions to be covered also while training the CNN network like ear covered with hair or earphones or jewels. Image augmentation is useful to generate multiple copies of images with slight variations to increase the dataset. For example, the data generator of Keras can be used.
DATADIR = "./dataset"CATEGORIES = ["Subject1", "Subject2", "Subject3", "Subject4"]for category in CATEGORIES:    path = os.path.join(DATADIR, category)        for img in os.listdir(path):        img_array = cv2.imread(os.path.join(path, img), cv2.IMREAD_GRAYSCALE)        break    breakIMG_WIDTH = 60IMG_HEIGHT = 100new_array = cv2.resize(img_array, (IMG_WIDTH, IMG_HEIGHT))training_data = []def create_training_data():    for category in CATEGORIES:        path = os.path.join(DATADIR, category)        class_num = CATEGORIES.index(category)        for img in os.listdir(path):            try:                img_array = cv2.imread(os.path.join(path, img), cv2.IMREAD_GRAYSCALE)                new_array = cv2.resize(img_array, (IMG_WIDTH, IMG_HEIGHT))                training_data.append([new_array, class_num])            except Exception as e:                passcreate_training_data()print(len(training_data))random.shuffle(training_data)
Convolutional Neural Network
As we are working with an image dataset (instead of .csv files or numeric data), CNN is used after the data preprocessing is done. The preprocessed data is passed to the architecture of the neural network. As the ear has less surface area than the face the computation time required for the ear is less. The grayscale images are taken by the input layer of the network.
Stride convolution and max-pooling is used to increase the depth (features) in each hidden layer.
Convolution filter reduces 9 image pixels to generate a single pixel without affecting the features.
a = []      #feature setb = []      #labelsfor features, labels in training_data:    a.append(features)    b.append(labels)a = np.array(a).reshape(-1, IMG_WIDTH, IMG_HEIGHT, 1)
# Create the modelmodel = Sequential()model.add(Conv2D(32, kernel_size=4, strides=1,activation='relu', input_shape=(IMG_WIDTH, IMG_HEIGHT, 1)))model.add(Conv2D(32, kernel_size=4, strides=2,activation='relu'))model.add(MaxPooling2D(pool_size=2))model.add(Conv2D(64, kernel_size=4, strides=1,activation='relu'))model.add(MaxPooling2D(pool_size=2))# model.add(Conv2D(64, kernel_size=4, strides=2,activation='relu'))# model.add(Dropout(0.5))# model.add(Conv2D(64, kernel_size=4, strides=1,activation='relu'))# model.add(Conv2D(64, kernel_size=4, strides=2,activation='relu'))model.summary()model.add(Flatten())model.add(Dense(32, activation='relu'))model.add(Dropout(0.5))model.add(Dense(4, activation='softmax'))model.summary()# Compile the modelmodel.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])# Train the modelmodel.fit(X_train, to_categorical(Y_train), batch_size=32, epochs=5)# Evaluate the modelscores = model.evaluate(X_test, to_categorical(Y_test))print('Loss: %.3f' % scores[0])print('Accuracy: %.3f' % scores[1])model.save('model_opt.h5')
In the final dense layer, Feature classification (FC) is done based on which the subjects are classified to a corresponding class of the network.
Graph shows the accuracy by the overlapping curves of the same class and distinct curves of different classes of the deep neural network.
Limitations
There would be cases of occlusions of different types like hair, earphones, or ear is injured or covered with cloth. The chances of an individual getting authenticated would become less. As the percent of pure ear taken for feature extraction is less so the accuracy would be affected. The system aims to concern small environment rooms like security rooms, board meetings, medical labs, etc.
Future Scope
The ear houses the smallest bones in the human body. That would be quite difficult for cybercriminals to duplicate. In addition, it can be used to measure temperature, blood flow, heart rate, and glucose levels. Ear biometric system will have diverse uses in the fields — authentication, access and attendance control, travel control, financial and other transactions requiring authorizations, remote voting, use of automated working devices, action control. e.g., nowadays airports use a combination of face & iris biometrics to authenticate a person, instead, a single ear biometric system can be used to achieve the authenticity of a person with the same accuracy. Every single feature of the body has an impact on how a person’s general personality is going to be like — face, eyes, palms & even handwriting. Personality analysis through handwriting is used to detect if the person is lying, etc. Similarly, the ear can be used for the personality analysis of a Person. Ear lobe has many features that can be used for Personality analysis — attached lobe, broad lobe, narrow ear, pointed ear, round lobe, square ear, ear sticking out, etc.
Summary
The most significant advantage of the Ear Biometric System is to identify the person without their knowledge/awareness. This feature will change the dynamics of passive biometrics and can bring complete automatism in the biometric field. Points to focus —
Large dataset (500 samples per subject) with a variety of images to train the learning model properly.
Preprocessing of images to reduce the computation time.
Good network architecture (without much loss or dropout).
Run enough epochs to train the network efficiently (with GPU, suggested).
Use the trained model to predict the person precisely.
Well, congratulations you made it to the end. Awesome. Click here to get the source code at my GitHub account.
Feel free to ask doubts because we are learning the same as the machines Hitesh Valecha..
References —
Ping Yan and Kevin W. Bowyer, Fellow, IEEE, “Biometric Recognition Using 3D Ear Shape,” IEEE Transactions on Pattern Analysis and Machine Intelligence, VOL. 29, NO. 8, AUGUST 2007John D. Bustard, Student Member, IEEE, and Mark S. Nixon, Associate Member, IEEE, “Toward Unconstrained Ear Recognition From Two-Dimensional Images,” IEEE Transactions on Systems, Man, and Cybernetics — Part A: Systems and Humans, VOL. 40, NO. 3, MAY 2010Bir Bhanu and Hui Chen, Center for Research in Intelligent Systems. Human Ear Recognition by Computer. California: University of California at Riverside [Online]. doi: 10.1007/978–1–84800–129–9.Eyad Haj Said,North Carolina Agricultural & Technical State University, Greensboro, NC, Ayman Abaza and Hany Ammar, Computer Science and Electrical Engineering Dep., West Virginia University, Morgantown, WV, “Ear Segmentation in Color Facial Images using Mathematical Morphology,” IEEE 2008.
Ping Yan and Kevin W. Bowyer, Fellow, IEEE, “Biometric Recognition Using 3D Ear Shape,” IEEE Transactions on Pattern Analysis and Machine Intelligence, VOL. 29, NO. 8, AUGUST 2007
John D. Bustard, Student Member, IEEE, and Mark S. Nixon, Associate Member, IEEE, “Toward Unconstrained Ear Recognition From Two-Dimensional Images,” IEEE Transactions on Systems, Man, and Cybernetics — Part A: Systems and Humans, VOL. 40, NO. 3, MAY 2010
Bir Bhanu and Hui Chen, Center for Research in Intelligent Systems. Human Ear Recognition by Computer. California: University of California at Riverside [Online]. doi: 10.1007/978–1–84800–129–9.
Eyad Haj Said,North Carolina Agricultural & Technical State University, Greensboro, NC, Ayman Abaza and Hany Ammar, Computer Science and Electrical Engineering Dep., West Virginia University, Morgantown, WV, “Ear Segmentation in Color Facial Images using Mathematical Morphology,” IEEE 2008.