3D landmark‐based face restoration for recognition using variational autoencoder and triplet loss

IET BiometricsVolume 10, Issue 1 p. 87-98 ORIGINAL RESEARCH PAPEROpen Access 3D landmark-based face restoration for recognition using variational autoencoder and triplet loss Sahil Sharma, Corresponding Author sahil.sharma@thapar.edu orcid.org/0000-0002-6694-3365 Computer Science and Engineering Department, Thapar Institute of Engineering and Technology, Patiala, India Correspondence Sahil Sharma, Computer Science and Engineering Department, Thapar Institute of Engineering and Technology, Patiala, India. Email: sahil.sharma@thapar.eduSearch for more papers by this authorVijay Kumar, Computer Science and Engineering Department, National Institute of Technology, Hamirpur, IndiaSearch for more papers by this author Sahil Sharma, Corresponding Author sahil.sharma@thapar.edu orcid.org/0000-0002-6694-3365 Computer Science and Engineering Department, Thapar Institute of Engineering and Technology, Patiala, India Correspondence Sahil Sharma, Computer Science and Engineering Department, Thapar Institute of Engineering and Technology, Patiala, India. Email: sahil.sharma@thapar.eduSearch for more papers by this authorVijay Kumar, Computer Science and Engineering Department, National Institute of Technology, Hamirpur, IndiaSearch for more papers by this author First published: 19 December 2020 https://doi.org/10.1049/bme2.12005Citations: 3AboutSectionsPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onEmailFacebookTwitterLinked InRedditWechat Abstract Restoration of a 3D face from the mesh image is highly demanded in computer vision applications. 3D face restoration is a challenging task due to the variation of expression, poses, intrinsic geometries, and textures. The proposed technique consists of two main components, namely face restoration and recognition. A novel three-dimensional (3D) landmark-based face restoration method is proposed. 3D facial landmarks are used in the face recognition technique. It uses the principle of reflection and mid-face plane for the restoration of facial landmarks. By using the restored 3D face, a deep learning-based face recognition system is developed. It utilizes the concept of deep features from variational autoencoders. Further, these deep feature embeddings are trained using triplet loss training to increase the distance between embeddings of different persons and decreasing the distance between embeddings of the same person. These trained embeddings are used in support vector machine for prediction. The proposed framework is compared with recently developed face recognition techniques in terms of computational time. The proposed technique is able to recognize the person's face with better accuracy than the existing methods. Further, ablation studies are conducted to test the robustness of the proposed technique. 1 INTRODUCTION Nowadays, researchers are working on three-dimensional (3D) face recognition. This modality has the potential for addressing the challenges of 2D face recognition viz. pose, illumination, occlusion, and scale [1]. 3D face recognition is also used in the entertainment industry viz. 3D animation, augmented reality, and virtual reality. Advancements in 3D deep learning have enabled researchers to achieve 3D fashion modelling [2] and building animated character through Generative Adversarial Networks (GANs) [3-5]. Various methods have been proposed to solve the problems associated with 3D facial restoration using facial geometry [6-8]. Shape from shading (SFS) method uses the shades on the face because of lightening. SFS is profoundly sensitive to lightening and does not perform well under pose variation. 3D morphable models (3DMM) are unable to generate accurate face shape [9-11]. To resolve the above-mentioned problems, a 3D restoration technique is proposed to regenerate the missing information. 3D face restoration is done through different approaches viz. using point cloud information, landmarks, or voxels. Intuitively, the whole of the 3D face mesh could be given as input into the deep learning model and the subject identification number can be predicted. This work has been done using voxel-based face reconstruction using sequential deep learning by the authors [46]. In presented work, 3D landmarks are used for restoration and recognition. There is a challenge in recognition through landmarks as the number of landmarks varies with pose or occlusion (see Figure 1). Especially in Figure 1a,b, the right nose saddle is visible and the left nose saddle is missing. However, the right nose saddle and left nose saddle points are visible in Figure 1c. The challenge of sparseness in landmarks information is handled using the proposed 3D landmarks-based face restoration technique. All the restored, as well as available landmarks, are labelled as ground-truth landmarks for face recognition. After the restoration of missing landmarks, deep learning is performed for extracting the deep features. A prediction model is applied to deep features for subject identity classification in the proposed framework. FIGURE 1Open in figure viewerPowerPoint Representation of 3D landmarks on different pose meshes The main contributions are as follows: One-shot 3D landmark-based face restoration technique is proposed the utilizes concept of mirroring. A novel deep learning framework is proposed that utilizes the variational autoencoders (VAE) for extracting the deep features. Triplet loss training is used for decreasing the distance between embeddings in intraclass and increasing the distance in an interclass manner. Gaussian support vector machine is used for prediction of the deep feature embeddings. The proposed framework is compared with three deep learning-based face recognition techniques over three well-known datasets, namely Bosphorus, KinectFaceDB, and UMBDB. The rest of the study is organized as follows. Section 2 describes the preliminary concepts of 3D face recognition using restoration, VAE, triplet loss, and support vector machine (SVM). Related work is presented in Section 3. Section 4 describes the proposed 3D deep learning-based face recognition framework. Experimental results and discussion are presented in Section 5. Ablation studies are done in Section 6, and the conclusion is drawn in Section 7. 2 BACKGROUND In this section, the preliminary concepts of 3D face recognition using restoration, VAE, triplet loss training, and SVM are discussed. 2.1 3D face recognition using restoration The process of 3D face recognition using the restoration technique on occluded faces is described. There are many publicly available 2D and 3D face datasets [39-41]. A 3D face dataset is collected and preprocessed for the training phase. In the next phase, the restoration of the occluded part of the face is done. The main focus of this study is the second phase. In this phase, 3D facial landmarks are used for restoration. Once the occluded information of the face is restored, the dataset is partitioned for training and testing phases [46]. The deep learning-based training is performed on the training dataset. After the training is completed, the test dataset is used for validation. The prediction accuracy is calculated based on the model's performance in the validation phase. The visual representation of these phases is shown in Figure 2. FIGURE 2Open in figure viewerPowerPoint Phases of 3D face recognition using restoration 2.2 Variational autoencoder Variational autoencoder (VAE) is a technique of reconstruction for the input image using a probabilistic encoder for generating the latent variables and probabilistic decoder for generating the output [53] (see Figure 3). The latent variables (R) are in the form of probability distributions instead of discrete values, as in the case of simple autoencoders. The goal of VAE is to find a distribution F ϕ ( R | Q ) of some latent variables (R), which we can sample from R ∼ F ϕ ( R | Q ) , to generate new samples Q ′ ∼ S θ ( Q | R ) . It attempts to decode the embedding which is not in the trained dataset to a reasonable output [12–13]. FIGURE 3Open in figure viewerPowerPoint Architectural design of variational autoencoders During the training of VAE, the hyperparameters which are used for tuning are the number of epochs, batch size, activation layer, regularization technique, latent dimension, and the optimizer is used. All the hyperparameter tuned values are presented in Table 2. 2.3 Triplet loss The embeddings generated by VAE are needed to be clustered based on their classes for efficient classification. In supervised learning, triplet loss is a technique of learning good embeddings of the face. A triplet is formed using three embeddings from two different classes jointly generating a positive pairing and a negative pairing. Anchor, Positive, and Negative are denoted by (A, P, N) as a triplet, where (A, P) holds the same label and (A, N) holds different labels. During triplet loss training, the distance of the positive pair is reduced in contrast to the negative pair (see Figure 4). FIGURE 4Open in figure viewerPowerPoint Distance between A and P is reduced and the distance between A and N is increased during triplet loss training Formally, triplet loss function can be defined as follows: [14]. 1 N ∑ i = 1 N max { | | f ( A ) − f ( P ) | | − | | f ( A ) − f ( N ) | | + α , 0 } (1)where N is the cardinality of the generated triplets used in the triplet loss training. f(.) is the embedding generated by variational autoencoder, α is the margin and | | f ( A ) − f ( P o r N ) | | represents the distance between two embeddings. 2.4 Support vector machine SVM is a classifier that performs the segregation of classes by finding an optimal hyperplane using a z number of support vectors. In the case of non-linear separability of classes, kernel-based methods such as gaussian SVM are used to generate higher-dimensional space or feature space by mapping the input vectors. This method makes linear separation possible in a higher dimension. To classify an input vector, SVM evaluates function f(x) as [15]: f ( x ) = s i g n ( ∑ i = 1 z p i α i N ( x i , x j ) + b ) (2)where z is the number of support vectors with model parameters pi, and a bias parameter (b). α i signifies the Laplacian coefficient from the dual optimization problem. N (xi, xj) is a kernel function. 3 RELATED WORK Gecer et al. [16] proposed a GAN-based and differentiable renderer-based 3D morphable model-fitting strategy. They showed that high-resolution statistical reconstruction of the full facial surface on a UV surface is successful in reconstructing arbitrary texture surfaces. UV surface is formed by the process of texture mapping of a 2D image to a 3D model. They also proposed a cost function that combines identity loss, content loss, pixel loss, and landmark loss. Tu et al. [17] proposed a 3D face reconstruction method using a 2D image as a self-supervised method by leveraging 2D landmarks out of noisy images. Prediction of 3D landmarks is done using the self-mapping process over the 3D morphable model (3DMM). The 2D-assisted self-supervised learning (2DASL) relieved the demands on 2D-to-3D conventionally paired annotations and giving 3D face models of higher quality without additional 3D annotations. Deng et al. [18] proposed a convolutional neural network-based single image face reconstruction method that exploits the image information at the hybrid level for weakly supervised learning. The benefit of combining perception-level loss and robustified image-level loss is demonstrated with state-of-the-art accuracy on multiple datasets. Tewari et al. [19] used supervision on 2D images, including the photometric loss and landmarks that achieved a self-supervised learning scheme for single image face reconstruction. Optimization-based methods vary in many respects for morphable model fitting. Some design choices vary with facial attributes are being used. A morphable model is used for generating the synthetic facial image and optimizing the parameters for minimizing the difference between synthetic and target image. This type of formulation depends on the initial sparse set of facial landmarks. Manually annotated landmarks were used in earlier methods [20]. Recently, automatic annotation of facial landmarks has made landmarks richer [21-24]. This posed a query if the morphable model fitting could be implemented by only using landmarks [25]. Texture based algorithms are computationally slow as compared to algorithms which are based on landmarks [26]. Liu et al. [27] proposed the recovery of pose and expression normalized (PEN) 3D face shapes by using 2D face landmarks via cascaded regressors. Liu et al. [28] used faces from CASIA-WebFace detected by [29] and 68 facial landmarks are located by [21]. Feng et al. [8] proposed an end-to-end method called Position map Regression Network (PRN) to predict dense alignment and shape of the 3D face without any restriction of low-dimensional solution space. 3D facial structure and the dense alignment were regressed using UV position map-generating the semantic meaning of each point on UV space. Wu et al. [30] proposed a 3D face expression reconstruction model based on a cascaded regression framework. Cao et al. [31] presented a novel 3D face reconstruction technique leveraging a sparse photometric stereo. Geometry proxy and semantic segmentation techniques were used to refine the input images in pre-processing the data. Feng et al. [7] used Epipolar plane images (EPIs) to learn convolutional neural networks (CNNs) for recovering horizontal and vertical 3D facial curves from respective EPIs. Hsu et al. [32] proposed a 3D face components reconstruction using a modified VGG pre-trained model instead of using a holistic approach. Richardson et al. [33] proposed a CNN framework capable of extracting the shape of the face in coarse-to-fine fashion using a single image. The framework was divided into two halves viz. CoarseNet and FineNet. Initially, CoarseNet recovered the facial geometry as coarse level. The renderer layer extracted the depth image from the given 3D face mesh. This depth image was responsible for refining the facial features of corresponding geometry using FineNet. Synthetic images were made in use for training the CoarseNet with matching pose and facial geometry. The advantage of synthetic data is to bypass the manual labelling of unlabelled facial images. Sharma et al. [52] proposed a game theory and simulated annealing based deep learning framework for 3D face reconstruction using the voxels. They have used Bosphorus, UMBDB, and the KinectFaceDB face datasets for verifying their technique. The main drawback of the existing techniques is computationally expensive. Very few techniques are based on reconstruction using a single 3D image. Most of the techniques are using 2D-to-3D conversion for the purpose of 3D face recognition and reconstruction. To resolve the above-mentioned problems, a novel face recognition technique is proposed. The proposed technique works directly on the 3D mesh and uses the 3D facial landmarks for restoration purposes. The proposed technique solves the problem of dependency on a large number of images needed for reconstruction of the face as it can be implemented on a single 3D input image. Second, the proposed technique is computationally efficient that the other techniques as reported in Section 5.5. 4 PROPOSED 3D FACE RECOGNITION FRAMEWORK This section discusses the motivation and the restoration-based 3D face recognition framework. 4.1 Motivation The concept of facial asymmetry has been studied in the field of plastic surgeries and reconstruction. Facial symmetry is one of the key factors in human attractiveness [34-37]. Linden et al. [38] presented the results of the study on the effect of age on facial symmetry. The motivation of the presented work comes from these studies. From a holistic point of view, facial geometry is such that more than 80% of the face is symmetrical in nature. Sharma and Kumar [46] presented 3D voxel-based face reconstruction (3DVFR). It uses using the full mesh as input and generated reconstructed voxels. This technique takes a full mesh as an input, does the voxelization process and then reconstruct the reconstructable voxels. The computational time is more for voxel reconstruction as compared to landmark reconstruction. It gives us a new direction to develop the computationally efficient face recognition technique. 4.2 Proposed face recognition framework The proposed 3D deep learning based face recognition framework is shown in Figure 5. It is based on deep learning and restoration of 3D landmarks. While designing the pipeline, once ground truth landmarks are fetched from a 3D mesh image. The number of landmarks is sparse for different images due to the variation of the pose. The proposed restoration technique restores the missing landmarks using the mirroring technique. The raw 3D landmarks are to be refined using deep learning for their use in the prediction phase. VAE uses the concept of the probabilistic encoder to remove the sparsity of the number of 3D landmarks. The deep features obtained from VAE are optimized by using the triplet loss training method. There are two phases namely, training and testing. The detailed description of these phases is as follows. FIGURE 5Open in figure viewerPowerPoint Proposed 3D deep learning based face recognition framework 4.2.1 Training phase The training phase of the proposed framework includes pre-processing of 3D mesh image, the restoration process, deep learning-based models, and the prediction of the face after restoration from occlusion. 4.2.1.1 Pre-processing 3D facial landmarks are extracted from the mesh image. These landmarks may or may not be present within the dataset. While extracting the face landmarks, the data augmentation technique plays a vital role in improving the quality of facial landmarks. Extracted landmarks serve as ground truth landmarks. Unique landmarks are generated for all the images in the dataset. The landmarks of each image are compared to a list of unique landmarks for generating the array of available and missing landmarks using landmark-to-landmark mapping. The number of landmarks varies for all images. As depicted in Figure 1, a face with yaw rotation left 90° has 19 landmarks initially. Whereas, a face with yaw rotation of left 45° has 24 landmarks. 4.2.1.2 Proposed restoration algorithm and computational complexity The restoration phase of the proposed framework utilizes the concept of 3D landmarks-based face restoration (3DLFR). Training (Tr) and testing (Ts) sets are randomly selected from the given dataset. Mesh image (Im) is taken as input from Tr. Initially, the array of ground truth landmarks (LGT) is fetched in correspondence to Im. Thereafter, an array LMid of mid-face landmarks is generated (Step 2). The selection of one of the landmarks (LM), and nose tip landmark (LNT) is done from array LMid (Step 3). The next step is to generate one landmark (L) such that it lies on the same mid-face plane as of LNT (Steps 4 and 5). In Step 6, the equation of the midface plane is generated using LNT, L, and LM. A list of reconstructable landmarks is calculated by the technique of (LGT − LMid) (Step 7). Finally, the reconstruction is achieved by mirroring technique (Step 8). Restoration using a mirroring technique through a mid-face plane is implemented from the list of reconstructable landmarks. Algorithm 3DLFR— 3D landmarks-based face restoration technique Input: Dataset D is divided into Train (Tr) and Test (Ts) sets randomly. Mesh image Im taken as input from the Tr. Output: Restored 3D landmarks 1. Fetch array of ground truth landmarks (LGT) corresponding to Im. a. LGT = [L1, L2, L3, …, Ln] 2. Generate an array LMid of mid-face landmarks from array LGT which are on mid-face plane. a. LMid = [LNT, LULUM, LULLM, LLLUM, LLLLM, LCM] 3. Select nose tip face landmark LNT from LMid and one more landmark LM from LMid as mid-face landmark from LGT. a. LNT = (x00, y00, z00) b. LM = (x11, y11, z11) 4. Find maximum and minimum z coordinate (zmin and zmax) from LGT. 5. Choose one random L (x00, y00, z) such that zmin < z < zmax. 6. Using LNT (x00, y00, z00), L (x00, y00, z), and LM (x11, y11, z11), calculate 3D mid-face plane. 7. Find the array (R) of all restorable points on face. R = (LGT − LMid)//List of landmarks not on mid-face 8. Restore the 3D face landmarks using the concept of mirroring through mid-face plane for all the R number of points on the face. a. For t = 1 to len(R) do b. P = R[t] = [x1, y1, z1]//P is restorable 3D point c. N = [x2, y2, z2]//PN is perpendicular to mid-face plane d. PN direction ratios = [a, b, c]//a, b, c are constants e. PN = [(x − x1)/a = (y − y1)/b = (z − z1)/c = k]//k is constant f. x = ak + x1, y = bk + y1, z = ck + z1 g. Since N lies on mid-face plane h. k = (−ax1 − by1 − cz1 − d)/(a2 + b2 + c2) i. x2 = ak + x1, y2 = bk + y1, z2 = ck + z1 j. Return [2x2 − x1, 2y2 − y1, 2z2 − z1]//Restored 3D landmark k. End for The face restoration phase aims to generate the missing landmarks by separating the landmarks on the mid-face plane (LMid) and the landmarks, not on the mid-face plane (R). By using the mid-face plane, the restoration of the landmarks is done in Step 8. Restoring the missing landmarks helps in increasing the deep learning accuracy by providing more number of parameters while training the model. Figure 6 illustrates how the three points are selected for calculating the mid-face plane. The first point is the nose tip LNT (x00, y00, z00) which is assumed to be available in all the 3D face meshes. The second landmark needed for the mid-face plane generation is one of the mid-face landmark LM (x11, y11, z11). It can be chin middle (LCM), upper lip upper-middle (LULUM), upper lip lower middle (LULLM), lower lip upper-middle (LLLUM), lower lip lower middle (LLLLM), the mid-point of the two eyebrows, or the mid-point of the two eyes landmarks. The third point is calculated by varying the value of z in the range of zmin and zmax, by keeping (x00, y00) as it is. Using this technique, the three points are not collinear and a unique midface plane can be easily formed for mirroring of reconstructable landmarks (R). Hence, the total number (T) of landmarks after reconstruction becomes T = ( 2 × l e n ( R ) + l e n ( L M i d ) ) (3) FIGURE 6Open in figure viewerPowerPoint Selection of three points on face to calculate mid-face plane FIGURE 7Open in figure viewerPowerPoint Visualization of landmarks before and after restoration FIGURE 8Open in figure viewerPowerPoint Convergence of accuracy for VAE and triplet loss with SVM. SVM, support vector machine; VAE, variational autoencoder This method increases the number of landmarks to approximately double. T number of landmarks are given as input to the VAE model for generating deep features. Figure 7 denotes the 3D landmarks before and after the restoration by mirroring. The time complexity of the proposed algorithm is as follows. The fetching of ground truth landmarks (Step 1) requires O(1) time for a single mesh. The generation of the array LMid (Step 2) requires O(L) time. In Step 3, selecting the LNT and LM requires O(1) time, and finding the maximum-minimum z coordinates (Step 4) requires O(L) time each. Choosing one random value (Step 5) between zmin and zmax requires O(1) time. In Step 6 and 7, finding the mid-face plane as well as the list of restorable points on the face requires O(1) time. In the final Step, the restoration process needs O(R) time. Hence, the asymptotic time complexity of the proposed technique is O(L), where L is the number of facial landmarks. The limitation of the proposed algorithm is that it does not work when there is no symmetry left. The face is said to be with no symmetry when the nose tip is missing, or the face is hidden for more than 50% of the area. Preferably, the nose tip should always be visible for proceeding in doing the restoration. 4.2.1.3 Deep learning phase After the restoration of the landmarks, the deep learning phase is implemented. The total number of T landmarks are given as input to VAE for the extraction of deep features from landmark values. VAE are used for the enhancement of the restored landmarks due to probabilistic encoding and decoding capability. Original 3D landmarks, as well as restored landmarks, are both combined for this phase. The main advantage of using the VAE after the restoration is that it removes the sparsity of the landmarks. Further, triplet loss training is performed on the deep features for fine-tuning of feature embedding usable in the prediction phase. 4.2.1.4 Prediction phase The success of the SVM in classification [47-50] made us to use it for predicting the subject identification number. 3D face landmarks after restoration are treated as a medium of coarse-to-fine features. Hence, the chosen variant of SVM is medium gaussian SVM. The details of the parameter setting of medium Gaussian SVM are presented in Section 5.2. 4.2.2 Testing phase In the testing phase, the concepts of validation and verification are represented in pre-processing and prediction sub-phases. 4.2.2.1 Pre-processing The validation is performed on the testing dataset. The ground truth landmarks are used for restoration. Enhanced embedding of the restored landmarks is done by using VAE and triplet loss training. The embeddings generated through deep learning are given as input to the final trained model of the training phase. 4.2.2.2 Prediction All the predictions are recorded and compared with ground truth values. Overall accuracy is calculated from the correctness of predictions. Similarly, for verification purposes, a single query image is processed for prediction and compared with ground truth value to generate a similarity score. 5 EXPERIMENTAL RESULTS AND DISCUSSION This section consists of all the experimental results and discussion. The details of datasets used, parameter setting of all the models, performance evaluation and metrics, computational time analysis of the proposed techniques, and the convergence are demonstrated. 5.1 Datasets used The experimentation of the proposed technique is implemented using the three datasets namely, Bosphorus [39], UMBDB [40], and KinectFaceDB [41]. These are described in Table 1. TABLE 1. Dataset description Dataset Bosphorus UMBDB KinectFaceDB Dimensions 2D, 3D 2D, 3D 2D, 2.5D, 3D Number of subjects 105 143 52 Number of images 4666 1473 936 Number of occluded images 381 590 312 Annotations Yes No Yes Classes of occlusion Eye, glasses, mouth, hair Hair, glass, mouth, cloth, paper, cigarette, bottle Paper, eye, mouth Extent of occlusion ≤50% ≤80% ≤50% Bosphorus dataset consists of the data of 105 subjects with a total number of images as 4666. The modalities available in this dataset are 2D as well as 3D. The total images with occlusion in this dataset are 381. UMBDB dataset holds the images of 143 subjects with 1473 total images. The dimensions of the face available in this dataset are 2D and 3D. KinectFaceDB dataset holds images of three modalities viz. 2D, 2.5D, and 3D. The total number of subjects being 52 and 936 as the total number of images, KinectFaceDB has 312 occluded images. The dataset is annotated for Bosphorus as well as KinectFaceDB, and not for UMBDB. The classes of occlusions for Bosphorus are four, UMBDB is seven, and KinectFaceDB is three. The extent of occlusion in UMBDB is the highest upto 80% and upto 50% for the other two datasets. 5.2 Parameter setting The parameters of the proposed technique and compared techniques are depicted in Table 2. The parameter setting of the proposed technique is set by trial and error method while keeping variational autoencoder [52], triplet loss training [14], and SVM [48, 49] in reference. In VAE, the number of epochs is set to be 1200, batch size as 50 with activation function as a rectified linear unit (ReLU). L2 regularization is employed along with adaptive moment (Adam) optimizer. In triplet loss training, the number of epochs is 2300 with alpha value as 0.2 using the Adam optimizer and calculating the convergence using mean absolute error (MAE). During the prediction phase, medium gaussian SVM is used with Gaussian kernel function and kernel scale as 9.1 (see Table 2). The data augmentation based pre-processing technique has been added for improving the initial 3D face landmarks. There are three parameters used while pre-processing the image viz. width-shift range, height-shift range, and zoom range. All the three parameters are applied with a value of 0.2 each. TABLE 2. Parameter setting Proposed technique/parameter Value Variational autoencoders Number of epochs 1200 Batch size 50 Activation ReLU Regularization L2 Latent dimension 200 Optimizer Adam Triplet loss training Number of epochs 2300 Optimizer Adam Alpha 0.2 Loss Mean absolute error Support vector machine