# 3D Approaches and Challenges in Facial Expression Recognition Algorithms--A Literature Review
## 3 Conventional and Deep Learning-Based Approaches
### Multi-Modal Algorithms Using 2D and 3D Data  
Algorithms that combine results from 2D and 3D data did not appear until about the early 2000s. Up to date, in this area, the most straightforward approaches use combining the features obtained independently from the bi-dimensional or three-dimensional methods, such as texture information, landmarks location, facial shapes, and curvature, for recognizing expressions and measuring their intensity [12]. Experiments show that merging features obtained in different modalities helps to catch the general characteristics of facial deformation and to enhance recognition accuracy, even though the texture information is affected by the illumination and pose variations. \begin{table}
\begin{tabular}{c c c c c c c c} \hline
**Author** & **Database 1** & **2D 2** & **Dynamic 3** & **FIs 4** & **Method 5** & **Expressions 6** & **Highlights** \\ \hline \hline [PERSON] et al. [67] & BU-3 DFE & No & No & 20 man. & SIFT descriptors + SVM & 6 BE & The neutral scans are not used & Curves used to capture the \\ [PERSON] et al. [60] & BU-3 DFE & No & No & 68 man. & & 6 BE & deformation on different faces \\ [PERSON] et al. [33] & Bosphorus & Yes & No & Not used & ROC curves & 25 AUs & 3D performs better for lower face \\ [PERSON] and [PERSON] [27] & BU-3 DFE & No & No & 11 man. & Fisher criterion + Neural & 6 BE & Pose-invariant algorithm \\ [PERSON] and & & & & & Geometry and curvature & 2D+3D images recorded in \\ [PERSON] [68] & Private & Yes & Yes & 81 aut. & features & 5-11 AUs & 2D+3D images recorded in \\ [PERSON] et al. [69] & BU-3 DFE & No & No & Not used & Spectral flow matrices as & 6 BE & No manual intervention or \\ [PERSON] et al. [70] & BU-3 DFE & No & No & Aut. & SIFT feature + SVM & 6 BE & Computerly automatic approach \\ [PERSON] et al. [44] & BU-4 DFE & No & Yes & Not used & Facial Level Curves + Hidden & 3 (happiness, sadness & \\ [PERSON] et al. [61] & BU-3 DFE & No & No & 60 man. & H\(\alpha\)G and H\(\epsilon\)S + SVM & 6 BE & Local shape descriptors are used \\ [PERSON] et al. [45] & BU-4 DFE & No & Yes & Not used & 3D facial geometry + Hidden & 3 (anger, happiness & \\  & & & & Markov Model & and surprise) & Fully automatic system \\ [PERSON] et al. [71] & BU-3 DFE and & & & & & Zemike moments assures & 6 BE & Real-time \\  & & & & & & & \\ [PERSON] et al. [52] & BU-4 DFE & Yes & Yes & Not used & DVF + Random Forest & 6 BE & Temporal analysis of facial \\ [PERSON] et al. [72] & BU-3 DFE & No & No & Not used & Multi-Scale Local Normal & 6 BE & Fully automatic approach \\ [PERSON] et al. [73] & BU-3 DFE and & & & & Minimum redundancy - & 6 BE & Person and gender independent \\  & & & & & maximum relevance + SVM & 6 BE & Temporal analysis \\ [PERSON] et al. [46] & BU-4 DFE & No & Yes & - & classifier and Hidden Markov & 6 BE & Temporal analysis \\ [PERSON] et al. [74] & & & & & Model & & Model \\  & & & & & & Depth map + APDM + SVM & 22 AUs & Temporal analysis \\ [PERSON] et al. [75] & BU-3 DFE & No & No & Not used & Differential Mean Curvature & 6 BE & Face normalization step \\ [PERSON] et al. [55] & BU-3 DFE & Yes & No & 3 aut. & CFI and MCI + 2D texture & 6 BE & Fully automatic \\  & & & & & & descriptors & \\ \end{tabular}
\end{table} 
Table 2: Feature-based algorithms. \begin{table}
\begin{tabular}{c c c c c c c c c} \hline
**Author** & **Database 1** & **2D 2** & **Dynamic 3** & **FLs 4** & **Method 5** & **Expressions 6** & **Highlights** \\ \hline [PERSON] et al. [41] & BP4D & No & Yes & Not used & HMM & 6 BE & Spontaneous expression data \\ [PERSON] and & & & & & & & No computationally expensive \\ [PERSON] [76] & & & & & & SVM & 6 BE & pre-processing step and any user \\  & FRGC v2.0, & & & & & & & interventions \\ [PERSON] et al. [77] & Bosphorus, & No & No & Not used & MSMC-LNP & 6 BE & Different expression intensity levels \\  & BU-3 DFE and 30-TE & & & & & & 3D flow of facial points + & 6 BE & Neutral 3D facial model for each \\ [PERSON] and [PERSON] [47] & BU-3 DFE & No & No & 83 man. & SVM & 6 BE & Subject \\  & & & & & 5 aut.