# Machine Learning-Based Lie Detector Applied to a Novel Annotated Game Dataset
## 4 Methodology
### Classical Machine Learning  
We use three different handcrafted features that are extracted from RGB facial images: local binary patterns (LBP), histogram of oriented gradients (HOG) and scale-invariant feature transform (SIFT). 1. We follow the approach of [41] to use LBP to model the facial area. Specifically, each image is divided into blocks, where LBP is independently computed and results in a LBP histogram. The histograms from all images' blocks are then concatenated to form the final descriptor. We also vary the number of (P) used to compute the final descriptor along with the number of blocks to see their impacts. We use the parameter values of P of 8, 12 and 16, with the number of blocks of 1 \(\times\) 1, 2 \(\times\) 2, 4 \(\times\) 4 and 8 \(\times\) 8. For technical implementation, we use Scikit-image library [42] to extract LBP descriptors given the image input. 2. Given the overall facial area, we calculate the HOG histogram, using fixed number of 8 bins, with different number of cells and blocks to test how their impact affects classification performance. We use 1 \(\times\) 1 and 2 \(\times\) 2 HOG cell sizes, and partition the images into 8 \(\times\) 8 and 16 \(\times\) 16 pixel sizes. We also use Scikit-image ([42]) for technical implementation. 3. We calculated the SIFT features by performing the centroids k-clustering that involves the computation of K-means for all SIFT keypoints. Specifically, the K-means algorithm is performed by splitting a set of \(N\) samples of \(X\) onto K separated clusters of \(C\) whereby each of instance is explained by the mean \(\mu_{j}\) of the samples in the cluster. This is done by minimizing the inertia criterion of \(\sum_{i=0}^{n}\min_{\mu_{j}\in C}\left(\left\|x_{i}-\mu_{j}\right\|^{2}\right)\), with n as the total input samples. In this case, we create histograms with different K bins values (100, 300, 500 and 800) for each key point in an image [43] to further evaluate how the histogram length and BoW size impact the classification. We use [44,45] for the technical implementation. Using these handcrafted features, we then employ three classifiers to predict the lie label (all implementations are based on Scikit-learn library [45]): support vector machine (SVM), AdaBoost, and linear discriminant analysis (LDA). Specifically, given the input features \(x_{i}\in\mathbb{R},\mathrm{i}=1,\ldots,\mathrm{n}\) and a vector \(y\) containing the binary label of \(-1\) (non lie) and \(1\) (lie) thus \(y\in\{1,-1\}^{n}\), the SVM is optimized by finding the normal vector \(w\in\mathbb{R}\), the kernel \(\phi\) (which can be linear or non-linear), and the bias \(b\in\mathbb{R}\) that results in the prediction of \(\mathrm{sign}\left(w^{T}\phi(x)+b\right)\) to be correct throughout the majority of samples. This process is formulated in Equation (1) below:  
Figure 3: The example of pairs of recording facial area of several participants when telling the truth (**top**) or lies (**bottom**). \[ \begin{split}\min_{w,b,\zeta}\frac{1}{2}w^{T}w+C\sum_{i=1}^{n}\zeta_{i} \\ \text{subject to }y_{i}\Big{(}w^{T}\phi(x_{i})+b\Big{)}\geq 1-\zeta_{i}\\ \zeta_{i}\geq 0,i=1,\ldots,n\end{split}  \tag{1}\]  
where \(\zeta_{i}\) is the distance from the evaluated features to the decision hyperplane and \(C\) is a penalty term that controls the amount of samples allowed to cross the learned boundary (to the region that corresponds to the opposite class). These operations maximize the margin by minimizing \(\|w\|^{2}=w^{T}w\), while including a penalty when a sample is misclassified or within the margin from the boundary. As for the AdaBoost method, we first form the decision tree that recursively splits the feature space such that samples with the targeted labels are grouped together.