# A Multi-Modal Person Recognition System for Social Robots
## 3 Architecture of the Person Recognition System
### Feature Extraction
#### 3.2.1 Vision-Based Feature Vectors  
The Configural Features Group  
The group consists of four feature vectors. The first feature vector is represented by the ratios of the Euclidian distances among the geometric position of a set of fiducial points on a face. These fiducial facial points are detected by \"OpenFace\";, an open source software for facial landmark detector [36]. The second feature vector is based on a cross ratio of the projection lines that are initiated from the corners of the polygon constructed from a set of predefined fiducial points on a face image. The third feature vector is constructed by computing the Euclidian distance among a set of selected skeleton joint positions. The fourth feature vector in this group is the surface-based feature, which is generated by computing the geodesic distances between the projections of selected pairs of skeleton joints on the point cloud that represent, an individual's body. It is worth mentioning that these feature vectors are purposely selected as they are easy to calculate and available early in perception process. The main purpose for these feature vectors are to limit the search scope and provide shortlisted candidates for the attended subject by biasing the top-ranked spiking neurons (see Section 3.4 for more details). The first feature vector in the configural group consists of eight facial feature ratios, as shown in the Appendix A (Table A1). Despite the simplicity of this geometric descriptor, it can be shown that they generate comparable performance in face clustering with respect to other feature vectors that describe face appearance such as EigenFace and Histogram of Oriented Gradients [37]. The second feature vector was constructed by employing the cross ratio theorem, which is a widely applied object and shape recognition algorithm in computer vision [38]. The cross ratio value stays invariant under geometric projection operations such as translation, rotation, and scaling changes [39]. The cross ratio of four collinear points A, B, C, and D in a line L, as shown in Figure 3, is given by:  
\[CR_{L}(A,B,C,D)=\frac{|\overline{AC}|\cdot|\overline{BD}|}{|\overline{BC}|\cdot |\overline{AD}|}\]  
Figure 3: The cross ratio relationship of two viewpoints. The same cross ratio, \(CR_{L}\), can also be expressed as ratio of the projection lines \(XA\), \(XB\), \(XC\), and \(XD\). By using the fact that the \(XAB\) triangle area can be calculated using the formulas: \(\frac{1}{2}*h*AB=\frac{1}{2}*XA*XB*sin\theta_{1}\) and some algebraic manipulation, the cross ratio from point \(X\), can be expressed as a function of the line segments as in (1) or as a function of projection angles as in (2), where \(h\) is the distance between the focus and the line AB, as depicted in Figure 3.  
\[CR_{X}(A,B,C,D)=\frac{|\overline{AC}|\cdot|\overline{BD}|}{|\overline{BC}| \cdot|\overline{AD}|} \tag{1}\]  
\[CR_{X}(\theta_{1},\theta_{2},\theta_{2})=\frac{sin\ (\theta_{1}+\theta_{2})\cdot sin \ (\theta_{2}+\theta_{3})}{sin\theta_{2}\cdot sin\ (\theta_{1}+\theta_{2}+\theta_{2})} \tag{2}\]  
Since the cross ratio value is independent of changes in the viewpoint, the cross ratio of the same four collinear points A, B, C, and D in a line L from point Y can be expressed in the same way as point \(X\) as in (3) and (4). \[CR_{Y}(A,B,C,D)=\frac{|\overline{AC}|\cdot|\overline{BD}|}{|\overline{BC}| \cdot|\overline{AD}|} \tag{3}\]  
\[CR_{Y}(\theta_{1},\theta_{2},\theta_{3})=\frac{sin\ (\theta_{1}+\theta_{2}) \cdot sin\ (\theta_{2}+\theta_{3})}{sin\theta_{2}\cdot sin\ (\theta_{1}+\theta_{2}+\theta_{3})} \tag{4}\]  
hence, \(CR_{X}(\theta_{1},\theta_{2},\theta_{3})=CR_{Y}(\theta_{1},\theta_{2},\theta_ {3})\). The reader may refer to [39] for detailed proof.