# Detection of Emotion Using Multi-Block Deep Learning in a Self-Management Interview App
## 3 Detection of Emotions Using Multi-Block Deep Learning in Self-Management Interviews
### Multi-Block Selection and Extraction of Main Area Features  
The multi-block selection process selects blocks to be used for actual recognition among the multi-blocks previously delivered through the feature numerical analysis. For accurate emotion analysis, the user's eyes, nose, and mouth, which are the main feature points, should be clearly identified, and they can be classified according to the degree of rotation of the face. If there is no information on specific points in the whole image, the rotation information should be detected during the multi-block process. Figure 6 shows the whole facial recognition and emotion analysis process. Face detection was made by moving a \(24\times 24\) pixel block; for simple patterns in multiple AdaBoost learning, basic patterns were used. In addition, the number of simple detectors to be searched by the learning process was selected as 160, and the learned detectors became serialized, in turn enhancing the processing speed. The learned detectors were serialized into 10 stages in which 16 learned detectors belong in an arbitrary manner. Parameters for each stage were adjusted, and as for images in multiple AdaBoost learning, \(24\times 24\) resolution was used. For detection by size, the input images were classified  
Figure 6: The whole facial recognition and emotion analysis process. * The figure included in this image is the author, who agreed to provide the figure. (based on the degree of down-sampling) into three types, and the face was detected from among the down-sampled images. In detection by rotation, facial images rotated at \(-5^{\circ}\) to \(+5^{\circ}\), \(+15^{\circ}\) to \(+25^{\circ}\), and \(-15^{\circ}\) to \(-25^{\circ}\) were learned by AdaBoost. Then, by using the serialized detector, they were each analyzed and, in order of detection, rotation of the face was classified. The detected faces were classified into nine types, and the information about the locations of the detected faces was provided as well. Figure 7 shows the face image-detection process. For face detection, \(80\times 60\) down-sampled images were used for detecting a large face, \(108\times 81\) down-sampled images were used for a medium-sized face, and \(144\times 108\) down-sampled images for a small face. The sequence of detection by size was selected to enhance the detection speed and was done as follows: Detection of \(80\times 60\) down-sampled images was first, followed by the \(108\times 81\) down-sampled images, and then, the \(144\times 108\) images. If a detected face overlapped the block detected in the face from the down-sampled image in the preceding step, that detection was not valid. The input image was searched for among the down-sampled images, and when it was detected, a block of the face from the detected image was cut and then normalized to the predesigned size and passed to the next process. At the time, principal component analysis (PCA) was used to measure similarity with the input image, verifying the face. This is a process of rotating an image by using the verified information on rotation of the face, until the rotation of the face in the image becomes almost zero. When the rotation of the face was verified to be between \(+15^{\circ}\) and \(+25^{\circ}\), the face was rotated by \(-20^{\circ}\), and when the rotation of the face was between \(-15^{\circ}\) and \(-25^{\circ}\), the face was rotated by \(+20^{\circ}\). When the user's face was extracted in the aforementioned process, the positions of the eyes and nose should be extracted. The patterns for the person's eyes could be extracted by using the facial image obtained through the face detection process. Eyes and nose extraction can be classified into three stages. The first stage was to designate a region to search for the eyes in the facial image obtained by the face detector. From this stage, it was possible to roughly estimate the position of the eyes, even if they were not precise, and such an estimated position could be defined as a certain domain.