# Posture Recognition Using Ensemble Deep Models under Various Home Environments
## 3 Database and Experiment Method
### Posture Recognition Using Ensemble CNN Models  
As more people live alone, there is a need for ways to cope with crime or dying alone. Posture recognition emerged as a solution because it contains some information about the person's situation. However, posture recognition using the conventional inertial sensor, the sensor is cumbersome to attach to the body. For the elderly, it is difficult to wear the sensor all the time and extra help is needed for them to do so. However, image-based posture recognition through a camera can be applied to the existing camera system and there is no need to attach any sensors. In addition, low-cost webcams can also be applied because the posture is recognized only by the image. It has been difficult to recognize posture by using only two-dimensional images, but recently, deep learning has proved excellent in various fields. We applied the technique to the 2D image based posture recognition. First, a daily domestic posture database was constructed. Then, the posture images were processed to different types of preprocessing. The CNN models were VGG16, VGG19 [19], ResNet50 [20], DenseNet201 [21], InceptionResNetV2 [22], and Xception [23]. Transfer learning is effectively used in cases where a neural network needs to be trained with a small database and there is insufficient time to train the network with new data. In this study, posture recognition was performed by training the CNNs using transfer learning. Most of the parameters used to extract the feature maps were fixed, and only the final FC layers and Softmax were trained. In addition, CNNs were trained by updating all the parameters (nontransfer learning). Figure 12 shows the posture recognition using the preprocessed images for different CNN models. Then CNNs of the deep models and CNNs of the preprocessing types are combined using the majority of outputs and average of scores as ensemble methods. The majority of outputs decides the most voted class among outputs of CNNs. The average of score decides the class with a max score from averaged scores of CNNs. Tables 1 and 2 show the ensemble methods of majority vote and score average. Figure 13 shows the various ensemble systems by input types and CNN models. Figure 11: Examples of five types of original and preprocessed images: (**a**) original image; (**b**) type 0; (**c**) type 1; (**d**) type 2; (**e**) type 4; (**f**) type 5. The number of postures is the number of classes. The number of defined postures was 10: \"standing normal\", \"standing bent\", \"sitting sofa\", \"sitting chair\", \"sitting floor\", \"sitting squat\", \"lying face\", \"lying back\", \"lying side\", and \"lying crouched\". The 10 posture classes were divided into four categories: standing, sitting, lying, and lying crouched. Instead of 10 and 4 classes, there were 11 and 5 when other images, such as the background, were considered as a separate class. The results of posture recognition under 10, 4, 11, and 5 classes were obtained. The other images were configured by programmatically cropping the original posture images randomly while avoiding the person to the greatest extent possible. These various numbers of classes have the advantage to extract context information. For example, a person does not sleep in a standing posture. Figure 14 shows the four categories. \begin{table}
\begin{tabular}{c c} \hline \hline
1. & Net results scores of each class as \(1\times n\) matrix. \\ \hline  & \(s_{c}=[x_{1}\ x_{2}\ x_{3}\dots\ x_{n}]\) \\ \hline
2. & Get the outputs by picking the class to the max value in scores as one hot matrix. \\ \hline  & \(o_{c}=[0\ 0\ 1\dots\ 0]\) \\ \hline
3. & Sum the one hot matrix of all classifiers. \\ \hline  & \(\mathrm{v}=\sum_{c=1}^{C}o_{c}\) \\ \hline
4. & Get the final output by picking the class to the max value in sum of one hot matrix. \\ \hline  & \(o_{v}=[1\ 0\ 0\dots 0]\) \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Ensemble method of majority vote. \begin{table}