# Human Skeleton Data Augmentation for Person Identification over Deep Neural Network
## 5 Data Augmentation for Person Identification  
### Human Bilateral Symmetry-Based Data Augmentation  
Here, we propose a novel data augmentation method by exploiting the bilateral symmetry of the human skeleton structure. In general, the length of the limbs on the left and right sides can be regarded as equal. As shown in Figure 7, eight pairs of limbs in the skeleton are symmetric. Using the limb definition in Table 2, the pairs are denoted as follows: (Limb #1, Limb #12), (Limb #2, Limb #13), (Limb #3, Limb #14), (Limb #4, Limb #15), (Limb #5, Limb #16), (Limb #6, Limb #17), (Limb #7, Limb #18), and (Limb #8, Limb #19). Using Equation (1), the average lengths for these limbs are calculated. In addition, from the feature concatenation process in Figure 6, the calculated values are located from the 1 st element to the 8 th element, and from the 12 th element to the 19 th element. Figure 7 depicts the proposed data augmentation method, where \(\mathbf{V}(t)\) is the \(t\)th element in its symbol color.  
Figure 6: Overall procedure to generate feature vector \(\mathbf{V}\) from skeleton sequence.  
Figure 7: \(\mathbf{V}(t)\) for the \(t\)th element is indicated in its symbol color.  
In the proposed method, several secondary versions are generated from \(\mathbf{V}\) by exchanging its elements corresponding to the symmetric limbs. In addition, this exchange process is performed with different combinations of pairs. For example, suppose that the exchange is performed for five out of the eight pairs. In addition, let \(C(p,q)\) be the number of \(q\)-combinations from a given set of \(p\) elements. Then, 56 combinations can be generated from the selected five out of eight pairs (i.e., \(C(8,5)=8!/(5!3!)=56\), where \(r!\) denotes the factorial of \(r\)). Therefore, because the number of pairs of symmetric limbs is eight, the number of \(q\)-combinations for all \(q\in\{1,2,\cdots,7,8\}\) is \(\sum_{q=1}^{8}C(8,q)=2^{8}-1=255\).  
In each of the 255 combinations above, \(q\) distinct index numbers represent the pairs selected from the eight pairs. In the proposed method, based on these index numbers, the elements of \(\mathbf{V}\) that need to be exchanged are selected. For example, suppose that Pair 1 is selected out of the eight pairs (i.e., \(q=1\)) for exchange. The first pair (Limb #1, Limb #12) is selected. The secondary version \(\mathbf{V}\) is then generated as shown in Figure 8a. For another example, two numbers indexed 3 and 8 are selected. The pairs (Limb #3, Limb #14) and (Limb #8, Limb #19) are then constructed. The secondary version \(\mathbf{V}\) is then generated by exchanging \(\mathbf{V}(3)\) and \(\mathbf{V}(14)\), and \(\mathbf{V}(8)\) and \(\mathbf{V}(19)\), as shown in Figure 8b.  
Therefore, for \(q\in\{1,2,\cdots,7,8\}\), the total number in the secondary version \(\mathbf{V}\) is \(\sum_{q=1}^{8}C(8,q)=255\). We present its pseudocode in Algorithm 1, where \(\mathbf{S}\) is the secondary version \(\mathbf{V}\) and its dimension is \(255\times 25\). According to the pseudocode, the total number of anthropometric feature vectors is increased up to 256 times (including the original vector) as shown in Figure 9.  
[htb]1 \(\times\) 25-dimensional feature vector of \(\mathbf{V}\)  
255 \(\times\) 25-dimensional matrix of \(\mathbf{S}\)  
1: count = 1;  
2: \(\mathbf{S}\) = zeros(255, 25);  
3: for q = 1:1:8 do  
4: comb = rchoosek(1:1:8, q);  
5: for b = 1:1:size(comb, 1) do  
6: \(\mathbf{S}\)(count, :) = \(\mathbf{V}\);  
7: temp1 = \(\mathbf{V}\)(comb(b, :));  
8: temp2 = \(\mathbf{V}\)(comb(b, :)+11);  
9: \(\mathbf{S}\)(count, comb(b, :)) = temp2;  
10: \(\mathbf{S}\)(count, comb(b, :)+11) = temp1;  
11: count = count + 1;  
12: endfor  
13: endfor  
Figure 8: Examples of methods to generate secondary versions \(\mathbf{V}\).