# Audio-Visual Tensor Fusion Network for Piano Player Posture Classification
## 4 Performance Evaluation
### Experiments and Results
\begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline  & **Accuracy** & **Precision** & **Recall** & **F1 Score** & **Error Rate** & **Training Time** & **Test Time** \\ \hline VN [7] & 84.9829 & 83.6057 & 81.7531 & 82.0542 & 0.6118 & 1.0963 & 0.6041 \\ AN & 80.137 & 82.8966 & 80.6059 & 81.0313 & 0.9951 & 1.7541 & 3.4616 \\ AVN-Concat [8] & 76.0274 & 79.8384 & 72.9123 & 73.53 & 0.9951 & 5.1612 & 5.2532 \\ AVN-Atten [9] & 81.57 & 84.8317 & 75.3428 & 75.0957 & 1.0746 & 5.1982 & 4.0455 \\ AV-TFN (Ours) & 87.7133 & 86.1329 & 84.7091 & 84.9085 & 0.3943 & 0.3454 & 0.2799 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Experiment results on comparison of AV-TFN with existing methods. **Experiment 3**. : _An experiment on estimating effect of tensor fusion Ratio._  
An experiment was conducted to test the synergy of audio and video information. As we mentioned in Section 3, when creating an audio-visual tensor, we mixed the colors of the audio and visual tensors and set the ratio to 0:100, 50:50, and 100:0. Here, the Tensor Fusion Rate (TFR) 0:100 means the audio tensor color ratio is 100%, while the visual tensor color ratio is 0%. Table 5 demonstrates changes in the F1 score for posture classification according to the TFR of the visual and audio tensors. Experimental results show that the F1 score in the case of 50:50 improves by about 5.4052 percentage points and 13.0826 percentage points when the two are combined than in the case of 0:100 and 100:0. This indicates that audio-visual tensors are more synergistic than they are used individually. \begin{table}
\begin{tabular}{c c c c c c c c c} \hline \hline \multirow{2}{*}{**Image Resolution**} & \multicolumn{4}{c}{**AN vs. AV-FFN**} & \multicolumn{4}{c}{**AVN vs. AV-FFN**} \\ \cline{2-9}  & **F1 Score** & **Error Rate** & **Training Time** & **Test Time** & **F1 Score** & **Error Rate** & **Training Time** & **Test Time** \\ \hline
20 \(\times\) 30 & 24.136 & 1.7118 & 0.008 & 0.0031 & 28.3329 & 3.251 & 1.9513 & 0.944 \\
40 \(\times\) 60 & 4.4681 & 0.3704 & 0.5471 & 0.7885 & 14.6703 & 0.7126 & 2.4798 & 1.4651 \\
80 \(\times\) 120 & 3.8772 & 0.6008 & 1.4087 & 3.1817 & 11.3785 & 0.6008 & 4.8158 & 4.9733 \\
120 \(\times\) 180 & 3.9825 & 0.3508 & 3.3398 & 5.668 & 12.7281 & 0.4891 & 9.2675 & 14.9646 \\ Average & 9.116 & 0.7447 & 1.3259 & 2.4103 & 16.7775 & 1.2634 & 4.6286 & 5.5868 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Superiority of AV-TFN compared with AN and AVN. Figure 10: Experiment results of AN, Audio-Visual Network (AVN), AV-TFN as image resolution increases: (**a**) F1 score; (**b**) error rate; (**c**) training time; (**d**) test time. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**IR** & **TFR** & **Accuracy** & **Precision** & **Recall** & **F1 Score** \\ \hline
20 \(\times\) & 0:100 & 64.7917 & 64.2071 & 61.1152 & 61.636 \\
30 & 50:50 & 70.2083 & 69.4851 & 66.4125 & 67.0412 \\
100:0 & 62.7083 & 58.9199 & 55.3345 & 53.9586 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Performance of AV-TFN as Tensor Fusion Rate (TFR) varied.