## 2 Materials and Methods
### Development of a Solution to Combine the Results of Two Tracking Methods  
The biggest problem in predicting the 3D human skeleton from a single image stream is the depth error. This paper proposes including an additional video stream perpendicular to the first video stream. An illustration of the proposed solution is presented in Figure 1. The red-dotted lines represent the frontal view, while the blue-dotted lines--the right side \begin{table}
\begin{tabular}{c c c c c c} \hline
**Data Set** & **Clothing** & **Exercise** & **Number of Tests** & **Duration, s** & **Number of Frames** & **Resolution** \\ \hline
1 st Data Set & Black & Upper arm addiction & 2 & 10 & 345 & 1920 \(\times\) 1080 \\
1 st Data Set & Black & Standing flexion of the upper arm & 4 & 25 & 786 & 1920 \(\times\) 1080 \\
1 st Data Set & Black & Standing brachial & 3 & 15 & 494 & 1920 \(\times\) 1080 \\
2 nd Data Set & Bright, Dark & Upper arm addiction & 2 & 7, 6 & 192, 206 & 640 \(\times\) 480 \\
2 nd Data Set & Bright, Dark & Standing flexion of the upper arm & 2 & 7, 7 & 222, 219 & 640 \(\times\) 480 \\
2 nd Data Set & Bright, Dark & Standing brachial & 2 & 6, 6 & 168, 182 & 640 \(\times\) 480 \\
2 nd Data Set & Bright, Dark & Standing flexion of both arms & 2 & 7, 6 & 220, 187 & 640 \(\times\) 480 \\ \hline \end{tabular}
\end{table} 
Table 2: Combined video summary of both datasets per person. view of the person. An additional video stream when filming a person from the side will provide additional depth information, with the help of this approach, it will be possible to adjust the coordinates of the predicted three-dimensional human skeleton obtained from the frontal video stream. Before applying the proposed algorithm, it is mandatory to synchronize the video streams. Ideally, both video streams should be shot with the same cameras, ensuring the same frame quality, image resolution, and frame rate. Synchronization can be carried out in two ways: inputting both real-time video streams at the input of the algorithm or using manually synchronized videos. The diagram of the proposed solution algorithm is presented in Figure 2. The steps of the algorithm are described below:  
1. The first video stream films a person from the front parallel to the second video stream;
2. The second video stream filmis a person from the side parallel to the first video stream;
3. The Transformer Encoder processes each video stream independently in a dual-stream encoder-decoder structure, extracting high-dimensional feature embeddings;
4. Feature embeddings from both views are fused, aligning the spatial features for enhanced accuracy;
5. Neural Implicit Fields are utilized to model the non-rigid deformations between the side-view and front-view skeletons;
6. Neural Radiance Fields are employed for depth error correction and depth prediction;
7. An RNN with LSTM cells is used to manage the temporal dynamics of the joint coordinates, reducing the impact of noise;
8. The final 3D human skeleton is reconstructed by integrating information from the processed streams, including the insertion of any missing joint points. Figure 1: An illustration of the proposed solution. In the 3D human skeleton prediction stage, we replaced traditional methods like MediaPipe Pose with a transformer-based architecture, following the recent advancements in Vision Transformers (ViTs) and Spatiotemporal Transformer Networks [27], for capturing intricate dependencies across both spatial and temporal dimensions, yielding a more accurate and robust estimation of 3D skeletons from multi-view video inputs.