## 2 Basic Theory
### Skeleton Simulation in Multi-Light-Source Scenarios
#### 2.1.1 3D Scenario Reproduction
Block matrix-based projection transformation is deployed based on the parallel computation of sub-transformations illustrated in Equation (13). The deployment algorithm of a sub-transformation is illustrated in Algorithm 1. ```
Input:\(M_{sub}^{S}=\{(u_{i},v_{i})\,,i=1,2,3,4\}\): coordinates set of marker positions for sub-block \(Sq_{sub}\); \(M_{sub}^{Im}=\{(x_{i},y_{i})\,,i=1,2,3,4\}\):corresponding pixel coordinates set of \(M_{sub}^{S}\) on image coordinate plane \(Im(x,y)\); \((x,y)\): image coordinates of captured pixel in human shadow silhouette Output:\((u,v)\): corresponding real-world coordinates of \((x,y)\)
1 foreach\(Sq_{sub}\)do
2\([\Delta x_{1},\Delta x_{2},\Delta y_{1},\Delta y_{2}]=[x_{2}-x_{3},x_{4}-x_{3},y _{2}-y_{3},y_{4}-y_{3}]\)
3\([\Delta x_{3},\Delta y_{3}]=[x_{1}-x_{2}+x_{3}-x_{4},y_{1}-y_{2}+y_{3}-y_{4}]\)
4\(T_{sub}=[ \begin{array}{cc}a_{31}&a_{32}\\ a_{31}&a_{23}\\ \end{array} ]^{T}=[ \begin{array}{cc}x_{1}&y_{1}\\ \end{array} ]^{T}\)
5\(P_{sub}=[ \begin{array}{cc}a_{13}&a_{23}\\ a_{13}&a_{23}\\ \end{array} ]^{T}=[\begin{array}{cc}\frac{\Delta x_{3}\Delta y_{2}-\Delta x_{ 2}\Delta y_{3}}{\Delta x_{1}\Delta y_{2}-\Delta x_{2}\Delta y_{1}},\frac{ \Delta x_{1}\Delta y_{3}-\Delta x_{3}\Delta y_{1}}{\Delta x_{1}\Delta y_{2}- \Delta x_{2}\Delta y_{1}}]^{T}\\ L_{sub}=\left[ \begin{array}{cc}a_{11}&a_{12}\\ a_{21}&a_{22}\\ \end{array} \right]^{T}=\left[ \begin{array}{cc}(x_{2}-x_{1}+a_{12}x_{2})&(y_ {2}-y_{1}+a_{13}y_{2})\\ (x_{4}-x_{1}+a_{12}x_{3})&(y_{4}-y_{1}+a_{23}y_{4})\\ \end{array} \right]^{T}\]
6\(A_{sub}=\left[ \begin{array}{cc}L_{sub}&P_{sub}\\ T_{sub}^{T}&1\\ \end{array} \right]\);
7 end foreach\((u,v)^{T}=f((x,y)^{T},A_{sub})\)
```  
**Algorithm 1**Block matrix based projection transformation deployment Algorithm  
For each sub-block area \(Sq_{sub}\), a distinctive sub-transformation thread is launched based on the specific calibration matrix \(A_{sub}\). The parallel computation of block matrix-based projection transformation contains multiple sub-transformation threads. For the simplicity of the parallel computation presentation, \(A_{mat}\) is introduced as the collection of all calibration sub-matrices \(\{A_{sub}\}\) for different sub-blocks. The overall transformation is simplified as Equation (14). \[(u,v)^{T}=F((x,y)^{T}\,,A_{mat}) \tag{14}\]  
Based on Equation (14), the real-world coordinates \((u,v)\) of human shadow silhouette \(Sh\) can be extracted from corresponding pixel coordinates \((x^{\prime},y^{\prime})\in Sh_{c}\) captured by a monocular camera. The block matrix-based projection transformation between the captured human shadow silhouette \(Sh_{c}\) and the corresponding real-world shadow silhouette \(Sh\) is illustrated in Equation (15). \[Sh=F(Sh_{c},A_{mat}) \tag{15}\]  
The benefits of the block matrix based projection transformation are obvious:  
* The positions of markers can be reused to simplify the scenario set up. For a scenario containing a \(m\times n\) square meter area, the number of markers is reduced from \((4\times m\times n)\) to \((m+1)\times(n+1)\). * Parallel sub-transformations on different sub-blocks can be processed synchronously to accelerate the overall projection transformation procedure. * Only when the position of camera is moved or the ground surfaced is repaved, will partial recalibration work be necessary for the affected sub-block \(Sq_{sub}\). Overall, all parameter matrices \(A_{sub}\) for different sub-blocks only need to be calculated once. Then all pixel coordinates in video frames can be transformed into the real-world coordinates on the ground surface plane. The block matrix-based structure also simplifies the parameter maintenance procedure when changes occur in the scenario.