## 2 Basic Theory
### Skeleton Simulation in Multi-Light-Source Scenarios
#### 2.1.1 3D Scenario Reproduction  
The silhouettes of human shadow are projected on the ground. To locate 2D shadow areas corresponding to different human joints, 3D scenario reproduction is launched to extract original 2D silhouette information \(Sh\) from the corresponding images \(Sh_{c}\) captured by camera \(C\). The extraction is launched through the a two step perspective transformation between the ground surface plane \(S(u,v)\) and the camera coordinate plane \(C(x^{\prime},y^{\prime})\). Due to the progressive road engineering and partial patching, the height levels between different road parts are normally discontinuous. Instead of deploying global perspective transformation between the ground surface plane \(S(u,v)\) and image coordinate plane \(Im(x,y)\), this work proposes a block matrix-based projection transformation optimized for uneven road surfaces. Before the 3D scenario reproduction procedure, the projection transformation parameter matrices are calculated once. Then frame-by-frame block matrix-based projection transformations are launched to extract original silhouette information. Plane-to-Plane Projection Transformation  
During the imaging process of a monocular camera, the projection transformation from ground surface plane to image coordinate plane is carried out in two steps. Firstly, each point \((u,v)\) on ground surface plane is projected to corresponding coordinates \((x^{\prime},y^{\prime})\) on the camera coordinate plane. Figure 1: Demo of silhouetted shadow-based skeleton extraction (SSSE) in a multi light source scenario: (**a**) A simulated dual light source scenario; (**b**) Scenario reconstruction. Secondly, a linear transformation happens inside the camera, transforming coordinates \((x^{\prime},y^{\prime})\) to pixel coordinates \((x,y)\) on the image coordinate plane. The projection transformation between the point coordinates \((u,v)\) on the ground surface plane and the corresponding coordinates \((x^{\prime},y^{\prime})\) on the camera coordinate plane is presented as below:  
\[[x^{\prime},y^{\prime},w^{\prime}]=[u,v,1]\;A \tag{1}\]  
\(w^{\prime}\) is a fixed camera internal parameter that affects the linear transformation from the camera coordinate plane to the image coordinate plane. Noticeably, \(A\) is the projection transformation calibration matrix that defines the relationship between ground surface plane and camera coordinate plane. Multiple transformations are taken into consideration in architecting the projection transformation calibration matrix \(A\). * Rotation transformation. Most surveillance cameras are not precisely set up at the horizontal angle which is parallel with the ground surface. The non-horizontal installation attitude brings a rotated field of view. The rotation transformation is introduced to calibrate the rotated field of view, ensuring the calibrated field of view parallel with the ground surface. * Scale transformation. The coordinate system of the ground surface plane is measured in centimeters. However, pixel is the basic unit of measurement in the image coordinate plane. Thus the scale transformation is introduced to bridge two different units of measurement, extracting ground surface plane coordinates from the pixel coordinates. Both rotation transformation and scale transformation are linear transformations. The coordinates of both transformations are combined into the linear parameter matrix \(L\). * Translation transformation. For the image coordinate plane, the origin of the coordinate system is fixed at the bottom left corner. For each captured frame, the origin of the coordinate system on the ground surface plane does not necessarily coincide with the origin of image coordinate plane. The translation transformation is introduced to calibrate the translation between two coordinate systems. The detailed parameters for translation transformation are given in parameter matrix \(T\). * Perspective transformation. Instead of the flat view, a perspective view is captured by each monocular surveillance camera in each frame.