# Motion-Induced Phase Error Compensation Using Three-Stream Neural Networks
## 2 Principle
### Dataset Establishment in Virtual FPP System  
Deep learning-based FPP methods usually require a dataset that consists of numerous fringe patterns modulated by different objects for high accuracy network prediction. However, the dataset collection in the real world will be limited in the hardware, number of objects, and other uncontrolled factors. To solve this problem, a dataset establishment flow in a virtual FPP system for dynamic scenes is proposed to produce synthetic data conveniently and flexibly. The availability of large image datasets has become a major bottleneck in deep learning-based techniques. Recently, synthetic dataset generation methods using computer graphics show their potential applications in various industrial use-cases, such as viewpoint estimation [28], image classification [29], single-shot FPP [30; 31], etc. In this section, our goal is to establish one-to-one mapping between the actual and virtual FPP system, and the method mentioned in [30] is extended from static scene to dynamic scene. A free and open source 3D creation suite named Blender [32] is introduced to build virtual FPP system and training dataset. Meanwhile, various 3D models including sculpture, toy, and industrial component in Thingi 10K dataset [33] are selected as the objects to be measured. The first step is to determine the positional relationship between camera and projector and their own inherent parameters, i.e., intrinsic and extrinsic matrixes. The experimental setup of FPP system and the schematic diagram of virtual FPP system are shown in Figure 2a,b, respectively. The actual experimental setup shown in Figure 2b provides the referenced structural parameters for the virtual one shown in Figure 2b and will be used in the performance evaluation of this work in Section 3. According to the traditional pinhole camera model, the image capture process from a point in the world coordinate to a pixel of the image plane can be described as:  
\[\lambda^{c}\left[ \begin{array}{c}u^{c}\\ v^{c}\\ 1\end{array} \right]=K^{c}[R^{c}|T^{c}]\left[ \begin{array}{c}X^{w}\\ Y^{w}\\ Z^{w}\\ 1\end{array} \right]. \tag{11}\]  
where the superscript \(c\) represents the camera imaging system. \(\lambda\) is a scaling factor. (\(u\),\(v\)) is the pixel coordinate on the image plane. \(K\) is the intrinsic matrix. \(R\) and \(T\) denote the rotation matrix and translation vector from the world coordinate (\(X_{w}\),\(Y_{w}\),\(Z_{w}\)) to camera coordinate in extrinsic matrix, respectively. \(K\), \(R\), and \(T\) can be further rewritten as:  
\[K^{c}=\left[ \begin{array}{ccc}f^{c}_{v}&s^{c}&u^{c}_{0}\\ 0&f^{c}_{v}&v^{c}_{0}\\ 0&0&1\end{array} \right], \tag{12}\]\[R^{c}= \begin{bmatrix}r_{11}^{c}&r_{12}^{c}&r_{13}^{c}\\ r_{21}^{c}&r_{22}^{c}&r_{23}^{c}\\ r_{31}^{c}&r_{32}^{c}&r_{33}^{c}\end{bmatrix} ,T^{c}= \begin{bmatrix}r_{1}^{c}\\ r_{2}^{c}\\ r_{3}^{c}\end{bmatrix} . \tag{13}\]  
\(f_{u}f_{v}\) are the focal length along \(u\) and \(v\) directions. \(s\) represents the axis skew. (\(u_{0}\),\(v_{0}\)) is the location of the principle point. \(R_{ij}\) and \(t_{i}\) are the parameters in the corresponding location of \(R\) and \(T\), respectively. Since the projector can be regarded as an inverse camera, the image projection process can be written as:  
\[\lambda^{p}\left[ \begin{array}{c}u^{p}\\ v^{p}\\ 1\end{array} \right]=K^{p}[R^{p}|T^{p}]\left[ \begin{array}{c}X^{w}\\ Y^{w}\\ Z^{w}\\ 1\end{array} \right]. \tag{14}\]  
where the superscript \(p\) represents the projector imaging system. Assume that the projector coordinate coincides with world coordinate, and its origin is (0,0,0)\({}^{T}\), the mapping from the origin of the camera coordinate to that of the projector can be obtained:  
\[\left[ \begin{array}{c}0\\ 0\\ 0\end{array} \right]=R^{c}\left[ \begin{array}{c}X_{0}^{c}\\ Y_{0}^{c}\\ Z_{0}^{c}\end{array} \right]+T^{c}. \tag{15}\]  
The camera location can be determined by a rearranging of Equation (15):  
\[\left[ \begin{array}{c}X_{0}^{c}\\ Y_{0}^{c}\\ Z_{0}^{c}\end{array} \right]=-(R^{c})^{T}T^{c}.