## 3 Pre-Deformed Projection Content with Simulation-Based Fitting Method
### Homography-Based Model of DLP-Based AR System  
As a projector-camera system with the elements of DLP, CCD, screen, and user, as shown in Figure 1, the image data is captured, projected, displayed, and perceived. To obtain the goal of letting the projection content equal the desired image in the warping AR system, the 3-dof (pan, tilt, screw) misalignment of modules has been addressed by pre-deforming the desired images. The perspective transform of images from the coordinate system \(\mathbf{O}^{i}\) to \(\mathbf{O}^{j}\) is modeled by [25],  
\[\mathbf{p_{O^{i}}}=\mathbf{H}^{i-j}\cdot\mathbf{p}^{\prime}_{\mathbf{O}^{i}} \tag{1}\]  
where \(\mathbf{p_{O^{i}}}=\left[x,y,1\right]^{\prime}_{\mathbf{O}^{i}}\) is a homogeneous coordinate of image point lying on the plane \(z=0\) in \(\mathbf{O}^{i}\); and \(\mathbf{O}^{i}\), \(i=C,P,S\) is the coordinate system of CCD, DLP, and screen, respectively; \(\mathbf{H}^{i-j}\) is the 3-order homography matrix from \(\mathbf{p_{O^{i}}}\) to \(\mathbf{p_{O^{i}}}\).  
When the real objects lie on the screen of size \(21\ \mathrm{cm}\times 28\ \mathrm{cm}\), \(\mathbf{I}^{5}=\left\{\mathbf{p_{O^{5}}}\right\}_{0\leq x\leq 28^{\prime}}^{0 \leq y\leq 21}\) the captured CCD image of size \(360\times 640\ \mathrm{pixels}\), and \(\mathbf{I}^{C}=\left\{\mathbf{p_{O^{c}}}\right\}_{x=1:360}^{y=1:640}\) is rewritten by Equation (1) using the homography matrix \(\mathbf{H}^{C\to 5}\) as,  
\[\mathbf{p_{O^{5}}}=\mathbf{H}^{C\to 5}\cdot\mathbf{p}^{\prime}_{O^{c}} \tag{2}\]  
Figure 1: The configuration of the proposed AR experimental platform consisting of a digital light processing (DLP), Beaglebone Black (BB), and CCD modules: (**a**) schematic diagram for illustrating identification, projection, and tracking; (**b**) geometric parameter of the setup, where the DLP with a distance/width (D/W) ratio of 1.6 is configured at the projection distance (D) of 44.8 cm; (**c**) an image of the prototype with three coordinate systems of projector, camera, and real world, [OP, OC, OS].  
Similarly, \(\mathbf{H}^{C\to P}\) and \(\mathbf{H}^{P\to S}\) denote the homography matrix of keystone correction and DLP projection in the AR system, respectively. Therefore, the goal of projecting the desired image \(\mathbf{I}^{D}\) onto the real image \(\mathbf{I}^{S}\) at right positions and scales is defined as the optimization of \(\mathbf{H}^{C\to P}\),  
\[\mathbf{H}^{*}=\min_{\mathbf{H}^{C\to P}}(|\boldsymbol{f}(\mathbf{I}^{P}( \mathbf{I}^{D}))-\boldsymbol{f}(\mathbf{I}^{S})|^{2})=\min_{\mathbf{H}^{C\to P }}(|\boldsymbol{f}(\mathbf{H}^{P\to S}\cdot(\mathbf{H}^{C\to P}\cdot \mathbf{I}^{D}))-\boldsymbol{f}(\mathbf{I}^{S})|^{2}) \tag{3}\]  
where \(\boldsymbol{f}(\cdot)\) is the function for extracting the coordinate of the feature points. Additionally, \(\mathbf{I}^{D}\) is the finite element method (FEM)-based plots of physical parameters, such as temperature T, with the indicative texts and markers based on the geometry parameter from \(\mathbf{I}^{S}\),  
\[\mathbf{I}^{D}=\text{FEM}\Big{(}\text{T};\mathbf{I}^{S}\Big{)}=\text{FEM} \Big{(}\text{T};\mathbf{H}^{C\to S}\cdot\mathbf{I}^{C}\Big{)} \tag{4}\]  
Here, \(\text{FEM}(\cdot)\) denotes the FEM model with output of plots under the boundary condition \(\mathbf{I}^{S}\), such as coordinates and shapes of the experimental parts, and \(\mathbf{I}^{S}\) is estimated from \(\mathbf{I}^{C}\) by Equation (2).  
According to Equations (1)-(4), the model of the proposed AR system is determined by homography matrix sets \(\big{\{}\mathbf{H}^{S\to C},\mathbf{H}^{P\to S},\mathbf{H}^{*}\big{\}}\) with the input of real image \(\mathbf{I}^{S}\) and output of the desired image \(\mathbf{I}^{D}\) related with the user-demanded model \(\text{FEM}(\cdot)\) in the AR experiment. Furthermore, the pre-deformed image \(\mathbf{I}^{F}=\mathbf{H}^{*}\cdot\mathbf{I}^{D}\) is projected onto the screen in real-time to obtain the superimposition of real objects with virtual assets when users operate the AR experimental platform.