# Off-Line Camera-Based Calibration for Optical See-Through Head-Mounted Displays
## 3 Methods
### Formulation of the Calibration Procedure
The intrinsic matrix for a different viewpoint position p\({}_{\text{i}}\) is:  
\[\begin{smallmatrix}\text{off-E}\text{K}_{\text{p}}=\begin{bmatrix}\frac{d^{ \text{C}-1\pi}}{d^{\text{E}-\pi}}&0&\frac{x_{\text{C}}^{\text{E}}}{d^{\text{E} -\pi}}\\ 0&\frac{d^{\text{C}-\pi}}{d  
where \({}^{C_{\rm{i}}}_{\rm{c}}\mathbf{t}=[x^{\prime},y^{\prime},z^{\prime}]\) is the translation from the old viewpoint position to the new viewpoint position and where \(d^{C_{\rm{i}}\cdot\pi}=d^{C\cdot\pi}+z^{\prime}\).  
4.4 Relation between the Optical Properties of the OST Display and Intrinsic Matrix of the Eye-Display Model  
In any HMD, the role of the collimation optics is to display the microdisplay image so that it appears magnified at a comfortable viewing distance [26]. Specifically, the imaging plane of many consumer level OST HMDs is at infinity (i.e., at a very far distance), and this happens if the microdisplay is located in close proximity of the focal point of the collimation optics. In this case, the light rays coming from each pixel of the microdisplay are arranged in a parallel pattern, and the virtual image perceived by the user, to a first approximation, is not subjected to any shift or scaling effect for different positions of the user's eye within the display eye-motion-box area.  
Thereby, for those OST displays, the position of the user's eye does not influence the intrinsic matrix of the off-axis eye-display pinhole camera model \({}^{\rm{off-E}}_{\rm{P}}\), and this implies that the intrinsic linear parameters of the eye-display can be considered as parallax-free:  
\[{}^{\rm{off-E}}_{\rm{P}}{\rm{=}}^{\rm{off-E}}_{\rm{P}}{\rm{=}}^{\rm{off-E}}_{ \rm{P}}{\rm{=}}^{\rm{off-E}}_{\rm{P}}{\rm{=}}^{\rm{off-E}}_{\rm{P}}{\rm{=}}^{ \rm{off  
mechanism (Figure 4). The LC panels were placed in front of the optical combiners of the HMD, and they could be electronically controlled so as to modify the transparency of the see-through display and switch from see-through state to occluded state. We used the occluded modality for the second step of the calibration, in which we projected a calibration checkerboard onto the imaging plane of the see-through display.  
Figure 5 shows the experimental setup. In our procedure, we used a single camera inside-out tracking mechanism. The tracking cameras had 109\({}^{\circ}\) diagonal FOV, which, was associated with a 1280 \(\times\) 720 image resolution, yielding \(\approx\) 4.8 arcmin/pixel angular resolution. As we will discuss in Section 5, this particular feature enabled us to have a reference limit in the evaluation of the overall calibration accuracy.  
Figure 4: The custom-made hybrid video-optical see-through head mounted display used in the experimental session. 1\(\rightarrow\)Pair of stereo cameras for the inside-out tracking. 2\(\rightarrow\)Pair of liquid-crystal optical shutters to drive the switching mechanism between occluded to optical see-through state. 3\(\rightarrow\)Plastic shell that incorporates all the components of the optical see-through visor. 4\(\rightarrow\)Optical combiner of the optical see-through visor.  
Figure 5: Experimental environment. 1\(\rightarrow\)Validation checkerboard. 2\(\rightarrow\)Optical see-through head mounted display. 3\(\rightarrow\)Tracking camera. 4\(\rightarrow\)Optical combiner. 5\(\rightarrow\)3D printed mounting template with the embedded viewpoint camera. 6\(\rightarrow\)Laptop where the calibration routines and the AR application run.  
In our tests, we performed the calibration on the right display. Therefore, we placed the viewpoint camera behind the right display at approximately the eye-relief distance from the flat combiner (i.e., \(\approx 30\) mm).  
The viewpoint camera was a SONY FCB-MA130, which had a \(1/2.45^{\prime\prime}\) CMOS sensor, a \(1280\times 720\) resolution, and a \(59^{\circ}\) diagonal FOV, which corresponded to \(\approx 2.67\) arcmin/pixel.  
Both the HMD and the validation checkerboard were locked to two rigid and adjustable holders. The validation checkerboard was a standard \(7\times 4\) OpenCV calibration checkerboard with a square size of 30 mm. The checkerboard was placed at approximately 650 mm of distance from the viewpoint camera. We attached the viewpoint camera to a 3D printed mounting template. The mounting template was equipped with fixing holes for placing the camera in eight pre-set positions radially arranged within the eye-motion-box of the see-through display (Figure 6). The template and the camera were both anchored to the translation bar of the HMD holder.  
We calibrated the display for the left position (reference position), and then, we computed the intrinsic matrix of the eye-display for the remaining seven positions using Equation (17), where \(d^{C\to\pi}\) is given by Equation (9) and where the translation vector \(\ce{c_{i}}{c_{i}}\) is computed using two stereo calibrations between the tracking camera (fixed) and the viewpoint camera in the reference and in the new position:  
\[\ce{c_{i}}{t}=[x^{\prime},y^{\prime},z^{\prime}]=\ce{c_{i}}{t}-\ce{c_{i}}{t} \tag{19}\]