# Off-Line Camera-Based Calibration for Optical See-Through Head-Mounted Displays
## 2 Related Works  
Research on how to achieve correct calibration of OST displays has been conducted over many years. In 2017, [PERSON] et al. [7] presented a comprehensive survey of all the calibration procedures proposed up to that time. In their work, the authors provided also useful insights into the fundamentals of calibration techniques, grouping them into three main categories: manual methods, semi-automatic methods, and automatic methods. We here provide a more focused overview of the different techniques in which the calibration workflow explicitly relies on an off-line phase. In 2002, [PERSON] et al. [12] proposed a simplified version of the SPAAM (a two phase SPAAM) in which the calibration process was split into two phases: an off-line stage that involves the estimation of the fixed projection parameters and a second on-line stage that aims to update the existing projection matrix partially by applying a 2D image warping of the screen image that includes scaling and shift. A vision based robust calibration (ViRC) method was proposed in 2013 [13]. The method clearly distinguishes two types of parameters: the ones estimated off-line that are associated with the device dependent parameters and that are based on an approximate position of the user's eye and those measured on-line that are instead related to the actual position of the user's eye (4 DoF user dependent parameters). These latter parameters are refined through a perspective-n-point algorithm whose inputs are: the image coordinates of a cross-hair cursor displayed on the see-through screen at different positions and the 3D coordinates of a fiducial marker that the user must visually align to such a cursor. The 3D coordinates of the marker are determined by querying a tracking camera attached to the HMD (i.e., adopting an inside-out tracking approach). In 2004, [PERSON] et al. [14] presented display relative calibration (DRC), a two phase camera based calibration method. In the first phase of the method, the authors replace the user's eye with a camera and use a mechanical jig to determine the projection parameters of the display system. In the second phase, the eye-display parameters are optimized on the position of the user's eye(s) by means of a SPAAM-like procedure. In 2005, [PERSON] et al. [15] presented a camera based method for calibrating a medical binocular OST HMD, the VarioscopeTM M5. The calibration system uses a precision spindle moving a mobile calibration grid via a stepping motor and a tracker probe to localize the moving grid in space with respect to the HMD (i.e., adopting an outside-in tracking approach). However, the proposed method does not have any calibration refinement step that accounts for the user's eye position. A similar camera based procedure was presented by [PERSON] et al. [16] that also employs an inside-out tracking mechanism. After the calibration, the authors did not include any user centered refinement step to account for the eye position, since in their method: \"the camera remains stationary by design, so we have constrained our method to minimize for only one camera pose\" (page 6). Overall, the authors declared to have achieved an average virtual-to-real alignment accuracy of about two pixels for three camera positions different from the calibration one. In 2014, [PERSON] and [PERSON] [8; 17] proposed an automatic method with inside-out tracking that employed an eye tracker to measure the position of the user's eye. The relative pose between the external camera and the eye tracking camera was predetermined during an off-line calibration session. The dynamic tracking of the user's eye was used for continuously refining the eye-display projection parameters, whose initial values once again must be computed off-line through a standard SPAAM-like method or a camera based procedure. A similar approach was proposed in 2015 by [PERSON] et al. [11; 18].