# A Projection-Based Augmented Reality System for Medical Applications
## 1 Introduction  
Prior to performing medical surgery, which requires both skills and experience, the surgeons usually perform re-examination of the patient's data to ensure the success of the surgery. There are many methods for visualizing patient data, and one such method involves the use of augmented reality (AR) [1]. The current state of AR uses dedicated devices, such as a head-mounted display (HMD) [2, 3], which places additional burden on the doctor, because the device needs to be worn constantly. In order to alleviate the doctor of the weight of the device, a projection-based single-view AR system is proposed in this paper. Such a system would interactively project a 3D model of cranial blood vessels onto a physical 3D model of a human head, using a 2D video projector. The difficulties of such an AR system includes a movable observer, who may accidentally block the projection or the phantom (occlusion), projecting a 2D image onto 3D surfaces with non-regular curves, and the demand for the accuracy required for medical applications, which can be around only 2.5 mm when using a HMD (3D model to phantom registration error) [4] as an interface. The problem of the movable observer may be resolved by solving for the observer's viewing perspective, and then updating the display accordingly. Once the perspective of the observer is found, the other problems can then be corrected by adjusting the projection to the surface curvatures based on the perspective of the observer. In this investigation, the prototype of such a system was built and then tested. Such an augmented reality system can aid medical professionals by being useful in many applications; includingpre-surgery simulations, the training of medical students, visualizations of lesions within the human body, etc. Our proposed method to calculate for the observer's perspective is through tracking the observer's eye(s). There are two categories of methods for tracking eye movements: contact-based and contactless. The methods most used in contact-based eye-tracking are the Search Coil method [5] and the Electro-oculography method [6]. The Search Coil method uses a specially designed soft contact lens, where an induction coil is added between the two soft lenses, and a fixed magnetic field is added around the eyeball. So, whenever the observer's eyeball rotates, it drives the rotation of the lens, and then the induction coil induces an electromotive force, due to the change of the magnetic flux. The magnitude of the induced electromotive force is used to calculate the deflection angle of the eyeball. However, the disadvantage of this method is that this measurement method is easily affected by the condition of the observer's eyeball, such as fluid secretion, etc., so it is not suitable for long-term application. Furthermore, the soft lens has a double-layer structure, which eventually adversely affects the observer's vision. The method of electro-oculography attaches electrodes to the skin around the observer's eyes to measure the voltage difference between the retina and the cornea. The difference in voltage between the electrodes is then used to calculate the movement pattern of the eyeball. However, the accuracy of this method can be affected by the secretion of keratin from the skin, resulting in unstable electrical signals. This implies that this method is also not suitable for public use. In contact-less methods, the most common methods are the Purkinje image tracking method (also called Dual-Purkinje-Image, DPI) [7], the Pupil Tracking method [8], and the Infra-Red Oculography (IROG) method [9]. The DPI method uses the property that different layers of tissues of the eyeball have different refractive indices, and their reflected images are different. This method can achieve high accuracy, but the cost of the necessary equipment is prohibitively high and, thus, cannot be widely used.