# AR in the Architecture Domain: State of the Art
## 5 AR System
### Tracking and Registration  
Tracking means following the user's point of view concerning a scene--real or virtual. In AR, this activity primarily aims to acquire data relevant to superimpose the user's view with the virtual one, positioning the virtual model in real space. The tracking registration, carried out mainly thanks to the camera, is essential for the correct real-time operation of AR. The precision and speed of acquisition fix the update rate and the superposition accuracy of the virtual model on the real one. The first step, defined as calibration, defines the orientation system in which the camera records the first scene. After this \"time zero\", the following steps are referred to as tracking activity, updating the initial reference system according to the system changes, and calculating the roto-translation matrix between the camera center and the real scene's position. This operation allows the correct alignment of the real and virtual models, called registration [1,11]. The elements in the scene that enable tracking can be artificial or natural. Artificial refers to all those elements explicitly created to support the camera in recognizing the scene. These include markers (standard or infrared) and tracking sensors external to the camera (electromagnetic or hybrid tracking). Natural elements are elements already present in the scene, whose shapes are recognized and used for tracking. In this case, therefore, no additional elements are needed, and we talk about markerless tracking. The automatic recognition of targets or shapes in space is strongly related to the point of view. Extreme perspective distortion of the subject through the device can cause failure in recognition. This limitation happens more frequently for flat markers than in recognition of 3D shapes. The assumption of an inconsistent point of view may cause the failure of the recognition process or a slowdown in processing, in case the camera moves quickly. For this reason, the choice of targets and their position in space must be carefully evaluated in the design phase [41,42,112], as well as the tracking system, still suggesting ubiquitous hybrid tracking systems [113]. The data fusion between different sensors allows increasing the device's capacity for scene recognition, solving specific bottlenecks in scene recognition at a large scale. Some problems arise if the GPS coverage is low, so tracking by location is not suitable, or if the geometric characteristics of the artifact do not allow easy recognition. A primary solution can be the hybrid image recognition technique, merging image-based markers referring to the monument pictures and QR codes [114]. [PERSON] et al. [115] suggest integrating an edge-based tracker with gyroscopic measurements and a back store of online reference frame selection to re-initialize automatically after dynamic occlusions or failures. Some tracking systems enlarge users' participation, activating co-design actions. In this sense, the Urban CoBuilder project shows how citizens can collaborate on urban environment projects using AR. The use of multiple markers and smartphone gyroscope sensors activates co-design strategies at an urban scale [116]. Some devices are based on a markerless approach, facilitating the outdoor experience. For example, the HMD HoloLens (Microsoft) has inbuilt processing units to handle all computational needs, offering a toolkit compatible with the Unity game engine. The user experience offered by HoloLens pushes a large-scale visualization, suggesting an immersive and natural feel of data visualization [117]. This capability also facilitates cultural heritage experiences, merging all the resources required for tracking, computation, and displaying virtual objects and audiovisual elements in the device [90].