## 3 Methodology
### Feature Extraction and Image Matching  
Feature points are extracted once in the pattern image (offline) and in every camera frame during runtime. The following combinations of detectors and binary descriptors are dealt with, within the process of image matching in ModAR: (i) the oriented FAST detector (i.e., FAST with an orientation assignment mechanism developed by the authors of  
Figure 1: Definition of the four coordinates of the pattern image in the world coordinate system.  
ORB) combined with the ORB descriptor; and (ii) the AGAST detector combined with the BRISK descriptor. ORB uses the oriented FAST detector, while the BRISK descriptor uses the AGAST operator. These combinations of detectors--descriptors have been proposed by the authors of these algorithms, i.e., [27; 28], respectively.  
The stage of finding correspondences between the extracted feature points is performed by matching their descriptors using a similarity measure. Whereas in case of floating-point descriptors, this measure is usually the Euclidean distance between the descriptors, in the case of binary descriptors, the Hamming distance is used. It is calculated as the number of positions at which the binary values of the descriptor vectors are different. However, the minimization of the Hamming distance does not suffice, and thus it is combined with other techniques in order to reject incorrect correspondences. In our work, a cross-check test is implemented. According to this test, the descriptor of each feature in the pattern image is compared with the descriptor of every feature in the real-world frame and some candidate correspondences are returned. These are additionally checked through reverse comparison. Thus, two feature points, \(i\) and \(j\), are matched if the nearest neighbor of the descriptor of point \(i\) in the pattern image is the descriptor of point \(j\) in the camera frame and reversely if the nearest neighbor of the descriptor of point \(j\) in the camera frame is the descriptor of point \(i\) in the pattern image. However, after cross testing, many outliers still remain. Therefore, a maximum accepted distance is defined as a threshold (equal to 1/3 of the descriptor length in bits) and the correspondences are rejected if the Hamming distance between the descriptors of the matched feature points is above this threshold. In this way, a significant number of outliers is removed, although a few incorrect matches are not detected. These are rejected using the RANSAC (random sample consensus) algorithm [49]. The latter is applied if at least 8 matches are detected. Otherwise, the real-world scene is not augmented, as it is considered that the pattern cannot be recognized in the frame. RANSAC calculates the parameters of a mathematical model using a data set, which may contain many errors, relying on the use of minimum data. In the case of ModAR, the dataset consists of the remaining correspondences and the model is the geometric relation between the pattern image and the image of the planar pattern object in the camera frame, i.e., a 2D projective transformation (homography). The correspondences that verify the homography computed by RANSAC constitute the inliers.