# A Novel Real-Time Match-Moving Method with HoloLens
## 3 Proposed System
### Evaluation of the Match-Moving Results  
In order to evaluate the match-moving results in both the proposed and Boujou methods, we performed experiments by placing a cardboard cube marked with a checkerboard pattern in a 2 m wide window and 1.5 m away from the camera-HoloLens system. The scene was captured for a duration of 45 s at a rate of 30 fps. Then, we recoded the video of the cardboard cube in two scenarios with different backgrounds and motion blur using the RGB camera mounted with HoloLens. In the first scenario, the cardboard cube was captured by placing it in front of a white background with cross-patterns in a regular manner. The cross-patterns were pasted on the background wall to facilitate the feature point method, which requires a maximal number of features in the scene to work properly. In the second scenario, we recorded the video of the cardboard cube without cross patterns and by moving the camera fast in order to create some motion blur in the recorded video. Motion blur causes the failure of feature point identification from the recorded video frames, which affects the Boujou method. The camera was moved from the center to the right and then left in a specific motion. We paused and resumed the camera motion at four points as marked in Figure 8. The camera was moved smoothly in a regular manner between positions 1 and 2, and then, from position 2 to 3, it was moved swiftly to create motion blur in the recorded video frames.  
The videos recorded in both scenarios were then processed using the Boujou method and the proposed method to synthesize the virtual-real images. First, the videos recorded by RGB camera were imported into Boujou software without any position information acquired from the HoloLens. The Boujou method uses a process to extract the feature points from the recorded video frames. In the next step, these feature points were processed to determine the \"camera solve\" or in other words the external parameters of the RGB camera, and this information was exported into 3 ds Max to create a virtual camera. However, the proposed method can obtain the position information from HoloLens at the same time as recording the videos and store it in a Max script. This information from the HoloLens in the form of a Max script is used to synthesize the virtual-real images in 3 ds Max. The synthesizing of virtual-real images was done by creating a virtual camera in 3 ds Max. The path of the virtual camera was generated from the Max script obtained from the HoloLens.  
In order to test the performance of the proposed system and Boujou under the circumstances of our experimental setup, we created a virtual scene exactly the same as the real scene in 3 ds Max. The cardboard cube in the virtual scene is identical to that in the real scene. The point where the camera was placed is considered as the origin in both the real and virtual scenes. In addition, the distance between the camera and the object was exactly the same in both scenes. This compares the concordance rate between the real object and virtual object based on the mismatch pixels in a specific range using  
Figure 7: Experimental setup for accuracy evaluation of camera trajectory. (**a**) experimental space in normal condition; (**b**) experimental space with occlusion. Oculus was used to record the ground truth for comparing camera trajectory with HoloLens and RGB camera.  
different images. In addition, it compares the external parameter of camera position obtained from each match-moving method.