Our method first trains a regression forest using random RGB features. At test time, the camera pose is initially estimated by the random forest predictions, and then is refined by sparse feature matching. We extend recent advances in scene coordinate regression forests [2] for camera relocalization in RGB-D images to use RGB features, enabling camera relocalization with only a single RGB image at test time. Furthermore, we integrate the random RGB features and sparse feature matching in an efficient and accurate way, broadening the method for fast sports camera calibration in highly dynamic scenes. 2 Methodology Fig.1 shows the pipeline of our method. During training, the scene information is encoded in a random forest. At test time, an initial camera pose is calculated using the random forest predictions with real-time response. A nearest neighbor (NN) image is queried using the initial camera pose. The camera pose is refined by sparse feature matching between the test image and the NN image. In our method, the labels can be any information associated with pixel locations. The random RGB features We use features based on pairwise pixel comparison: where δ is a 2D offset and I(p, c) indicates an RGB pixel lookup in channel c. Our feature does not require depth information, and so is suitable for large scale sports camera calibration. Pose Refinement The 2D-3D correspondences are found by SIFT feature matching between the test image and the NN image. Then, the camera pose P is optimized by minimizing the reprojection error: where x are the feature locations in the image, and X are the correspondent 3D world coordinates associated with the NN image. Our method is evaluated on the 7 Scenes dataset and a new basketball dataset using standard metrics. Table 1 shows quantitative results in 7 Scenes dataset. Fig.2 illustrates qualitative results in the basketball dataset. Experiment results demonstrate the efficacy of our approach, showing superior or on-par performance with the state of the art.
Only RGB pixel comparison features are used in the forest in REF , eliminating the dependency on the depth image at test time.