For two consecutive frames in a video, we identify which pixels in the first frame become occluded in the second. Such general-purpose detection of occlusion regions is difficult and important because one-to-one correspondence of imaged scene points is needed for many tracking, video segmentation, and reconstruction algorithms. Our hypothesis is that an effective trained occlusion detector can be generated on the basis of i) a broad spectrum of visual features, and ii) representative but synthetic training sequences. By using a Random Forest based framework for feature selection and training, we found that the proposed feature set was sufficient to frequently assign a high probability of occlusion to just the pixels that were indeed becoming occluded. Our extensive experiments on many sequences support this finding, and while accuracy is certainly still scenedependent, the proposed classifier could be a useful preprocessing step to exploit temporal information in video.
Humayun et al. REF determine occlusions post-hoc by training a classifier on a broad spectrum of visual features and precomputed optical flow.
Learning to find occlusion regions
CVPR 2011
CVPR 2011
Computer Science