# Background Subtraction for Moving Object Detection in RGBD Data: A Survey
## 3 Methods
Depth keying is adopted to segment moving objects based on depth information, comparing the current depth image with a depth background image (constructed by averaging several ToF-images). MoG is adopted to segment moving objects based on color information. The two segmentations are weighted using two types of reliability measure for depth measurements: the depth variance and the amplitude image of the ToF-camera. The weighted average of the color and depth segmentations is used as matting alpha value for blending foreground and background, while its thresholding (using a user-defined threshold) is used for evaluating moving object segmentation. [PERSON] and [PERSON] [56] use only the depth information provided by a Kinect device to extract the foreground. For each pixel, minimum and maximum depth values \(d_{m}\) and \(d_{M}\) are computed by a set of training images to form a background model. For a new frame, each pixel is compared against the background model, and those pixels which lie outside the range [\(d_{m}-1\), \(d_{M}+1\)] are considered foreground. In [9], [PERSON] et al. present a human detection and tracking system for a smart environment application. Background subtraction is applied only on the depth images as frame-by-frame difference, assisted by a clustering algorithm that checks the depth continuity of pixels in the neighborhood of foreground pixels. Once the object has been located in the image, visual features are extracted from the RGB image and are then used for tracking the object in successive frames. In the surveillance system based on the Kinect proposed by [PERSON] et al. [57], a per pixel background subtraction technique is presented. The authors propose a background model based on a four-dimensional Gaussian distribution (using color and depth features). Then, user and object candidate regions are detected and recognized using robust statistical approaches. In the gesture recognition system presented by [PERSON] et al. [10], the foreground objects are extracted by the depth data using frame difference. [PERSON] et al. (2013) [59] refine ViBe segmentation of the color data by adding to the achieved foreground mask a compensation factor computed based on the color and depth data obtained by a stereo camera. In the object detection system presented by [PERSON] et al. [60], background subtraction is achieved by single Gaussian modeling of the depth information provided by a Kinect sensor. [PERSON] et al. [58] adopt Codebook as background model and consider data captured by Kinect cameras. They analyze two approaches that differ in the depth integration method: the four-dimensional Codebook (CB4D) considers merely depth as a fourth channel of the background model, while the Depth-Extended Codebook (DECB) adds a joint RGBD fusion method directly into the model. They proved that the latter achieves better results than the former. In [15], the authors consider stereo disparity data, besides color. To get the best of color and depth features, they extend the DECB algorithm through a post-processing stage for mask fusion (DECB-LF), based on morphological reconstruction using the output of the color-based algorithm. [PERSON] et al. [61] adopt two background models for depth data, separating valid values (modeled by a single Gaussian model) and invalid values (holes). The Gaussian mean is updated to the maximum valid value, while the standard deviation follows a quadratic relationship with respect to the depth. This leads to a depth-dependent foreground/background threshold that enables the model to adapt to the non-uniform noise of range images automatically. In [17], [PERSON] and [PERSON] propose an approach, named CL\({}_{W}\), based on a combination of color and depth classifiers (CL\({}_{C}\) and CL\({}_{D}\)) and the adoption of the MoG model. The combination of classifiers is based on a weighted average that allows to adaptively modifying the support of each classifier in the ensemble by considering foreground detections in the previous frames and the depth and color edges.