Depth data should use CV_INTER_NN interpolation.

When using freenect backend, raw depth data is not interpolatable. Approximately speaking, 0~1023 represents normal values, while 2047 represents invalid depth. So liner interpolation between 2047 and other values is meaningless.