Patent ID: 8395642

Claim:
A method for synthesizing virtual images from a sequence of texture images and a sequence of corresponding depth images, and wherein each depth image includes pixels, and each pixel stores a depth d at a pixel location I(x, y), comprising a processor for performing steps of the method, comprising the steps of: preprocessing, each depth image, to produce a corresponding preprocessed depth image; selecting, a first reference image and a second reference image from the sequence of texture images; applying, to the preprocessed depth images corresponding to the reference images, depth-based 3D warping to produce a corresponding warped images; applying, to the first and second reference images, depth-based histogram matching to produce a corresponding histogram image using the warped images, wherein the depth-based histogram matching reduces illumination differences between the two reference images by adjusting a distribution of pixel intensities; applying, to each histogram image, base plus assistant image blending to produce a corresponding virtual image; and applying, to each virtual image, depth-based in-painting to synthesize a virtual image, and further comprising: constructing, for the first and second reference images, a cumulative reference histogram of pixel intensities; determining a cumulative warped histogram of pixel intensities based on a linear combination of cumulative reference histograms corresponding to each reference image; generating, for the first and second reference images, a mapping function based on the corresponding cumulative reference histograms and cumulative warped histogram; and applying, for each pixel in the first and second reference images, the corresponding mapping function to produce the histogram images.