Patent ID: 7286143

Claim:
A computer-implemented process for generating an interactive viewpoint video, comprising using a computer to perform the following process actions: inputting three or more synchronized video streams each depicting a portion of the same scene captured from different viewpoints, wherein said viewpoints form an array; inputting calibration data defining geometric and photometric parameters associated with each video stream; and for each group of contemporaneous frames from the synchronized video streams, generating a 3D reconstruction of the scene, using the reconstruction to compute a disparity map for each frame in the group of contemporaneous frames, and for each frame in the group of contemporaneous frames, identifying areas of significant depth discontinuities based on its disparity map, identifying background pixel information and foreground pixel information for pixels in said areas of significant depth discontinuities, generating a main layer comprising pixel information associated with areas in the frame that do not exhibit depth discontinuities exceeding a prescribed threshold and the background pixel information from areas having depth discontinuities above the threshold, generating a boundary layer comprising the foreground pixel information associated with areas having depth discontinuities that exceed the threshold, to produce a layered representation for the frame under consideration, and storing the main layer and boundary layer in a computer readable medium that has a physical form.