Patent Document ID: 9117310
Application ID: 13638969
Patent Status: 1

Claim One:
1. A method for creating a virtual viewpoint of a plurality of images of a scene captured by a plurality of cameras, the images comprising a plurality of current frames and previous frames, the method comprising: creating background frames by combining a set of previous frames; creating a plurality of foreground masks by performing segmentation on the plurality of current frames, the segmentation comprising separating foreground objects from background objects using the created background frames; extracting the foreground objects by AND-ing the plurality of current frames with the created plurality of foreground masks; obtaining a position estimate within the scene for at least one foreground object; projecting representations of the created foreground masks for the at least one foreground object onto a two-dimensional ground plane of the scene, wherein the projecting is clipped by using the obtained position estimate to restrict a distance over which the representations of the created foreground masks are projected, thereby reducing a likelihood of false intersections; constructing a three-dimensional depth-box for the at least one foreground object based on an intersection of the projected representations of the foreground masks in the two-dimensional ground plane; rendering the at least one foreground object using a depth-box constrained plane sweeping algorithm on the plurality of current frames, wherein a plane sweeping depth range for the at least one foreground object is constrained by the constructed three-dimensional depth-box for the at least one foreground object; rendering a view-dependent background image by performing homographic transformations on the created background frames; and creating the virtual viewpoint by superimposing the rendered at least one foreground object on the rendered view-dependent background image.