A computer-implemented method for generating depth data for a six degree of freedom, 6DoF, video of a scene, the method comprising: obtaining a first set of images of the scene; generating a first set of depth components based on the first set of images, wherein each depth component comprises a plurality of depth values; analyzing the first set of depth components to determine completeness of the depth components; obtaining a second set of images of the scene, wherein each set of images are obtained from a group of cameras among all the available cameras imaging the scene, which are fixed; and generating a second set of depth components based on the second set of images, wherein: if the analysis determines the first set of depth components to be overcomplete, the number of depth components in the second set is selected to be smaller than the number of depth components in the first set, wherein the overcomplete first set of depth components has more than enough depth information of the scene to render a 6DoF frame, and if the analysis determines the first set of depth components to be undercomplete, the number of depth components in the second set is selected to be larger than the number of depth components in the first set, wherein a portion of the depth information of the scene is missing from the undercomplete first set of depth components to render a frame of the 6DoF scene. obtaining a first set of images of the scene; generating a first set of depth components based on the first set of images, wherein each depth component comprises a plurality of depth values; analyzing the first set of depth components to determine completeness of the depth components; obtaining a second set of images of the scene, wherein each set of images are obtained from a group of cameras among all the available cameras imaging the scene, which are fixed; and generating a second set of depth components based on the second set of images, wherein: if the analysis determines the first set of depth components to be overcomplete, the number of depth components in the second set is selected to be smaller than the number of depth components in the first set, wherein the overcomplete first set of depth components has more than enough depth information of the scene to render a 6DoF frame, and if the analysis determines the first set of depth components to be undercomplete, the number of depth components in the second set is selected to be larger than the number of depth components in the first set, wherein a portion of the depth information of the scene is missing from the undercomplete first set of depth components to render a frame of the 6DoF scene. if the analysis determines the first set of depth components to be overcomplete, the number of depth components in the second set is selected to be smaller than the number of depth components in the first set, wherein the overcomplete first set of depth components has more than enough depth information of the scene to render a 6DoF frame, and if the analysis determines the first set of depth components to be undercomplete, the number of depth components in the second set is selected to be larger than the number of depth components in the first set, wherein a portion of the depth information of the scene is missing from the undercomplete first set of depth components to render a frame of the 6DoF scene.