Patent ID: 12206838

DETAILED DESCRIPTION

As described above, a conventional volumetric capture system having multiple cameras may end up capturing a lot of redundant data and any changes in the camera settings may necessitate recapture of the entire performance and all the data associated with each camera.

Certain implementations of the present disclosure provide for systems and methods of reducing the data used during capture by selectively activating cameras from a virtual view, and enabling reframing of already captured data without having to re-capture new data.

After reading the below descriptions, it will become apparent how to implement the disclosure in various implementations and applications. Although various implementations of the present disclosure will be described herein, it is understood that these implementations are presented by way of example only, and not limitation. As such, the detailed description of various implementations should not be construed to limit the scope or breadth of the present disclosure.

FIG.1Ais a diagram of a volumetric capture system100including a plurality of image capture devices102-118in accordance with one implementation of the present disclosure. In the illustrated implementation ofFIG.1A, the plurality of image capture devices102-118is arranged in a dome-like structure within which is a capture volume. In another implementation, the plurality of image capture devices102-118is arranged sequentially on posts. In a further implementation, any and all combinations of arrangements of the plurality of image capture devices102-118are used. In one implementation, an image capture device is a video camera having a lens.

FIG.1Bis a diagram of the volumetric capture system100in communication with a virtual camera120in accordance with one implementation of the present disclosure. In the illustrated implementation ofFIG.1B, the virtual camera120is configured in a personal computer122or similar device such as a tablet computer, which is used to control the virtual camera120.

In one implementation, the virtual camera120provides a birds-eye-view of the volumetric capture system100with a capability to move around the system100and activate or deactivate each of the plurality of image capture devices. The virtual camera120has a number of settings associated with it including aperture, focal length, focus, and other device parameters such as lens distortion and image sensor format.

In the illustrated implementation ofFIG.1B, the virtual camera120receives information about the physical volumetric capture system100and visualizes the capture system100and the physical cameras in the virtual world.

To do this, the computer122(which controls the virtual camera120) is in communication with the capture system100and has data about each of the plurality of image capture devices within the system100. In one implementation, the computer122initially calculates a view frustum which is a region of 3-D space within the capture volume that would appear on a view screen. Thus, the view frustum would only include a region that would appear on a view screen (of a game, movie, etc.). Regions that would not appear on a view screen would not be included in the view frustum.

In one implementation, the settings on the virtual camera120define a view frustum of the camera120, which intersects with the physical cameras defined in the virtual scene. In one example, only those physical cameras124,126,128that are intersected by the virtual view frustum will be activated for capture. Thus, in this example, other physical cameras are deactivated for capture. Depending on the type of capture that is being performed, this could greatly reduce the number of physical cameras actually used while still getting enough coverage, thereby greatly reducing the amount of data that is captured.

FIG.1Cshows a physical capture space, e.g. a dome, represented in a virtual 3-D world viewport130(e.g., in a game) in accordance with one implementation of the present disclosure. In this virtual 3-D viewport130there is a virtual camera, which depending on lens parameters (focal length, etc.), has a view frustum of a certain size. This view frustum intersects with virtual representations of the physical image capture devices132-142depending on the view directions of the virtual cameras. These intersections determine which physical image capture devices (e.g., image capture devices134,138,140,142) of the total physical image capture devices132-142actually capture any image data.

In one implementation, each of the physical image capture devices (e.g., image capture devices132-142inFIG.1C) includes a large enough image sensor (e.g., a full frame sensor) and a wide enough lens to capture the entire scene of the capture system. Thus, the physical image capture devices (each device having a large image sensor and wide lens) provide the ability to digitally crop the image to a different focal length. For example, the scene can be cropped from 24 mm to 35 mm or 50 mm while still retaining enough pixel information so as not to degrade the image quality. The data is always captured in full resolution with widest lens settings. Therefore, by using the virtual camera120and changing the camera and lens parameters accordingly, the already-captured data can be re-processed to use different parameters. For example, the data can be re-processed to use a different focal length to change the framing of the already-captured data. Accordingly, by moving the virtual camera120to intersect with other physical cameras from the original capture, it is possible to effectively do a “re-shoot” using different viewing angle than what was used in the original capture without physically re-capturing the physical scene.

FIG.1Dshows a virtual camera150(through which a user looks at the scene) which intersects with cameras defined in the virtual space. Cameras152-158represent the cameras activated for capture in the physical capture space represented in the virtual 3-D world viewport130.

FIG.1Eis a view160of the virtual camera150and what the camera150is intersecting. This is effectively looking through the lens of the virtual camera150in the 3-D viewport130of the application that controls the system.

FIG.2is a block diagram of a system200for reducing data used during capture in accordance with one implementation of the present disclosure. In the illustrated implementation ofFIG.2, the system200includes a volumetric capture system210in communication with a processor220and a visual display230. In one implementation, the visual display230displays a visual representation of a virtual camera232. In one implementation, the display of the virtual camera232is controlled by the processor220residing in a personal computer or similar device such as a tablet computer.

In one implementation, the volumetric capture system210includes a plurality of image capture devices arranged in a dome-like structure. In one implementation, the virtual camera232provides a birds-eye-view of the volumetric capture system210with a capability to move around the system210and activate or deactivate each of the plurality of image capture devices. The virtual camera232has a number of settings associated with it including aperture, focal length, focus, and other device parameters.

In the illustrated implementation ofFIG.2, the virtual camera232receives information about the physical volumetric capture system210and visualizes the capture system210and the physical cameras in the virtual world. To do this, the processor220(which controls the display of the virtual camera232on the visual display230) is in communication with the capture system210and has data about each of the plurality of image capture devices within the system210. In one implementation, the processor220calculates a view frustum which is a region of 3-D space within the capture volume that would appear on a view screen. Thus, the view frustum would only include a region that would appear on a view screen (of a game, movie, etc.). Regions that would not appear on a view screen would not be included in the view frustum.

In one implementation, the settings on the virtual camera232define a view frustum of the camera232, which intersects with the physical cameras defined in the virtual scene. In one example, only those physical cameras that are intersected by the virtual view frustum will be activated for capture. Thus, in this example, other physical cameras are deactivated for capture. Depending on the type of capture that is being performed, this could greatly reduce the number of physical cameras actually used while still getting enough coverage, thereby greatly reducing the amount of data that is captured.

In one implementation, the system200is a system configured entirely with hardware including one or more digital signal processors (DSPs), general purpose microprocessors, application specific integrated circuits (ASICs), field programmable gate/logic arrays (FPGAs), or other equivalent integrated or discrete logic circuitry. In another implementation, the system200is configured with a combination of hardware and software.

FIG.3is a flow diagram of a method300for reducing data used during capture in a physical capture volume by selectively activating image capture devices from a virtual view, and enabling reframing of already-captured data without having to re-capture new data in accordance with one implementation of the present disclosure. To do this, a plurality of image capture devices is arranged in a dome-like structure surrounding a physical capture volume. In another implementation, the plurality of image capture devices is arranged sequentially on posts. In a further implementation, any and all combinations of arrangements of the plurality of image capture devices are used. In one implementation, an image capture device is a video camera having a lens.

In the illustrated implementation ofFIG.3, a virtual camera is setup, at step310, to receive information about and visualize the physical capture volume and a plurality of image capture devices in the virtual view. In one implementation, the virtual view (e.g., a birds-eye view) of the physical capture volume is provided to the virtual camera, at step320, with a capability to move around the volume and activate or deactivate each of the plurality of image capture devices. The virtual camera has a number of settings associated with it including aperture, focal length, focus, and other device parameters.

In one implementation, a view frustum, which is a region of 3-D space within the capture volume that would appear on a view screen of the virtual camera, is calculated, at step330. Thus, the view frustum would only include a region that would appear on a view screen (of a game, movie, etc.). Regions that would not appear on a view screen would not be included in the view frustum. The view frustum of the virtual camera which intersects with the physical image capture devices defined in the virtual view is defined, at step340, using the settings on the virtual camera. In one example, only those physical cameras that are intersected by the virtual view frustum will be activated for capture. Thus, in this example, other physical cameras are deactivated for capture. Depending on the type of capture that is being performed, this could greatly reduce the number of physical cameras actually used while still getting enough coverage, thereby greatly reducing the amount of data that is captured.

FIG.4Ais a representation of a computer system400and a user402in accordance with an implementation of the present disclosure. The user402uses the computer system400to implement a virtual camera application490for reducing data used during capture as illustrated and described with respect to the system200inFIG.2and the method300inFIG.3.

The computer system400stores and executes the virtual camera application490ofFIG.4B. In addition, the computer system400may be in communication with a software program404. Software program404may include the software code for the virtual camera application490. Software program404may be loaded on an external medium such as a CD, DVD, or a storage drive, as will be explained further below.

Furthermore, the computer system400may be connected to a network480. The network480can be connected in various different architectures, for example, client-server architecture, a Peer-to-Peer network architecture, or other type of architectures. For example, network480can be in communication with a server485that coordinates engines and data used within the virtual camera application490. Also, the network can be different types of networks. For example, the network480can be the Internet, a Local Area Network or any variations of Local Area Network, a Wide Area Network, a Metropolitan Area Network, an Intranet or Extranet, or a wireless network.

FIG.4Bis a functional block diagram illustrating the computer system400hosting the virtual camera application490in accordance with an implementation of the present disclosure. A controller410is a programmable processor and controls the operation of the computer system400and its components. The controller410loads instructions (e.g., in the form of a computer program) from the memory420or an embedded controller memory (not shown) and executes these instructions to control the system, such as to provide the data processing. In its execution, the controller410provides the virtual camera application490with a software system. Alternatively, this service can be implemented as separate hardware components in the controller410or the computer system400.

Memory420stores data temporarily for use by the other components of the computer system400. In one implementation, memory420is implemented as RAM. In one implementation, memory420also includes long-term or permanent memory, such as flash memory and/or ROM.

Storage430stores data either temporarily or for long periods of time for use by the other components of the computer system400. For example, storage430stores data used by the virtual camera application490. In one implementation, storage430is a hard disk drive.

The media device440receives removable media and reads and/or writes data to the inserted media. In one implementation, for example, the media device440is an optical disc drive.

The user interface450includes components for accepting user input from the user of the computer system400and presenting information to the user402. In one implementation, the user interface450includes a keyboard, a mouse, audio speakers, and a display. The controller410uses input from the user402to adjust the operation of the computer system400.

The I/O interface460includes one or more I/O ports to connect to corresponding I/O devices, such as external storage or supplemental devices (e.g., a printer or a PDA). In one implementation, the ports of the I/O interface460include ports such as: USB ports, PCMCIA ports, serial ports, and/or parallel ports. In another implementation, the I/O interface460includes a wireless interface for communication with external devices wirelessly.

The network interface470includes a wired and/or wireless network connection, such as an RJ-45 or “Wi-Fi” interface (including, but not limited to 802.11) supporting an Ethernet connection.

The computer system400includes additional hardware and software typical of computer systems (e.g., power, cooling, operating system), though these components are not specifically shown inFIG.4Bfor simplicity. In other implementations, different configurations of the computer system can be used (e.g., different bus or storage configurations or a multi-processor configuration).

The description herein of the disclosed implementations is provided to enable any person skilled in the art to make or use the present disclosure. Numerous modifications to these implementations would be readily apparent to those skilled in the art, and the principals defined herein can be applied to other implementations without departing from the spirit or scope of the present disclosure.

All features of each of the above-discussed examples are not necessarily required in a particular implementation of the present disclosure. Further, it is to be understood that the description and drawings presented herein are representative of the subject matter which is broadly contemplated by the present disclosure. It is further understood that the scope of the present disclosure fully encompasses other implementations that may become obvious to those skilled in the art and that the scope of the present disclosure is accordingly limited by nothing other than the appended claims.