System and method for bullet-time photography

Systems and methods for bullet-time photography are disclosed. According to certain embodiments, a method for bullet-time photography may include arranging a plurality of cameras around a subject of the bullet-time photography. The method may also include projecting a light beam from each of the plurality of cameras to the subject. The method may further include aligning an imaging direction of each of the plurality of cameras according to a direction of the respective light beam. The method may further include triggering the plurality of cameras to capture images of the subject.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is based upon and claims priority to Chinese Patent Application No. 201610024066.3, filed Jan. 14, 2016, the entire contents of which are incorporated herein by reference.

TECHNICAL FIELD

The present disclosure relates generally to photography technologies, and more particularly, to a system and method for bullet-time photography.

BACKGROUND

“Bullet time,” also known as “time slice” and “time freeze,” is a visual effect of stopped time or slowed time. Bullet time is famous for being used in the movie series “the Matrix” to feature, for example, actors dodging bullets, and received its time because of these scenes. Bullet time is characterized both by its extreme transformation of time (slow enough to show normally imperceptible events, such as flying bullets) and space (by way of the ability of the camera angle, i.e., the audience's point-of-view, to move around the scene at a normal speed while events are slowed). Bullet time has been widely utilized in movies, commercials, video games, sports broadcasting, music videos, etc.

Bullet time can be achieved using a series of cameras, such as a camera array consisting of 25-150 cameras, surrounding the filmed subject. When the subject starts to make an action, the cameras are fired sequentially, or all at the same time, depending on the desired effect. The frames generated by each camera are then arranged and displayed consecutively to produce an orbiting viewpoint of an action frozen in time or in slow motion.

Despite all the fascination it has excited, bullet time is expensive and time-consuming to create. A typical bullet-time setup requires multiple cameras accurately lined up on a carefully designed rig. Both building the rig and installing the cameras can be daunting tasks. Moreover, a lot of efforts need to be spent on interpolating frames (i.e., creating frames between frames) using a special software program to achieve the final effect. Therefore, “the Matrix” type of bullet time is teamwork by at least a software engineer, an imbedded-system engineer, photography director, a rig installation crew, and other assistants. The production periods may last from 24 hours to several weeks and the production cost can easily exceed 1 million dollars.

The disclosed methods and systems address one or more of the problems listed above.

SUMMARY

Consistent with one disclosed embodiment of the present disclosure, a method for bullet-time photography is provided. The method may include arranging a plurality of cameras around a subject of the bullet-time photography. The method may also include projecting a light beam from each of the plurality of cameras to the subject. The method may further include aligning an imaging direction of each of the plurality of cameras according to a direction of the respective light beam. The method may further include triggering the plurality of cameras to capture images of the subject.

Consistent with another disclosed embodiment of the present disclosure, a system for bullet-time photography is provided. The system may include a controller and a camera accessory attached to a camera. The camera accessory may include a light projector configured to project a light beam to a subject of the bullet-time photography. An imaging direction of the camera may be aligned according to a direction of the light beam. The controller may be configured to: control the light projector to project the light beam; and trigger the camera to capture images of the subject when the subject starts an action.

DETAILED DESCRIPTION

FIG. 1Ais a schematic diagram illustrating a front view of a camera assembly100for bullet-time photography, according to an exemplary embodiment. Referring toFIG. 1, camera assembly100may include a camera110and a bullet-time camera accessory120.

Camera110may be any type of image capturing device. For example, camera110may be an action camera, a digital camera, a web camera. Camera110may also be imbedded in another device, such as a smartphone, a computer, a personal digital assistant (PDA), a monitoring device, a video gaming console, etc.

Camera110can be configured to capture one or more images in a variety of ways. For example, camera110can be configured to capture images by a user, by programming, by hardware setting, or by a combination thereof. In some embodiments, when camera110is configured to capture images by software or hardware programming or by hardware setting, image capturing can be performed at one or more predetermined conditions. For example, a set of predetermined conditions (for example, the sensing of an actor/actress starting an action) can trigger camera110to capture images. Alternatively, or additionally, camera110can capture images in response to a user's operation (for example, the use pushes a control button). In some embodiments, capturing images may include that camera110is in a mode or setting capable of capturing one or more images. As used herein, an “image” can refer to, in part or in whole, a static or dynamic visual representation including, but not limited to, a photo, a picture, a graphic, a video, a hologram, a virtual reality image, an augmented reality image, other visual representations, or combination thereof.

Camera110may be configured to form wireless or wired communications with other devices, including a controller that can remotely control the operation of camera110and a displaying device that can display the images taken by camera110. For example, camera110may include a built-in Wi-Fi or Bluetooth module for wireless connection. Also for example, camera110may include a micro Universal Serial Bus (USB) interface by which other device may be connected to camera110. For yet another example, camera110may include a High-Definition Multimedia Interface (HDMI) for outputting video and/or audio data to other devices.

With continued reference toFIG. 1A, camera110may include a base member112configured to mount camera110to a structure. In some exemplary embodiments, base member112may include one or more universal clips, cavities, or apertures designed for easy attachment to the structure. The structure may be a rig specially built for bullet-time photography. The structure may also be any common structures in a photography studio, such as the edge of a table, a door frame, etc. In some exemplary embodiments, base member112may also include one or more joints and/or gears configured to allow adjustment of the position and/or orientation of camera110. Base member112may even include one or more electronic actuators and/or motors that can be controlled by a controller to change the position and/or orientation of camera110. Base member112may be permanently or detachably secured to the bottom of camera110. AlthoughFIG. 1Ashows base member112being directly attached to camera110, it is contemplated that base member112may also be part of a separate case, such as a protective case or a waterproof case, that houses camera110.

Still referring toFIG. 1A, bullet-time camera accessory120may be a device attached to camera110to facilitate the bullet-time photography. Bullet-time camera accessory120may include a power source, such as a battery, to supply power to each component of bullet-time camera accessory120and/or camera110. Bullet-time camera accessory120may also include a charging port for connecting to an external charging device. In some exemplary embodiments, bullet-time camera accessory120may include a micro USB plug (not shown) configured to connect bullet-time camera accessory120to camera110via the micro USB interface on camera110. In one embodiment, bullet-time camera accessory120may be attached on a side of camera110and may have an “L” shape, with the tail of the “L” shape extending to the back of camera110.

The front end of bullet-time camera accessory110may be a light projector122configured to project a focused and collimated light beam on a subject of the bullet-time photography. The direction of the light beam may be configured to be parallel to the imaging direction of camera110. For example, light projector122may be a laser pointer that includes a laser diode emitting a narrow coherent low-powered laser beam of visible light. As described in more detail below, the visible light may be used to form a small bright spot on the subject to facilitate the alignment of the imaging direction of camera110.

FIG. 1Bis a schematic diagram illustrating a back view of the camera assembly100shown inFIG. 1A. Referring toFIG. 1B, as described above, the tail of bullet-time camera accessory120may extend to the back of camera110, and include a micro USB interface124and/or wireless transceiver126.

Micro USB interface124may be similar to the micro USB interface on camera110. Micro USB interface124may be configured to receive a cable for various purposes. Micro USB interface124may function as a charging port to receive a charging cable to charge bullet-time camera accessory120. Micro USB interface124may also function as a data port to exchange data and signals with another device. For example, micro USB interface124may receive a data cable to transfer the images taken by camera110to another device, such as a computer, for further processing. For another example, micro USB interface124may be connected to a controller to receive control signals associated with bullet-time photography.

Wireless transceiver126may be an infrared signal transceiver, a near field communication (NFC) signal transceiver, a Bluetooth antenna, and/or a wireless router to facilitate short-range two-way communications between bullet-time camera accessory120and another device, such as a controller of camera110. Similar to micro USB interface124, wireless transceiver126may exchange data and signals with another device.

In exemplary embodiments, bullet-time camera accessory120may relay data and signals between camera110and other devices. That is, after bullet-time camera accessory120receives data and signals from other devices through micro USB interface124or wireless transceiver126, bullet-time camera accessory120may transmit the received data and signals to camera110. Similarly, bullet-time camera accessory120may also obtain data and signals from camera110and send them to other devices.

The plurality of camera assemblies100forms a camera array for bullet-time photography. The camera array may include any number of camera assemblies100and may be arranged in any configuration as needed. In the example illustrated inFIG. 2, the camera array may be mounted on a straight track (not shown), with each camera assemblies100being separated by an equal distance, for example, 10 cm.

Each camera assembly100may be connected to controller240using a data cable230. Specifically, controller240may include a plurality of micro USB ports, such that each data cable230may connect micro USB interface124in each bullet-time camera accessory120with controller240. This way, controller240may send control signals to each camera assembly100for performing the bullet-time photography related functions.

Alternatively, each camera assembly100may receive control signals from controller240through wireless transceiver126. Correspondingly, controller240may include a wireless communication module configured to facilitate the wireless communications.

Controller240may be an independent device specially designed for bullet-time photography. Alternatively, controller240may be part of a larger device, such as a computer. Moreover, controller240may be implemented through hardware, software, or a combination of hardware and software. Referring toFIG. 2, in one exemplary embodiment, controller240may be a remote control that allows a camera operator to control the plurality of camera assemblies100. For example, controller240may include a touchscreen, a touch pad, and/or a keyboard to enable user input.

To perform the bullet-time photography, the operator may push a power button on controller240to power and activate each camera assembly100. The operator may then activate light projector122in each bullet-time camera accessory120to project a light beam to the subject of the time-bullet photography. Each light beam may form a light spot on the subject. The operator may adjust the orientation of each camera assembly100such that all the light spots converge to a position on the subject. In this manner, the imaging direction of each camera110may be aligned toward the same point.

The operator may input in controller240the camera parameters used for controlling the plurality of cameras110. For example, the operator may enter the synchronization parameters for the plurality of cameras110. The synchronization parameters may be used by controller240to trigger each camera110to take photos at the same time or sequentially. The operator may also set the exposure time (i.e., ISO), shutter speed, and aperture for each camera110. After all the camera parameters have been set. The operator may use the controller240to control the plurality of cameras110to take photos of the subject.

FIG. 3is a schematic diagram illustrating an implementation of the system200for bullet-time photography, according to an exemplary embodiment. The system200may be used to film the actions of a subject10, such as a human actor/actress or an object. Referring toFIG. 3, a plurality of camera assemblies100may be mounted on a rig (not shown). The rig may be built to form a specially designed camera path. Industry standard trussing and grip gear can be used in conjunction with various custom rigging solutions to allow substantial flexibility with positioning, height, and camera movement. The rig may include multiple circles and curves with various diameters, straight tracks, incline/decline angles, overhead rigging, etc. However, consistent with the present disclosure, system200may only require a simple shaped rig, such as a single straight or curved track. For example, in a low-budget embodiment, the rig may be configured to allow the plurality of camera assemblies100to form a straight line, a 180° arc, or a 360° circle.

Upon the establishment of the rig, the plurality of camera assemblies100may be mounted to the rig through base members112. Cameras110may be any types of cameras. In a low-budget embodiment, cameras110may be action cameras (also called sports cameras). Action cameras allow fast and continuous capture of the actions of a subject, and are suitable for both indoor and outdoor photography. Action cameras are typically compact, light-weight, and small sized, and are thus easy to be mounted on any structures. In a high-budget embodiment, cameras110may be digital single-lens reflex (DSLR) cameras typically used for film making.

After all the camera assemblies100are mounted on the rig, controller240may activate light projector122in each bullet-time camera accessory120. As described above, controller140may send control signals to each bullet-time camera accessory120in a wired or a wireless manner (i.e., through micro USB interface124or wireless transceiver126). Each bullet-time camera accessory120may then project a laser beam to subject10. The direction of each laser beam is configured to be parallel to the imaging direction of the respective camera110(i.e., the axis of the lens in the respective camera110). Each laser beam may form a crosshair on subject10to allow accurate alignment of cameras110. In cases that subject10is far away from camera assemblies100, light projector122may be configured to form a bright laser dot, instead of a crosshair, on subject10to provide better illumination of subject10.

Various mechanisms may be provided to align the imaging direction of each camera110based on the laser beams generated by light projectors122. In one embodiment, the operator may manually adjust each base member112to tune the imaging direction of the corresponding camera110. Base members112may include any mechanical structures know in the art that allow the operator to adjust the orientations of cameras110, such that all the laser beams point to a pre-chosen reference position12on subject10. In another embodiment, base members112may be equipped with electronic actuators and/or motors so that the operator may preciously control the orientations of cameras110through controller240or another computer. For example, controller240or the computer may be configured to allow the operator to remotely control the orientation of each camera110separately or simultaneously.

After all cameras110are aligned in the desired imaging directions, system200may turn off light projectors122and start to shoot the footage. When subject10starts to make actions, controller240may trigger each camera110sequentially or simultaneously to take photos of subject10. Controller240may also control other operation parameters of each camera110, such as the aperture, the shutter speed, and the ISO. These parameters may be set by the operator before the shooting starts or adjusted by the operator during the shooting.

FIG. 4is a block diagram of a system400for bullet-time photography, according to an exemplary embodiment. Referring toFIG. 4, system400may include a plurality of camera assemblies100, a controller440, and one or more video cameras450. Video cameras450may be used to shoot videos of subject10, which may be combined with the photos taken by cameras110to produce the desired effects. Controller440may perform functions similar to controller240. Controller440may further include a communication module441, an accessory control module442, a camera calibration module443, a shutter control module444, a data acquisition module445, and an image processor446.

Communication component441may be configured to facilitate communication, wired or wirelessly, between controller440and other devices, including camera assemblies100and video cameras450. Communication module441may access a wireless network based on one or more communication standard, such as Wi-Fi, LTE, 2G, 3G, 4G, 5G, etc. In one exemplary embodiment, communication module441may further include a NFC module to facilitate short-range communications. In other embodiments, communication module441may be implemented based on a radio frequency identification (RFID) technology, an infrared data association (IrDA) technology, an ultra-wideband (UWB) technology, a Bluetooth (BT) technology, or other technologies. For example, communication module441may form Wi-Fi communications with each camera110and wireless infrared communications with each bullet-time camera accessory120.

Accessory control module442may be configured to generate signals to control the operation of each bullet-time camera accessory120. For example, accessory control module442may remotely turn on or turn off each light projector122. Accessory control module442may also adjust the light intensity, light shape, and/or light color generated by each light projector122.

Camera calibration module443may be configured to adjust the orientation and operation parameters of each camera110. Camera calibration module443may remotely control the pan, tilt, zoom, focus, and iris of each camera110. For example, camera calibration module443may control a servo motor in each base member112to adjust the imaging direction of the respective camera110. In one embodiment, cameras110may take images of the areas on subject10that are illuminated by the plurality of light projectors122and transmit the images to camera calibration module443. Based on the images, camera calibration module443may adjust the orientation of each camera110such that all the illuminated areas converge to reference position12.

Shutter control module444may be configured to synchronize the shutter operation in each camera110during the footage shooting. Shutter control module444may trigger each camera110to capture the images of subject10simultaneously or in a specified sequence. Shutter control module444may also control the shutter speed of each camera110to achieve the desired effects.

Data acquisition module445may be configured to collect the footage generated by cameras110and video cameras450. Data acquisition module445may also be configured to determine the metadata associated with the footage, such as the time and camera position at which the footage is shot. Data acquisition module445may supply the footage and the associated metadata to image processor446for further processing.

Image processor446may be configured to process the footage to generate the desired bullet-time effects. Image processor446may run any suitable software to process the footage. In one exemplary embodiment, image processor446may directly combine the desired sets of images based on the time when the images are taken. In another exemplary embodiment, image processors446may process the footage to interpolate between each camera110at one point in time with heavy frame-to-frame interpolation. Such treatment may ensure the bullet-time effect to be smooth and prevent image artifacts due to the spatial separation of cameras110.

In exemplary embodiments, controller440may be implemented with one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, micro-controllers, microprocessors, or other electronic components, for performing the above described functions.

In exemplary embodiments, controller440may also include a non-transitory computer-readable storage medium including instructions for performing above-described functions. For example, the non-transitory computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a CD-ROM, a magnetic tape, a memory chip (or integrated circuit), a hard disc, a floppy disc, an optical data storage device, or the like

FIG. 5is a flowchart of a method500for bullet-time photography, according to an exemplary embodiment. For example, method500may be used in system400(FIG. 4). Referring toFIG. 5, method500may include the following steps510-550.

In step510, controller440may form communications with each camera assembly100. In one embodiment, controller440may form direct communication channels with cameras110and bullet-time camera accessories120separately. In another embodiment, controller440may form direct communication channels with bullet-time camera accessories120only and exchange data/signals with cameras110through bullet-time camera accessories120.

In step520, controller440may control each light projector122to project a light beam to subject10. Controller440may send control signals to each bullet-time camera accessory120to turn on the corresponding light projector122. In exemplary embodiments, light projectors122may be laser pointers that can generate focused and collimated laser beams.

In step530, controller440may align the imaging direction of each camera110to a predetermined direction. The direction of the light beam generated by each light projector122is set parallel to the imaging direction of the corresponding camera110. Controller440may adjust the orientation of each camera110such that all the light beams point to reference position12. This way, the imaging direction of each camera110is aligned towards reference position12.

In step540, controller440triggers each camera110to capture images of subject10. After the imaging directions of cameras110are aligned, controller440may turn off light projectors122. When subject10starts an action, controller440may trigger each camera110simultaneously or in a specified sequence to capture images of subject10from various positions. Controller440may control the shutter speed, ISO, aperture of each camera110to achieve desired result. Controller440may start the footage-shooting process in various manners. For example, an operator of system400may press a “start” button on a remote control to initiate the footage-shooting process. For another example, controller440may be connected to a sensor, such as a pressure sensor, configured to detect the action of subject10. When the action starts, the sensor may generate a signal and transmit the signal to controller440, which in turn fires cameras110.

In step550, controller440may process the footage to generate the bullet-time effect. Controller440may interpolate between each camera110at one point in time to generate extra frames, so as to improve the smoothness of the bullet-time effect. In some embodiments, controller440may also drop some frames to speed up the action.

The disclosed systems and methods provide a fast and cost-effective solution for time-bullet photography. In particular, the disclosed systems and methods minimize the requirement for rig design and construction. The light projectors provide easily observable guidance for aligning the imaging direction of each camera. Moreover, the base members enable the quick installment and alignment of the cameras. Therefore, not only the cameras may be mounted on any structures, but also the installation and alignment of the cameras may be accomplished by one person in a short time period. In addition, when budget-friendly cameras, such as action cameras, are used, the expenses for conducting the bullet-time photography may be further lowed. For example, in one implementation consistent with the disclosed embodiments, the bullet-time footage may be shot using 25 action cameras mounted on a straight track and separated from each other at a distance of 10 cm. This system may be set up in less than an hour and the budget may be controlled within a few thousand dollars.