Systems and methods for generating stabilized visual content using spherical visual content

Spherical visual content represented in an image space may be obtained. The spherical visual content may have been captured by image sensor(s) during a time duration. The spherical visual content may include phenomena caused by motion of the image sensor(s) and/or optical components that guide light onto the image sensor(s). A capture path taken by the image sensor(s) during the time duration may be determined. The capture path may reflect positions and orientations of the image sensor(s) during the time duration. A smoothed path may be determined based on the capture path. The smoothed path may have smoother changes in positions and/or orientations than the capture path. The image space may be warped based on a difference between the capture path and the smoothed path. The stabilized visual content may be determined by projecting the spherical visual content represented in the warped image space to a spherical projection space.

FIELD

This disclosure relates to systems and methods that generate stabilized visual content using spherical visual content.

BACKGROUND

Video applications may allow a user to increase the playback speed of a video. The image sensor(s) that captured the video may have been moving during the capture of the video. Movement of the image sensor(s) during the capture of the video may cause sped-up playback of the video to appear jerky/shaky.

SUMMARY

This disclosure relates to generating stabilized visual content using spherical visual content. The spherical visual content may be obtained. The spherical visual content may include pixels represented in an image space. The spherical visual content may have been captured by image sensor(s) during a time duration. The spherical visual content may include phenomena caused by motion of the image sensor(s) and/or optical components that guide light onto the image sensor(s) during at least a part of the time duration. A capture path taken by the image sensor(s) during the time duration may be determined. The capture path may reflect positions and orientations of the image sensor(s) during the time duration. The capture path may include capture viewpoints from which the image sensor(s) captured the visual content during the time duration. A smoothed path may be determined based on the capture path. The smoothed path may have smoother changes in positions and/or orientations than the capture path. The smoothed path may include smoothed viewpoints. The image space may be warped based on a difference between the capture path and the smoothed path. The stabilized visual content may be determined by projecting the spherical visual content represented in the warped image space to a spherical projection space. Presentation of the stabilized visual content on a display may be effectuated.

A system that generates stabilized visual content using spherical visual content may include one or more physical processors, and/or other components. The one or more physical processors may be configured by machine-readable instructions. Executing the machine-readable instructions may cause the one or more physical processors to facilitate generating stabilized visual content using spherical visual content. The machine-readable instructions may include one or more computer program components. The computer program components may include one or more of a spherical visual content component, a capture path component, a smoothed path component, a warp component, a stabilized visual content component, a display component, and/or other computer program components.

The spherical visual content component may be configured to obtain spherical visual content. Spherical visual content may include visual content obtained by a spherical capture. Visual content may refer to media content that may be observed visually. Visual content may include one or more of an image, a sequence of images, a frame of a video, a video, and/or other visual content.

Spherical visual content may have been captured by image sensor(s) during a time duration. In some implementations, the image sensor(s) may be carried by an unmanned aerial vehicle. The spherical visual content may include phenomena caused by motion of the image sensor(s) and/or optical components that guide light onto the image sensor(s) during at least a part of the time duration. The spherical visual content may include pixels represented in an image space. The image space may include a projection point inside the image space. In some implementations, the projection point may coincide with a center of the spherical visual content represented in the image space. The spherical visual content may be transformed into a spherical projection space by projecting the pixels in the image space to the spherical projection space along lines including the projection point.

The capture path component may be configured to determine a capture path taken by the image sensor(s) during the time duration. The capture path may reflect positions and orientations of the image sensor(s) during the time duration. The capture path may include capture viewpoints from which the image sensor(s) captured the spherical visual content during the time duration. The capture path may include a first capture viewpoint from which the spherical visual content was captured at a first point in time within the time duration. In some implementations, the capture path taken by the image sensor(s) during the time duration may be determined based on an analysis of the spherical visual content and/or motion and orientation information for the image sensor(s). The motion and orientation information for the image sensor(s) may be generated by a motion and orientation sensor.

The smoothed path component may be configured to determine a smoothed path based on the capture path. The smoothed path may have smoother changes in positions and/or orientations than the capture path. The smoothed path may include smoothed viewpoints. The smoothed path may include a first smoothed viewpoint at the first point in time within the time duration. In some implementations, the smoothed path having smoother changes in positions and/or orientations than the capture path may be characterized by a maximum rate of changes in positions and/or orientations of the smoothed path being smaller than a maximum rate of changes in positions and/or orientations of the capture path. In some implementations, the smoothed path having smoother changes in positions and/or orientations than the capture path may be characterized by the smoothed path having less jitters in positions and/or orientations than the capture path.

The warp component may be configured to warp the image space. The image space may be warped based on a difference between the capture path and the smoothed path. The difference between the capture path and the smoothed path may include a difference between the positions of the first capture viewpoint and the first smoothed viewpoint at the first point in time. In some implementations, warping the image space may include warping the image space for a portion of the time duration.

The stabilized visual content component may be configured to determine the stabilized visual content. The stabilized visual content may be determined by projecting the spherical visual content represented in the warped image space to the spherical projection space. Views of the stabilized visual content may appear to be from the smoothed viewpoints such that a view of the stabilized visual content corresponding to the first point in time appears to be from the first smoothed viewpoint.

The display component may be configured to effectuate presentation of the stabilized visual content on a display. A user may be presented with the stabilized visual content through a graphical user interface of a visual application. In some implementations, presentation of the stabilized visual content on the display may include playback of the stabilized visual content at a play rate faster than a capture rate at which the image sensor(s) captured the spherical visual content for at least a portion of the time duration. In some implementations, the smoothed viewpoints may include a viewing field of view and a viewing rotation of the stabilized visual content, and the presentation of the stabilized visual content on the display may include playback of portions of the stabilized visual content corresponding to the viewing field of view and the viewing rotation of the stabilized visual content.

DETAILED DESCRIPTION

FIG. 1illustrates system10for generates stabilized visual content using spherical visual content. System10may include one or more of processor11, electronic storage12, bus13, and/or other components. The spherical visual content may be obtained by processor11. The spherical visual content may include pixels represented in an image space. The spherical visual content may have been captured by image sensor(s) during a time duration. The spherical visual content may include phenomena caused by motion of the image sensor(s) and/or optical components that guide light onto the image sensor(s) during at least a part of the time duration. A capture path taken by the image sensor(s) during the time duration may be determined. The capture path may reflect positions and orientations of the image sensor(s) during the time duration. The capture path may include capture viewpoints from which the image sensor(s) captured the visual content during the time duration. A smoothed path may be determined based on the capture path. The smoothed path may have smoother changes in positions and/or orientations than the capture path. The smoothed path may include smoothed viewpoints. The image space may be warped based on a difference between the capture path and the smoothed path. The stabilized visual content may be determined by projecting the spherical visual content represented in the warped image space to a spherical projection space. Presentation of the stabilized visual content on a display may be effectuated.

Electronic storage12may include electronic storage medium that electronically stores information. Electronic storage12may store software algorithms, information determined by processor11, information received remotely, and/or other information that enables system10to function properly. For example, electronic storage12may store information relating to spherical visual content, phenomena caused by motion of image sensor(s) and/or optical components that guide light onto the image sensor(s), capture path, capture viewpoints, smoothed path, smoothed viewpoints, image space, warping of image space, spherical projection space, stabilized visual content, and/or other information.

Processor11may be configured to provide information processing capabilities in system10. As such, processor11may comprise one or more of a digital processor, an analog processor, a digital circuit designed to process information, a central processing unit, a graphics processing unit, a microcontroller, an analog circuit designed to process information, a state machine, and/or other mechanisms for electronically processing information. Processor11may be configured to execute one or more machine readable instructions100to facilitate generating stabilized visual content using spherical visual content. Machine-readable instructions100may include one or more computer program components. Machine readable instructions100may include one or more of spherical visual content component102, capture path component104, smoothed path component106, warp component108, stabilized visual content component110, display component112, and/or other computer program components.

Spherical visual content component102may be configured to obtain spherical visual content. Spherical visual content may include visual content obtained by a spherical capture. Visual content may refer to media content that may be observed visually. Visual content may include one or more of an image, a sequence of images, a frame of a video, a video, and/or other visual content. Spherical visual content may be obtained by a spherical capture through use of one or more cameras/image sensors. For example, spherical visual content may be captured by using multiple cameras/image sensors to capture images/video from a location and stitching the images/videos together.

In some implementations, the image sensor(s) may be carried (e.g., attached to, supported, held, disposed on, and/or otherwise carried) by an object (e.g., a gimbal,). In some implementations, the image sensor(s) may be carried by a vehicle (e.g., a car, a bike, a boat, an airplane, etc.). In some implementations, the image sensor(s) may be carried by a remote controlled vehicle (e.g., remote controlled airplane, remote controlled car, remoted controlled submarine, etc.). In some implementations, the image sensor(s) may be carried by an unmanned aerial vehicle (e.g., drones, etc.). In some implementations, the image sensor(s) may be carried by a person. In some implementations, the image sensor(s) may be carried by an animal. Other carryings of the image sensor(s) are contemplated.

Spherical visual content may have been captured by image sensor(s) during a time duration. The spherical visual content may include phenomena caused by motion of the image sensor(s) and/or optical components that guide light onto the image sensor(s) during at least a part of the time duration. Optical components may refer to components that directly and/or indirectly guide light onto the image sensors. Optical components may include one or more of a lens, a mirror, a prism, and/or other optical components.

Phenomena may refer to one or more visually observable characteristics captured within the spherical visual content. Phenomena may be captured within one or more visual portions (e.g., visual extents, etc.) of the spherical visual content. Phenomena may be captured within one or more moments within the spherical visual content and/or one or more durations within the spherical visual content. For example, phenomena may include one or more shakings of the visuals within the spherical visual content due to shaking(s) of the image sensor(s) and/or optical components that guide light onto the image sensor(s), one or more movements of the visuals within the spherical visual content due to movement(s) of the image sensor(s) and/or optical components that guide light onto the image sensor(s), one or more rotations of the visuals within the spherical visual content due to rotation(s) of the image sensor(s) and/or optical components that guide light onto the image sensor(s), and/or other phenomena.

Spherical visual content may include pixels represented in an image space. The image space may include a projection point inside the image space. In some implementations, the projection point may coincide with a center of the spherical visual content represented in the image space. The spherical visual content may be transformed into a spherical projection space by projecting the pixels in the image space to the spherical projection space along lines including the projection point. One or more portions of the image space may be smaller than the spherical projection space. One or more portions of the image space may be the same size as the spherical projection space. One or more portions of the image space may be larger than the spherical projection space.

FIG. 3Aillustrates a non-limiting example of image space30and spherical projection space32. InFIG. 3A, image space30is shown as a solid line and spherical projection space32is shown as a dashed line. Pixels of spherical visual content may be represented in image space30. Image space30may include projection point31. Projection point31may coincide with the center of the spherical visual content represented in image space30. The center of image space30and the center of spherical projection space32may be located at the same point. For example, the center of image space30and the center of spherical projection space32may be located at projection point31.

One or more portions of image space30may be smaller than spherical projection space32. One or more portions of image space30that are smaller than spherical projection space32may lie inside spherical projection space32. For example, a bottom portion of image space30(image portion B34) may be smaller than spherical projection space32. Image portion B34may lie inside spherical projection space32.

One or more portions of image space30may be the same size as spherical projection space32. One or more portions of image space30that are the same size as spherical projection space32may lie on spherical projection space32. For example, sides of image space30may be the same size as spherical projection space32. The sides of image space30may lie on spherical projection space32.

One or more portions of image space30may be larger than spherical projection space32. One or more portions of image space30that are larger than spherical projection space32may lie outside spherical projection space32. For example, a top portion of image space30(image portion A33) may be larger than spherical projection space32. Image portion A33may lie outside spherical projection space.

Pixels of the spherical visual content represented in image space30may be transformed into spherical projection space32by projecting the pixels in image space30to spherical projection space32along lines including projection point31.FIG. 3Billustrates a non-limiting example of spherical visual content represented in image space30that is projected onto spherical projection space32. For example, one or more pixels of the spherical visual content may be located along image portion A33. One or more pixels from image portion A33may be projected to projected portion A37of spherical projection space32. For example, two pixels from image portion A33may be projected to projected portion A37along projection lines A35. Projecting pixels from outside of spherical projection space32to spherical projection space32may decrease dimensions of the projected pixels. For example, projecting pixels from image portion A33to projected portion A37may decrease dimensions of the pixels to squeeze into a smaller space.

One or more pixels of the spherical visual content may be located along image portion B34. One or more pixels from image portion B34may be projected to projected portion B38of spherical projection space32. For example, two pixels from image portion B34may be projected to projected portion B38along projection lines B36. Projecting pixels from inside of spherical projection space32to spherical projection space32may increase dimensions of the projected pixels. For example, projecting pixels from image portion B34to projected portion B38may increase dimensions of the pixels to expand into a larger space.

One or more pixels of the spherical visual content may be located along spherical projection space32. The locations of such pixels projected to spherical projection space32may coincide with the locations of such pixels in image space30. Dimensions of such pixels may remain the same between image space30and projection space32.

Capture path component104may be configured to determine a capture path taken by the image sensor(s) during a time duration. A capture path may reflect positions and/or orientations of the image sensor(s) during the time duration. A capture path may reflect changes in positions and/or orientations of the image sensor(s) in a two-dimensional space, a three-dimensional space, and/or other dimensional spaces.

FIG. 4Aillustrates a non-limiting example of capture path A40taken by image sensor(s) during a time duration. Capture path A40may reflect position(s) and/or orientation(s) of the image sensor(s) during the time duration. For example, capture path A40may reflect changes in positions of the image sensor(s) in a two dimensional space and/or a three-dimensional space as the image sensor(s) move past wall42and barrel43, towards tree44.

A capture path may include capture viewpoints from which the image sensor(s) captured the spherical visual content during the time duration. Capture viewpoints may correspond to particular moments and/or durations within the time duration. Capture viewpoints may reflect a position and/or orientation of the image sensor(s) at particular moments or durations within the time duration. For example, a capture path may include a first capture viewpoint from which spherical visual content was captured at a first point in time within the time duration. The first capture viewpoint may reflect the position and/or orientation of the image sensor(s) at the first point in time.

For example, capture path A40may include capture viewpoint A41and/or other capture viewpoints. Capture viewpoint A41may correspond to a particular moment and/or duration within the time duration. Capture viewpoint A41may reflect a position and/or orientation of the image sensor(s) at the particular moment and/or duration within the time duration.

FIG. 4Billustrates a non-limiting example of capture orientations of image sensor(s) during a time duration.FIG. 4Billustrates changes in capture pitch orientation45and capture yaw orientation46of the image sensor(s) as a function of progress over capture path A40and/or the time duration. Capture pitch orientation45may reflect changes in pitch orientation of the image sensor(s) as the image sensor(s) move past wall42and barrel43, towards tree44and/over the time duration. Capture yaw orientation45may reflect changes in yaw orientation of the image sensor(s) as the image sensor(s) move past wall42and barrel43, towards tree44and/or over the time duration.

The capture path taken by the image sensor(s) during the time duration may be determined based on an analysis of the spherical visual content and/or motion and orientation information for the image sensor(s). For example, a sparse depth map may be computed from the spherical visual content. The sparse depth (e.g., including 100-200 points, etc.) may by computed for one or more frames of the spherical visual content. The sparse depth map may be used to compute the capture path taken by the image sensor(s).

The motion and orientation information for the image sensor(s) may be generated by one or more motion and orientation sensors. Motion and orientation information for the image sensor(s) may characterize one or more motion and/or orientation of the image sensor(s). Motion of the image sensor(s) may include one or more of movement of the image sensor(s), change in position of the image sensor(s), and/or other motion of the image sensor(s) at a time or over a period of time. Orientation of the image sensor(s) may include one or more of yaw, pitch, and/or roll of the image sensor(s), change in yaw, pitch, and/or roll of the image sensor(s), and/or other orientations of the image sensor(s) at a time or over a period of time. As a non-limiting example, a motion and orientation sensor may include one or more of a global positioning system, an accelerometer, a gyroscope, a magnetometer, an inertial measurement unit, a magnetic position sensor, a radio-frequency position sensor, and/or other motion and orientation sensors.

Smoothed path component106may be configured to determine a smoothed path. A smoothed path may be determined based on a capture path and/or other information. Smoothed path component106may stabilize the motion of the image sensor(s) within a two-dimensional space and/or a three-dimensional space over time (e.g., the time duration during which the image sensor(s) moved within the capture path). One or more smoothed path algorithms may be used to determine the smoothed path based on the capture path. By way of non-limiting examples, smoothed path algorithms may include cubic Bezier curve, spline, BSpline, and/or other smoothed path algorithms.

A smoothed path may have smoother changes in positions and/or orientations than a capture path. In some implementations, the smoothed path having smoother changes in positions and/or orientations than the capture path may be characterized by a maximum rate of changes in positions and/or orientations of the smoothed path being smaller than a maximum rate of changes in positions and/or orientations of the capture path. In some implementations, the smoothed path having smoother changes in positions and/or orientations than the capture path may be characterized by the smoothed path having less jitters in positions and/or orientations than the capture path.

FIG. 5Aillustrates a non-limiting example of capture path A40taken by image sensor(s) during a time duration and smoothed path A50. Smoothed path A50may be determined based on capture path A40, and/or other information. Smoothed path A50may reflect changes to the positions of the image sensor(s) reflected in capture viewpoints of capture path A40. As shown inFIG. 5A, smoothed path A50may have smoother changes in positions than capture path A40. Smoothed path A may be characterized with a maximum rate of changes in positions that is smaller than a maximum rate of changes in positions of capture path A40. Smoothed path A may be characterized by less jitters in positions than capture path A40.

FIG. 5Billustrates a non-limiting example of capture orientations of image sensor(s) during a time duration and smoothed orientations. Smoothed pitch orientation55and/or smoothed yaw orientation56may reflect changes to the orientations of the image sensor(s) reflected in capture viewpoints of capture path A40. As shown inFIG. 5B, smoothed pitch orientation55may have smoother changes in orientations than capture pitch orientation45. Smoothed yaw orientation56may have smoother changes in orientations than capture yaw orientation46. Smoothed pitch orientation55may be characterized with a maximum rate of changes in orientations that is smaller than a maximum rate of changes in orientations of capture pitch orientation45. Smoothed yaw orientation56may be characterized with a maximum rate of changes in orientations that is smaller than a maximum rate of changes in orientations of capture yaw orientation46. Smoothed pitch orientation55may be characterized by less jitters in orientations than capture pitch orientation45. Smoothed yaw orientation56may be characterized by less jitters in orientations than capture pitch orientation46.

A smoothed path may include smoothed viewpoints. Smoothed viewpoints may correspond to particular moments or durations within the time duration. Smoothed viewpoints may reflect changes to the positions and/or orientations of the image sensor(s) reflected in capture viewpoints. For example, a smoothed path may include a first smoothed viewpoint at the first point in time within the time duration. The first smoothed viewpoint may reflect changes to the position and/or orientation of the image sensor(s) reflected in the first capture viewpoint.

For example, as shown inFIG. 5A, smoothed path A50may include smoothed viewpoint A51and/or other smoothed viewpoints. Smoothed viewpoint A51may correspond to a particular moment and/or duration within the time duration. Smoothed viewpoint A51may reflect changes to the position and/or orientation of the image sensor(s) at the particular moment and/or duration within the time duration. Smoothed viewpoint A51may reflect changes to the position and/or orientation of the image sensor(s) reflected in capture viewpoint A41. For example, smoothed viewpoint A51may reflect changes to the position of capture viewpoint A41being moved to the left from capture path A40. Other changes in the positions and/or orientations of the image sensor(s) are contemplated.

Warp component108may be configured to warp the image space. The image space may be warped based on one or more differences between the capture path and the smoothed path. For example,FIG. 7Aillustrates a non-limiting example of spherical visual content represented in image space A70. The spherical visual content represented in image space A70may have been captured during a movement of image sensor(s) through capture path A40.FIG. 7Amay illustrate the spherical visual content represented in image space A70at capture viewpoint A41. Image space A70may include pixels for wall portion72(corresponding to wall42), barrel portion73(corresponding to barrel43), tree portion74(corresponding to tree44), and/or other portions of the spherical visual content.

Warp component108may warp image space A70based on one or more differences between capture path A40and smoothed path A50. Differences between capture path A40and smoothed path A50may include one or more differences in the positions between capture viewpoints and smoothed viewpoints. A difference between capture path A40and smoothed path A50may include a difference between position of capture viewpoint A41and position of smoothed viewpoint A51at a particular point in time. For example, as shown inFIG. 5A, smoothed viewpoint A51may be positioned to the left of capture viewpoint A41by a certain distance. Position of capture viewpoint A41and position of smoothed viewpoint A51may include the same and/or different distances to objects near capture viewpoint A41and/or smoothed viewpoint A51. For example, smoothed viewpoint A51may be closer to wall42than capture viewpoint A41. Smoothed viewpoint A51may be further away from barrel43than capture viewpoint A41. Smoothed viewpoint A51may be at a same distance to tree44as capture viewpoint A41. Other differences between the capture path and the smoothed path are contemplated.

Warp component108may warp the image space based on the amount of difference(s) between the positions between capture viewpoints and smoothed viewpoints. Warp component108may warp the image space with larger deformations for larger differences between the positions between capture viewpoints and smoothed viewpoints. Warp component108may warp the image with smaller deformations for smaller differences between the positions between capture viewpoints and smoothed viewpoints.

Warp component108may warp the image space based on the differences in positions between capture viewpoint A41and smoothed viewpoint A51. For example, based on smoothed viewpoint A51being located to the left of capture viewpoint A41, warp component108may warp image space A70to warped image space A75, as shown inFIG. 7B. In warped image space A75, dimensions of wall portion72may be decreased. Wall portion72may be pushed in so that wall portion72in warped image space A75is located inside dimensions of image space A70. In warped image space A75, dimensions of barrel portion73may be increased. Barrel portion73may be pushed out so that barrel portion73in warped image space A75is located outside dimensions of image space A70. In warped image space A75, dimensions of tree portion74may remain the same. Tree portion74may remain the same so that tree portion74in warped image space A75is located along dimensions of image space A70.

In some implementations, one or more sparse depth maps may be used to determine the amount of deformation for the image space. The amount of deformation for a particular smoothed viewpoint may correspond to a deformation that solves for the sparse 3D points and keeps their depth from the smoothed viewpoint based on the deformed image space. For example, image space A70may be warped so that the amounts of deformation correspond to the depths of objects from smoothed viewpoint A51.

In some implementations, the amounts of deformation may be measured by a norm defined based on one or more objects around the image sensor(s). For example, a norm may be defined as a distance between the image sensor(s) and one or more objects closest to the image sensor(s). A small difference between the positions between capture viewpoints and smoothed viewpoints may be defined as distances between the capture viewpoints and the smooth viewpoints that are smaller than the norm. A big difference between the positions between capture viewpoints and smoothed viewpoints may be defined as distances between the capture viewpoints and the smooth viewpoints that are bigger than the norm.

In some implementations, a difference between the capture path and the smoothed path may include one or more differences in the orientations between capture viewpoints and smoothed viewpoints.FIG. 6illustrates a non-limiting example of differences in orientations for a capture viewpoint and a smoothed viewpoint. Capture viewpoint A orientation61may reflect the orientation of the image sensor(s) at capture viewpoint A41. Capture viewpoint A orientation61may be tilted to the right. Capture viewpoint A orientation61may be tilted to the right based on the image sensor(s) at capture viewpoint A41being tilted to the right. Smoothed viewpoint A orientation62may reflect changes to the orientation of the image sensor(s) reflected at capture viewpoint A41. Smoothed viewpoint A orientation A62may be upright. Other changes in positions and/or orientations of the image sensor(s) are contemplated.

In some implementations, warping the image space may include warping the image space for a portion of the time duration. In some implementations, warping the spherical visual content may include warping the spherical visual content for the time duration. Warping the image space may include warping the image space for one or more parts of the time duration or for the entire time duration. For example, spherical visual content may be captured for thirty minutes. Image space for the spherical visual content may be warped for a part of the thirty minutes (e.g., from one-minute mark to four-minute thirty-second mark, etc.), multiple parts of the thirty minutes (e.g., from two-minute mark to five-minute mark and from ten-minute twenty second-mark to twenty-minute forty-second mark, etc.), or for the entire thirty minutes.

Stabilized visual content component110may be configured to determine stabilized visual content. The stabilized visual content may be determined by projecting the spherical visual content represented in the warped image space to a spherical projection space. The spherical visual content represented in the warped image space may be transformed into the spherical projection space by projecting the pixels in the warped image space to the spherical projection space along lines including the projection point.

FIG. 8illustrates a non-limiting example of projecting spherical visual content represented in warped image space A75to spherical projection space A80. Spherical projection space A80may have the same dimensions as image space A70. Other dimensions of spherical projection space A80are contemplated.

Pixels of spherical visual content may be represented in warped image space A75. Pixels of the spherical visual content represented in warped image space A75may be projected to spherical projection space A80along lines (shown as dotted arrows inFIG. 8) including a projection point (not shown). The projection point may coincide with the center of the spherical visual content represented in the image space A70. The center of image space A70and the center of spherical projection space A80may be located at the same point.

Wall portion72of warped image space A75may be projected out to spherical projection space A80. Projection of pixels from inside spherical projection space A80to spherical projection space A80may increase the dimensions of the projected pixels. For example, the dimensions of pixels corresponding to wall portion72of warped image space A75may increase to expand into a larger space in spherical projection space A80.

Barrel portion73of warped image space A75may be projected into spherical projection space A80. Projection of pixels from outside spherical projection space A80to spherical projection space A80may decrease the dimensions of the projected pixels. For example, the dimensions of pixels corresponding to barrel portion73of warped image space A75may decrease to squeeze into a smaller space in spherical projection space A80.

Tree portion74of warped image space A75may be projected onto spherical projection space A80. Projections of pixels located along spherical projection space A80may not change the dimensions of the projected pixels. For example, the dimensions of pixels corresponding to tree portion74of warped image space A75may not change when projected into the same sized space in spherical projection space A80.

Views of the stabilized visual content may appear to be from the smoothed viewpoints such that a view of the stabilized visual content corresponding to a particular point in time appears to be from a corresponding smoothed viewpoint. For example, inFIG. 5A, a view of the stabilized visual content at a particular point in time may appear to be from smoothed viewpoint A51, rather than capture viewpoint A41.FIGS. 10-11illustrates non-limiting examples of original image150and warped image160. Original image150may include a view of the visual content from capture viewpoint A41. Warped image160may include a view of the corresponding stabilized visual content (e.g., a view of the visual content from smoothed viewpoint A51, etc.).

For example, smoothed viewpoint A51may be closer to wall42than capture viewpoint A41. Warped image160may include visuals representing warped wall161, which may be larger than visual representing wall151in original image150. Smoothed viewpoint A51may be further away from barrel43than capture viewpoint A41. Warped image160may include visuals representing warped barrel163, which may be smaller than visual representing barrel153in original image150. Smoothed viewpoint A51may be at a same distance to tree44as capture viewpoint A41. Warped image160may include visuals representing non-warped tree162, which may be the same as visual representing tree152in original image150.

Display component112may be configured to effectuate presentation of the stabilized visual content on a display. A user may be presented with the stabilized visual content through a graphical user interface of a visual application. A visual application may refer to one or more software, one or more software running on one or more hardware, and/or other applications operating to present visual content on a display. As a non-limiting example, a visual application may include one or more of visual content viewer, visual content editor, and/or other visual applications. As a non-limiting example, a visual application may run on one or more of a mobile device, a desktop device, a camera, and/or other hardware.

In some implementations, presentation of the stabilized visual content on the display may include playback of the stabilized visual content at a play rate faster than a capture rate at which the image sensor(s) captured the spherical visual content for at least a portion of the time duration. The stabilized visual content may be presented at the play rate faster than the capture rate for one or more parts of the time duration or for the entire time duration. For example, spherical visual content may be captured for thirty minutes. The stabilized visual content may be presented at the play rate faster than the capture rate for a part of the thirty minutes (e.g., from one-minute mark to four-minute thirty-second mark, etc.), multiple parts of the thirty minutes (e.g., from two-minute mark to five-minute mark and from ten-minute twenty-second mark to twenty-minute forty second mark, etc.), or for the entire thirty minutes. A relationship between a play rate and a capture rate may be linear or non-linear.

In some implementations, a play rate may be constant. For example, a constant play rate may be a multiple of a capture rate (e.g., a stabilized visual content is played at 10× speed, etc.). In some implementations, a play rate may vary. For example, a play rate may be varied so that motions of varying speeds captured within a visual content appears to be moving at a constant speed during playback of the stabilized visual content (e.g., visual content includes capture of a duration in which the image sensor(s) are moving at 10 meters/second and another duration in which the image sensor(s) are moving at 20 meters/second, and the stabilized visual content is played back so that motion captured within the stabilized visual content appears to be moving at the same speed during both durations, etc.). In some implementations, a play rate may vary based on objects and/or scenes detected within the stabilized visual content. For example, stabilized visual content may include capture of a duration in which a moment of interest occurs and another duration in which a moment of interest does not occur. The play rate may be varied so that the duration in which the moment of interest occurs is played more slowly than the duration in which the moment of interest does not occur.

In some implementations, the smoothed viewpoints may include a viewing field of view and a viewing rotation of the stabilized visual content, and the presentation of the stabilized visual content on the display may include playback of portions of the stabilized visual content corresponding to the viewing field of view and the viewing rotation of the stabilized visual content. The viewing field of view and the viewing rotation of the stabilized visual content may define an extent of the stabilized visual content to be displayed.

FIG. 9illustrates non-limiting examples of viewing fields of view and viewing rotations for two smoothed viewpoints. First smoothed viewpoint may include viewing field of view A91and viewing rotation A92. Second smoothed viewpoint may include viewing field of view B93and viewing rotation B94. Viewing field of view A91may be smaller than viewing field of view B93. Viewing field of view A91and viewing field of view B93may be centered at a same point of a stabilized visual content or may be centered at different points of the stabilized visual content. Viewing rotation A92may include an upright rotation and viewing rotation B94may include a tilted rotation. Presentation of the stabilized visual content corresponding to the first smoothed viewpoint may include presentation of the extent of the stabilized visual content defined by viewing field of view A91and viewing rotation A92. Presentation of the stabilized visual content corresponding to the second smoothed viewpoint may include presentation of the extent of the stabilized visual content defined by viewing field of view B93and viewing rotation B94.

In some implementations, stabilized visual content may be orientated to keep as central view the direction of movement in the smoothed path. Such orientation of the stabilized visual content may reduce adverse effects of viewing the stabilized visual content (e.g., motion sickness, etc.).

Although processor11and electronic storage12are shown to be connected to a bus13inFIG. 1, any communication medium may be used to facilitate interaction between any components of system10. One or more components of system10may communicate with each other through hard-wired communication, wireless communication, or both. For example, one or more components of system10may communicate with each other through a network. For example, processor11may wirelessly communicate with electronic storage12. By way of non-limiting example, wireless communication may include one or more of radio communication, Bluetooth communication, Wi-Fi communication, cellular communication, infrared communication, or other wireless communication. Other types of communications are contemplated by the present disclosure.

Although processor11is shown inFIG. 1as a single entity, this is for illustrative purposes only. In some implementations, processor11may comprise a plurality of processing units. These processing units may be physically located within the same device, or processor11may represent processing functionality of a plurality of devices operating in coordination. Processor11may be configured to execute one or more components by software; hardware; firmware; some combination of software, hardware, and/or firmware; and/or other mechanisms for configuring processing capabilities on processor11.

It should be appreciated that although computer components are illustrated inFIG. 1as being co-located within a single processing unit, in implementations in which processor11comprises multiple processing units, one or more of computer program components may be located remotely from the other computer program components.

The electronic storage media of electronic storage12may be provided integrally (i.e., substantially non-removable) with one or more components of system10and/or removable storage that is connectable to one or more components of system10via, for example, a port (e.g., a USB port, a Firewire port, etc.) or a drive (e.g., a disk drive, etc.). Electronic storage12may include one or more of optically readable storage media (e.g., optical disks, etc.), magnetically readable storage media (e.g., magnetic tape, magnetic hard drive, floppy drive, etc.), electrical charge-based storage media (e.g., EPROM, EEPROM, RAM, etc.), solid-state storage media (e.g., flash drive, etc.), and/or other electronically readable storage media. Electronic storage12may be a separate component within system10, or electronic storage12may be provided integrally with one or more other components of system10(e.g., processor11). Although electronic storage12is shown inFIG. 1as a single entity, this is for illustrative purposes only. In some implementations, electronic storage12may comprise a plurality of storage units. These storage units may be physically located within the same device, or electronic storage12may represent storage functionality of a plurality of devices operating in coordination.

FIG. 2illustrates method200for generating stabilized visual content using spherical visual content. The operations of method200presented below are intended to be illustrative. In some implementations, method200may be accomplished with one or more additional operations not described, and/or without one or more of the operations discussed. In some implementations, two or more of the operations may occur substantially simultaneously.

Referring toFIG. 2and method200, at operation201, spherical visual content may be obtained. The spherical visual content may include pixels represented inside an image space. In some implementations, operation201may be performed by a processor component the same as or similar to spherical visual content component102(shown inFIG. 1and described herein).

At operation202, a capture path taken by one or more image sensors may be determined. In some implementations, operation202may be performed by a processor component the same as or similar to capture path component104(shown inFIG. 1and described herein).

At operation203, a smoothed path may be determined based on the capture path. In some implementations, operation203may be performed by a processor component the same as or similar to smoothed path component106(shown inFIG. 1and described herein).

At operation204, the image space may be warped based on a difference between the capture path and the smoothed path. In some implementations, operation204may be performed by a processor component the same as or similar to warp component108(shown inFIG. 1and described herein).

At operation205, a stabilized visual content may be determined. In some implementations, operation205may be performed by a processor component the same as or similar to stabilized visual content component110(shown inFIG. 1and described herein).

At operation206, presentation of the stabilized visual content on a display may be effectuated. In some implementations, operation206may be performed by a processor component the same as or similar to display component112(shown inFIG. 1and described herein).