Representing high-resolution media content in a lower resolution system

Representing high-resolution media content in a lower resolution system is disclosed. For a first portion of a high-resolution media content which is associated with a content boundary that does not align with an interval boundary of the lower resolution system, a new media content file that includes at least said first portion of the high-resolution media content is created. The new media content file and a second file comprising at least a second portion of the original media content file, which second portion begins at a first interval boundary and ends at a second interval boundary of the lower resolution system, are provided to the lower resolution system.

BACKGROUND OF THE INVENTION

Software applications and/or hardware systems have been provided to enable audio, video, and other media content to be edited, manipulated, enhanced, modified, mixed, and arranged to provide a desired media output. For example, such applications/systems may be used to combine and sequence selected portions of video, audio, animation, and/or other media content files—sometimes referred to herein as “clips”—to provide a desired output. Examples of such applications include without limitation Soundtrack® Pro (“STP”) and Final Cut Pro® (“FCP”) available from Apple Inc.

FIG. 1shows an example of a media editing/mixing interface100. In the example shown, media clips denoted “Clip1”, “Clip2”, “Clip3”, and “Clip4” have been arranged along a time or other sequence line indicated by the numbers “0” to “29”, with the start and stop of each numbered interval being indicated by a vertical dashed line. In the example shown, Clip1and Clip3would begin being rendered at the start of interval “3”; Clip2would begin being rendered at the start of interval “7”; , Clip3would end at the end of interval “15”; Clip4would start at the beginning of interval18; Clip2would end at the end of interval “18”; Clip1would end at the end of interval “25”; and finally Clip4would end at the end of interval “26”. The Clips1-4may comprise any media content. For example, Clip1might be video content, and Clips2,3, and4audio desired to be rendered with corresponding portions of video content in Clip1.

Some media editing/design applications have a higher resolution, i.e., the ability to place a clip at finer gradations along a time or other sequence line, than other such applications. For example, an application focused primarily on audio content, such as STP, may provide the ability to specify start/stop times to a precision of some small fraction of a second (e.g., a 64 bit floating point value in the case of STP), while a video application such as FCP may only support starting/stopping a clip at an indicated frame number or index, typically at a relatively more course granularity, such as 24-30 frames per second (i.e., timeline gradations of 1/30- 1/24 seconds). In some cases, it may be desirable or necessary to represent in a lower resolution application and/or system media content and/or a project created in a higher resolution system, for example to enable an audio track or mix created in STP to be combined in FCP with video edited and/or arranged in FCP to create a combined multi-media project and/or output. However, the start/stop boundaries of clips as arranged in the higher resolution system, for example at a granularity of some small fraction of a second, may not line up neatly with the gradations of the lower resolution system.

FIG. 2illustrates the potential misalignment of start/stop boundaries when higher resolution media content is represented in a lower-resolution application or system. As used herein the terms “higher resolution system” and “lower resolution system” include without limitation media editing and/or design software and/or hardware and/or combinations thereof. Note that in this sense the higher and lower resolution “systems” may reside in the same physical computer, such as would be the case if each were an application running on the same machine. In the example shown inFIG. 2, a first clip202and a second clip204have been arranged in a media editing interface200of a first, higher resolution application, associated inFIG. 2with the lower time or sequence line indicated by the numbers “0” through “29”. The clips are desired to be represented in a second, lower-resolution system, represented inFIG. 2by the upper, lower-resolution frame indexes/numbers (“Frame0” through “Frame5”) and the corresponding thick (non-dashed) vertical lines. In this example, there are an even five intervals of the higher resolution system for each interval (i.e., frame) of the lower resolution system. However, in many cases there will not be such an even ratio. As the example shown inFIG. 2illustrates, the higher resolution content must be represented in the lower resolution system in a manner that enables the lower resolution system to render the content in a manner that (at least most nearly) matches the designer's intent and is within the capabilities of the lower resolution system. In this example, the lower resolution system can only start or stop clips at frame boundaries, and cannot directly render the clips202and204as arranged in the higher resolution system, because that would require clip202to start in the middle of Frame0, for example. In the example shown inFIG. 2, clip202comprises a portion of an underlying media file206that includes other content not included in clip202, as will be discussed further below.

FIGS. 3A-3Cillustrate prior art techniques for representing higher resolution media content in a lower resolution system. InFIG. 3A, the clips202and204have been shifted and truncated to provide shifted clips322and324; but this approach results in the clips starting and stopping later/earlier than the designer intended, potentially resulting in their being out of synch with other/related content. Also, in this example the end of each of clips202and204(portions “23-25” and “Q-S”, respectively) has been lost. In the approach shown inFIG. 3B, clip202ofFIG. 2has been truncated at each end to create a truncated clip332, which starts and stops on the lower resolution frame boundaries. Likewise, clip204has been truncated to create a truncated clip334that aligns with the frame boundaries of the lower-resolution system. In this approach, the portions included are rendered at the time originally desired but the truncation results in parts of the media content—in this example portions “13” and “14” at the beginning of clip202and portion “25” at the end, as well as portions “G-I” at the beginning of clip204, being lost (i.e., not represented and therefore not rendered) in the lower-resolution system. Finally, in the approach shown inFIG. 3C, content comprising clips arranged in a higher resolution system is incorporated into one or more clips expanded in length at the beginning and/or end, as required, for example by adding silence or other null content, to create one or more expanded clips that begin and end on frame (or other lower resolution system) boundaries, such as the expanded clip342ofFIG. 3C. The approach shown inFIG. 3C, however, may limit flexibility, e.g., the ability to further edit or manipulate clips in the destination (lower-resolution) system. Typically a clip such as clip202comprises a view or other representation and/or designation of a selected portion of an underlying media content file, such as an audio or video content file, for example the media file206shown inFIG. 2. By way of example, the clip202may include a selected scene or other portion of an audio or video file that include other content before or after the portion included in the clip. The lower resolution system may allow a clip to be modified (or a new clip defined) to include such additional portions, e.g., by moving the clip start boundary to the left or the clip end boundary to the right, but such flexibility to further edit would not be available if only a static combined and expanded clip such as clip342were provided to the lower-resolution system.

DETAILED DESCRIPTION

Representing high-resolution media content in a lower resolution system is disclosed. In some embodiments, one or more “loose end” and/or intersection files having boundaries aligned with time or other sequence line increments of a lower resolution system are combined with a truncated clip of one or more original media files to provide a set of two or more adjacent clips, each of which starts and stops at the beginning/end of an interval of the lower resolution system and which together include all of the originally selected and desired media content with each selected portion being rendered at a time/point as indicated in the higher resolution system.

FIG. 4Aillustrates an embodiment of a technique for representing high-resolution media content in a lower resolution system. In the example shown, the clips202and204ofFIG. 2are represented in the lower resolution system (associated with the frames0through5ofFIG. 4A) by clips402,404,406, and408. In some embodiments, each of clips402-408is associated with a corresponding file. For example, clip404is formed by generating a shortened clip, aligned with the frame (or other) boundaries of the lower resolution system, based in this example on the original media file206on which clip202is based. Likewise, clip408is a shortened clip based on the same underlying media file as clip204. In addition to the shortened clips404and408and the underlying original media files, a “loose end” clip402and an “intersection” clip406, as associated files, are created. In some embodiments, clips402and406, and associated underlying files, are generated programmatically, for example by a higher resolution system, in response to receiving an indication that high resolution media content, such as clips202and204, is to be represented (e.g., exported to) a lower resolution system. In the example shown, loose end clip402includes a silence, filler, or other null content portion (represented inFIG. 4by the large dots at the beginning of clip402), which starts at the beginning of the lower resolution boundary immediately preceding the start time/location of original clip202, in this case Frame0, and ends at the first lower resolution (frame) boundary crossed by the original clip202, in this case the beginning of Frame1. Those portions of clip202that were truncated from the beginning of clip202to create clip404are included in the loose end clip402at a point such that they will be rendered in the lower resolution system, as part of clip402, beginning at a point in (absolute and/or relative) time corresponding to the placement of clip202in the higher resolution system. In some embodiments, clip402is associated with an underlying loose end file, not shown, that includes at least the media content included in clip402. In some embodiments, the underlying loose end file includes additional content, for example at least an additional portion of the content file206on which the original clip202was based, such as portions “15” through “19” or some other additional portion(s), to enable the loose end clip to be edited and/or otherwise manipulated to at least some extent in the destination (lower resolution) system. The intersection clip406includes in this example the portion removed from the end of clip202to create clip404, i.e., portion “25”, a null content portion (large dot), and the portions of clip204removed from the beginning of clip204to create clip408, i.e., portions “G”, “H”, and “I” in this example. As with loose end clip402, intersection clip406in some embodiments is based on and/or otherwise associated with an underlying file created by combining the included portions of clips202and204with null content. As with the file underlying loose end clip402, in some embodiments the file underlying intersection clip406includes additional adjacent portions of clip202(e.g., portions “20” through “24”) and/or clip204(e.g., portions “G” through “I”) to enable at least limited editing and/or other manipulation of the intersection clip in the destination (lower resolution) system.

In some embodiments, a shortened clip such as clip404is based on a media file other than the complete underlying media file on which a corresponding clip in the higher resolution system is based. For example, in some embodiments the clip404may be based on a specially-generated media file that includes some but not necessarily all additional portions of the original media file. In the example shown inFIG. 4A, for example, in some embodiments the clip404is based on a generated media file that includes just portions “10” through “29” of original media file206. Using this approach, limited additional content is provided to enable limited editing in the lower resolution system. In some embodiments, the specially-generated file is or may be generated such that the beginning and end boundaries of a clip comprising the entire file would align with corresponding frame (or other) boundaries of the lower resolution system. In the example shown inFIG. 4A, for example, a specially-generated file may include portions “10” through “29” because a clip that included the entire file would align with the beginning of Frame0and the end of Frame5in the example shown. In some embodiments, silence or other null content may be included at the beginning and/or end of the file to ensure alignment with the boundaries of the lower resolution system. In some embodiments, the original file may be provided, instead of a specially-generated file, in certain circumstances, for example based on a user-configurable preference or in situations in which multiple clips share the same media file. In some embodiments, a specially-generated file is created and provided to the lower resolution system instead of the original media file if the original media file is large and/or large relative to the size of an associated clip. In some embodiments, a specially-generated file with boundaries that align with frame (or other) boundaries of the lower resolution system is provided, instead of the original file, at least in cases where the lower resolution system is not able to use a section of the original media file, if any, that does not begin and/or end on a frame (or other) boundary of the lower resolution system.

FIG. 4Billustrates an embodiment of a technique for representing high-resolution media content in a lower resolution system. In the example shown inFIG. 4B, the clip404has been expanded to create an expanded clip404B that begins at the beginning of Frame0and ends at the end of Frame3and includes items “10” through “29” of the original media file206. The clip404B may be based on either a specially-created file that includes just that or possible additional content, or the original media file206, depending on the embodiment.

FIG. 5is a flow chart illustrating an embodiment of a process for representing high-resolution media content in a lower resolution system. In some embodiments, the process ofFIG. 5is used to generate clips such as those shown inFIG. 4. In the example shown, an indication that one or more clips arranged along a timeline (or other sequence line) in a higher resolution system are to be exported to a lower resolution system is received (502). The resolution required and/or supported by the destination system is determined (504). One or more loose end and/or intersection files and/or truncated views of original media files, and/or associated clips, are generated (506). A description of how the files are to be arranged and/or used in the lower resolution system is created (508). The files and description are sent to the lower resolution system (510), after which the process ends.

A benefit of the approach described above in connection withFIGS. 4 and 5is that the original media content files underlying clips202and204are provided to the lower resolution system, in connection with clips404and408, which enables full editing and/or other manipulation of the original files, as desired, in the destination system. The approach described herein in some embodiments preserves the intent of the designer in the higher resolution system and enables high resolution media content to be represented in a lower resolution system without loss of content, flexibility, and/or synchronization.

In some embodiments, the approach disclosed herein is used to represent in a destination system a cross-fade and/or other effect created in a source system.FIGS. 6A and 6Billustrate a prior art approach to exporting to a destination system an effect, such as cross-fade, from a system in which the effect was created. In the example shown, a “cross-fade” effect is created in a source system (e.g., editing interface600) by causing a first clip (Clip1) to fade out while a second clip (Clip2) fades in. In a prior art approach shown inFIG. 6B, a combined clip/file that includes statically a representation of a cross-fade portion610is created and provided to the destination system. Note that the example shown has been simplified by having Clip1start at a frame (i.e., lower resolution interval) boundary and Clip2end at such a boundary, to set aside other issues.

FIGS. 7A and 7Billustrate an embodiment of representing in a destination system cross-fade and/or other effects created in a source system. In the example shown, the original Clips1and2have been used to create truncated clips Clip1′ and Clip2′ and intersection clip702, which latter clip includes the cross-fade and/or other effect. As noted above, the approach shown inFIG. 7Benables the original media files underlying Clips1and2(or at least relevant additional potentially desirable portions thereof) to be provided to the destination system, with the overlapping portions associated with the cross-fade or other effect being provided in a separate file underlying clip702. In this way, the original files (or additional portions thereof) are available at the destination system for further editing and/or other manipulation.