System and method for audio video pattern sequence diagnostic tool

There is provided a system and method for confirming correct encoder processing for a media file. There is provided a method comprising preparing a diagnostic clip, embedding the diagnostic clip in the media file to create a modified media file, encoding the modified media file according to a set of encoder settings to create an encoded media file embedded with an encoded diagnostic clip, displaying the encoded diagnostic clip to a user, and assessing the accuracy of the encoder settings based on a feedback received from the user. In this manner, users may quickly and accurately diagnose encoder settings after encoding the media file. Users may more easily identify which encodes caused which problems and avoid the costly overhead of backtracking in the production process.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates generally to media file creation. More particularly, the present invention relates to verifying the correct encoding of media files.

2. Background Art

Modern media production workflows increasingly rely on digital formats, such as compressed video data on flash media, rather than analog formats of the past, such as tapes. Digital media provides several advantages over analog media, such as ease of distribution and increased reliability. However, the transition to digital media production workflows has introduced several new problems, particularly with respect to format encoding and transcoding.

For example, specific hardware devices or encoding steps used within a production workflow may require the video data to be in a specific format. Thus, the video data must be manipulated, for example by encoding or transcoding to the appropriate format. Each manipulation, however, increases the risk of introducing encoding errors in the video data. For example, due to differences between hardware, software, production environments, and other variables, encoder settings at particular manipulation steps may be improperly configured. Thus, while an encoded video file may include a valid file structure, the video content may contain encoding errors resulting in video or audio degradation. In addition, pinpointing an encoding step that introduced an error may be difficult in extended multi-step workflows.

Moreover, while some encoding errors may be quickly found using standard manual or automated quality control procedures, other encoding errors may not be immediately apparent or amenable to automatic detection. Some errors, such as detecting incorrect cadence, often require close manual examination and knowledge of the specific problem. Other errors, such as audio down-mixing, may be difficult to detect even with knowledge of the problem. Thus, encoding mistakes may go undetected until later stages of a production process, necessitating expensive and time consuming backtracking to restore proper video and audio quality.

Accordingly, there is a need to overcome the drawbacks and deficiencies in the art by providing a diagnostic tool that enables users to quickly and accurately confirm the correct video and audio encoding of media content.

SUMMARY OF THE INVENTION

There are provided systems and methods for a media content diagnostic tool comprising audio and video pattern sequences embedded in a media file to confirm correct encoder processing after encoding, substantially as shown in and/or described in connection with at least one of the figures, as set forth more completely in the claims.

DETAILED DESCRIPTION OF THE INVENTION

The present application is directed to a system and method for a media content diagnostic tool comprising audio and video pattern sequences embedded in a media file to confirm correct encoder processing after encoding. The following description contains specific information pertaining to the implementation of the present invention. One skilled in the art will recognize that the present invention may be implemented in a manner different from that specifically discussed in the present application. Moreover, some of the specific details of the invention are not discussed in order not to obscure the invention. The specific details not described in the present application are within the knowledge of a person of ordinary skill in the art. The drawings in the present application and their accompanying detailed description are directed to merely exemplary embodiments of the invention. To maintain brevity, other embodiments of the invention, which use the principles of the present invention, are not specifically described in the present application and are not specifically illustrated by the present drawings.

FIG. 1presents a system for a media content diagnostic tool comprising audio and video pattern sequences, according to one embodiment of the present invention. Diagram100ofFIG. 1includes diagnostic clip110, media file120, modified media file130, media encoder140, encoded media file150, display160, and user170. Diagnostic clip110includes center-cut extraction112, safe area113, field display order114, cadence detection115, interpolation116a, interpolation116b, interpolation116c, interpolation116d, audio slates117, and additional patterns118. Modified media file130includes media file120and diagnostic clip110. Media encoder140includes encoder settings145, encoding processor146, such as a microprocessor, and memory (not shown) for storing instructions for execution by processor146and data storage and retrieval. Encoded media file150includes encoded diagnostic clip155and encoded media content156. Display160includes speakers165.

Diagnostic clip110may comprise a ten second video sequence comprising a set of graphical patterns and audio tracks, such as center-cut extraction112, safe area113, field display order114, cadence detection115, interpolation116a-116d, audio slates117, and additional patterns118. Diagnostic clip110may be used as a diagnostic tool for a media file undergoing encodes or transcodes by providing immediate visual and aural feedback.

According to one embodiment of the present invention, diagnostic clip110may be a file object within a media encoding system. However, in alternative embodiments, the components of diagnostic clip110may comprise a standalone process or method. Thus, depending on the desired detection features, any combination of center-cut extraction112, safe area113, field display order114, cadence detection115, interpolation116a-116d, audio slates117, and additional patterns118may be applied as a standalone process to any media content for diagnostic testing.

Center-cut extraction112may aid in exposing any issues with inaccurate 4×3 center-cut extraction from 16×9 source material. Because a 16×9 source material is a larger resolution than can be displayed on a 4×3 screen, a 4×3 video may be extracted from the 16×9 material. Center-cut extraction112may help determine whether the correct material was extracted. Safe area113may provide a similar function. Certain images may have to stay within a safe area of the video screen, which may be a smaller area within the 4×3 center-cut. Safe area113may help determine whether correct images remain within the safe area.

Field display order114may assist in identifying incorrect field display order or field dominance. Media file120may need to be broadcast to an interlaced output device, such as an older television set. Interlaced devices receive video as a stream of fields. A field contains data for every other scanline of the display, in contrast with a frame, which contains data for every scanline. Half of the fields are upper fields, which are the fields that contain data from the odd-numbered scanlines starting with the top scanline. Lower fields contain the even-numbered scanlines, starting with the second scanline from the top. The upper and lower fields may then be paired as a frame, forming a complete set of scanlines. Field dominance determines whether a frame starts with its upper or lower field. Incorrect upper/lower field dominance may produce stuttered video motion.

Cadence detection115may aid in detecting incorrect cadence. The sequence of fields in the media file must have the correct cadence to be displayed properly. For example, a source material may comprise frames A, B, C, and D in sequence. If that sequence of frames underwent an encoding process, such as doubling each frame into field pairs, the cadence must remain intact. The resulting sequence should read A-A-B-B-C-C-D-D. Thus, a sequence such as A-A-B-C-B-C-D-D would be incorrect cadence.

Interpolation116a-116dmay help in detecting interpolation. Interpolation, specifically temporal interpolation, occurs when adjacent frame images are blended together. The interpolated image may contain elements of both blended images. Interpolation116a-116dmay indicate if interpolation was induced during encoding.

Audio slates117may comprise several audio tracks that may aid in diagnosing audio problems. Audio slates117may comprise audio slates for each of the 24 standard audio channels. Each track may have a voice announcing the track number, which corresponds to the original channel the track resides on. For example, spoken words “channel1” may be heard when playing back channel1. The tracks may also be staggered, thus a track may never be heard in sync with another track. Encodes may re-assign or re-map audio tracks from the original channel positions to new ones. Audio slates117may assist in the difficult and error prone task of confirming what audio track exists on which channel. For example, channel3may have been re-mapped to channel7. With audio slates117, “channel3” may be heard when playing back channel7. Hearing another track number, hearing no voice, or hearing “channel3” on another channel may indicate an audio mapping issue.

Additionally, audio slates117may also help discover unintentional down-mixing. During the encoding of media file120, several tracks may be accidentally sent to the same channel, or down-mixed. Normally, listening to the audio is a difficult way to detect unintentional down-mixing. However, when audio slates117undergoes down-mixing, the error may be easier to discover. Because audio slates117are staggered, several asynchronous voices may be heard if several tracks have been down-mixed. In addition, the staggered audio simplifies the process of identifying the track numbers that were down-mixed.

Additional patterns118may comprise additional graphical patterns or audio tracks, or variations of existing patterns and tracks. Additional tools, such as additional letter-box demarcations, may be necessary to diagnose issues not detectable with a default pattern set. Additional patterns118may also comprise other information, such as timestamps or identifiers.

Media file120may comprise media content which will undergo encodes or transcodes by media encoder140. It may, for example, comprise original source material. Alternatively, media file120may comprise media content that has already been encoded and will be encoded again. Because encodes may introduce unwanted results in its media content, media file120may require verification of its media content.

Modified media file130may comprise media file120and diagnostic clip110. Diagnostic clip110may be placed at the head or tail of media file120, similar to bars and tone signals used in video masters, to form modified media file130. In alternative embodiments, diagnostic clip110may be used as a stand-alone clip. Regardless of the method, diagnostic clip110may be encoded by the same encoder settings145as media file120.

Media encoder140may comprise a computing device capable of performing encodes or transcodes on modified media file130. Media encoder140may also be configured with encoder settings145. Unfortunately, the incorrect settings may result in unwanted changes to modified media file130. However, encoder settings145may be correctly reconfigured with correct settings through additional input, such as from user170though a computer interface. Then, media encoder140may re-encode modified media file130. Media encoder140may additionally comprise encoding processor145. Encoding processor145may comprise a processor capable of performing encodes or transcodes on modified media file130according to encoder settings145. After encoding or transcoding modified media file130into encoded media file150, media encoder140may output encoded media file150to display160. Alternatively, media encoder140may also store encoded media file150for future reference.

Encoded media file150may comprise the result of media encoder140's encodes on modified media file130. Encoded media file150may comprise encoded diagnostic clip155and encoded media content156, which may comprise the encoded versions of diagnostic clip110and media file120, respectively. Although depicted inFIG. 1as elements of encoded media file150, in alternative embodiments encoded diagnostic clip155and encoded media content156may each comprise separate files.

Display160may comprise any device capable of displaying encoded media file150. Display160may comprise the same device as media encoder140, such as a server. Alternatively, display160may be a remote machine, such as a workstation. Display160may also comprise speakers165. Speakers165may comprise any device capable of outputting the audio from encoded media file150. Speakers165may be part of display160, as shown inFIG. 1, or may comprise a separate device, such as standalone speakers.

User170may comprise a person or persons trained to use diagnostic clip110. User170may be able to view encoded diagnostic clip155and interpret its graphical patterns and audio tracks. In addition, user170may be able to reconfigure encoder settings145.

Diagnostic clip110may first be customized by user170. For example, center-cut extraction112, safe area113, field display order114, cadence detection115, interpolation116a-116d, audio slates117, and additional patterns118may all vary to suit the needs of user170. Diagnostic clip110may then be bundled with media file120, either as one file or separate files, to form modified media file130. Modified media file130may then be sent to media encoder140to undergo encodes.

Media encoder140encodes modified media file130to encoded media file150according to encoder settings145. Encoder settings145may be preconfigured with settings for a particular encode. After the particular encoding, user170may then view and hear encoded media file150immediately on display160. Undesired results in encoded diagnostic clip155may be symptoms of common encoding issues, such as an inaccurate 4×3 center-cut extraction from 16×9 source material, a reversed field display order, an incorrect video cadence, an interpolation, an improper audio channel mapping, or an unwanted audio down-mixing. By recognizing any anomalous graphical patterns and audio tracks in encoded diagnostic clip155, user170may determine what settings in encoder settings145require adjustment.

Encoded diagnostic clip155may also reveal which encodes caused problems. For example, if after the particular encode, encoded diagnostic clip155developed a graphical anomaly, user170may know that that particular encode caused the error. User170may then reconfigure encoder settings145to correct the issues revealed by encoded diagnostic clip155. After re-encoding modified media file130, user170may immediately watch encoded diagnostic clip155again and reconfigure encoder settings145based on that feedback. User170may repeat this cycle until encoded diagnostic clip155is free of errors. Alternatively, media encoder140may perform all encoding jobs needed on modified media file130while storing intermediary encodes. User170may later review the intermediary encodes to identify specific problematic encodes.

When encoded diagnostic clip155no longer displays graphical or audio anomalies, encoded media content156may be error-free as well, because both undergo encodes with the same encoder settings145. The resultant encoded media file150may then be ready for broadcast or additional processing.

Diagnostic clip frame210may represent one frame from diagnostic clip110's video sequence. Diagnostic clip frame210may comprise an image of 1920 pixels across, and1080pixels down. In the embodiment depicted inFIG. 2, diagnostic clip frame210comprises a frame with a 16×9 black background. In alternative embodiments, diagnostic clip frame210may comprise a frame with a different background, such as a 16×9 video image. Diagnostic clip frame210may comprise graphical patterns from center-cut extraction220, safe area230, field display order240, cadence detection250, and interpolation260a-260d, which may all vary to suit user170's needs. Diagnostic clip frame210may also comprise additional patterns118fromFIG. 1, depending on user170's needs.

Center-cut extraction220, which may correspond to center-cut extraction112inFIG. 1, may comprise markers that delineate a certain area within the frame. As shown inFIG. 2, center-cut extraction220may comprise an outer white rectangle within the frame. Center-cut extraction220may represent the exact area for the active video of a correctly extracted 4×3 image. In one embodiment, the top line of center-cut extraction220may reside on line21and the bottom line on line1123, when displayed as a 1080 23.98 PsF signal. The left-side line may reside at pixel240, and the right-side line at pixel1679. User170may verify the center-cut extraction by viewing the resultant active video. If an image falls short or exceeds any of center-cut extraction220's white borders, the image may not be sized correctly. Alternatively, user170may look for the white borders at all four sides of the 4×3 image.

Safe area230, which may correspond to safe area113inFIG. 1, may also comprise markers which delineate a certain area within the frame. As shown inFIG. 2, safe area230may comprise a white rectangle inside the outer white rectangle, or center-cut extraction220. Safe area230may delineate an area which is a fraction of center-cut extraction220, such as 90%. Thus, a 90% safe area may delineate 90% of the area represented by center-cut extraction220. User170may then verify any images that must remain in the safe area by checking to see if the images stay within safe area230's white border. User170may also be able to determine if parts of the frame's image, such as its center, have been disproportionately expanded compared to the rest of the frame's image, such as its edges.

Field display order240, which may correspond to field display order114in FIG.1, may comprise visual cues that loop through the entirety of diagnostic clip110's sequence. In the embodiment depicted inFIG. 2, field display order240comprises a black vertical bar on a white box background. Field display order240may be smaller than safe area230, and placed near the center of the frame. The black bar may move horizontally within the white box. Starting from the left, the black bar may move to the right every second, and then loop back. The black bar's stuttering or jerky motion, instead of fluid motion, may indicate an incorrect field display order.

Cadence detection250, which may correspond to cadence detection115inFIG. 1, may comprise various markings capable of displaying information about a frame cadence. InFIG. 2, cadence detection250comprises the letters “A,” “B,” “C,” and “D” near the bottom of the screen. Each letter may correspond to successive frames, starting with the “A-frame,” and returning to A after D. The underscore, seen below A inFIG. 2, may indicate which frame is currently viewed. As diagnostic clip110's sequence progresses, the white underscore may shift from A to B to C and so forth, until it loops back to A after D. User170may then be able to recognize incorrect cadence by viewing the underscore's movement. Additionally, user170may verify that the cadence begins on the A-frame at the top of each second, which may be as critical as the correct cadence itself. Cadence detection250allows the user to verify cadence without regard to the display hardware or player software being used.

Interpolation260a-260d, which may respectively correspond to interpolation116a-116dinFIG. 1, each comprise a marker which may assist user170in detecting interpolation within video images. As shown inFIG. 2, interpolation260a-260dmay each comprise a dash pattern, in a vertical orientation, at the corners within safe area230. With each successive frame, interpolation260a-260dmay alternate from vertical to horizontal orientations. If interpolation was induced during an encode, the interpolated frames may blend the alternating orientations together, forming a cross shape instead of a single dash. User170may then be able to quickly determine if interpolation was induced, by viewing cross shapes instead of horizontal or vertical dash shapes.

Frame set310may comprise a sequence of five consecutive frames from encoded diagnostic clip155. Frame set310may comprise frame311-315. Frame311-315, which may each correspond to diagnostic clip frame210inFIG. 2, each comprises a frame from encoded diagnostic clip155's video sequence. Accordingly, each of frame311-315may comprise similar elements to diagnostic clip frame210.

Likewise, frame set320may also comprise a sequence of five consecutive frames from encoded diagnostic clip155. Frame set320may comprise frame321-325. Frame321-325, which may each correspond to diagnostic clip frame210inFIG. 2, each comprises a frame from encoded diagnostic clip155's video sequence. Accordingly, each of frame311-315may comprise similar elements to diagnostic clip frame210.

Because many of the problems revealed by diagnostic clip110may involve multiple frames, user170may need to view several frames to detect certain problems. For example, verifying cadence after a 2:3 pull-down requires viewing multiple frames. NTSC video is broadcast at 30 or 29.97 frames per second (fps). However, a source material may be filmed at 24 or 23.98 fps. Content at 24 fps must be converted to 30 fps. The frame ratio is 24/30, or 4/5, which means for every 4 frames at 24 fps, there should be 5 frames for 30 fps. 2:3 pull-down is a method of stretching the 4 frames into 5. For example, if the 24 fps content had source frames A, B, C, and D, then the corresponding fields are normally made by placing the same frame across the upper and lower field pair. The cadence would be A-A-B-B-C-C-D-D, yielding frames AA, BB, CC, and DD. To get that extra frame, every other frame is placed across three, rather than two fields. Thus, the2:3cadence would now be A-A-B-B-B-C-C-D-D-D, and the five frames would consist of AA, BB, BC, CD, and DD.

Frame set310and320may each represent a possible diagnostic tool sequence after a 2:3 pull-down of a 24 fps source. The four corner patterns in each of frame311-315,321-325may each correspond to interpolation116a-116dinFIG. 1, and interpolation260a-260d inFIG. 2; the four letters and underscores in each of frame311-315,321-325may correspond to cadence detection115inFIG. 1, and cadence detection250inFIG. 2; and the white boxes containing vertical bars in each of frame311-315,321-325may each correspond to field display order114inFIG. 1, and field display order240inFIG. 2.

Frame set310may comprise frame311-315shown in “full frame,” wherein both fields of a frame are shown together. Frame311may show no sign of interpolation, as the corner marks are full white dashes instead of crosses, which indicates both fields are from the same source frame. In addition, frame311may be the A-frame, as indicated by the underscore below A. Therefore frame311may be frame AA. In a similar fashion, frame312and315may be frame BB and DD, respectively. Frame313may show signs of interpolation, because the corner marks are gray crosses instead of white dashes, which indicates the blending of black and white. The gray underscores beneath letters B and C suggest the blending of fields from frames B and C, indicating frame313may be frame BC. In a similar fashion, frame314may be identified as frame CD. Frame313and314are jitter frames because they may have been blended from two different source frames. Since frame311-315may comprise frames AA, BB, BC, CD, DD from the 2:3 cadence discussed above, frame set310may follow the correct 2:3 cadence, which includes starting on the A-frame. Finally, the vertical black bar moves from left to right in the frame sequence, indicating correct field order.

Frame set320may comprise frame321-325shown in “field mode,” wherein only one of the two fields per frame is displayed. Here, frame set320may display the second field only, and thus no jitter frames may be present. Since the corner dashes alternate according to the frame, none of frame321-325show signs of interpolation, which may be expected in field mode. The underscores indicate frame321-325may comprise frames A, B, C, D, and D again, respectively. A correct cadence in field mode, after taking out the first fields from the 2:3 cadence, is A-B-C-D-D. Thus, frame set320may follow the correct cadence, starting with the A-frame as well. According to frame set310and320, user170may not need to adjust encoder settings145with respect to cadence, field order, or interpolation.

Moving toFIG. 4,FIG. 4shows a flowchart describing the steps, according to one embodiment of the present invention, by which a trained user may utilize a diagnostic tool to verify media content after an encode. Certain details and features have been left out of flowchart400that are apparent to a person of ordinary skill in the art. For example, a step may comprise one or more substeps or may involve specialized equipment or materials, as known in the art. While steps410through450indicated in flowchart400are sufficient to describe one embodiment of the present invention, other embodiments of the invention may utilize steps different from those shown in flowchart400.

Referring to step410of flowchart400inFIG. 4and diagram100ofFIG. 1, step410of flowchart400comprises preparing diagnostic clip110inFIG. 1for encodes. Diagnostic clip110may be preset with a default set of graphical patterns. However, user170may also want to add or modify graphical or audio patterns. For example, user170may wish to modify safe area113by reducing its size, or add additional patterns118. User170may know what graphical and audio patterns have been set, thus expecting a certain output from encoded diagnostic clip155.

Referring to step420of flowchart400inFIG. 4and diagram100ofFIG. 1, step420of flowchart400comprises embedding diagnostic clip110in media file120to form modified media file130. As previously discussed, diagnostic clip110may be placed at the head or tail of media file120before any encodes. Alternatively, diagnostic clip110may be used as a stand-alone clip that would follow media file120through a tapeless workflow. Diagnostic clip110may undergo the same encodes as media file120, ensuring that user170's diagnosis of diagnostic clip110applies to media file120as well.

Referring to step430of flowchart400inFIG. 4and diagram100ofFIG. 1, step430of flowchart400comprises media encoder140's encoding processor146encoding modified media file130according to encoder settings145to create encoded media file150. Encoded media file150may comprise encoded diagnostic clip155and encoded media content156, which may comprise the encoded versions of diagnostic clip110and media file120, respectively. Encoder settings145may not be properly configured to correctly encode modified media file130. Encoder settings145may require adjustments, after which modified media file130may be re-encoded.

Referring to step440of flowchart400inFIG. 4and diagram100ofFIG. 1, step440of flowchart400comprises displaying encoded diagnostic clip155on display160to user170. As previously discussed, display160may comprise a separate device from media encoder140. User170may also need to hear encoded diagnostic clip155through speakers165. Speakers165may comprise a separate device from display160or may comprise the same device. Display160may faithfully display encoded diagnostic clip155, including any errors created from encoding.

Referring to step450of flowchart400inFIG. 4, diagram100ofFIG. 1, and diagram200ofFIG. 2, step450of flowchart400comprises user170assessing the accuracy of encoder settings145by viewing encoded diagnostic clip155. User170may view encoded diagnostic clip155to receive immediate visual and aural feedback as to the unwanted effects of encoding. As previously discussed, the elements of diagnostic clip frame210may provide user170with various indicators about common issues that may arise after encodes. For example, user170may notice that parts of center-cut extraction220do not appear, indicating an error with a center-cut extraction. User170may then correct the center-cut extraction settings in encoder settings145. Display160may allow user170to adjust encoder settings145. Alternatively, user170may adjust encoder settings145through media encoder140directly. Once the known issues are resolved, diagnostic clip110may then be re-encoded according to the adjusted encoder settings145and subsequently reviewed by user170to verify the corrections made. In the example, user170may redo the center-cut extraction and then confirm that center-cut extraction220appears correctly on screen.