Abstract:
This document describes, among other things, systems and methods for visual representation and review of media files. A method comprises: 1) accessing a media file that includes at least an audio track; accessing a media review file, wherein the media review file is associated with the media file and further wherein the media review file includes annotation data, the annotation data including two or more users&#39; annotation comments; 2) presenting the media file and the media review file using a graphical user interface, wherein the graphical user interface includes a vertical time bar, wherein the vertical time bar includes an indicator to indicate a position in the audio track included in the media file and further wherein annotation data displayed in the user interface vertically scrolls in synchronization with the indicator on the vertical time bar; and 3) saving any modifications to annotation data in the media review file.

Description:
TECHNICAL FIELD 
       [0001]    This patent document pertains generally to graphical user interfaces, and more particularly, but not by way of limitation, to visual representation and review of media files. 
       BACKGROUND 
       [0002]    Media files, including audio or video files, are continually being created for many different types of applications. Telemarketer conversations, video broadcasts, books recorded on tape, university lectures, and video recordings of golf swings are only a few examples of the various types of media files that exist. 
     
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         [0003]      FIG. 1  illustrates a functional block diagram of a media file reviewing system in accordance with an example embodiment. 
           [0004]      FIG. 2  illustrates an example of the data stored in an authorization file in accordance with an example embodiment. 
           [0005]      FIG. 3  illustrates an example of data stored in a media review file in accordance with an example embodiment. 
           [0006]      FIG. 4  illustrates a further example of data stored in a media review file after a review by reviewer “Joe” in accordance with an example embodiment. 
           [0007]      FIG. 5  illustrates an example search graphical user interface (“GUI”) in accordance with an example embodiment. 
           [0008]      FIG. 6  illustrates an example list GUI in accordance with an example embodiment. 
           [0009]      FIG. 7  illustrates an example reviewer software GUI in accordance with an example embodiment. 
           [0010]      FIG. 8  illustrates a flow diagram of a method of creating and maintaining a database in a media file reviewing system in accordance with an example embodiment. 
           [0011]      FIG. 9  illustrates a flow diagram of a method of using the media file reviewing system in accordance with an example embodiment. 
           [0012]      FIG. 10  illustrates a diagrammatic representation of a machine capable of performing the methods or implementing the systems/devices described herein. 
       
    
    
     DETAILED DESCRIPTION 
       [0013]    In the following detailed description of example embodiments of the invention, reference is made to specific example embodiments of the invention by way of drawings and illustrations. These examples are described in sufficient detail to enable those skilled in the art to practice the invention, and serve to illustrate how the invention may be applied to various purposes or embodiments. Other embodiments of the invention exist and are within the scope of the invention, and logical, mechanical, electrical, and other changes may be made without departing from the subject or scope of the present invention. Features or limitations of various embodiments of the invention described herein, however essential to the example embodiments in which they are incorporated, do not limit other embodiments of the invention or the invention as a whole, and any reference to the invention, its elements, operation, and application do not limit the invention as a whole but serve only to define these example embodiments. The following detailed description does not, therefore, limit the scope of the invention, which is defined only by the appended claims. 
         [0014]    Often media files are reviewed and annotated. For example, telemarketer conversations may be recorded as an audio file, with subsequent reviewers analyzing the recorded conversations and annotating, for example, mistakes, profanity, and sales events. News or sports broadcasts may be recorded as a video file with important aspects of the broadcast highlighted with annotations. There are many other examples of situations where media files are reviewed. 
         [0015]    A media file may be reviewed and annotated by multiple reviewers. For example, an advertising agency may have multiple marketing executives analyzing an audio file of a telemarketing phone call. In some cases, it is useful for each marketing executive to analyze a previous reviewer&#39;s annotations, while at the same time, contributing his own annotations. Moreover, reviewing a media file is a time-consuming process. What is needed is a better way to facilitate the review of media files by multiple reviewers. 
         [0016]    The present invention is a system for, and method of, visual representation and review of media files. In particular, a media file reviewing system provides mechanisms for reviewing media files in a visual fashion, displaying media files and the accompanying annotations in a highly readable fashion, searching media files, and facilitating the review of media files by multiple reviewers. 
         [0017]      FIG. 1  illustrates a functional block diagram of a media file reviewing system  100  in accordance with an example embodiment. In an embodiment, media file reviewing system  100  includes a management computer  110 , a network  112 , and a user computer  114 . Management computer  110  further includes a database  116  that further contains a plurality of media files  118 , a plurality of media review files  120 , and an authorization file  122 ; a first instance of reviewer software  124 , which is represented by reviewer software  124   a;  and automated analysis software  126 . User computer  114  further includes a second instance of reviewer software  124 , which is represented by reviewer software  124   b.    
         [0018]    In embodiments, management computer  110  is any standard computer used to store, search, or authorize access to media files. Management computer  110  may contain industry-standard components, including databases (e.g., database  116 ), as well as software (e.g., reviewer software  124   a ). Network  112  is a communications network, such as a local area network (LAN), wide area network (WAN), intranet or Internet. User computer  114  includes any standard networked computer and may be used to access management computer  110  via network  112 . User computer  114  includes software, such as reviewer software  124   b.    
         [0019]    In embodiments, each instance of reviewer software  124  is a software program that graphically represents a media file for review and annotation, determines which media files a reviewer is authorized to access, and processes media file queries. Reviewer software  124  may be a client-based software module, as is depicted with reviewer software  124   b,  or a server-based software module, as is depicted with reviewer software  124   a,  in various embodiments. 
         [0020]    In an embodiment, database  116  is a database that contains a comprehensive collection of related data. In an embodiment, various data associated with media files is stored in database  116 . In some embodiments, database  116  contains media files  118 , media review files  120 , and authorization file  122 . In some embodiments, database  116  is implemented as a relational database, such as an Oracle database by Oracle Corporation (Redwood Shores, Calif.). Media file  118  may be a standard audio, video, or advanced content source file, including, but not limited to, a WAVE form audio format (WAV) file, an Audio Video Interleave (AVI) file, an MPEG Audio Layer-3 (MP3) file, or an MPEG-4 Part 14 (MP4) file. Audio files may include, for example, recorded phone calls, university lectures, or books on tape. Video files may include, for example, television broadcasts or recorded golf swings. In an embodiment, each media review file  120  is associated with a media file  118 , where the media review file  120  contains a record of any reviewer annotations that are related with the associated media file  118 . Authorization file  122  contains, in some embodiments, data that identifies which media files, if any, each reviewer is authorized to view and annotate. 
         [0021]    In an embodiment, automated analysis software  126  includes a software program that analyzes each media file  118  for data that may be detected automatically, such as profanity, meta-data, whether a sale was made or voice inflections (e.g., yelling). For example, automated analysis software  126  may review media file  118  and determine calls that did not result in a sale and contained profanity or yelling. 
         [0022]    In operation, automated analysis software  126  evaluates media files  118  and stores automated analysis annotations in the associated media review files  120 , in an embodiment. A user may then initiate a review using review software  124   b  on user computer  114 , connect to the management computer  110 , and enter a query using reviewer software  124   b.  Reviewer software  124   b  may then access authorization file  122  of database  116  to determine which media files  118  and media review files  120  the user is authorized to access. In an embodiment, reviewer software  124   b  determines the user&#39;s authorization by cross-referencing the user log-in data (e.g., a username) with authorization file  122 . Reviewer software  124   b  then uses the query information to perform a search of authorized media files  118  and media review files  120 . A list of authorized media files  118  and media review files  120  that have the associated notations is displayed by reviewer software  124   b  on user computer  114 . The user may then select a media review file  120  of interest for review. Using reviewer software  124   b,  the user may add annotations to the selected media review file  120 . Upon completion, the selected media review file  120  is saved and the updated file containing the revised annotations is stored in database  116 . Alternatively, the above-described operations are performed using server-based reviewer software  124   a  of management computer  110 . 
         [0023]      FIG. 2  illustrates an example of the data stored in an authorization file  122  in accordance with an example embodiment. The “Log-in ID” field  200  identifies a user or reviewer. The “Media file  120  Authorization” field  202  identifies one or more media review files  120  a user or reviewer is authorized to review. The “Notes Authorization” field  204  identifies one or more notes (e.g., annotations) which a user or reviewer is permitted to review (e.g., reviewer  1 , reviewer  2 , or automated annotations). 
         [0024]    As illustrated in  FIG. 2 , Joe may access media files  120   a  and  120   b.  Further, Joe may view the notes of reviewer  1  (“Rev. 1”), reviewer  2  (“Rev. 2”), and any automated annotations (“Auto”) in the media files  120   a  or  120   b.    
         [0025]      FIG. 3  illustrates an example of data stored in a media review file  120  in accordance with an example embodiment. Media review file  120  may, in some embodiments, be stored in a relational database (e.g., database  116 ) or alternatively, as a file in a file system. In an embodiment, media review file  120  contains descriptive information  300 , such as a date and time of the recording, an place of the recording, a title, an outcome, or other descriptive or identifying information. In addition, one or more notes may be included in media review file  120 . In an embodiment, the notes are embedded in media review file  120 . In an alternative embodiment, media review file  120  contains pointers or other references to notes, which may be contained, for example, in a related table in a relational database. 
         [0026]    In embodiments, media review file  120  includes a media file timestamp  302 , and a division for each reviewer, such as automated annotation data  304 , a first reviewer  306 A, a second reviewer  306 B, and a third reviewer  306 C. Each note  310  has an associated timestamp  308  and reviewer  306 C. 
         [0027]      FIG. 4  illustrates a further example of data stored in a media review file  120  after a review by reviewer “Joe”  404  in accordance with an example embodiment. The reviewer Joe  404  has not previously reviewed media review file  120  and so another division (e.g., column) is represented that contains review notes provided by Joe. In this example, Joe has provided “Note 11”  400  at the 0:51 timestamp  402 . 
         [0028]      FIG. 5  illustrates an example search graphical user interface (GUI)  500  in accordance with an example embodiment. One or more controls (e.g.,  502 A,  502 B,  502 C) may be used by a user to define one or more constraints. Constraints may include a start date  502 A, an end date  502 B, a product  502 C, an 800 number  502 D, a terminating number  502 E, a recording hour range  502 F, a recording length range  502 G, a call outcome  502 H, a call comment type  502 I, a review status  502 J, a caller identification (ID)  502 K, a recording ID  502 L, or a returned call count  502 M. After a user configured the controls  502  to reflect the desired search constraints, the user can activate a submission control  504  to submit the query. To reset the search parameters  502 , the user can activate the reset control  506 . 
         [0029]      FIG. 6  illustrates an example list GUI  600  in accordance with an example embodiment. For example, as a result of a search performed using the search GUI ( FIG. 5 ), list GUI  600  displays any matching media review files  120  and allows the user to select one or more media review files  120  to review. 
         [0030]      FIG. 7  illustrates an example reviewer software GUI  700  in accordance with an example embodiment. Reviewer software GUI  700  includes a vertical time bar  702 , a time bar mark  704 , zero or more comment marks  706  (e.g., comment marks  706 A,  706 B,  706 C, and  706 D), zero or more comments fields  708  (e.g., comments fields  708 A,  708 B,  708 C, and  708 D), an insert comment control  710 , a text input control  712 , a call data field  714 , and control icons  716 . 
         [0031]    Vertical time bar  702  includes a graphical representation of the time-based length of a media file  118  in a vertical orientation. In this example, a media file  118  with duration of 11:09 is being reviewed. 
         [0032]    Time bar mark  704  includes a graphical indication of a current location within a media file  118 . In an embodiment, time bar mark  704  may be manipulated by a user, for example, sliding the bar up (e.g., toward the beginning of the recording) or down (e.g., toward the end of the recording). In an embodiment, time bar mark  704  gradually moves (e.g., an animated control) as the recording is played to the user. In this example, time bar mark  704  is at the 6:59 mark, indicating that the recording is 6 minutes and 59 seconds from the beginning. 
         [0033]    During operation, in an embodiment, a user may provide a comment in text input control  712  to insert or revise a comment using insert comment control  710 . When a comment is inserted or revised, time bar mark  704  is used to determine the associated time of the comment and the comment is displayed in comment control  718 . The user may provide a category for the comment using a category control  720 . Different categories may be available depending on the use of the reviewer software, the user or user privilege level, or other factors. For example, categories available when the reviewer software is used to review call center sales calls may include “observation,” “accolade,” “criticism,” or the like. In another example, some users with more or fewer rights or privileges may be able to utilize more or fewer comment categories. 
         [0034]    In this example, four comments are associated with media review file  120  and are represented graphically by comment marks  706 A,  706 B,  706 C, and  706 D in comment control  718 . Comment fields  708 A,  708 B,  708 C, and  708 D represent text that is associated with each comment mark  706 . In the example illustrated in  FIG. 7 , comment field  708 A is associated with comment mark  706 A, comment field  708 B is associated with comment mark  706 B, and so on. A user may view comment fields  708  to quickly ascertain details of the recording. In an embodiment, during playback of media file  118 , comments in comment control  718  scroll to correspond with the position of time bar mark  704  on vertical time bar  702 . In an embodiment, the user may filter on the comment categories by using a filter control  722 . In the example shown, the filter control  722  is set to “All” so that any available comments in any category are shown to the user. In some embodiments, comments are presented with one or more colors to indicate the comment&#39;s category, the comment&#39;s author, or other characteristics of the comment. 
         [0035]    In an embodiment, call data field  714  displays information associated with media file  120 . Control icons  716  may include controls to manage playback of media file  120 , such as play, stop, pause, fast forward, rewind, and the like. 
         [0036]      FIG. 8  illustrates a flow diagram of a method  800  of creating and maintaining a database  116  in a media file reviewing system  100  in accordance with an example embodiment. At block  802 , a media file  118  is created. For example, an audio or video event is digitally recorded and stored in an associated media file  118  of database  116 . 
         [0037]    At block  804 , a media review file header is created. In an embodiment, each media file  118  stored in database  116  includes a media review file  120  with a media review file header. In embodiments, the media review file header includes descriptive information of the associated media file  118  such as a date, a time, a title, an agent, or the like. In an embodiment, media review file  120  and its header data are generated at the time that media file  118  is created. In a further embodiment, header data is not updateable by reviewer software  124 . 
         [0038]    At block  806 , automated analysis software  126  analyzed the media file  118 . In an embodiment, automated analysis software  126  updates media review file  120  with results of the automated analysis. Automated review includes, in various embodiments, detection of profanity, voice inflection, or the like. Incidents of detected events may be recorded at block  808  in database  116  (e.g., media review file  120 ), in some embodiments. 
         [0039]    At block  810 , the method  800  determines whether a user (e.g., reviewer) desires to review a media file  118 . If the user does not wish to review a media file  118 , then the method  800  ends. 
         [0040]    At block  812 , a user (e.g., reviewer) reviews a media file  118  by use of reviewer software  124 . At block  814 , notes created by the user are stored in database  116 . In an embodiment, the notes are stored in media review file  120  along with an indication of the reviewing user. 
         [0041]    At block  816 , the method determines whether any subsequent reviewer desires to review the media file  118 . If there are no subsequent reviewers, the method  800  ends. 
         [0042]    At block  818 , a subsequent user reviews media file  120  using reviewer software  124 . Any number of subsequent reviewers may review media file  118  and its associated media review file  120 . At block  820 , any notes created by the subsequent users in block  818  are stored in media review file  120 . 
         [0043]    Method  800  of creating and maintaining database  116  advantageously provides efficient searching through the use of media review file headers, efficient reviewing of media review files  120  and any related or associated notes or comments, and better facilitation of multiple reviews by simultaneously displaying reviewer notes regarding a particular media file  118 . 
         [0044]      FIG. 9  illustrates a flow diagram of a method  900  of using the media file reviewing system  100  in accordance with an example embodiment. At block  902 , the method  900  receives user authentication information (e.g., a username and password) and if the user is authenticated, then a query is received. In an embodiment, the query includes one or more search constraints and is directed to search for matching media files  118  in database  116 . 
         [0045]    At block  904 , using the user&#39;s authorization information (e.g., username), a set of zero or more media files  118  from the set of all media files  118  matching the user&#39;s query from block  902  are found. The subset of media files  118  are those that the user has access or authorization to review. In an embodiment, reviewer software  124  accesses authorization file  122  in order to determine to which media files  118  and media review files  120  the user is authorized to access. Reviewer software  124  may do so by cross-referencing the user log-in data with authorization file  122 . 
         [0046]    At block  906 , the subset of media files  118  to which the user is authorized to review are displayed for the user. In an embodiment, the subset of media files  118  is displayed using a list view, such as illustrated in  FIG. 6 . 
         [0047]    At block  908 , one or more indicators are received, where each indicator represents a media file  118  that the user wishes to review. In an embodiment, the user may use a control (e.g., a checkbox) to indicate which files are of interest. 
         [0048]    At block  910 , selected media files  118  are presented to the user for review. In an embodiment, the reviewer reviews media files  118  and media review files  120  using reviewer software  124 . 
         [0049]    At block  912 , review notes are received and stored by the management computer  110 . In an embodiment, the notes include annotations, such as observations, comments, event indications, and the like. In an embodiment, notes for a media file  118  are stored in an associated media review file  120  in database  116 . 
         [0050]    Thus, as described above, the inventive subject matter provides a way to increase the speed at which media files are reviewed. The appearance of a graphical user interface (GUI) while reviewing media files contributes to the effectiveness of the reviewer. The review and annotation of the media file are made more efficient and convenient. Further, upon completion of a review and annotation of the media file, embodiments of the inventive subject matter provide for searching the various media files and annotations for specific occurrences. For example, all telemarketing calls that are recorded as audio files and result in a sale may be searched for further analysis. According to other aspects of the inventive subject matter, a system for and method of easily facilitating the review of media files by multiple reviewers is provided. 
         [0051]      FIG. 10  illustrates a diagrammatic representation of a machine  1000  capable of performing the methods or implementing the systems/devices described herein. In alternative embodiments, the machine may comprise a computer, a network router, a network switch, a network bridge, a Personal Digital Assistant (PDA), a cellular telephone, a web appliance, a set-top box (STB), or any machine capable of executing a sequence of instructions that specify actions to be taken by that machine. 
         [0052]    The machine  1000  includes a processor  1002 , a main memory  1004 , and a static memory  1006 , which communicate with each other via a bus  1008 . The machine  1000  may further include a video display unit  1010  (e.g., a liquid crystal display (LCD) or a cathode ray tube (CRT)). The machine  1000  also includes an alphanumeric input device  1012  (e.g., a keyboard), a cursor control device  1014  (e.g., a mouse), a disk drive unit  1016 , a signal generation device  1018  (e.g., a speaker), and a network interface device  1020  to interface the computer system to a network  1022 . 
         [0053]    The disk drive unit  1016  includes a machine-readable medium  1024  on which is stored a set of instructions or software  1026  embodying any one, or all, of the methodologies described herein. The software  1026  is also shown to reside, completely or at least partially, within the main memory  1004  and/or within the processor  1002 . The software  1026  may further be transmitted or received via the network interface device  1020 . 
         [0054]    For the purposes of this specification, the term “machine-readable medium” shall be taken to include any medium which is capable of storing or encoding a sequence of instructions for execution by the machine and that cause the machine to perform any one of the methodologies of the inventive subject matter. The term “machine-readable medium” shall accordingly be taken to include, but not be limited to, solid-state memories, optical and magnetic disks, and carrier wave signals. Further, while the software is shown in  FIG. 10  to reside within a single device, it will be appreciated that the software could be distributed across multiple machines or storage media, which may include the machine-readable medium. 
         [0055]    Although specific embodiments have been illustrated and described herein, it will be appreciated by those of ordinary skill in the art that any arrangement that achieves the same purpose, structure, or function may be substituted for the specific embodiments shown. This application is intended to cover any adaptations or variations of the example embodiments of the invention described herein. It is intended that this invention be limited only by the claims, and the full scope of equivalents thereof. 
         [0056]    The Abstract is provided to comply with 37 C.F.R. §1.72(b), which requires that it allow the reader to quickly ascertain the nature of the technical disclosure. It is submitted with the understanding that it will not be used to interpret or limit the scope or meaning of the claims. Also, in the above Detailed Description, various features may be grouped together to streamline the disclosure. This should not be interpreted as intending that an unclaimed disclosed feature is essential to any claim. Rather, inventive subject matter may lie in less than all features of a particular disclosed embodiment. Thus, the following claims are hereby incorporated into the Detailed Description, with each claim standing on its own as a separate embodiment.