Source: http://www.google.com/patents/US20050042591?dq=6,998,619
Timestamp: 2016-08-25 08:19:34
Document Index: 40660243

Matched Legal Cases: ['ART>0', 'ART>0', 'ART>0', 'ART>2', 'ART>2', 'ART>3', 'ART>3', 'ART>3', 'ART>3']

Patent US20050042591 - Methods and apparatus for use in sound replacement with automatic ... - Google PatentsSearch Images Maps Play YouTube News Gmail Drive More »Sign inPatentsDigital audio and video files are created corresponding to selected scenes from a creative production and are provided with a processing system that enables dialog to be selected from a scene and replaced by a user's dialog which is automatically synchronized with the original dialog so as to be in synchronism...http://www.google.com/patents/US20050042591?utm_source=gb-gplus-sharePatent US20050042591 - Methods and apparatus for use in sound replacement with automatic synchronization to imagesAdvanced Patent SearchPublication numberUS20050042591 A1Publication typeApplicationApplication numberUS 10/695,596Publication dateFeb 24, 2005Filing dateOct 28, 2003Priority dateNov 1, 2002Also published asCA2477697A1, CA2477697C, EP1559104A1, US8009966, WO2004040576A1, WO2004040576A8Publication number10695596, 695596, US 2005/0042591 A1, US 2005/042591 A1, US 20050042591 A1, US 20050042591A1, US 2005042591 A1, US 2005042591A1, US-A1-20050042591, US-A1-2005042591, US2005/0042591A1, US2005/042591A1, US20050042591 A1, US20050042591A1, US2005042591 A1, US2005042591A1InventorsPhillip Bloom, William EllwoodOriginal AssigneeBloom Phillip Jeffrey, Ellwood William JohnExport CitationBiBTeX, EndNote, RefManPatent Citations (27), Referenced by (135), Classifications (19), Legal Events (2) External Links: USPTO, USPTO Assignment, EspacenetMethods and apparatus for use in sound replacement with automatic synchronization to images
BRIEF DESCRIPTION OF THE DRAWINGS [0062] FIG. 1 is a block diagram of the typical components found in a typical home PC system. [0063] FIG. 2 is a block diagram illustrating prior art software components in a typical PC environment. [0064] FIG. 3 is an example of a graphical user interface with controls that the revoicing system application software presents on a PC monitor to the unskilled system end user according to one embodiment of the present invention. [0065] FIG. 4 is a block diagram schematically illustrating, according to one embodiment of the present invention, the main elements within the various computer systems involved in creating an end-user program which is transferred to a user's PC by removable media and provides revoicing of embedded media. [0066] FIG. 5 illustrates schematically the relationships between various media data streams that are present in a selected audio video clip, according to one embodiment of the present invention. [0067] FIG. 6 illustrates an example of the audio waveform display in a typical commercial audio editing system used by the Media Preparation Editor in which the Start- and Stop-Times of two Lines are shown marked by two pairs of dashed lines. [0068] FIG. 7 illustrates an example of the audio waveform display in a typical commercial audio editing system used by the Media Preparation Editor in which the Start- and Stop-Times of two Words in one Line are shown marked by the dashed lines. [0069] FIG. 8 illustrates in block form the main operational software modules, data storage devices and hardware components which implement the operations of rehearsing, recording, processing, and playback of the user's voice in a PC-based revoicing system according to one embodiment of the present invention. [0070] FIG. 9 illustrates schematically the various media sources, actions with these sources, the relative timing of events in the media and the output destinations involved in the Rehearse Phase operation according to one embodiment of the present invention. [0071] FIG. 10 illustrates schematically the various media sources, actions with these sources, the relative timing of events in the media and the output destinations involved in the Record Phase operation according to one embodiment of the present invention. [0072] FIG. 11 illustrates in block form further details of elements in FIG. 8 which implement the Recording and Alignment Processing block 830 and create the Time Aligned version of the User's Recording according to one embodiment of the present invention. [0073] FIG. 12 illustrates schematically the various media sources, actions with these sources, the relative timing of main events in the media and the output destinations involved in the Play Phase operation according to one embodiment of the present invention. [0074] FIG. 13 illustrates in block form further details of the data stream processing which takes place in the block 840 in FIG. 8 during Scene and Line playback according to one embodiment of the present invention. [0075] FIG. 14 illustrates in block form further details of the data stream processing which takes place in the block 840 in FIG. 8 during the recording of a Line according to one embodiment of the present invention. DESCRIPTION OF THE PREFERRED EMBODIMENTS [0076] According to the present embodiment of this invention, to provide simple operation for the end user, there are data preparation steps which can, in one example, be made by skilled editors who prepare and organise the original audio and video data as well as create auxiliary data such as data files describing the text and timing of the words in the original dialog audio for which an end user may wish to record a replacement. Such auxiliary text timing data is used to provide cues to the end user as to what to say and when to say it during rehearse and recording passes. [0077] In other embodiments such preparation could be performed automatically by a combination of speech recognition systems with computerised timing measurements and automatic generation of a database of results. [0078] In further embodiments, the audio being replaced need not be limited to speech. The principles and processes of the invention can be applied to replacing singing, musical instruments and sound effects. [0079] Description of User Interface and Operation [0080] FIG. 3 is a schematic diagram illustrating one example of a graphical user interface which includes basic controls required to operate the revoicing system for digitized film or video clips. A similar interface can be used for replacing singing or sound effects, with the names given to various controls and screen areas changed to suit the industry terms normally associated in describing elements of the type of content being replaced. [0081] Solely for convenience, the following defined terms will be used in the description. A Programme (as distinct from a computer program) is a collection of Scenes from one or more digital video and audio titles e.g. from single or multiple film or TV productions. The Scenes may be from contiguous or non-contiguous sections of the production. A Scene is several consecutive lines of dialog spoken by one or more characters. A Line is a sequence of consecutive words that are spoken or sounds that are uttered by one character in a Scene. Sounds or words that are not made by the same character can overlap parts of a Line. A Digital Video file is a digitized and typically compressed version of the video content that, when played with the original audio files, reconstructs a Scene from a Programme in its original form. Typical compressed audio/video file formats are represented by .avi or .mov extensions in computer systems. An Original Recording is a digitally stored version of a character's utterance, whether speech or other character action related sound, that is being replaced in the Digital Video and corresponds to a Line. A R v iced Take (or simply Take) is the result of the end user initiating a recording operation, recording his/her speech or other sounds through a microphone into computer storage, and the stored digital audio being edited automatically according to instructions from a time alignment algorithm to match substantially the time varying features of the Original Recording. Revoicing—the operation of recording and automatically time aligning a user's voice to convincingly replace the original voice in a Programme. [0090] In the following description reference will be made to the computer system 100 of FIG. 1 as modified to embody one aspect of the present invention by software produced by a first system embodying a second aspect of the invention. [0091] A graphical user interface 320 as it appears on the display screen 130 in this embodiment is shown in FIG. 3. The interface 320 is divided into five main areas: a Scene display and selection area 340; a main video display window 330; a Line display and selection window 350; Line operation buttons 361 through 363; a Stop button 364 and a Revoiced Take display and selector or Recorder section 370, which is labelled “Recorder” for the end user. Only the main video display window 330 does not serve as a control. The Recorder section 370 indicates the available user Revoiced Takes, such as 371, 373 and 374 and the Original Recording 372. The pointing device 120 is used to position a cursor (not shown) on and to select and operate (e.g. if a mouse, by clicking) any of the controls shown in ways to be described below. [0092] The user can operate the interface controls in any order, but a typical initial sequence would be as follows: [0093] Firstly, the user would select a Scene to work with or review by selecting in the Scene selection area 340. To select the Scene, the user clicks on the small window showing a still image 341 from that Scene. The heavy border around the image 341 indicates that the associated Scene is selected. Alternatively the user can drag a Scene selector slider control 343 up and down with the device 120 within the boundaries of a slider 342, to select from all the Scenes available. When a Scene is selected, the user sees a still image from that Scene displayed in the main video display window 330. [0094] If the user clicks a second time on the selected Scene window, the modified computer system 100 plays the digital video file of that Scene to provide the video display in the Video window 330 and plays back the original audio through the computer's audio system 150 and speakers 156. Each Scene typically extends over one or more Lines of dialog for one or more characters. At the same time, the text of the Lines of dialog for that Scene is displayed in the Line display and selection window 350. [0095] Secondly, the user can select a Line from that Scene in the Line selector window, 350 by clicking on any visible line of text. The currently selected Line, in this example, Line 353, is shown selected by being highlighted (which is represented in FIG. 3 by the bold text). The user can bring non-visible Lines into the Line selector window for selection by dragging the Line selector control 352 within the boundaries of the slider control 351. When a Line is selected, an image of the Character who speaks the Line in the original audio/video track is shown in a Character window 355, with the character name (optionally) shown below the image for further identification. [0096] Thirdly, once the user has selected a Scene and Line he wishes to revoice, he can play the selected Line with the video and original characters recorded voice by clicking on the Rehearse button 361. When the user clicks on the Rehearse button 361, the video and the original audio begin to play at a point a short time before the beginning of the chosen Line. To give the user an accurate indication of when the first word starts and, more importantly, when to start speaking during the recording phase, the computer system automatically generates and presents graphically within the interface 320, a sequential countdown of the numbers “3”, “2”, “1”, and the word “GO”. The successive countdown images are separated by an equal time period (such as 0.75 seconds) and are timed such that “GO” appears at the instant before the first word (or sound) in the selected Line's dialog starts. A further guide to the timing of words and sounds as they occur in the selected Line is given by the system showing the displayed words in the window 350 with additional highlighting (represented by underlining 354). The additional highlighting of a word occurs at the time that word is encountered in the original audio track (whether or not that original audio signal is being played and heard). The additional highlighting of a word in the line lasts for the duration of that word in the audio track. This gives the user a running visual guide to the timing of the events in the line during both the Rehearse phase and in the Record phase to be described next. In the Rehearse phase, the video and the original audio stop playing a short time after the end of the Line. [0097] Fourthly, when the user is ready to record his performance of the selected Line, he clicks on the Record button 362. When the user clicks on the Record button 362, the video and the original audio begin to play at the same point a short time before the beginning of the chosen Line, as in the Rehearse phase. The system again, as in the Rehearse mode, provides the user with a visual graphic countdown to indicate when to start speaking and also display the moving additional highlighting of words in the window 350 as they occur in the Original Recording. During the recording phase, however, the original dialog track audio is not played through the loudspeaker 156 so the microphone 159 does not pick it up. The original dialog audio can be played as an audible guide, however, through headphones (not shown) which the user may wear. To provide further cues and the right audio ambience, the original music and sound effects track and/or the original characters' voice(s) which occur before (but not during) the selected Line starts are played through the loudspeaker 156 during the Record phase. [0098] Consequently, if, during the Record phase, the user follows the countdown to start speaking, starts speaking at “GO” and then follows the timing of the highlighted words, this will help create a recording with words and vocal sounds that have similar timing to, but are not precisely in time with, those in the Original Recording. [0099] In the current example, the digital recording thus made by the user is stored temporarily on the hard disk 140 or into RAM 118, and that audio data will be processed and edited automatically using, for example, a method according to U.S. Pat. No. 4,591,928 to create and store a processed i.e. revoiced, audio recording. It is this processed version which will now be referred to as the Aligned audio that will be accurately in synchronism with the Original Recording and, hence, with the corresponding section of original video. [0100] In this embodiment, the Aligned audio is stored on the hard disk 140 (or in RAM 118) and its existence and identity will be displayed in one of the previously “empty” Take recording holders 371, 373, 374 in the Recorder section 370. As shown in FIG. 3, for example, the user would see a name in the position of the selected Recording holder 371, labelled here “Recording M+1”. In fact, only data pointing to the location of the Aligned audio on the disk 140 (or in RAM) will be held. The user can, optionally, generate commands via the pointing device 120 to audition, “erase” or name any of these recordings by using, for example, a right click menu if the device is 120 is a mouse. A name is entered by use of the keyboard 125 in a naming mode selected by the device 120. [0101] By clicking on one of the Take recording holders 371, 373 and 374, the user can choose which one is selected for playback when the Line is played with the Play button 363, or when the Scene is played with the Play Scene button 344. In this example, there are three Take recording holders 371, 373, 374 which the user can record into and choose from for each line that is revoiced. Any non-zero number of these recording holders may be provided. For easy use, when a new recording is initiated, the system automatically finds which of the recording holders is empty (i.e. available) and uses it. When a new Take is created, it is automatically made the Selected Take. [0102] If the Play Scene button 344 is actuated, the interface image in the display screen 130 is replaced by a full-screen version of the video display window 330. [0103] Other playback options are available: If the user presses the Play All button 346, the videos of all of the Scenes will be played in their normal running order using the selected Takes for each Line in each Scene. [0104] One advantage of this system is that the Lines and Scenes do not have to be recorded in any particular order, but when a Scene is played back, the system plays the Lines in the Scenes in their correct original sequence. Similarly, Play All 346 will play the Scenes in their originial order. [0105] A Stop button 364 can be pressed at any time to stop the audio and video playback in the Rehearse, Record, and Play Line modes. Additionally, for the Play Scene and Play All modes, a key on the keyboard 125 such as the spacebar, can be designated as a Stop key so that the full screen video image can be stopped during these playback modes and the graphical user interface 320 restored. [0106] For improved operation, the user can adjust the microphone input level by clicking on and dragging a microphone level control slider 382 within the boundary of a mike level control range shown by 383. The level of the mike input is displayed “live” on the screen in real time by a graphic microphone level meter 381. The user can also adjust the overall audio playback level by dragging a volume level control slider 384 within the boundary of a volume control range 385. [0107] The interface illustrated by FIG. 3 can be extended to offer more options such as the selection of different Programmes, each containing different Scenes and the addition of further audio signal processing to change the recorded voice characteristics. In addition, the recorded Takes can be shown in many ways, including, for example, time-sequenced blocks or waveforms. [0108] In the present embodiment, the media data, media-related data and the run-time application program which carries out the operations described above are assembled together and delivered to the User's system 100 on a CD ROM or DVD ROM 165 (FIG. 1) or other high capacity memory device. The Internet 175 could be used as an alternative delivery method for one, both or any part of these components. The user's system 100 can also be provided by other forms of connected sets of electronic components capable of receiving the data, displaying a graphical interface, recording, playing back and processing digital signals, and manipulating data. [heading-0109] Overview of Preparation Phase and Revoicing Computer Program Creation [0110] There is no guarantee that voice audio streams in an original audio visual media recording synchronize accurately with the lip movements in the video. For example, the audio could contain a foreign language dubbed dialog track. However, the media selected for use in this invention should be chosen specifically for having an adequate proportion of sufficiently accurate lip movement synchronization between original audio and video tracks to make revoicing worthwhile in terms of entertainment. [0111] Media data preparation and creation of a revoicing computer program are represented schematically in FIG. 4. Media data preparation simplifies the revoicing operation for the end user. [0112] A media preparation computer 105 a is shown in FIG. 4 which, in one example, can be a personal computer system such as the system 100 shown in FIG. 1. A second similar computer 105 b and its hard disk 140 is shown and data created in computer 105 a can be transferred to computer 105 b via normal data transfer means including removable disks or networks. In this case, computer 105 b can be used to test the program operations on the media and confirm the media has been edited correctly before creating a copy of the program on removable media 165 that can transfer the program and media to an end-user's computer 100. In some circumstances, it can be efficient for the computers 105 a and 105 b to be the same computer, thereby allowing testing of the program and refining of the media editing and markup as necessary within the same computer. [0113] In this embodiment, a person skilled in media data handling, who will be referred to as the Media Preparation Editor, carries out media data preparation steps using a variety of Media Preparation programs 410. Media preparation programs 410 include known software such as the audio editing programs listed previously, video/audio editing programs such as Premiere from Adobe Systems, Inc., still image editing programs such as Photoshop from Adobe Systems, Inc. and digital video file (e.g. MPEG) playback and still image generation programs such as PowerDVD from CyberLink Corp. or Premiere. [0114] In the preferred embodiment, media preparation can be achieved as follows: Firstly, video and audio source media for a Programme are digitized and are stored in media preparation computer 105 a as Digitized Original Audio and Video Data 420. Using programs such as Premiere or Sound Forge, the digital video files are trimmed, sized and compressed using codecs such as Indeo� Video 5, and the appropriate original audio tracks described earlier are added to the video files into avi format files. The Media Preparation Editor then decides what sections of the digitized original audio and video data 420 are to be used as Scenes and selects those portions of the digitized original audio and video data 420 which include the Scenes. The extent of a Scene is largely determined by artistic and creative decisions on the part of the Media Preparation Editor and naturally based on the programme content. Using Sound Forge, Premiere or other digital audio and or video editing programs, the relative start and stop times of the desired Scenes and Lines and other media-related data which defines and identifies these divisions in the digitized media are logged and included in Media Related Data files 430 for later use. Those portions of the digitized original audio and video data 420 which do not include Scenes can be discarded. [0115] Representative digital still images are created from the video files as snapshots to be used to identify Scenes and similar still images are created to identify the original characters. These images are included in the Media Related Data files 430. Other Media-Related Data includes media file names. The preparation of further media related data will be described hereinafter with reference to FIG. 11. The Media Related Data is entered into one or more XMLdata files as described in more detail hereinafter. [0116] In this embodiment, some of the preparation is manual and some can be done automatically. Such steps of media digitisation, media measurements, and media preparation steps can take place in a plurality of multi-media PCs so long as the resulting media data and/or other data can be transferred or made available to the computer 105 b where the testing and building of the end-user program are achieved. The manual preparation steps may be automated in alternative embodiments and may even run within the end user PC system 100. [0117] The run-time computer program 450, whose application specific modules are written in C++, utilizes the data 440 and 430 and carries out the system operations in the end user's computer system 100. [0118] A further optional process can be run in computer 105 b on the data 430 and 440 and on the program 450 to compile these elements into a form which also includes restructuring the data into a compact format for commercial release. This step also reduces the risk of reverse engineering. [0119] This complied version of the program and data can also be copied to removable medium 165 and that version will run on the end user's PC system 100 efficiently. [0120] In either case, the program on medium 165 is loaded into the user's system 100 to run in the computer 110 either directly from the removable media storage device 165 or after being copied to the hard disk 140. [heading-0121] Details of Media Preparation Phase [0122] The first step of media preparation is to convert the Programme's media, which often is provided in a film or standard analog video and audio format, to digital media files suitable for this process. [0123] If the source Programme is in a film format, a commercial audio/video transfer facility can convert the film's content via a standard telecine process to analog format on tape or to a digital video format such as NTSC or PAL Digital Betacam� tapes. To turn analog video and audio signals on tape into digital video and audio data, either a commercial video transfer facility can create the files with dedicated hardware, or this can be achieved on a specialized PC. Suitable video and audio hardware cards and software codecs are known. Such processes will be familiar to those skilled in the art of audio and video conversion techniques. [0124] The digital media files are created in a format that can be transferred to the program preparation computer 105 a for example via CD ROM or a network, so that the Media Preparation Editor can access them to create the prepared data required for the revoice processing. Examples of suitable file formats are given in a table hereinafter. [0125] It is preferred that along with the video, the following audio tracks are obtained as separate tracks either on the same video tape, on separate digital or analog audio tapes, or by other means of audio data transfer such as CD ROMS or digital networks. Such separate tracks are generally available because during the making of a large proportion of the original programmes to which this invention may be applied, separate audio tracks are recorded and or generated and kept on multitrack tapes or in a multitrack format on a digital audio recording and editing system. In these cases the following audio tracks would be obtainable and then can be digitized via standard studio techniques. The Dialog track (or tracks), optimally of each character separately if there are any characters speaking at the same time, or combined if there are no such overlaps. The Music and Sound Effects (“M&E”) tracks, which can be mixed into a stereo pair or a single monophonic track. For added flexibility, the music tracks and the effects tracks can remain separated, but this would not alter the functionality required. The mixed original track (“Full Mix”)— containing the “final” mix of all the dialogs and the M&E tracks. For revoicing singing, it is also useful to obtain mixes with only harmony and/or backing tracks and without the principal voice that is to be revoiced. [0130] In the cases where the source media is not available in multitrack form, it is possible to either accurately recreate the required backing tracks or solo vocal tracks in the format required or apply digital audio signal processing techniques to obtain some or all of the elements required. For example, sometimes simple phase reversal of one channel of a stereo recording and summing it with the other channel can remove voices which were equally present on both channels and leave the out of phase components of a background track. [0131] The following table indicates examples of the file formats and codecs used in the present embodiment to digitise the signals listed for each Scene. Digital Media Description File Type for Each Scene Used Digital Coding Used Digital Video File .avi Indeo � Video5 compression Digital Audio - .wav or .avi 16 bit PCM Main Dialog File Digital Audio - .wav or .avi 16 bit PCM Auxiliary Dialog File (required in the case of overlapping dialog) Digital Audio - .wav or .avi 16 bit PCM Music and Sound Effects (“M&E”) Digital Audio - .wav or .avi 16 bit PCM Mixed Dialog and Music and Sound Effects (“Full Mix”) [0132] A set of such digital media files for each Scene provides a library of Scene data from which, when one Scene is selected by the user, the appropriate data can be combined by computer software modules into synchronized streams of digital audio and video data which, when fed through the user's computer video and sound systems, appears to the user as a complete original “video” clip. [0133] For music videos, the term Vocals replaces the term Dialog, and the Music and Effects data stream would simply be backing Music. An additional optional Music track, containing the backing music plus any background and harmony vocals, can also be created for playing back with the user's vocal in order to most accurately recreate the song. [0134] If the video is in an avi format file with no audio, the Media Preparation Editor can, on a current standard PC used as the computer 105 a in FIG. 4, insert into the avi file (using a sound and video editing program such as the Sound Forge� program version 4.5 from Sonic Foundry�) one or more of the audio streams (shown in the above table) to create a single avi file with all or most of the digital media data related to a Scene. This creates a multiplexed audio/video file that has advantages in convenience of data management and improving data access during streaming. [0135] The avi file format has the further advantage that, by using DirectX�) components available from Microsoft� along with the Microsoft Windows� operating system, which provides the required codecs, the revoicing computer program supplied on the removable media 165 to the end user can instruct standard Microsoft software modules in the end user's PC 100 to create, select, decode and play back avi files simultaneously and synchronously with wav files at the correct sample rate and/or frame rate. Also, when required, playback can be started from a specified offset from the start of the media data file. Generally, the first sample or video frame in each file is defined as time zero for that signal, unless there is data representing a start time offset present in the file or maintained elsewhere in a corresponding database. [0136] Apart from the advantages listed above, it is not functionally significant whether the selected audio and video data are combined into a single data file or remain in multiple files, so long as enough information is maintained in a database to ensure that the appropriate media data can be located, combined and played back synchronously through the end user's computer system 100 when required. [0137] Once the digital audio and video files for a Programme are created and accessible to the Media Preparation Editor, the next step in data preparation is to measure and log certain details of the signals. [0138] In the data preparation stage, the Media Preparation Editor decides on an XML internal reference name for each Scene and other names as required. He then, using software on the Program Preparation Computer 105 a, manually measures and logs the start and end times of the Scenes and then does the same for the Lines for each Character in the Scene. The duration of a Scene can typically be from 20 or 30 seconds to one or two minutes. The Media Preparation Editor makes these measurements by viewing and/or auditioning the audio/video files (avi's) and the corresponding synchronized digital audio streams using a commercially available PC-based digital audio editor such as Sound Forge or Pro Tools. This is discussed further below. [0139] What determines Line segmentation and boundaries is somewhat arbitrary but it is based on skilled decisions by the Media Preparation Editor who must judge what length would be easy for the typical user revoicing that particular dialog to repeat without errors. Thus a Line can sometimes be as short as a single word or sound, or as long as a sentence or multiple sentences. Typically, one Line is a phrase in a sentence or a whole sentence. [0140] In FIG. 5, for an example the extent of “Scene N” has been determined and is shown to contain five lines in audio tracks 520 and 530. The relationships of the signals in the various digital media streams, as they might exist in the signal data files described above, are shown schematically, along with sample text transcribed from two character voice dialog streams. [0141] The length of the streams of digitized original audio and video data containing this Scene is shown by an original video stream 510 in FIG. 5. [0142] The start of the original Video stream 510, which corresponds to slightly more than the length of example Scene N, defines a reference start time, T=0, for both the video and the related audio streams. Character Voice 1, original audio stream 520, is shown comprising three separated Lines and Character Voice 2, original audio stream 530 is shown with two Lines. In this example the end of the first Line of Character Voice 2 overlaps the second Line of Character Voice 1. Lastly, the M&E (Music and Sound Effects) stream 540 is shown as a continuous signal, regardless of whether or not there is actually audio present in this stream. Scene N in FIG. 5 is marked as starting at a start time TSTARTTIME(N) and ending at a stop time TSTOPTIME(N). Line Start and Stop times are chosen to be near the boundaries of the hatched blocks in the audio streams 520 and 530, but to allow for added lead-in time and lead out time to give the user added visual cues from the video stream at the start and spill over time at the end respectively. The Media Preparation Editor determines these boundary times by listening to the signals as well as by observing their waveforms in the audio editing software programs. He then can note the offset measured from the start of the file, T=0, as will be discussed further below. [0143] Because the Character voices are generally closely synchronized with the lip movements in the video, Original Character audio will act as and frequently be referred to in this description as the “GUIDE” signals, to which a replacement (user recording or “DUB”) will be edited in the revoicing process so that the edited version will consequently be synchronized with the video. [0144] During media preparation, with reference to FIG. 4, images representing a Scene to be used in the Scene Selector 340 (FIG. 3), or images representing characters in the character window 355 can be captured from the digital video file using commercial digital video and image editing programs on PC 105 a. Which images are captured is based on artistic decisions on the part of the Media Preparation Editor. The captured images are saved as graphic image data files in .jpg or .bmp file formats and are included in the Media Related Data 430. Alternatively, the Computer Program 450 can “grab” frames of video data from the appropriate original video data file in 440 at the times corresponding to the starts of Scenes or Lines in order to create automatically snapshots required for the main video display window 330. [0145] In addition to measuring the waveform, the Media Preparation Editor creates as part of the process carried out by the Media preparation programs 410 a database of names, data descriptors and measurements that make up part of the media related data 430 and will be used eventually as input data by the application computer program 450 to obtain the operational results described above. In this embodiment, the Media Preparation Editor uses a commercially available XML editing program such as xmlspy� available from www.altova.com to populate the database of Media Related Data 430 by entering such data into an XML (Extended Markup Language) file. [0146] XML data structures can be easily defined, and they expedite the means by which the data can be processed by the computer program 450, which is written to recognise and act on the variables so defined. Once the computer programmer defines the data structure and variables in a DTD file, as described fully in references such as “Learning XML” by Erik T. Ray, O'Reilly & Associates, 1st edition (2001), the xmispy program enables the Media Preparation Editor to enter efficiently the required data into the defined structures. [0147] The main definitions of XML variables (in bold, uppercase or capital letters and between brackets < >) which the editor fills in for each audio/video PROGRAMME are given below. Time measurements are made relative to the VIDEO start time, taken as T=0, unless otherwise stated. Indenting in the list below indicates that the indented variable is related to the immediately preceding non-indented variable and defines further information about that parent variable. A reference name not emboldened represents a specific instance of such a variable that would refer to an actual file. This is clarified further in the example .xml file that is given following the definitions below. <PROGRAM> - Contains all the information and static data that will be transformed into a form that will be used by the end-use application program to process the data to effect the opeartions described in this embodiment. A PROGRAM.XML file will generally contain information relating to one or more scenes from a single film, video or a music video. <TAKE_DIR> - Identifies the directory to be used to store the processed (aligned) user audio data recordings on the computer running the application. <VIDEOS> - List of video files used by the application program. <VIDEO> - Information follows defining a video file used by the program. <VIDEO_REF> - Unique identifier string used to reference this entry from other parts of the PROGRAM.XML file. <VIDEOFILE> - Name and path of the video file. The path is relative to the location of the PROGRAM.XML document. <GUIDES> - List of Guide file names used by the application program. <GUIDE> - Information follows defining a data file whose contents are used to align the user's recording with the moving images in the video file. <GUIDE_REF> - Unique identifier string used to reference this entry from other parts of the PROGRAM.XML file. <GUIDEFILE> - Name and path of the file containing the alignment information. The path is relative to the location of the PROGRAM.XML file. <STILLS> - A list of still image data files used in the interface. <STILL> - Information follows defining a still image. The images are representative snapshots of each Scene or Character whose dialog will be replaced by the user. <STILL_REF> - Unique identifier string used to reference this entry from other parts of the PROGRAM.XML file. <STILLFILE> - Name and path of the file which contains the image. The path is relative to the PROGRAM.XML file. <WIDTH> - Width of the image in pixels when it is displayed in the user interface. <HEIGHT> - Height of the image in pixels when it is displayed in the user interface. <SCENES> - Lists of the Scenes in the programme <SCENE> - contains the information required by the application to control activity related to the Scene <SCENE REF> - Unique identifier string used to reference this entry from other parts of the PROGRAM.XML file. <NAME> - name of the scene displayed in the user interface. <VIDEO_REF> - Identifier used to select the entry in the <VIDEO> array which references the source file of the video and edited audio data streams for this Scene. <STILL_REF> - Identifier used to select the entry in the <STILL> array which references the still image file representing the scene in the Scene selector (in FIG. 3 at 341) <STARTTIME> and <STOPTIME> - Are the Start time and Stop time respectively of the Scene in the source video file. These define the limits of the Scene's video and audio data played when the user presses the Play Scene control; <CHARACTERS> - List of the Characters used in the Scene. <CHARACTER> - Information follows below about a Character used in the scene <CHAR_REF> - Unique identifier string used to reference this entry from other parts of the PROGRAM.XML file. <NAME> - Name of the Character displayed to the user as shown in Figure 3 at 355. <STILL_REF> - Reference to the still image used to represent the Character in Figure 3 at 355. <LINES> - List of the lines that make up the Scene. <LINE> - Information follows below about an individual line. <CHAR_REF> - Identifier used to select the Character in the <CHARACTERS> array who is speaking the line. <STARTTIME> and <STOPTIME> - Playback start and stop time for the line within the source video. These include the editor's added extensions to allow lead-in and lead-out. <GUIDE_REF> - Identifier used to select the entry in the <GUIDES> array which contains the Guide data used to align the user's recording. <GUIDE_START> and <GUIDE_STOP> - When the alignment process takes place, these variables define the start and stop time of the data to use in the Guide file. (These times are relative to the start of the Guide file.) <DUB_START> When recording the user input, only audio after this time will be used in the alignment process. This time is also used to switch off the Full Mix signal and switch on the M&E signal in Figures 10 and 12, <DUB_STOP> - When recording the user input, this specifies the time after which the user input will be discarded. This time is also used to switch on the Full Mix signal and switch off the M&E signal in Figures 10 and 12. <SPOT_TIME> - Time relative to the start of the playback of the video at which to start playing the Aligned user Take when playing back the resulting Line in the selected Scene as shown in Figure 12. <WORD> - Defines information which follows below about a word or group of sounds visually represented to the user to cue the timing of their recording. Sometimes this may be several words or part of a word. <TEXT> - Strings of dialog text or transcriptions of the sounds made by the Character which are displayed to the user in Figure 3 at 353. <START> - Start time of the word or utterance. This is used to highlight the word in the user interface as the video reaches this time. <DURATION> - Length of time that the text will be highlighted. If no DURATION is defined for a TEXT entry, that entry lasts until the time the next TEXT string starts. [0148] The Guide and Dub identifiers and times are required to implement many of the voice replacement functions. The use of these variables will be discussed further hereinafter. [0149] A sample of the XML file structure using the variables defined above is given below. The Media Preparation Editor uses xmlspy or equivalent to place the data definitions or measured values between the variable's name <Variablename> and the “end of variable” marker </Variablename>. Sample XML program file contents with Media Content and timing variables <PROGRAM> <TAKE_DIR>\ReVoice\MovieTitle</TAKE_DIR> <VIDEOS> <VIDEO> <VIDEOFILE>AVIs\MovieClip01.avi</VIDEOFILE> </VIDEO> <VIDEO> <VIDEOFILE>AVIs\MovieClip02.avi</VIDEOFILE> </VIDEO> </VIDEOS> <GUIDES> <GUIDE> <GUIDEFILE>Guide Audio\Character1.wav</GUIDEFILE> </GUIDE> <GUIDE> <GUIDEFILE>Guide Audio\Character2.wav</GUIDEFILE> </GUIDE> </GUIDES> <STILLS> <STILL> <STILL_REF>Scene1 Image</STILL_REF> <FILE>Stills\Scene1 Image.bmp</FILE> <WIDTH>83</WIDTH> <HEIGHT>75</HEIGHT> </STILL> <STILL> <STILL_REF>Character01 Image</STILL_REF> <FILE>Stills\Character01.bmp</FILE> <WIDTH>72</WIDTH> <HEIGHT>69</HEIGHT> </STILL> <STILL> <STILL_REF>Character02 Image</STILL_REF> <FILE>Stills\Character02.bmp</FILE> <WIDTH>72</WIDTH> <HEIGHT>69</HEIGHT> </STILL> </STILLS> <SCENES> <SCENE> <SCENE_REF>Scene1</SCENE_REF> <NAME>Scene1 Title</NAME> <VIDEO_REF>MovieClip01</VIDEO_REF> <STILL_REF>Scene1 Image</STILL_REF> <STARTTIME>0</STARTTIME> <STOPTIME>23.14</STOPTIME> <CHARACTERS> <CHARACTER> <CHAR_REF>Character 01</CHAR_REF> <NAME>Fred</NAME> <STILL_REF>Character01 Image</STILL_REF> </CHARACTER> <CHARACTER> <CHAR_REF>Character 02</CHAR_REF> <NAME>Betty</NAME> <STILL_REF>Character 02 Image</STILL_REF> </CHARACTER> </CHARACTERS> <LINES> <LINE> <CHAR_REF>Character 01</CHAR_REF> <STARTTIME>0</STARTTIME> <STOPTIME>1.5</STOPTIME> <GUIDE_REF>Character1.wav</GUIDE_REF> <GUIDE_START>0.36</GUIDE_START> <GUIDE_STOP>1.497</GUIDE_STOP> <DUB_START>0.26</DUB_START> <DUB_STOP>−1</DUB_STOP> <SPOT_TIME>0.36</SPOT_TIME> <WORD> <TEXT>Hello</TEXT> <START>0.36</START> <DURATION>0.6</DURATION> </WORD> </LINE> <LINE> <CHAR_REF>Character 02</CHAR_REF> <STARTTIME>1.0</STARTTIME> <STOPTIME>5.0</STOPTIME> <GUIDE_REF>Character2.wav</GUIDE_REF> <GUIDE_START>2.856</GUIDE_START> <GUIDE_STOP>4.516</GUIDE_STOP> <DUB_START>2.484</DUB_START> <DUB_STOP>−1</DUB_STOP> <SPOT_TIME>2.856</SPOT_TIME> <WORD> <TEXT>Hey!</TEXT> <START>3.0</START> <DURATION>.5</DURATION> </WORD> <WORD> <TEXT>How</TEXT> <START>3.7</START> </WORD> <WORD> <TEXT>are</TEXT> <START>3.84</START> </WORD> <WORD> <TEXT>you?</TEXT> <START>3.95</START> <DURATION>0.5</DURATION> </WORD> </LINE> </LINES> </SCENE> </SCENES> </PROGRAM> [0150] It will be appreciated that the Scene <STARTTIME> in FIG. 5 is at a positive offset from the start of the video file which defines T=0. [0151] An example of the output display on display screen 130 of a PC using a commercial sound editing program such as Sound Forge by the Media Editor to measure and log two lines' STARTTIMEs and STOPTIMEs is shown in FIG. 6. The timing measurements for Scenes (not shown in FIG. 6), Lines and Words are made relative to the Video start time, i.e. T=0.0. The logged entries for this example are shown below. LINE TEXT STARTTIME STOPTIME Hello, everybody. 25.861 sec 26.981 sec So, Scott, why don't we start with you? 27.823 sec 29.437 sec [0152] Each Line's STARTTIME is marked initially as the time measured from T=0, just before a Character begins to utter a sound associated with a section of their dialog, and the same Line's STOPTIME is defined as the time measured from T=0 just after the same Character finishes uttering a sound associated with the same Line of their dialog. However, the editor can add in extra time at the beginning and end of a line to allow the user to get cues from previous dialog and to overrun at the end. The region of the recorded line to submit for processing (without the audio recorded during the lead-in and, optionally, lead-out) will be determined and logged as described previously as the DUB_START and DUB_STOP times. [0153] Measurements of the start and stop times for each Word or character's sound in each Line are similarly made by listening to the audio and observing the waveform. [0154] An illustration of the measuring and logging of the STARTTIMEs and DURATIONS of two words using a waveform editor such as Sound Forge is shown in FIG. 7 and the logged entries are shown below. WORD STARTTIME DURATION Hello 25.978 sec 0.883 sec Everybody 26.233 sec 0.780 sec [0155] A technique using speech recognition software in conjunction with an automatic word timing extraction program, such as that described as a component in U.S. Pat. No. 6,185,538 (Schulz), can alternatively be used to automate this word mark up preparation phase. [0156] Once the markup is complete, the file for each programme, which can contain data for several scenes, is saved as a Media Related Data file 430 in an XML format. It is this data that will subsequently be used in the computer program 450. [0157] To summarise the media preparation phase, at this point, a Media Preparation Editor has created .avi files held as data 440 for each Scene, each Scene file containing a selected portion of the Original Video, and the corresponding portions of audio tracks, i.e. an M&E track, one track comprising the Original Dialog tracks mixed with the M&E track (the Full Mix), and separate .wav files for each characters Original Dialog audio also held as data 440. He has also created XML data files held in files 430 containing all the logged data and text for each Scene, Line and Character, all of which serves as database information for the program 450. [heading-0158] User's Computer Program Modules [0159] Computer program modules provide in this embodiment the system control, data processing, recording and playback functions using the media and processed XML data described above, and provide the interoperability between modules and the computer hardware of the end user. [0160] FIG. 8. is a schematic block diagram containing the main operational software modules 810 to 840 of the Run-time Computer Program 450 and data 430 and 440 which have been assembled as an executable PC application program, copied to CD ROM 165, and loaded from CD ROM 165 onto the users PC system 100 (FIG. 1). The program 450 when run on system 100 in this embodiment carries out the user-initiated operations of the revoicing system described previously in conjunction with the graphical user interface 320 in FIG. 3. [0161] The computer program modules 810 through 830 and application-specific Direct Show modules in the module block 840 can be written in C++, which is described in books such as The C++ Programming Language (Special 3rd Edition), by Bjarne Stroustrup, Addison-Wesley, 2000. The same or similar functions may equally be implemented with modules written in other programming languages and environments. [0162] In FIG. 8, blocks bordered by solid lines represent original application-specific software modules written in C++, which when compiled and running on the users computer system 100 or on the computer 105 b of FIG. 4: a) provide the graphical user interface 320; b) access audio, video and other data; and c) carry out all the required data processing. Blocks shown bordered by dashed lines represent hardware 130, 151, 152, data storage modules 850 or commercially available software modules included in the module block 840 normally-provided with PC operating systems or as add-ons to operating systems which, among other essential functions, support synchronized playback of multiple digital audio and video signal streams and audio recording. [0163] One example of the software modules of block 840 are those components and drivers provided in the Microsoft� DirectX� versions 8.0 or higher and in the Windows� range of operating systems. DirectX provides access to Microsoft� DirectShow� services, which are used to playback multimedia streams from the local files and capture audio data streams from an audio input device such as the microphone 159. These modules also enable playback of compressed video and audio content (compressed in various formats, including Audio-Video Interleaved (avi), and wav). The DirectShow services provide a modular system of software components called filters, which are arranged into a configuration called a filter graph by a filter graph manager which is a component amongst the modules of block 840 that oversees the connection of these filters and controls the data stream flow. An application program module 824 controls the DirectShow filter graph's activities by communicating with the filter graph manager. These Microsoft modules and their documentation can be obtained from http://www.microsoft.com. In this embodiment, the DirectShow modules and Windows operating system primarily provide the video and audio media handling functions in the module block 840. However, some application-specific DirectShow modules have been written in C++ to provide functions unavailable in the Microsoft library of DirectShow services. These modules reside in the module block 840 and will be described hereinafter. [0164] Other computer operating systems generally have functionally equivalent software modules or can be provided with such modules. [0165] A user interface software module 810 creates and displays the contents of the graphical user interface 320 shown in FIG. 3, services and updates the controls in the interface 320, and supports the video and the audio input and output. Sub-modules 812 to 818 within the module 810 provide specific interface programs: sub-module 812 supports the on-screen control operations of the graphical user interface 320; sub-module 816 co-ordinates the Video and Graphics Display output which is presented to the user via the Display Hardware and Monitor 130. Other sub-modules 814 and 818 control respectively user-initiated Audio Recording and Audio Playback through the PC Audio Input and Output systems 151 and 152 which together form the Audio I/O hardware 150 of FIG. 1. The audio input hardware module 151 includes analog to digital conversion, and the audio output hardware module 152 includes digital to audio conversion. The control sub-module 812 generates user interface commands in response to actuation of controls in the graphical user interface 320. All these application-specific modules and sub-modules can be implemented by those skilled in the art using conventional programming techniques. [0166] A Command Interpreter module and main program 820 responds to the user interface generated commands from the control sub-module 812 to determine which data sources are to be combined to produce the required program action and media output. The main commands requiring different media source combinations are Rehearse, Record, Play, Play Scene and Play All commands, the use of which is described hereinbefore with reference to FIG. 3. [0167] A database management module 828 maintains and accesses a database of files and directories containing the static data which comprises edited Audio/Video files 440, Text Data 853, Scene and Line data 854 including timing data derived from XML data and image files, Guide Signal Feature Data 855, and dynamically created data comprising temporary User Dialog Recordings 856 and Aligned User Dialog 858 (or Takes). The data 853, 854, and 855 are the media related data 430 described with reference to FIG. 4 hereinbefore. In this embodiment, the selected Audio/Video files 440 and media related data 430 are supplied on the CD or DVD ROM or other removable storage device 165 and the user recordings 856 are written to and maintained on a hard disk 140, or in other forms of data storage devices, including RAM, as appropriate. In FIG. 8, the data storage area is generically indicated as data storage 850 and can be implemented using such various or multiple forms of data storage. [0168] Essential to all the operations described herein is the means to maintain exact timing relationships between all the digital media streams during playback and recording. The main modules that enable this are modules 824 and 840 which act together as a Playback and Record engine and in effect drive the other modules of the system. DirectShow Modules in the module block 840 are configured in different ways in dependence upon the operation instructed by the user, as will be explained hereinafter in relation to the Rehearse, Record and Playback operations. [0169] Module 824 receives instructions from the Command Interpreter module 820, calculates any required timing data and assembles lists of control data instructing the audio and video sub-system in module block 840 to construct sequences of selected source data streams from the stored data. The control data includes file names, relative timing and positions of the requested media data content. The control data and media data streams are fed to module block 840, which assembles the media data into the requisite number of output and/or input data streams in correct temporal order, and maintains their relative synchronicity as they are fed through the modules 816 and 818 respectively to the components 130 and 152 that render them into the appropriate audio, video and graphics outputs for the user. Module block 840 also provides mixing and gain processing for the different audio streams as discussed in more detail below. [0170] Further software components in module 824 access, through the database management module 828, text and timing data initially logged in the XML files and stored as text or binary data in storage 850 and present highlighted text on the display 130 synchronized with the audio via module 816. [heading-0171] System Module Actions in Each User-Initiated Phase of Operation [0172] The following descriptions outline the resulting actions that the system hardware and software modules perform in response to the user activating the Rehearse, Record, and Play controls in the graphical user interface 320 of FIG. 3. These actions implement what has previously been described as taking place in the user interface 320. [0173] State changes at buttons and selection areas in the interface 320 are detected by module 812 in FIG. 8 and the resulting system activity is initiated by the command interpreter and main program module 820 on the media and media related data in the data storage 850 with appropriate timings and data access requests computed in module 824. [0174] FIGS. 9, 10, and 12 illustrate schematically how in this embodiment the various media and data sources are combined in the Rehearse, Record and Play phases to create the sequence of audio, video and graphic outputs that the user hears and sees. [heading-0175] Rehearse Phase [0176] When the user activates the Rehearse button 361 in FIG. 3, the database management module 828 in FIG. 8 accesses the required media, text and timing data for the selected Scene and Line. [0177] FIG. 9 illustrates schematically the relationships between the data sources, the media event timings, and the Output Devices. Module 824 uses Line N's STARTTIME (TSTARTTIME(N)) to begin the playing of the selected video data stream 910 and the associated Full Mix audio data stream 920 in sync via module block 840. [0178] A Gain Control signal 930 is also computed which, via DirectShow application-specific software modules represented by multiplier 925 in FIG. 9, and similarly by 1026, 1027, 1028 and 1029 in FIGS. 10 and 12, applies audio gain commands to the output audio. Such a software multiplier is given a list of gain time events as control data before the operation starts, and converts the event times from time in seconds into a sample offsets at the data rate of the audio stream. During playback, the software multiplier multiplies the incoming audio data stream by the appropriate gain factor defined at the specified sample times and maintains that gain factor until the next gain event in the corresponding control data occurs. [0179] In FIG. 9 at TSTARTTIME(N), the Full Mix audio signal is set to a gain of 1.0. This lets the user hear the original Lines via the audio output system 152 and see on the Video Display 130 the corresponding video action preceding and during the selected Line, which helps the user to prepare and refine his performance. The system stops playing the Line's media at TSTOPTTIME(N). Generally, to give the user extra time to complete his reading, TSTOPTTIME(N) is set a second or so beyond the ending of the original dialog in the Full Mix. [0180] The module 824 also computes the time TCUE (N) at which to start the graphic display 940 of the countdown characters “3”, “2”, “1”, and “GO”, and times the display of the characters individually in sequence so that “GO” appears at the time the actual Line starts (defined by TSPOT — TIME (N)). This cues the user to begin speaking. These graphic cues are displayed, in the present embodiment, in the Line display and selection window 350. Optionally audible signals such as “beep tones” can also be emitted at the same times as the 3, 2, 1 graphics—but not at “GO” since they would then interfere with recording in the Record phase. At the time of the Line start, the system also highlights, as indicated at 950, in the Line display and selection window area 350, the text of the Line's words or transcribed sounds individually in sync with the original audio signal, giving the user further timing cues during this Rehearse pass. All the timings and text used in this and the remaining phases are derived from the data measurements and text entered in the preparation phase into the XML file. [heading-0181] Record Phase [0182] When the user activates the Record button 362 (FIG. 3), modules 812 and 820 in FIG. 8 respond and engage module 824 in referring to the database management module 828, and this in turn accesses the same media, text and timing data for the selected Scene and Line as was accessed in the Rehearse phase. Again, the system uses Line N's STARTTIME (TSTARTTIME(N)) which includes the lead-in time at which module block 840 is instructed to begin playing the video 910 and the Full Mix audio 920 data streams in sync, as shown schematically in FIG. 10. [0183] In this phase, unlike in the Rehearse phase, the command interpreter and main program module 820 controls recording of the user's audio and, at TSPOT — TIME(N), causes the following additional functions to be performed: Reduction to 0.0 i.e. zero of the gain of the Full Mix signal 920 using Full Mix Signal Gain control data 930 computed by module 824 and fed to module block 840 via the software module multiplier 1026; Further M&E Signal Gain control data 1040 applied to multiplier 1027 raises from 0.0 to 1.0 the gain of the M&E (Music and Effects) signal 1030 so that the user does not hear the original dialog during the recording phase but hears the rest of the background audio. [0186] Note that although the user can hear the background audio during a recording, the microphone is held sufficiently close to the user's mouth so that the level of airborne background (M&E) signal received by the microphone is negligible compared to the level of the voice and therefore the background audio is nearly inaudible in the Takes. [0187] The module 824 again computes the time TCUE(N) to display the countdown characters “3”, “2”, “1”, and “GO” graphically represented at 940 and times their sequence so that “GO” appears at the time TSPOT — TIME(N) at which the dialog starts and the user should begin speaking. In practice, recording has already started, as indicated at the start of user recording 1060, which allows the user to be recorded even if he starts speaking too early. During this phase, the module 824 again controls highlighting of the displayed text 950 of the Line's words or sounds in sync with the muted dialog audio signal, giving the user further timing cues during the recording phase. [0188] Recording of the User Recording 1060 through the microphone 159 connected to the Audio Input 151 is started at TSTARTTIME(N) and ended at TSTOPTIME(N). Recording employs the Input Audio Processor Module 814 which uses the DirectShow recording modules in the module block 840. The module 824 stops the playback through the module block 840 of the Line's video and audio media at TSTOPTIME(N) but to give the user extra time to complete his reading, TSTOPTIME(N) is normally set a second or so beyond the ending of the original dialog in the Line. These recording modules also synchronize the timing of the digitized user input audio with the playback of audio and video in the other modules 816 and 818. [0189] The user's voice recording is stored as a temporary audio file 856, which is subsequently used as the source of Dub data for the alignment process. Recording and alignment processing module 830 measures and stores the time offset between the start time TSTARTTIME(N) of the Full Mix audio output 920 starting and the actual start time of the recording of the user. In this embodiment, the temporary audio data file will be processed immediately according to methods that will be described below. [0190] If further Recordings of a Line are initiated, the management module 828 in FIG. 8 maintains a record of the Takes and presents, through the display sub-module 816, a representation of the additional Takes 373 and 374 for that Line on screen as described hereinbefore with reference to FIG. 3. [heading-0191] Creating the Aligned Replacement Audio Streams [0192] The process of creating a version of the user's dialog recording to serve as an accurate and convincing replacement (in terms of lip-sync) of the original Dialog will be described next. We refer to the original Dialog audio as the Guide audio because its time-varying features will be used to guide the alignment process. A user's edited replacement audio signal is described as “aligned” to the Guide audio if, when measured from corresponding start points in each signal, a substantial proportion of the main audible time-varying acoustic features in the replacement signal occur at the same relative times as corresponding features in the Guide audio. [0193] To achieve such alignment of a user's recording, an automatic audio alignment algorithm is used which analyses both audio signals and automatically edits a Temporary User's recording 856 in FIG. 8 (one of which will be referred to herein as the “Dub” audio because that is a known term in the audio-post production industry) to align to the Guide audio. Such an algorithm has been described in GB2117168 and U.S. Pat. No. 4,591,928 (Bloom et al). It comprises the essential steps of: (a) measuring the same time-varying features of both the Guide and Dub audio and saving these measurements as digital feature data; (b) processing the resulting time-varying feature data to determine a time distortion path that best aligns the Dub audio features to the Guide audio features; and (c) passing the time distortion path to an automatic waveform editor which edits the waveform of the Dub audio according to the time distortion path, whilst also taking into account the properties of the waveform data of the Dub audio in order that edits made in the new aligned waveform do not introduce audible artefacts. [0194] In the simplest case, a Line contains only uninterrupted original dialog from a single character in addition to the M&E audio. One example of the Guide audio in such a case was illustrated in FIG. 5 as Character Voice 1, audio stream 520. This Guide audio stream, if played with the Line's Video data (although this does not take place in this embodiment), would run synchronously with the corresponding Video data and be in sync with the image of the Character's lip movements. [0195] The use of the alignment algorithm 830 in FIG. 8 will now be described with reference to FIG. 11. In this embodiment, the selected Line's edited Original Character audio 440 in FIG. 11 serves as the Guide audio and is accessed from data storage 1110. [0196] In this embodiment, the Guide audio is pre-processed to extract the time-varying feature data and therefore data storage 1110 represents the disk storage 140 in computer 105 b in FIG. 4. Because the Guide audio is not played during alignment in the User's PC, only the time-varying guide signal feature data 855 is required to represent the Guide audio during time alignment in the end-user computer program. Pre-processing during the media and program preparation phase is advantageous because it reduces the amount of audio data storage in the computer program image on removable media 165 in FIG. 4 and the processing required when the User runs the program. The elements involved in this optional pre-processing of the Guide audios are shown within the box with dashed lines 1150. The alternative to pre-processing, which is to implement the activities in 1150 in the User's PC 100 in FIG. 4, will be described later. [0197] In one example of pre-processing, the Media Preparation Editor initiates the computer-based processing of the digitized version of the first Line “I am the first,” shown as the: first hatched block in stream 520 in FIG. 5, as Original Character Line N of audio data 440 in FIG. 11. This digital audio stream is first put through a Feature Analysis computer program 1130, implemented in this embodiment by a 4 band digital filterbank (with octave-wide bands starting at 250 Hz, 500 Hz, 1000 Hz and 2000 Hz) and sampling the measured 4 band audio spectrum every 10 ms (as described in U.S. Pat. No. 4,591,928). The output from this analysis of the Guide audio is FG(N,n) where G represents Guide, N indicates line N, and n is the n-th 10 ms analysis time frame n, where n starts at 0. The analyzed data is stored as data file 855 in the system's data storage 1120 for that Programme, along with the other previously discussed Media Related Data including Guide signal identification, Line text data and the GUIDE_START and GUIDE_STOP timing information measured manually by the Media Preparation Editor as indicated by Media Related Data 430 in FIG. 4. Data storage 1120 can either be in the Data Storage 140 in PC 105 b if pre-processed or in Data Storage in the User's PC 100 such as in RAM or Hard Disk 140. [0198] In a further embodiment, if the program and data in storage 140 FIG. 4 are compiled into a binary form, the pre-processing to create the feature analysis data as described above can be automated to take place during the compilation phase in PC 105 b, using the XML data to define the file name and the extent of the audio within the file to be pre-processed, based on the Guide-related data entered into the XML file by the Media Editor. [0199] Whether the Guide audio is pre-processed or not, the majority of the alignment process occurs in this embodiment after the completion of the digital recording of the user's version of Line N 856 as described hereinbefore with references to FIGS. 10 and 8. The Main Program 820 in FIG. 8 controls the alignment processing module 830 in cooperation with the database management module 828, which controls the access of the required media and the storage of the processed results. [0200] The digital waveform data for this user's audio recording plus its stop- and start-time information are stored in Data Storage 140 as a block 856 in FIG. 11. Alternatively, other forms of Data Storage such as RAM in the User's PC can also be used. Once all or even a portion of the digitized User's audio data becomes available, it can be processed by the same or identical Feature Analysis computer program 1130 as that used on the Guide audio. A program 1130 corresponding to the Feature Analysis computer program 1130 therefore forms part of the recording and alignment processing module 830 shown in FIG. 8. [0201] The additional time variables DUB_START and DUB_STOP (measured and logged as data by the Media Editor as previously described), are supplied by the database management module 828 from Scene and Line Data 854 in storage 850, to the Recording and Alignment [0202] Processing module 830 in FIG. 8 to select the range over which to process the User's recordings instead of using the Line's STARTTIME and STOPTIME variables, which may have been set to occur somewhat before and after the Guide audio of the Line. Normally DUB_START is set slightly ahead of SPOT_TIME (say, by 0.2 seconds), and DUB_STOP is normally set to the Line's STOPTIME. The values of GUIDE_START and GUIDE_STOP are provided to the Recording and Alignment Processing module 830. [0203] The time varying feature data streams for Line N from the selected ranges of the Guide, FG(N,n) and the selected range of the User Recording, Fu(N,n), are processed by a digital Time Alignment Analysis program 1160 (described in U.S. Pat. No. 4,591,928) to create a time alignment path P(N,n) 1165, where n is again the feature stream sample time. The sampling period for n used here is 0.01 second. The alignment path data P(N,n), which defines the time distortion that must be applied to the selected range of the User Recording 856 to make its measured time-varying features best match those of the Guide audio 440, is input to an Automatic Waveform Editing program module 1170 which operates in the manner described in U.S. Pat. No. 4,591,928. The Time Alignment Analysis program 1160 and the Automatic Waveform Editing program module 1170 are further parts of the Recording and Alignment processing module 830. [0204] The function of the Automatic Waveform Editing module 1170 is to edit the User's Audio Recording 856 of Line N and time-compress or time-expand the audio signal waveform where it is required according to P(N,n) 1165, but with the added condition imposed that each edit used to create the output waveform is made pitch synchronously wherever the signal has a measurable pitch at the joining point. Put another way, when cutting out or repeating a section of the original User's waveform to time compress or expand it respectively, the module 1170 analyses the signal waveform at the proposed cut and splice points and adjusts the edit point to ensure that the end of a signal block joins the beginning of the next non-contiguous signal block at a point which best aligns the two waveform's phases in order to not introduce audible artefacts. Further details of this process are available in U.S. Pat. No. 4,591,928 or in papers on time-scale modification of speech in Speech Enhancement, edited by J. S. Lim, Prentice Hall, 1983. Other methods of time distorting an audio signal waveform or analysis and resynthesis techniques may be used to create the required time-aligned waveform. [0205] The resulting, time-aligned digital audio User Recording data is then stored, along with associated data defining the timing of the start and end of the new audio, as a waveform data file 858 in digital storage 140. The file 858 and other such files are represented collectively as Takes 858 in FIG. 8. [0206] The preceding example describes the simple situation when there is a single Character's audio stream contained in the Line. In a more complex situation, as illustrated in FIG. 5, when the Dialog stream 530 of a second Original Character overlaps with the first Character, a separate data file can provide the second Character's Original Dialog stream. This second stream will also have a Line STARTTIME defined relative to the start of the video, but to avoid having large amounts of zero value audio data, the STARTTIME of this second stream can be combined with a non-zero offset time variable which is measured relative to the start of the Video Data in order simply to use less data storage space. The audio stream of the second Character can be treated the same as that of the first, using simple arithmetic to add the offset value to compute the starting point for data access and processing. [0207] For simplicity, it has been assumed that during the Recording phase the user reads and records the words in the script correctly, but does not achieve perfect sync with the Original Character's voice in the original dialog audio and the corresponding lip movements. It is also assumed that, in the event the user makes a substantial error in reading the text, the user will stop and make further attempts until he gets it essentially correct apart from timing. If, however, there is an error in the reading, or the user records words different from those in the script, and the audio alignment algorithm achieves results similar to that aimed for in the algorithm in U.S. Pat. No. 4,591,928, then it is likely that significant time-varying features of the Replacement audio will still be aligned to other significant time-varying features of the original audio, thereby providing some level of lip-synchronization. [0208] Lastly, it should be clear that pre-processing the Guide audio is not required or even always possible and, in alternative embodiments of this invention, the Feature Analysis of the Guide audio can take place at the same time as the feature analysis of the Dub audio which can be during or after the Recording pass. In particular, pre-processing would not be feasible in circumstances where the Guide audio is not pre-determined. [0209] For example, one instance of this is a further embodiment in which the programme 450 in FIG. 4 is provided without the audio/video Media to be revoiced and the user can select the media to be revoiced. In this embodiment, the User can apply the revoicing methods and apparatus described herein to digital audio/video media that has the appropriate characteristics to allow it to be analysed by computer-based digital signal analysis techniques to extract the information required to create the data used for revoicing as described in this invention. Such analysis and data creation is feasible using known techniques including speech recognition and timing measurement (to automate the manual transcription and text entry step), extraction of voice signals from noise or unwanted background audio when necessary (to create Guide audio), and removal of voice signals from background audio or generation of background audio music and effects by synthesis or other techniques (to create replacement M&E tracks). [0210] Such procedures could be applied, as an example, to certain films or music videos on DVDs, in which there are often multiple audio tracks that could be separately analysed to improve the quality and effectiveness of the Guide audio extraction and removal, speech to text generation, and provision of background audio without voice during playback stages. Similar procedures could be applied to other digital audio/video media carriers. [0211] Similarly, the individual steps in the processing steps described above could be distributed between the User's PC and other computers, for example, a program based on a server on the Internet. [0212] In other embodiments, in cases where either or both speech to text transcription or background audio replacement are not feasible, the revoicing process can still be applied in its most basic form by analysing the available Guide audio (including background signals if present) for alignment purposes only, creating the aligned User 's recording, and playing only the aligned User's audio with the Video, but without any background Music and Effects audio. In other words, it is preferable but not absolutely essential to provide highlighted text and recreate the audio background present in the original audio/video programme as part of the revoicing process. [heading-0213] Playback of the Replacement Aligned Audio [0214] There are three main modes of Playback using the Aligned audio that the user can initiate via Commands Play, Play Scene and Play All in the User interface 320. [0215] FIG. 12 illustrates schematically what takes place in response to a Play Scene command but also contains the events in a simpler Play Line function. FIG. 12 again illustrates the relationships between the same data sources, Media files and Output Devices that were described in FIG. 10 for the Record Pass, but provides the elements of a simple Scene containing two Aligned User Recordings 1270 and 1271 as the available media for output. How these various media files are combined during the playback phases will be described next. [heading-0216] Play Line [0217] If the User selects Line N via the Line Selector 350 in the User Interface 320 (FIG. 3) and clicks the Play button 363, this initiates the Play Line function. During Play Line, the sequence of events results in the User hearing a single Aligned User Recording 1270 (FIG. 12) of Line N played back simultaneously with the Original Video 910 and M&E 1030 tracks, but without the original dialog contained in the Full Mix 920. The detailed events are as follows. [0218] Following the user activating the Play control 363, the Command Interpreter module and main program 820 in FIG. 8 accesses the Database Management module 828 to obtain from storage 850 the same media, text and timing data for the selected Line N as in the recording phase. Next, module 824 in FIG. 8 computes from Line N's data for TSTARTTIME(N) (FIG. 12) the times at which to begin playing the Original Video 910 and the Full Mix audio 920 data streams in sync via module 840. A Full Mix Signal Gain control signal 930 is applied to the Gain control software multiplier 1026 as a gate on the Full Mix audio 920. This allows any audio before the replaced audio to be heard via the software mixing device 1028, which is implemented in DirectShow in module 840, and which sums the incoming audio signals and feeds the summed signal to the Audio Output 152 through the Output Audio Processor module 818 in FIG. 8, not shown in FIG. 12. [0219] The Aligned User Recording for Line N 1270 in FIG. 12 is retrieved from storage 850 in FIG. 8 by the database management module 828 and played through the module 840 starting from the stored timing data TSPOT-TIME(N), and at a corresponding time (plus any additional amount of samples required to allow the fade-in and fade-out to take place), two Gain Control signals 1040 and 1280 for the M&E track 1030 and the Aligned User Recording 1270 respectively, are ramped up to 1.0 to let these signals be mixed and heard via the audio output 152. At the same time that 1040 and 1280 are ramped up to 1.0, the Full Mix Gain 930 is ramped to 0.0 and thereby turned off. When the Aligned User Recording stream 1270 comes to an end, the Gain Control signals 1040 and 1280 for the M&E stream 1030 and Aligned User Recording 1270 respectively ramp back to 0.0, and the Gain Control signal 930 for the Full Mix 920 (which has continued to run in sync) ramps up to 1.0 allowing this track to be heard again. In this embodiment, the Gain Control ramps provide a fade-up or fade-down of 10 ms in length at the transitions to ensure a smooth crossover. At the calculated switching times such as TSPOT-TIME (N), the gain control ramps are at a value of 0.5. Thus, for switchover calculated at a switching time TSWITCH, the gain transitions are calculated to start at TSWITCH−5 ms and to stop at TSWITCH+5 ms as indicated schematically in FIG. 12. [0220] The Full Mix and M&E signals 920 and 1030 are edited by the Media Preparation Editor to sufficient sample accuracy to ensure that both the M&E signal waveform contained in the Full Mix signal and the separate M&E signal waveform are phase locked, ensuring that the summed output of the mixer 1028 does not contain phase cancellations creating artefacts in the audio output. [heading-0221] Play Scene [0222] The Play Scene phase allows the user to hear a sequence of Lines with Aligned User Recordings when available in context with the Video, the M&E audio and, when no User Recording is present, the Full Mix with the Original Voices. [0223] The User Interface 320 in FIG. 3 allows the user to record one or more Takes 371, 373, 374 for each of the Lines such as line 353 in the Scene. The Database Management module 828 in FIG. 8 keeps track of the last user-Selected Take for each Line and loads and controls sequential playing of the Selected Takes during a Play Scene operation, which is initiated by the user clicking on the Play Scene button 344 of the interface 320. If a Line in a Scene has not been recorded and the Take Selector points to an “empty” (i.e. unrecorded Take), or the user has clicked on Take 372 to select the Original Voice Recording as the Selected Take for a specific Line, then the Command Interpreter and Main Program 820 will play the Original Voice Recording 372 for that Line. [0224] FIG. 12 shows schematically the timing of events when the System in FIG. 8 assembles in sequence two Aligned User Recordings, shown as audio block 1270 for Line N and audio block 1271 for Line N+1 during the playback of the example Scene M between times TSCENESTART(M) and TSCENESTOP(M). Again, the Aligned User Recording Gain control signal 1280 is used to fade up to Gain=1.0 (i.e. ON) both the Aligned User Recordings and the M&E track when there is Aligned User audio present and ramp the Full Mix to Gain=0.0 (i.e. OFF) at these times, applying the cross fades as shown and described hereinbefore. [0225] If a Scene has multiple Character Voices which sometimes overlap as shown in FIG. 5 at stream 520 and stream 530, then the modules 820 and 824 enable additional simultaneous streams of Aligned User Recordings such as the stream containing blocks 1270 and 1271 and corresponding Gain control signals such as the signal 1280 to be created during the Recording Phase, and all of the gated audio signals are summed at mixer 1028 in FIG. 12 to form the audio output that heard via the Audio Output device 152 during a Play Scene phase. At areas of multiple overlapping User Recordings, the module block 840 in FIG. 8 maintains playback of simultaneous synchronized audio streams and has more than one User Recording Gain ON at 1.0. [0226] Apart from the use of a different start and stop time, and possible multiple Aligned User Recording audio streams with overlapping being controlled in parallel, any remaining details of the Play Scene command are essentially the same as the Play Line phase previously described. [heading-0227] Play All [0228] When the user selects the Play All button 346 in FIG. 3, the module 824 in FIG. 8 will initiate a sequence of Play Scene commands starting with Scene 1 (or alternatively from any particular Scene selected in the Scene Selector 340) and continuing playing through all of the Scenes shown in the Scene Selector 340. During the playback of each Scene, as described previously in the Play Scene phase, Selected Takes are used for each successive Line in each Scene. Following selection of the Play All button 346, the system in FIG. 8 will continue playing Scenes in this embodiment until the user presses the keyboard Stop command (i.e. the spacebar) or playback of the last Scene is completed. [0229] Depending on the type of media, the Play All command can utilize a further higher level of structured data organization such as a “Song” or “Movie” which includes data describing which Scenes are contained in that Song or Movie. Playing a Song or Movie, as a “Play All” function, would initiate the contiguous playback of all Scenes in that structure sequentially and consequently this would trigger the sequential playback of each successive Line in each Scene with the Lines' Selected Takes. [0230] In the present embodiment, Play All is only available if the Scenes all come from a single continuous video file, but such restrictions may be removed in further embodiments. In a further embodiment (not shown here), it is possible to arrange for a higher level of audio organization: for example, a sequence of Songs or Movies to be played back similar to a “juke box”. [heading-0231] Detail of System Data Processing Operations During Playback [0232] Reference will now be made again to FIG. 8 and especially to the module 824 and module block 840 which create the audio streams for playback in both the record and playback operations. [0233] Lines are the smallest element that can be selected for Playback and are the “building blocks” for all other forms of Playback and Recording. For every Line in a Scene, the Database Management module 828 maintains dynamically a data array of Takes including, among other data, references to the file names for each Aligned User Recording as well as data specifying which one Take is the Selected Take for playback. [0234] Once the user has recorded Lines, as described previously, for each Line the User Interface 320 indicates which is the Selected Take for playback 371 by a graphical frame around it. The user can manually make any one of the available Takes for a Line, including the original character's recording, the Selected Take. [0235] For each Scene, the Database Management module 828 dynamically maintains and accesses data which includes the array of Lines and Takes associated with that Scene. [0236] In any Play command, the application module 824 in FIG. 8 will receive an instruction specifying whether it is to output a single Line (with its Selected Take), or a Scene (containing at least one Line), or a Song or Movie (containing one or more Scenes) or other higher level collection of Songs or Movies. In any of these cases, the application module 824 will go through the lists of data structures associated with the request and ultimately Line by Line construct a list of all the requested Selected Takes. The required static data defining the timings and media file names are obtained from the data structures on the CD/DVD ROM 165 FIG. 4 that were created from the original XML data files, and other dynamically-created data related to the User Takes created and maintained by the database management module 828 in FIG. 8. [0237] During the playback list construction process, the filenames of Takes are passed to one or more Take File Reader modules 1330 shown in FIG. 13. A Take File Reader module 1330 is an application specific program which, in the present embodiment maintains a list of user Take files to be played, accesses the appropriate files, and then streams the audio data into the next module (as will be described further below) when the associated pre-calculated time slots occur. The Take File Reader modules 1330 are created by the application program module 824 and function as an active buffer between the Data Storage 850 and Media Stream Creating modules in the module block 840. [0238] A Take File Reader 1330 works as follows. Prior to playback, if a Take selected for playback has an associated data file which exists, as described above, module 824 adds the appropriate data to the list of Takes to be played by a Take File Reader. [0239] During playback, a Take File Reader 1330 creates a single continuous Aligned User Take Audio Data Stream 1335. Before playback, the module 824 examines the Start and End Times data of each Selected Take that is requested and, if the start or end times for a playback request would cause one Take to overlap another Take in time, deploys an additional instance of another Take File Reader to provide a parallel Take data stream. [0240] The module 824 creates and deploys the minimum number of Take File Readers 1330 that will allow all Selected Takes to be played at the required times under the constraint that each Take File Reader can only play Takes sequentially (i.e. without overlaps). The use of multiple Take File Readers 1330 illustrated in FIG. 13 is used in this embodiment to support the required overlapping (parallel) playback. Other mechanisms for playback could be devised to achieve the same result. [0241] Once the request to Start playback is received, the name of the selected Take is passed to a Take File Reader 1330. The Take File Reader uses Operating System function calls to read out a stream of digital samples from the Take file and to put this data into a format suitable for DirectShow data streams in module block 840. [0242] Those skilled in the art will understand that the process described is implemented as a demand fed output system. When a Take File Reader 1330 is instructed to supply data to the next module in line, the Take File Reader 1330 reads the number of samples that have been processed and, for each sample that has been asked for, the Take File Reader tests whether a file exists with the data required to fill the request. If the data exists, the Reader 1330 supplies the request with data and counts samples until it reaches the end of the file's data. When there are no more samples, the module sends an End of Stream message to the Direct Show filter object and when all active streams have sent this message, the application module 824 terminates the playback function. [0243] If, in order to fill a downstream request for data, a Take File Reader 1330 is left without Take audio data for any duration, then the Take File Reader will supply digital zero data and keep the stream running in sync and ready to inject and output Take data at a later time. At these times when there is no Take data required during a playback request, the application module 824 instructs the module 840 to revert to playing back the original Full Mix data or the Music and Effects track as appropriate to the specific playback mode. [0244] The same mechanisms for creating and outputting demand driven data streams are also used for the Video data, the Full Mix and the Music & Effects audio streams accessed in FIG. 13 from the Multiplexed Video & Audio storage 440. Here, the media data is shown as multiplexed in this embodiment, which is an optional matter, but is advantageous for efficient data access and streaming. [0245] The AVI De-multiplexer 1310 reassembles the individual media data streams. The resulting compressed Video goes through a Video De-Compressor 1320 before being output to the Video Display Software and Hardware 1350 (incorporating both 816 and 130 in FIG. 8). The resulting Full Mix and Music & Effects audio pass through separate audio decompressors 1325 before being streamed through their individual Audio Gain Controls 1026 and 1027, and being summed in 1028 with any User Take audio. The summed audio stream from 1028 is output through Audio Output system 1360 which includes modules Output Audio Processor Software 818 and Audio Output Hardware 152 in FIG. 8. [0246] This process recreates the data streams shown schematically in FIG. 12 for Playback and, if the User Take data 858 and associated Take File Readers are not engaged, the remaining modules and data streams depict the Rehearse function. [0247] A Direct Show filter object module in the module block 840 builds up a list of switching times from the lists of timing data originally provided in the XML program data file. When no Take is available, the instruction put into the list will switch the output audio data stream of the module block 840 back to the original dialog stream (i.e. the Full Mix audio data) at the correct time position for the appropriate length of time. As the list is built, the filter object module also pre-computes the times for gain values to change for ach stream's Audio Gain Controls and enters these values into a further lists to drive the Gain control signals shown in FIGS. 9, 10, and 12. [heading-0248] Detail of System Data Processing Operations During Recording [0249] During the Recording of a Line, module block 840 is configured slightly differently to its configuration during Playback as just described. The difference is shown in FIG. 14 where the Audio Recorder module 1420, receiving data from the Input Audio Processor 814 and creating Temporary User Recordings 856, replaces the User Take playback processing modules shown in FIG. 13. The Video and Audio output stream processing in the upper portion of FIG. 14 however is identical to FIG. 13 and the description above for these modules and data streams in FIG. 13 also applies here. [heading-0250] Clocking and Synchronization of Output Media [0251] The audio output hardware 152 in FIG. 8 provides the clocking of the media during both Record and Playback operations and creates a demand driven chain from the hardware 152 back through all the other audio media data handling modules and processes. [0252] When a command to start outputting is received in the application module 824, the final output module for each type of media asks for sufficient data to fill an input buffer of that output module. The individual modules in the chain before the last module do not need to know the current output “time”. Each module in the chain only responds to and fulfils requests for supplying the type of data that the module handles. [0253] All of the modules responding to data requests independently count the number of samples being passed onward to the next module in the output sequence chain in order to determine if they need to request more data from downstream or not. This includes the Video data in addition to the Take audio data, and the original Scene audio with and without the dialog (Full Mix and M&E). [0254] Prior to the output of media related to one or more Lines, when a Start Playback or Record request is received, buffers in the module block 840 that feed the output sub-modules 816 and 818 are filled with the required audio and video data by Direct Show operating system modules in the module block 840. [0255] The video and audio output data synchronization is handled in this embodiment by the so-called Operating System modules implemented by Direct Show within the module block 840. These OS modules also implement the system buffering and output requests. Other modules in the module 840 are clients of Direct Show. Direct Show functions and operation are described in the DirectX SDK (Software Development Kit) which is provided and documented by Microsoft, Inc. and in other publications. [0256] The method and apparatus described hereinbefore provide for an untrained user a single integrated, simple system which: allows a member of the general public to replace the voices of characters or singers that accompany a video clip with their own voice, which is automatically processed and placed during playback such that it is accurately synchronized with displayed lip movements. Patent CitationsCited PatentFiling datePublication dateApplicantTitleUS3743391 *May 14, 1971Jul 3, 1973White DSystem for dubbing fresh sound tracks on motion picture filmsUS4479150 *Nov 2, 1981Oct 23, 1984Basf AktiengesellschaftMethods and arrangements for changing the audio information in an audio/video recording/reproduction systemUS4591928 *Mar 23, 1983May 27, 1986Wordfit LimitedMethod and apparatus for use in processing signalsUS4839733 *Dec 15, 1987Jun 13, 1989Karamon John JMethod and system for synchronization of an auxiliary sound source to motion picture film, video tape, or other picture source containing a sound trackUS5111409 *Jul 21, 1989May 5, 1992Elon GasperAuthoring and use systems for sound synchronized animationUS5553864 *May 22, 1992Sep 10, 1996Sitrick; David H.User image integration into audiovisual presentation system and methodologyUS5613909 *Jul 21, 1994Mar 25, 1997Stelovsky; JanTime-segmented multimedia game playing and authoring systemUS5634020 *Dec 31, 1992May 27, 1997Avid Technology, Inc.Apparatus and method for displaying audio data as a discrete waveformUS5649060 *Oct 23, 1995Jul 15, 1997International Business Machines CorporationAutomatic indexing and aligning of audio and text using speech recognitionUS5782692 *Mar 22, 1997Jul 21, 1998Stelovsky; JanTime-segmented multimedia game playing and authoring systemUS5809454 *Jun 28, 1996Sep 15, 1998Sanyo Electric Co., Ltd.Audio reproducing apparatus having voice speed converting functionUS6034679 *Apr 9, 1997Mar 7, 2000Sony CorporationMethod and apparatus for controlling editing of recorded audio and/or video materialUS6151441 *Dec 19, 1994Nov 21, 2000Sony CorporationSystem for storing and reproducing multiplexed dataUS6166731 *Sep 24, 1997Dec 26, 2000Sony CorporationEditing digitized audio/video data across a networkUS6185538 *Jul 29, 1998Feb 6, 2001Us Philips CorporationSystem for editing digital video and audio informationUS6260011 *Mar 20, 2000Jul 10, 2001Microsoft CorporationMethods and apparatus for automatically synchronizing electronic audio files with electronic text filesUS6283760 *Aug 24, 1998Sep 4, 2001Carl WakamotoLearning and entertainment device, method and system and storage media thereforUS6352432 *Mar 23, 1998Mar 5, 2002Yamaha CorporationKaraoke apparatusUS6425825 *Nov 2, 1998Jul 30, 2002David H. SitrickUser image integration and tracking for an audiovisual presentation system and methodologyUS6469711 *Jul 23, 2001Oct 22, 2002Avid Technology, Inc.Graphical user interface for a video editing systemUS6535269 *Jun 29, 2001Mar 18, 2003Gary ShermanVideo karaoke system and method of useUS6661496 *Jan 8, 2003Dec 9, 2003Gary ShermanVideo karaoke system and method of useUS6816858 *Oct 27, 2000Nov 9, 2004International Business Machines CorporationSystem, method and apparatus providing collateral information for a video/audio streamUS7165029 *May 9, 2002Jan 16, 2007Intel CorporationCoupled hidden Markov model for audiovisual speech recognitionUS7243062 *Oct 25, 2002Jul 10, 2007Canon Kabushiki KaishaAudio segmentation with energy-weighted bandwidth biasUS7424204 *Jul 16, 2002Sep 9, 2008Pioneer CorporationVideo information summarizing apparatus and method for generating digest information, and video information summarizing program for generating digest informationUS20020031334 *Nov 14, 2001Mar 14, 2002Sony CorporationEditing system and method and distribution medium* Cited by examinerReferenced byCiting PatentFiling datePublication dateApplicantTitleUS7372536Mar 8, 2005May 13, 2008Microsoft CorporationPhotostory 3—automated motion generationUS7400351Oct 6, 2004Jul 15, 2008Microsoft CorporationCreation of image based video using step-imagesUS7457880Sep 27, 2004Nov 25, 2008Ximeta Technology, Inc.System using a single host to receive and redirect all file access commands for shared data storage device from other hosts on a networkUS7483967Apr 22, 2004Jan 27, 2009Ximeta Technology, Inc.Scalable server architecture based on asymmetric 3-way TCPUS7555131 *Jun 30, 2009Harris CorporationMulti-channel relative amplitude and phase display with loggingUS7725828 *Oct 15, 2003May 25, 2010Apple Inc.Application of speed effects to a video presentationUS7725830Jul 19, 2004May 25, 2010Microsoft CorporationAssembling verbal narration for digital display imagesUS7746895 *Jul 29, 2005Jun 29, 2010Dell Products L.P.Guided discovery of media contentUS7746900Jul 22, 2005Jun 29, 2010Zhe Khi PakLow-level communication layers and device employing sameUS7783761Aug 24, 2010Zhe Khi PakScheme for dynamically connecting I/O devices through networkUS7787058 *Aug 31, 2010The Weather Channel, Inc.System and method for providing information synchronized to a playlistUS7792923Oct 9, 2001Sep 7, 2010Zhe Khi PakDisk system adapted to be directly attached to networkUS7821574 *Oct 26, 2010Apple Inc.Method and apparatus for synchronizing audio and video streamsUS7849153Dec 7, 2010Zhe Khi PakDisk system adapted to be directly attachedUS7849257Jan 6, 2006Dec 7, 2010Zhe Khi PakMethod and apparatus for storing and retrieving dataUS7860943Feb 23, 2006Dec 28, 2010Zhe Khi PakEnhanced network direct attached storage controllerUS7870225Jan 11, 2011Zhe Khi PakDisk system adapted to be directly attached to networkUS7899802Apr 28, 2004Mar 1, 2011Hewlett-Packard Development Company, L.P.Moveable interface to a search engine that remains visible on the desktopUS7966636May 22, 2002Jun 21, 2011Kangaroo Media, Inc.Multi-video receiving method and apparatusUS8033831 *Nov 22, 2004Oct 11, 2011Bravobrava L.L.C.System and method for programmatically evaluating and aiding a person learning a new languageUS8042140Oct 18, 2011Kangaroo Media, Inc.Buffering content on a handheld electronic deviceUS8051452Nov 1, 2011Kangaroo Media, Inc.System and methods for enhancing the experience of spectators attending a live sporting event, with contextual information distribution capabilityUS8051453Jul 21, 2006Nov 1, 2011Kangaroo Media, Inc.System and method for presenting content on a wireless mobile computing device using a bufferUS8112711 *Oct 6, 2004Feb 7, 2012Disney Enterprises, Inc.System and method of playback and feature control for video playersUS8131100 *Oct 11, 2007Mar 6, 2012Apple Inc.Representing high-resolution media content in a lower resolution systemUS8209612Apr 19, 2010Jun 26, 2012Apple Inc.Application of speed effects to a video presentationUS8221126Jul 17, 2012Bravobrava L.L.C.System and method for performing programmatic language learning tests and evaluationsUS8228372 *Jan 5, 2007Jul 24, 2012Agile Sports Technologies, Inc.Digital video editing systemUS8272874Nov 22, 2004Sep 25, 2012Bravobrava L.L.C.System and method for assisting language learningUS8365083Jun 25, 2004Jan 29, 2013Hewlett-Packard Development Company, L.P.Customizable, categorically organized graphical user interface for utilizing online and local contentUS8391773 *Mar 5, 2013Kangaroo Media, Inc.System and methods for enhancing the experience of spectators attending a live sporting event, with content filtering functionUS8391774Jul 21, 2006Mar 5, 2013Kangaroo Media, Inc.System and methods for enhancing the experience of spectators attending a live sporting event, with automated video stream switching functionsUS8391825 *Mar 5, 2013Kangaroo Media, Inc.System and methods for enhancing the experience of spectators attending a live sporting event, with user authentication capabilityUS8392004Apr 30, 2009Mar 5, 2013Apple Inc.Automatic audio adjustmentUS8432489Jul 21, 2006Apr 30, 2013Kangaroo Media, Inc.System and methods for enhancing the experience of spectators attending a live sporting event, with bookmark setting capabilityUS8472633 *Nov 15, 2005Jun 25, 2013Microsoft CorporationDetection of device configurationUS8483397Sep 2, 2010Jul 9, 2013Hbc Solutions, Inc.Multi-channel audio displayUS8484027Jun 10, 2010Jul 9, 2013Skyreader Media Inc.Method for live remote narration of a digital bookUS8515757 *Mar 20, 2007Aug 20, 2013Nuance Communications, Inc.Indexing digitized speech with words represented in the digitized speechUS8532423 *Jan 26, 2012Sep 10, 2013Apple Inc.Representing high-resolution media content in a lower resolution systemUS8558953Sep 20, 2010Oct 15, 2013Apple Inc.Method and apparatus for synchronizing audio and video streamsUS8583165 *Jul 30, 2007Nov 12, 2013Bindu Rama RaoSystem for cartoon creation and distribution to mobile devicesUS8621355Feb 2, 2011Dec 31, 2013Apple Inc.Automatic synchronization of media clipsUS8645134 *Nov 18, 2010Feb 4, 2014Google Inc.Generation of timed text using speech-to-text technology and applications thereofUS8645336 *Mar 20, 2007Feb 4, 2014Magnaforte, LlcDigital interactive phrasing system and methodUS8701147Sep 19, 2011Apr 15, 2014Kangaroo Media Inc.Buffering content on a handheld electronic deviceUS8706490Aug 7, 2013Apr 22, 2014Nuance Communications, Inc.Indexing digitized speech with words represented in the digitized speechUS8731370 *Jan 12, 2009May 20, 2014At&T Intellectual Property I, L.P.Method and device for transmitting audio and video for playbackUS8744239Aug 6, 2010Jun 3, 2014Apple Inc.Teleprompter tool for voice-over toolUS8751022Apr 14, 2007Jun 10, 2014Apple Inc.Multi-take compositing of digital media assetsUS8775480May 19, 2011Jul 8, 2014Apple Inc.Media clip managementUS8782176Apr 12, 2012Jul 15, 2014Fusic Ltd.Synchronized video systemUS8802957 *Sep 3, 2010Aug 12, 2014Boardwalk Technology Group, LlcMobile replacement-dialogue recording systemUS8810728Oct 14, 2013Aug 19, 2014Apple Inc.Method and apparatus for synchronizing audio and video streamsUS8842842Feb 1, 2011Sep 23, 2014Apple Inc.Detection of audio channel configurationUS8849432 *May 31, 2007Sep 30, 2014Adobe Systems IncorporatedAcoustic pattern identification using spectral characteristics to synchronize audio and/or videoUS8862254Jan 13, 2011Oct 14, 2014Apple Inc.Background audio processingUS8886015May 19, 2011Nov 11, 2014Apple Inc.Efficient media importUS8949123Apr 11, 2012Feb 3, 2015Samsung Electronics Co., Ltd.Display apparatus and voice conversion method thereofUS8954175 *Aug 26, 2009Feb 10, 2015Adobe Systems IncorporatedUser-guided audio selection from complex sound mixturesUS8954477May 19, 2011Feb 10, 2015Apple Inc.Data structures for a media-editing applicationUS8965774Aug 23, 2011Feb 24, 2015Apple Inc.Automatic detection of audio compression parametersUS8988489Sep 15, 2010Mar 24, 2015Hewlett-Packard Development Company, L. P.Determining a synchronization relationshipUS9014544Dec 19, 2012Apr 21, 2015Apple Inc.User interface for retiming in a media authoring toolUS9065984Mar 7, 2013Jun 23, 2015Fanvision Entertainment LlcSystem and methods for enhancing the experience of spectators attending a live sporting eventUS9099161May 19, 2011Aug 4, 2015Apple Inc.Media-editing application with multiple resolution modesUS9111579Jan 30, 2012Aug 18, 2015Apple Inc.Media editing with multi-camera media clipsUS9123337Mar 11, 2014Sep 1, 2015Nuance Communications, Inc.Indexing digitized speech with words represented in the digitized speechUS9215539 *Nov 19, 2012Dec 15, 2015Adobe Systems IncorporatedSound data identificationUS9237176Apr 4, 2014Jan 12, 2016At&T Intellectual Property I, LpMethod and device for transmitting audio and video for playbackUS9251855May 19, 2011Feb 2, 2016Apple Inc.Efficient media processingUS9412392Jan 27, 2014Aug 9, 2016Apple Inc.Electronic devices with voice command and contextual data processing capabilitiesUS20040190874 *Mar 25, 2003Sep 30, 2004Phoury LeiMethod of generating a multimedia discUS20040255251 *Jul 19, 2004Dec 16, 2004Microsoft CorporationAssembling verbal narration for digital display imagesUS20050081155 *Oct 4, 2004Apr 14, 2005Geoffrey MartinVirtual player capable of handling dissimilar contentUS20050091597 *Oct 6, 2004Apr 28, 2005Jonathan AckleySystem and method of playback and feature control for video playersUS20050123886 *Nov 26, 2003Jun 9, 2005Xian-Sheng HuaSystems and methods for personalized karaokeUS20050149682 *Jan 5, 2005Jul 7, 2005Han-Gyoo KimVirtual multiple removable media jukeboxUS20050193017 *Feb 18, 2005Sep 1, 2005Han-Gyoo KimPortable multimedia player/recorder that accesses data contents from and writes to networked deviceUS20050193189 *Feb 17, 2005Sep 1, 2005Han-Gyoo KimDevice and method for booting an operating system for a computer from a passive directly attached network deviceUS20050226429 *Mar 31, 2004Oct 13, 2005Hollowbush Richard RMulti-channel relative amplitude and phase display with loggingUS20050228663 *Mar 31, 2004Oct 13, 2005Robert BomanMedia production system using time alignment to scriptsUS20050246655 *Apr 28, 2004Nov 3, 2005Janet SailorMoveable interface to a search engine that remains visible on the desktopUS20050265339 *May 31, 2005Dec 1, 2005Hiroki KatoServer, contents processor, contents processing system, contents processing method, program for executing contents processing and recording medium for recording the programUS20050289475 *Jun 25, 2004Dec 29, 2005Geoffrey MartinCustomizable, categorically organized graphical user interface for utilizing online and local contentUS20060026626 *Jul 30, 2004Feb 2, 2006Malamud Mark ACue-aware privacy filter for participants in persistent communicationsUS20060041632 *Aug 23, 2004Feb 23, 2006Microsoft CorporationSystem and method to associate content types in a portable communication deviceUS20060069884 *Feb 27, 2005Mar 30, 2006Han-Gyoo KimUniversal network to device bridge chip that enables network directly attached deviceUS20060072017 *Oct 6, 2004Apr 6, 2006Microsoft CorporationCreation of image based video using step-imagesUS20060110711 *Nov 22, 2004May 25, 2006Bravobrava L.L.C.System and method for performing programmatic language learning tests and evaluationsUS20060110712 *Nov 22, 2004May 25, 2006Bravobrava L.L.C.System and method for programmatically evaluating and aiding a person learning a new languageUS20060111902 *Nov 22, 2004May 25, 2006Bravobrava L.L.C.System and method for assisting language learningUS20060155805 *Apr 22, 2004Jul 13, 2006Netkingcall, Co., Ltd.Scalable server architecture based on asymmetric 3-way TCPUS20060203199 *Mar 8, 2005Sep 14, 2006Microsoft CorporationPhotostory 3 - automated motion generationUS20060204214 *Mar 14, 2005Sep 14, 2006Microsoft CorporationPicture line audio augmentationUS20060218488 *Mar 28, 2005Sep 28, 2006Microsoft CorporationPlug-in architecture for post-authoring activitiesUS20060224778 *Apr 4, 2005Oct 5, 2006Microsoft CorporationLinked wizardsUS20070008988 *Feb 23, 2006Jan 11, 2007Han-Gyoo KimEnhanced network direct attached storage controllerUS20070011279 *Mar 8, 2006Jan 11, 2007Eliram HaklaiSystem and method for providing and displaying multimedia content in correlation with music files in an audio discUS20070019068 *Jul 21, 2006Jan 25, 2007Marc ArseneauSystem and Methods for Enhancing the Experience of Spectators Attending a Live Sporting Event, with User Authentication CapabilityUS20070021056 *Jul 21, 2006Jan 25, 2007Marc ArseneauSystem and Methods for Enhancing the Experience of Spectators Attending a Live Sporting Event, with Content Filtering FunctionUS20070021057 *Jul 21, 2006Jan 25, 2007Marc ArseneauSystem and Methods for Enhancing the Experience of Spectators Attending a Live Sporting Event, with an Audio Stream Selector Using a Priority ProfileUS20070058674 *Jul 29, 2005Mar 15, 2007Zermatt Systems, Inc.Guided discovery of media contentUS20070092200 *Nov 20, 2006Apr 26, 2007Black David RMethod and apparatus for synchronizing audio and video streamsUS20070110251 *Nov 15, 2005May 17, 2007Microsoft CorporationDetection of device configurationUS20070201815 *Jan 5, 2007Aug 30, 2007Christopher GriffinDigital video editing systemUS20070239754 *Mar 20, 2007Oct 11, 2007Schnitman Myles JDigital interactive phrasing system and methodUS20080115063 *Nov 13, 2006May 15, 2008Flagpath Venture Vii, LlcMedia assemblyUS20080200191 *Jul 30, 2007Aug 21, 2008Bindu Rama RaoSystem for cartoon creation and distribution to mobile devicesUS20080235021 *Mar 20, 2007Sep 25, 2008Cross Charles WIndexing Digitized Speech With Words Represented In The Digitized SpeechUS20080255687 *Apr 14, 2007Oct 16, 2008Aaron EppolitoMulti-Take Compositing of Digital Media AssetsUS20080256136 *Mar 31, 2008Oct 16, 2008Jerremy HollandTechniques and tools for managing attributes of media contentUS20080256448 *Apr 14, 2007Oct 16, 2008Nikhil Mahesh BhattMulti-Frame Video Display Method and ApparatusUS20080263433 *Apr 14, 2008Oct 23, 2008Aaron EppolitoMultiple version merge for media productionUS20080263450 *Apr 14, 2008Oct 23, 2008James Jacob HodgesSystem and method to conform separately edited sequencesUS20090043971 *Oct 21, 2008Feb 12, 2009Ximeta Technology, Inc.Data integrity for data storage devices shared by multiple hosts via a networkUS20090097770 *Oct 11, 2007Apr 16, 2009Apple Inc.Representing high-resolution media content in a lower resolution systemUS20100178036 *Jul 15, 2010At&T Intellectual Property I, L.P.Method and Device for Transmitting Audio and Video for PlaybackUS20100216437 *Oct 26, 2007Aug 26, 2010Jean ChouraquiMethods and system for transferring multimedia content using an existing digital sound transfer protocolUS20100275121 *Apr 19, 2010Oct 28, 2010Gary JohnsonApplication of speed effects to a video presentationUS20100280638 *Apr 30, 2009Nov 4, 2010Ken MatsudaAutomatic audio adjustmentUS20110013084 *Jan 20, 2011David Robert BlackMethod and apparatus for synchronizing audio and video streamsUS20120041759 *Sep 3, 2010Feb 16, 2012Boardwalk Technology Group, LlcMobile Replacement-Dialogue Recording SystemUS20120128262 *Jan 26, 2012May 24, 2012Apple Inc.Representing high-resolution media content in a lower resolution systemUS20130121511 *Aug 26, 2009May 16, 2013Paris SmaragdisUser-Guided Audio Selection from Complex Sound MixturesUS20130121662 *May 31, 2007May 16, 2013Adobe Systems IncorporatedAcoustic Pattern Identification Using Spectral Characteristics to Synchronize Audio and/or VideoUS20130132074 *Nov 11, 2010May 23, 2013Byung Chan KimMethod and system for reproducing and distributing sound source of electronic terminalUS20140114919 *Oct 19, 2012Apr 24, 2014United Video Properties, Inc.Systems and methods for providing synchronized media contentUS20140140517 *Nov 19, 2012May 22, 2014Adobe Systems IncorporatedSound Data IdentificationUS20160042766 *Aug 6, 2014Feb 11, 2016Echostar Technologies L.L.C.Custom video contentUSRE43601Nov 4, 2011Aug 21, 2012Kangaroo Media, Inc.System and methods for enhancing the experience of spectators attending a live sporting event, with gaming capabilityWO2006057896A2 *Nov 18, 2005Jun 1, 2006Bravobrava, L.L.C.System and method for assisting language learningWO2012023951A1 *Sep 10, 2010Feb 23, 2012Boardwalk Technology Group, LlcMobile replacement-dialogue recording systemWO2014018652A2 *Jul 24, 2013Jan 30, 2014Adam PolakMedia synchronizationWO2014018652A3 *Jul 24, 2013Apr 17, 2014Adam PolakMedia synchronization* Cited by examinerClassifications U.S. Classification434/307.00A, 434/308, 434/307.00R, G9B/27.012, G9B/27.017International ClassificationG11B27/34, G10L21/06, G11B27/036, G11B27/10, G09B5/00, G11B27/034Cooperative ClassificationG11B27/036, G11B27/034, G10L2021/105, G11B27/34, G11B27/10European ClassificationG11B27/34, G11B27/10, G11B27/034Legal EventsDateCodeEventDescriptionFeb 14, 2009ASAssignmentOwner name: SYNCHRO ARTS LIMITED, UNITED KINGDOMFree format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:BLOOM, PHILLIP JEFFREY;ELLWOOD, WILLIAM JOHN;REEL/FRAME:022258/0372;SIGNING DATES FROM 20090120 TO 20090127Owner name: SYNCHRO ARTS LIMITED, UNITED KINGDOMFree format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:BLOOM, PHILLIP JEFFREY;ELLWOOD, WILLIAM JOHN;SIGNING DATES FROM 20090120 TO 20090127;REEL/FRAME:022258/0372Feb 3, 2015FPAYFee paymentYear of fee payment: 4RotateOriginal ImageGoogle Home - Sitemap - USPTO Bulk Downloads - Privacy Policy - Terms of Service - About Google Patents - Send FeedbackData provided by IFI CLAIMS Patent Services