Abstract:
A system for the production of video signals includes a playback device for playing back prerecorded video and audio signals from a prerecorded storage medium, a source of user supplied video and audio signals, a video and audio mixer for combining the prerecorded and user supplied signals to provide combined video and audio outputs, a production monitor connected to the mixer to display to the user the mixed signals, and a storage or reproduction device receiving a mixed video signal output from the mixer. The prerecorded storage medium, wherein the prerecorded storage medium stores, as well as a video channel and at least one audio channel, at least one prompting channel, the video signals stored on the prerecorded medium being prekeyed to indicate areas to be overlaid in the mixer by the user supplied video signals, and the mixer being operative to convert signals from the prompting channel into production control signals.

Description:
RELATED APPLICATION 
     This application is a continuation-in-part of my application Ser. No. 08/399,013, filed Mar. 6, 1995, now abandoned. 
    
    
     FIELD OF THE INVENTION 
     This invention relates to the production of video sequences, in which users own video signal can be combined with a prerecorded video signal to provide a composite sequence, in which prerecorded images are integrated with the user&#39;s own material. 
     REVIEW OF THE ART 
     So called video karaoke systems are well known in which a video recording is provided, with the video display overlaid by text indicating the words of a song, usually with some form of marker to indicate the synchronization of the words of the song with a musical accompaniment recorded on the sound channel or channels of the video recording. This enables persons viewing the recording to “sing along” with the musical accompaniment. 
     It is also well known to produce composite video signals by overlaying one signal on another, utilizing one of several keying techniques of which those known as chroma-keying and luminance keying are the most common. In chroma-keying, essential elements of a foreground scene, typically a person or persons, are imaged against a background having a higher level of saturation of a particular colour than is likely to occur in the foreground objects. Typically an intense blue background is utilized, but other colours may be used provided that, in the particular application, they enable the foreground and background to be reliably differentiated by signal processing circuitry. An alternative approach is known as luminance keying, in which it is arranged that the luminance level of the background against which the foreground objects are imaged is consistently and detectably lower than that of the wanted foreground objects. During the combination process, boundaries between the foreground objects and the background are detected on the basis of the above-mentioned difference in colour content or luminance level, so as to produce a switching signal which switches a second video signal, synchronized with the first, into the background areas. These techniques are well known and understood in the art, but require good quality and hence expensive equipment to operate reliably. 
     In U.S. Pat. No. 5,099,337 (Cury) it is proposed to provide a selection of background audio and video recordings which can be selected from separate libraries and combined with foreground audio and video signals provided by a user, so as to enable the user to provide customized video recordings in which the users own foreground images and audio signals are superimposed upon selected background audio and video signals. This in effect provides system in which the user, as well as providing a foreground audio signal, also provides a foreground video signal, and is provided with means for recording the result. A performer performs in front of a blue screen, so that the performer&#39;s image may be chromakeyed into the background video signal, thus providing the illusion that the performer is performing in the selected background. During performance, prompt information is provided to a performer through a prompt monitor from a prerecorded prompt library. 
     A limitation of such systems is that, by their nature, they can only provide background for a user&#39;s performance, and may require the user,s signal to be keyed which, as mentioned above, may be difficult to achieve reliably with consumer quality equipment. 
     SUMMARY OF THE INVENTION 
     In its broadest aspect, the present invention relates to a system in which a prerecorded video signal is prekeyed to define background areas which, on playback by a user of a recording medium carrying the keyed signal on apparatus configured to recognize the prekeyed background areas, will generate a signal into which may be inserted, in those background areas, a local signal provided by the user, which need not itself by keyed. 
     The present invention further seeks to provide a system in which a keyed video prerecording is used to provide a prerecorded signal which is combined with a user provided background signal to provide a final combined signal, the prerecording including prompt channel, which can be suppressed in the final combined signal, to assist a user and/or a user&#39;s equipment to provide a background signal compatible with the prerecorded signal. 
     According to the invention, there is provided a system for the production of video signals, comprising a playback device for playing back prerecorded video and audio signals from a prerecorded storage medium, a source of user supplied video and audio signals, a video and audio mixer for combining the prerecorded and user supplied signal to provide combined video and audio outputs, a production monitor connected to the mixer to display to the user the mixed signals, and a storage or reproduction device receiving a mixed video signal output from the mixer, wherein the prerecorded storage medium stores, as well as a video channel and at least one audio channel, at least one prompting channel, the video signals stored on the prerecorded medium being prekeyed to indicate areas to be overlaid in the mixer by the user supplied video signals, and the mixer being operative to convert signals from the prompting channel into production control signals. Typically the production control signals include prompts displayed on the production monitor but absent from the combined video output. 
     The invention extends to a recording medium providing multiple channels of information, including a video channel, a least one audio channel, and at least one prompting channel, the video channel being recorded with a video signal prekeyed to indicate picture areas available for overlay by a user provided video signal, and the prompting channel including data translatable into instructions for control of the user provided video signal. 
     In order to permit production of prerecorded tapes (or discs or other media) which will perform reliably with low cost user equipment, I employ by preference a modified luminance keying system in producing the prerecorded tapes. In simple terms, the brightness level of at lease the lowlights of portions of images which are not to be overlaid by the user signal is artificially enhanced so that the “black level” of retained image portions is well above the normal black level of a recorded signal, thus enabling the keyed portion which is at or below the normal black level, to be readily distinguished. The user supplied signals are also brightness enhanced in a similar manner prior to mixing with the signal from the prerecorded tape, and the brightness level of the mixed signal is then returned to normal so as to restore the original black levels. 
     The word “video signal” as used in this specification, means a television type video signal consisting of a sequence of frames which when reproduced in radial succession are capable of providing a moving picture. It does not include bit-mapped or vector digital representations of a single static image. 
    
    
     
       SHORT DESCRIPTION OF THE DRAWINGS 
       In the drawings: 
         FIG. 1  is a simplified block diagram of the system; 
         FIG. 2  is a simplified block diagram of a mixer unit used in the system; 
         FIG. 3  is a more detailed block diagram of an exemplary mixer unit; and 
     
    
    
     DESCRIPTION OF THE PREFERRED EMBODIMENT 
     Referring to  FIG. 1 , which has been simplified by omitting any consideration of sound as opposed to video channels, a prerecorded tape or video disk, generically referred to as a prerecording  2 , is replayed to provide a foreground video signal upon which a user signal may be overlaid using chroma or luminance keying or any other system enabling effective identification of areas of the recorded image available for overlay by a user signal (henceforward generically referred to as “keying”) It should be understood that the keying signal, whatever form it takes, for example a high saturation of blue, or a very low luminance level, is already built into the prerecording  2  so as to predefine those areas of the prerecorded signal  4  available for overlay by user provided video signals: the user provided signals will not usually themselves be keyed, and if they are keyed, it will be for purposes extraneous to the present invention. It should also be understood that overlay effects depend on successive fields of the prerecorded video being keyed in certain areas, fields may also be either wholly keyed or not keyed at all, thus permitting scenes provided wholly either by the user or wholly by the prerecorded tape. The prerecorded tape also preferably carries, as well the video signal channel, at least one audio channel for audio signals, and at least one prompting channel  6  for prompting signals prompting of instructing the user and/or the users equipment so as to assist in rendering the content of the user provided signal compatible with the foreground content of the prerecording. Prompts to the user may include text prompts, e.g. the words of a song and/or stage directions, or position of dimension indicators to assist the user in controlling direction or zooming of a video camera providing the user signal. It should further be understood that the playback of the prerecording may occur at a remote point, with the signal being distributed to users over a distribution network such as a CATV network or the internet. 
     A controllable user video source, usually a camera  8  (or one of multiple selectable cameras) has a signal output to a mixer  12  which combines the video signals by inserting the user signal output  10  in those areas of the prerecorded signal  4  which are identified by the keying, or by mixing the user signal with the prerecorded signal, depending upon the effect desired. The superimposition provided by mixing may be useful for some applications for example training videos. The prompting signals on channel  6  are translated by the mixer  12  into signals displayed on a user monitor  14  so that a user may control the camera  8 , or sing (or otherwise perform) along with a prerecorded artist; or the user monitor may implement a camera control function (for example a power zoom control) which is applied directly to the camera to control its input to the mixer  12 , which replaces the keyed portions of the signal from the prerecording  2  with the signal  10  from the camera  8 . It is however preferred that zooming be performed electronically within the mixer, because of the lack of standardization of camera controls. The signals  4  and  10  must of course be synchronized to a common set of scanning signals before combination. This is preferably achieved as discussed below with reference to  FIG. 3 , but in an alternative arrangement the camera  8  receives a signal via control  14  which synchronizes it to the signal  6  from the prerecorded source. The output signal from the mixer  12  is passed to a suitable recorder and/or display  16 . 
     Referring now to  FIG. 2 , this shows schematically one embodiment of the mixer  12  in somewhat more detail. Video inputs  4  (including the signal  6 ) and  10  are applied to a timebase control unit  20  which ensures synchronization between the video signals, preferably by applying them to synchronized frame memories as described below, since this requires no feedback control of the timing of either input signal. A brightness enhancement circuit  21  may be provided in one advantageous embodiment of the invention to enhance the brightness of the lowlights of the user supplied video signal so that its black levels are well above the pedestal level of the video signal by an amount equal to a similar brightness boost already present in unkeyed image areas of the prerecorded video signal in unkeyed areas, keyed areas being maintained at of slightly below normal black level. This distortion of the prerecorded video signal enables the keying switcher to detect keyed areas very simply and reliably since there will be a substantial minimum difference of the level of the video signal in keyed and unkeyed areas which can readily be detected even in the presence of some distortion of the video signal. Typically, this enhancement may be about 20% or more the normal difference between black and white levels of the signal. The brightness enhancement applied to the user signal by the circuit  21  should of course be substantially the same as that applied during manufacture of the tape providing the signal  4 . Downstream of the switcher  22 , a further brightness control circuit  23  applies an equal and opposite correction to restore proper brightness levels in the signal. A level switching circuit  22  responds to the keying signal extracted from the signal  4  to switch between levels of the signals  4  and  10  and thus provide a combined video output signal  24  to the recorder and/or display  16  (FIG.  1 ). For full keying the different levels of the two signals are full/zero and zero/full respectively, but other level combinations can be utilized with suitable digital mixing technology, in a manner known per se. The video output signal is also applied, in this embodiment, to a closed caption decoder  28  which extracts control signals and which are encoded into an available channel in the prerecording  2 , in this case that portion of the video interval between frames normally reserved for closed captioning. The control signals are overlaid on a video signal  26  sent to the monitor  14  (FIG.  1 ), so as to provide directions (either text prompts or instructions or positioning marks) to the user of the system so that the user may “sing along” or otherwise perform in synchronization with the prerecorded signal, or exercise suitable control over the camera  8 . The signal  26  may also be decoded to provide a camera control output, for example a zoom control signal. The mixer  12  typically also incorporates an audio effects and mixer unit  30  which receives and mixes inputs from an audio channel or channels  32  from the prerecording  2 , inputs from an audio channel or channels  34  from a user microphone or microphones, and provides an output audio channel or channels at  36  to the monitor  14  and at  38  to the recorder  16 . 
     Referring now to  FIG. 3 , one version of the mixer  12  is shown in more detail. It should be understood that, in the main, the mixer  12  shown in  FIG. 3  is similar to known digital video mixers incorporating chroma or luminance keying facilities except that it responds additionally to an additional prompt channel included in the prerecorded video input  4 . Typically the closed caption channel, the data for which is encoded into certain lines of the video signal in the vertical interval outside of the normally displayed area, is used to convey prompt messages and control data to the system, and thus a closed caption decoder  100  is incorporated into the mixer  12  to recover this data. Such decoders are well understood in the art and need not be described further. 
     The mixer operates under control of a microcomputer  102  including appropriate working memory. An example of an suitable device is the 87C752 from Intel Corporation, and in general it controls the mixer in a manner similar to that of known video mixers. Accordingly, the mixer will be described primarily with a view to explaining how it differs from conventional digital video mixers, such as the MX-1 digital video mixer from Videonics (although it should be understood that many of the functions provided by such mixers are not essential to the present application and may be omitted to reduce costs), and so as to explain its relationship to the essential functions of the invention. 
     The video input  4  from the prerecorded source  2  may be either composite or S-video. In the latter case the video chroma and luminance signals are applied to separate analog to digital converters  104 ,  106  under control of a clock generator  108  which also controls a decoder  110  passing digital Y (luminance) and U and V (chrominance) signals to a frame memory  112  under control of a video memory write controller  114 . If a composite video signal is provided, it is applied to the converter  106  and converter  104  is not used. 
     Similarly, the input  10  from the camera  8  is applied to A/D converters  124  and  126 , or converter  124  only if a composite signal, under control of a clock generator  128  also controlling a decoder  130  passing signal to a memory  132  under control of controller  134 . 
     A synchronized memory read controller  140  reads the contents of the memories  112  and  132  in synchronism under control of the microcomputer  102 , the Y, U and V signals read from the memories being selected by a multiplexer  142 , under control of a luminance or color keyer  144  itself controlled by signals read from the memory  112 . When the selected keying signal, be it luminance or chroma, is present at a level denoting background, then the keyer causes the multiplexer  142  to pass at least part of the signals derived from input  10  and block at least part of the signals from input  4 ; otherwise it passes at least part of the signals derived from input  4  and blocks at least part of the signals derived from input  10 . By this means signals from the camera input  10  are inserted into the keyed portions of the video input  4 . 
     The Y (luminance) signal derived from the video input  4  is also passed to the closed caption decoder  100 , which decodes the closed caption data in conventional manner to recover data contained therein and pass it to microcomputer  102  which interprets the data and either forwards it to a conventional overlay generator  146  and/or generates signals applied on a line  148  to control zooming (for example) of camera  8 , and/or to the scalar  162  or interpolator  160  discussed below, if provided. 
     The Y, U and V signals from multiplexer  142  are passed to output encoders  150  and  152  under control of an output clock and synchronization separator  158 , providing the video outputs  24  and  26  via amplifiers and filters  154  and  156 , the V signal to encoder  152  being overlaid by text or video regenerated by the overlay generator  146  so that control instructions are passed to the user monitor  14 . 
     The microcomputer  102  may control additional optional processing circuits between the multiplexer  142  and the encoders  150  and  152 , these being of types known in digital video mixers; in this case there is shown a zoom interpolator  160 , a scaler  162  controlled by a clock generator  164 , and a lower bit switch  166 . The zoom interpolator and scaler provide an electronic zoom effect which is preferred to optical zoom controlled by the line  48 . The electronic zoom interpolator and scaler will act on the combined signal, and not just the local camera input as would an optical zoom. The lower bit switch  166  can be activated in known manner to provide a posterization effect. The interpolater  160  and scaler  162  may also be configured to be controlled manually by the user, since they act conjointly on the prerecorded and user provided signals. 
     Audio inputs  32  from local microphones are processed by potentiometer  168 , preamplifiers  170 , mixer  172 , and a master potentiometer  174  before being applied to an A/D converter  176  while audio inputs associated with the prerecorded video signal are applied to A/D converters  278 , the outputs from the A/D converters  176  and  178  being combined and optionally processed by a digital signal processor  180  in known manner under control of microcomputer  102 . The processed digital audio signals are then passed through digital to analog converters  182  and preamplifiers  184  to the outputs  36  and  38 . 
     The programming of microcomputer  102  has not been described, since except for any processing of the signals from the closed caption decoder  100 , it is similar to that for known digital mixers. Processing by microcomputer  102  of closed caption data merely consist of intercepting data encoded on the closed option line  20  which provides control signals for passage by the microcomputer to the output  148  or the processing circuits  160 ,  162  and  166 . 
     In use, the mixer  12  will operate much like a conventional digital video mixer, except that the keying function controlled by the keying signal in the input  4  is a default function, and control signals or messages in the closed caption field of the input  4  are decoded and output either as video overlays on the monitor  14  or as camera or mixer control signals. 
     It will therefore be appreciated that a users local generated video (and audio) signals may be combined with the video input from a prerecorded tape or disc to provide video and audio outputs in which user contributed images and sounds are combined with those on the prerecorded tape to provide a composite output in which prerecorded images are inserted into images provided by the user so that for example a famous singing star or animated cartoon character may appear to be performing together with a user in the user&#39;s own home or the user&#39;s choice of surroundings. The control or prompting channel, for example close captions appearing on the monitor  14 , may provide on screen prompts to a user, which are invisible on the output passed to a recorder  16 , either in the form of words, or indicators indicating how the user should place a locally generated image on the screen. This channel may also carry data which can be converted by the microcomputer  102  into data output on the line  148  in the form of camera control signals, for example to control of a zoom function of the camera  8 . Unlike prior systems, the user does not need to perform against a blue screen or other means to generate keying of the user signal, since the keying is prerecorded into the prerecorded foreground signal. Such prerecorded, prekeyed signals have numerous potential applications of which those discussed above are merely exemplary.