Patent Publication Number: US-RE45406-E

Title: Program encoding and counterfeit tracking system and method

Description:
CROSS-REFERENCE TO RELATED APPLICATION 
     The present application is a reissue of U.S. Pat. No. 7,818,257, granted from U.S. patent application Ser. No. 10/893,508, filed Jul. 16, 2004. 
     This invention relates to the encoding of audio-visual programs such as motion pictures and the tracking of counterfeit program copies, particularly counterfeit copies made from releases of a motion picture in video form. 
     “Motion picture”, as that term is used herein, includes any type of program material using moving images as a medium of expression. It can include episodes of broadcast television programs, corporate events, as well as “movies”, etc. 
     When a new movie is released, it usually is released in both film and video versions. The first copies, both film and video copies, usually are “screeners” sent to studio executives, advertising agencies, Motion Picture Academy Members and others. Later, film copies are sent to theaters and video copies are sent to airlines and hotels, then to pay-per-view distributors, and, finally, to free television broadcasters. 
     The production and sale of counterfeit copies of motion pictures is a serious problem of long standing. Counterfeit copies of new motion pictures sometimes are sold to the public even before the motion picture has been released by the motion picture studio. These counterfeit copies are hard to trace, and it is a difficult job to identify and bring the counterfeiters to justice. 
     A particularly crude but effective type of counterfeiting of film version of movies is the use of a video camera to copy a motion picture from a movie screen. The copy made by this technique then is converted to video tape or DVD records, and the records are sold to the public and otherwise distributed, such as by way of the internet. 
     Video program counterfeiting usually is done by obtaining a legitimate copy of the program and copying it with the use of video capture cards or use of means known for the purpose. 
     Attempts have been made in the past to stem the tide of such counterfeits. An effective system and method for tracing the sources of counterfeits of motion pictures is described in U.S. patent application Ser. No. 10/657,287 filed on Sep. 9, 2003, which is assigned to Deluxe Laboratories, Inc., the assignee of this patent application. The disclosure of that patent application is hereby incorporated herein by reference. 
     The method described in that patent application uses coded arrays of artifacts to uniquely identify each copy distributed. Detection of the coded identification numbers from the counterfeit copies aids in tracking the source of the counterfeits. As successful as that method is, it is desired to improve on it, especially when detecting the sources of counterfeits of video distribution copies of a new motion picture. 
     In certain prior proposals for video copy protection, noise has been embedded in the video signals in particular patterns to use in identifying the source. This has certain disadvantages, such as by requiring a substantial amount of extra hardware for use in recording the protected video copies. 
     Other so-called “electronic water-marking” schemes have been proposed and are not believed to have been successfully employed in solving the problems intended to be solved by the present invention. 
     In accordance with the present invention, the images appearing in specific pre-selected places in the video program are slightly altered so as to be essentially unnoticeable to the viewer. The presence or absence of an altered picture at each of a number of pre-selected locations forms a code which represents a unique number applied to that copy of the motion picture. The unique number then is recorded and stored, together with an identification of the person or entity who receives the copy when it is distributed, so as to enable law enforcement personnel to review suspected counterfeit copies of the motion picture to read the code number and identify the recipient of the copy and thereby track down the source of the counterfeit copies. 
     Preferably, the alteration of the images consists of slightly shifting an image such as by changing the aspect ratio of the image. It is preferred to do this by enlarging an image slightly so that one or more edges of the image is moved relative to the same edge in the video master. Sometimes an image boundary overlaps a boundary of the frame and becomes invisible. 
     This alteration procedure helps enforcement personnel to compare the frame of a suspected counterfeit copy with an unaltered frame from an unaltered master copy of the motion picture to determine which of the images has been altered. When the pattern of alterations is detected, this will indicate the number of the copy which has been counterfeited and will lead to the possible source of the counterfeit. 
     In a preferred embodiment, the alteration of images is applied to each frame of a complete scene. This makes the alteration very difficult to detect without access to the unaltered master video. 
     It also is preferable that each of the pre-determined locations for alterations consisted of a plurality of successive or separated scenes in order to ensure detectability of the alterations by enforcement personnel. 
     In accordance with another aspect of the invention, the code recorded by image alteration is combined with the separate code provided by the invention of the above-identified co-pending U.S. patent application so that enforcement personnel have two different codes to look for in identifying the source of a counterfeit and confirming the number of the copy which has been counterfeited. 
     In making video copies containing the codes described above, a particularly advantageous method is used. 
     First, a video copy with images altered at pre-determined locations is prepared. Then, the master video which has not been altered is played back in one playback device, such as a video server, while the altered version is played back on another playback device such as a second video server. The output of both servers is delivered to a video router which controls a large number of different video recorders to simultaneously prepare copies in relatively large quantities. 
     A random number generator is employed to generate an identification number for each copy, and the number so generated is used to control one output channel of the router so as to alternatingly deliver the master video signal and the altered video signal and thereby record the altered images in a particular pattern which gives that copy a unique identification number. 
     The code made of artifacts is added by modifying the output signals of the second video server, in copies using both forms of coding. 
     The foregoing and other objects and advantages of the invention will be apparent from or explained in the following description and drawings. 
    
    
     
       IN THE DRAWINGS 
         FIG. 1  is a schematic view of several unaltered frames of video programs; 
         FIG. 2  is a schematic view of the same frames as those shown in  FIG. 1 , after alteration; 
         FIG. 3  is a schematic view of a system used to perform the encoding and prepare uniquely encoded video distribution copies of a motion picture or other program; and 
         FIG. 4  is a schematic view of another encoding method used in tandem with the method illustrated in  FIGS. 1 and 2 . 
     
    
    
     GENERAL DESCRIPTION 
       FIG. 1  shows a series  10  of three successive frames  12 ,  14  and  16  from a single scene of a video program such as a motion picture. 
     Each frame contains an image of a square or block  26 . It is moving from left to right and thus is shown progressively closer to the right edge of the frame as one moves from frame  12  to frame  16 . 
       FIG. 2  shows the same three frames of the video program shown in  FIG. 1  after alteration, in accordance with the present invention. The alteration, in this example, is an enlargement of the image by a relatively small amount, e.g., from less than 4% to 10%. The enlargement of the object  26  to form the image  20  shown in the drawings is greater than that so as to more clearly illustrate the principle of the process. 
     In each of the altered frames  20 ,  22  and  24  the right edge  20  of the enlarged object  28  is closer to the right-hand border of the frame than the unaltered object  26  in the corresponding frame of  FIG. 1 . In fact, in the right-most frame  24 , the right edge  30  extends beyond the right edge of the frame and thus is not visible. 
     In accordance with a highly advantageous feature of the preferred embodiment of the invention, the same alteration of the images is applied to all frames of each scene in which alteration is applied. 
     The fact that a frame of the video program has been altered can be determined by viewing the altered frame in comparison with the same frame in the unaltered master video copy, such as by juxtaposing the two frames near one another, in the manner illustrated by  FIGS. 1 and 2 . However, it is believed to be very difficult for a counterfeiter to detect the alterations without having the unaltered master to compare the copy with. 
     This superior result is due, in part, to the fact that the same alteration is applied to every frame within the scene in which it is applied. Were this not done, a counterfeiter might be able to detect the alteration as a sudden change within a scene and take steps to correct his counterfeit copy to eliminate it. Instead, the sudden change from one scene to the next masks the sudden change caused by the alteration. 
     Since it is relatively easy to keep the unaltered master out of the hands of counterfeiters, the encoding of the invention is very difficult for counterfeiters to detect and defeat. 
     Coding Scheme 
     A variety of coding schemes can be used to encode a unique identification in each video distribution copy of a program. 
     In the preferred coding scheme, 64 different locations in the program are pre-selected. The locations can be selected at random. The time code of each location is stored. Thus, during review of a suspected counterfeit, the master can be run to each code location and still-framed when the code location is reached. When the corresponding portion of the copy is found and still-framed, frames in the master can be compared with corresponding frames of the copy. 
     A random number generator is used to generate a unique combination of signals forming a digital one or zero at each of the 64 locations. If the images are unaltered at a location, that is taken to be a digital zero. If the images are altered, that is taken to be a digital one. 
     The number of genuine copies made of many programs, such as movies, usually is relatively low, e.g., in the hundreds or low thousands. A 64-unit code is not necessary in order to uniquely identify each such copy. However, by providing so many digits, distinct advantages are obtained. 
     First, there is no need to access the code at its start because virtually any sequence of 10 to 15 digits can be compared with the codes stored for the copies until a unique match is found. A match can be found quickly and easily using conventional computer software. 
     Secondly, a longer code sequence is harder to completely delete when making an illegal copy. Thus, it is more likely that enough code will survive the counterfeiter&#39;s effort to destroy it. 
     The alterations preferably are repeated in each of a plurality of scenes at each of the 64 locations in order to avoid the loss of code due to cuts or deletions made in the copy by the counterfeiter, either deliberately or accidentally. Also, it facilitates detection of alterations by enforcement personnel because the alterations may be more visible in some scenes than others. 
     In addition, if a scene is repeated at a location very near its first appearance, both of the appearances will be altered and comparison of the two will yield nothing for the counterfeiter. 
     Encoding and Duplication System 
       FIG. 3  is a schematic diagram of an encoding and video program duplication system  32 . 
     The system includes a first video server  34 , a second video server  36 , a video router  40 , a computer  42  with an input keyboard and screen  60 , with a storage unit  44 , and a plurality of video recorders  50 ,  52 ,  54 ,  56 , etc., for making distribution video copies. 
     Although it does not take part in the production of copies, also shown in  FIG. 3  are an image processor  48  which is used to alter the images output from the server  36  in accordance with instructions received from the computer  42 . 
     The unaltered master video is stored in the first or “master” server  34 , and the altered video is stored in the second or “modification” server  36 . The altered video preferably has altered scenes at each of the  64  locations selected randomly by the computer  42 . 
     Signals from the unaltered master video are sent from the master server  34  to the router  40  while altered video signals are sent from the modification server  36  to the router  40  in synchronism with the master video signals. 
     The video router  40 , as it is well known, has the ability to switch rapidly back and forth between two input video signals to produce on each of a large number of output terminals  58  a unique video signal which is made up of signals from each source alternating with signals from the other source. 
     The computer  42  contains a random number generator which is used to generate a unique 64-digit random number which is assigned to and stored for each of the output terminals of the router. 
     Each of the recorders  50 ,  52 , etc., has a unique bar code identifying it. Each blank DVD or tape on which the program is recorded also has a bar-code identifying it. These bar codes are read by the use of bar code readers  62 ,  64 ,  66  and  68 , and associated with one another and stored in memory (e.g., a disk file)  44 . 
     The random number code also is stored and associated with the bar codes for the recorder and the record. Similarly, the identification for the recipient of each copy is stored and associated with the copy identification. 
     The time codes for the 64 locations are stored once for each program and made available to enforcement personnel in order to check a suspected counterfeit. 
     It should be understood that the alterations to the images can take forms other than enlargements. For example, the images can be enlarged in one dimension only instead of in both height and width, images can be slightly reduced in size in both or only one dimension, etc. 
     Second Coding Method 
     Although the foregoing method can be used alone, it is preferred to use a second coding method in addition. 
       FIG. 4  shows a segment  70  of a motion picture video master in which one frame bears an image, represented by the line  78 , and a coded pattern  80  represents an alphanumeric character. One full frame  72  and parts of two adjacent frames  74  and  76  are shown in  FIG. 4 . 
     Preferably, the pattern is made of a plurality of very small dots  82 ,  84 ,  86  and  88  in a 3×3 dot matrix, although patterns and code elements other than those described can be used instead, as it will be explained in detail below. 
     Although any desired information can be encoded, in this invention, it is preferred that the pattern represent one of the digits of an identification number which is assigned to each video copy. Preferably, this is a 5 digit number starting with 00001 and increasing by one for each additional copy made. Thus, if 128 recorders are connected to the router  40  during a production run, the numbers used will be 00001 to 00128. In the second production run, the numbers assigned will be 00129 to 00257, etc. 
     For example, the dot pattern shown may represent the first digit “0” of the number “01736” which has been given to the copy. Similarly, a different dot pattern is used to represent the “1”, the “7”, and each other digit in the number. 
     During a record production run, the modification server  36  begins by generating the pattern for the number “0” in the first “placeholder” location. This pattern will appear for two film frames, immediately following an edit. 
     Locating a pattern immediately following an edit makes the dots harder to see. The patterns can be generated at any edit point throughout the program with the exception of edits within modified scenes; that is, within scenes in which the images have been altered as described above. 
     There are ten specific locations for each of the numbers 0-9 at the first placeholder location, ten more specific locations at the second placeholder location for the second number, ten more at the third placeholder location, and ten more at each of the fourth and fifth place holder locations. 
     Each of the specific locations is selected by the computer, and its time code is stored. Thus, if a pattern is found at a particular time code position in a suspected counterfeit program, it will not be necessary to be able to read the pattern; it will be known what the number is simply by its location. Of course, its place in the identification number (that is, whether it is the first, second, third, fourth or fifth digit) also is known, from the stored time codes. 
     In the production of a batch of copies (e.g., 128 copies), the modification server  36  first generates the pattern for the number “0” in the first placeholder position. All recorders that have been assigned the number “0” in the first placeholder position will now be switched by the router to receive an input feed from the modification server  36 . 
     Next, the modification server  36  will generate the number “1” for the first placeholder. Once again, any recorders requiring the number “1” in the first placeholder will now be switched to the output of the server  36 . 
     In this fashion, the system steps through all of the numbers in all of the placeholders; 0 to 9 in placeholder number 1; 0 to 9 in placeholder number 2, etc., until the patterns forming all the numbers to represent a different five-digit number for each copy have been recorded. 
     The numbers generated for each placeholder will not exceed what is required before moving on to the next placeholder. In the example “01736”, there is no need to generate anything higher than a “1” in the second placeholder. 
     This process is repeated anywhere from 5 to 10 times throughout the duration of the program, depending on the run time of the program. Thus, the coded copy number is repeated 5 to 10 times during the program. This protects against destruction of the code patterns which frequently occurs due to data compression or deliberate destruction by the counterfeiter. 
     Preferably, the artifacts forming each code are formed by altering specific pixels of the video picture signals being transmitted from the second server  36  ( FIG. 3 ). This can be done by simply reducing the luminance level to zero in a few pixels to form small dark dots, by controlling the video color level of the pixels on the video card&#39;s output board in the server  36 . Preferably, this is done by programming the server  36 . 
     Pixel luminance and chroma values may be brought to zero or any other level greater than zero. For example, it can be desirable to subtract a predetermined luminance value from the existing luminance value of the pixel. The amount to be subtracted is the minimum amount required to enable enforcement personnel to see the dots in a pirated copy. In this method, no more pixel level change is used than is necessary, and this makes the pixels less obvious. Similarly, colors can be used to create the necessary marks. For example, a combination of luminance and color can be used to create the color brown, which is less obvious than black. 
     In other words, the contrast between the artifact and the program material is made just high enough to make the artifacts visible to enforcement personnel who know where to look. 
     The dots  82 ,  84 ,  86 ,  88  are visible to the naked eye. Therefore, the print copy number can be read without any special equipment. However, reading is greatly aided by use of a stored record of where the codes are recorded in the film. With the aid of a time code reader and a video display unit, enforcement personnel can fast-forward to the locations in the program wherein the coded frames are located, and still-frame the codes frames for reading. 
     In this way, the other digits of the program code number can be read, and the records kept showing the entity or person to which each copy was sent will then help to quickly narrow the search for the counterfeiter. 
     The dots need not be circular. However, it is preferred that they look like specks of dirt. This prevents them from being too evident to ordinary movie patrons, and makes them harder to find by people who do not know where to look. 
     As it is explained in greater detail in the above-identified co-pending patent application, small marks other than dots can be used as code elements. Small marks which look like small scratches, elongated spots or color artifacts can be used 
     When both encoding methods are used, enforcement personnel can use one method to check the accuracy of the other, or can select which one is easiest to read in a given counterfeit copy. 
     The above description of the invention is intended to be illustrative and not limiting. Various changes or modifications in the embodiments described may occur to those skilled in the art. These can be made without departing from the spirit or scope of the invention.