Abstract:
Digital watermarking provides a chain of custody for video in movie studios. Several novel improvements in workflow processes for digital watermarking provide chains of custody. In a first implementation, a digital watermark payload includes an embedder ID and time stamp. The embedder ID is associated with a controlling PC user log, including, e.g., the user log being linked to a biometric of the user. In a second implementation, a video player is forced to run a video digital watermark embedder based upon a rights command, where an ID embedded in a watermark payload is determined solely by the embedder. A third implementation encrypts a conglomerate of watermarked segments of video so that a rendering device can decrypt the segments in a manner to produce different results in each usage dependent upon the key. A fourth implementation embeds digital watermarks in DVD disk images during delivery to a duplication house.

Description:
RELATED APPLICATION DATA  
       [0001]     This application is a continuation-in-part of U.S. patent application Ser. No. 10/278,742 (published as U.S. 2003-0138127 A1), filed Oct. 21, 2002. This patent application also claims the benefit of U.S. Provisional Patent Application No. 60/554,748, filed Mar. 19, 2004. This application is also related to U.S. patent application Ser. No. 09/858,336 (published as U.S. 2002-0124024 A1), filed May 15, 2001. Each of these patent documents is herein incorporated by reference. 
     
    
     TECHNICAL FIELD  
       [0002]     The present invention relates generally to digital watermarking. In a specific example, the present invention embeds digital watermarks in media content to provide a chain of custody.  
       BACKGROUND AND SUMMARY  
       [0003]     Movie studios have a problem in that they have not established a successful mechanism to provide a chain of custody for their video or other media. In other words, their video is not linked to a current person, usually an employee, whom is in possession of the video. This problem is described even more fully in a paper by Simon Byers, et al. titled “Analysis of Security Vulnerabilities in the Movie Production and Distribution Process,” Sep. 13, 2003, (http://lorrie.cranor.org/pubs/drm03-tr.pdf), incorporated herein by reference.  
         [0004]     Digital watermarks can be used to link video to a specific person. I disclose herein several processes to make this linking efficient in terms of a conventional video workflow.  
         [0005]     Digital watermarking is a process for modifying physical or electronic media to embed a hidden machine-readable code into the media. The media may be modified such that the embedded code is imperceptible or nearly imperceptible to the user, yet may be detected through an automated detection process. Most commonly, digital watermarking is applied to media signals such as images, audio signals, and video signals. However, it may also be applied to other types of media objects, including documents (e.g., through line, word or character shifting), software, multi-dimensional graphics models, and surface textures of objects.  
         [0006]     Digital watermarking systems typically have two primary components: an encoder that embeds the watermark in a host media signal, and a decoder that detects and reads the embedded watermark from a signal suspected of containing a watermark (a suspect signal). The encoder embeds a watermark by subtly altering the host media signal. The reading component analyzes a suspect signal to detect whether a watermark is present. In applications where the watermark encodes information, the reader extracts this information from the detected watermark.  
         [0007]     Several particular watermarking techniques have been developed. The reader is presumed to be familiar with the literature in this field. Particular techniques for embedding and detecting imperceptible watermarks in media signals are detailed in the assignee&#39;s U.S. Pat. Nos. 6,122,403 and 6,614,914, which are hereby incorporated by reference.  
         [0008]     One aspect of the invention provides workflow efficiency by matching a time based log of personal computer (PC) users with a watermark embedder ID (e.g., a number or plural-bit identifier) and date/time stamp. The embedder ID is uniquely associated with the PC, so that a user can be matched to a computer via the embedder ID and a log of user activity on the PC. This system works automatically, without an operator needing to keep track of IDs and video that is processed on the system.  
         [0009]     Another aspect of the invention is a system responsive to a rights command. The rights command requires a video player to embed a forensic ID in content, where a watermark embedder—and not the rights command—controls which data is embedded. More specifically, a rights language, such as MPEG-21 REL (ISO/EEC 21000-5 draft, incorporated herein by reference) is used to mandate watermark embedding, but is not used to specify a particular identifier or data to be embedded.  
         [0010]     Still another aspect is an encrypted video file with certain video segments duplicated therein. The duplicated segments can be encrypted according to different keys. The segments are used to generate different unique watermarks when the content is rendered by applying device specific decryption keys. Consider a typical movie trail. A movie is passed from one executive to another, perhaps even to editors and directors, each giving their approval prior to release of the movie. The video is played on a rendering device (e.g., video player). To establish an audit trail the player preferably generates a unique ID that is associated with the player or executive in the video, based upon selective decryption of the duplicated video segments as dictated by device-specific keys. An embedded identifier is then dependent upon which player is rendering the video (e.g., which movie studio executive is currently viewing the video as the video is passed along to each other after approval of a master version for release). The embedding happens during viewing (a.k.a. rendering) in the player. 
        (Related information for embedding is found, e.g., in EP 1134977 entitled “Method and system for providing copies of scrambled content with unique watermarks, and system for descrambling scrambled content” (incorporated herein by reference) and J. Benaloh, et al&#39;s, “Efficient Fingerprinting to Protect Digital Content,” a copy of which is found at http://research.microsoft.com/crypto/FingerMark.ppt, which is incorporated herein by reference.)        
 
         [0012]     Yet another aspect of the present invention is identifying an intended recipient of a DVD disk by embedding a recipient identifier in a DVD disk master file during transport of the disk to the recipient.  
         [0013]     The foregoing and other features, aspects and advantages of the present invention will be more readily apparent from the following detailed description, which proceeds with reference to the accompanying drawings. 
     
    
     BRIEF DESCRIPTION OF THE DRAWINGS  
       [0014]      FIG. 1  is flowchart illustrating a conventional workflow during video creation.  
         [0015]      FIG. 2  is illustrates an architecture for automatic logging.  
         [0016]      FIG. 3  is a flowchart illustrating a rights command controlled embedding. 
     
    
     DETAILED DESCRIPTION  
       [0017]     Workflow Overview  
         [0018]      FIG. 1  illustrates a workflow during video creation. Camera segments (e.g., images or frames) are recorded to film in step  101 . The segments are edited by using, e.g., software editing tools (e.g., Avid editing tools for Macs and Windows PCs) in step  102 . A finished version is written to a digital master file (e.g., stored on DigiBeta) via a channel (e.g., SDI) in step  103 . The master may include a final version (or semi-final like a rough cut version) to which sound and effects have been added. (Of course, other editing software, such as is provided by Apple and Adobe, and other master storage formats can be suitably interchanged with these aspects of the invention.). Steps  101 - 103  are typically repeated as additional camera segments are captured. And, of course, these steps can be substituted or augmented with digital video as well.  
         [0019]     In the process of shooting film on a day-to-day basis, reviews are often performed on previous work. These reviews are provided in video segments and are referred to as “dailies.” (Dailies are rough cuts or segments that may or may not include sound tracks or effects. Dailies can be viewed by directors and crew to determine what else needs to be accomplished for a completion of a particular scene or to track a movie timeline.). Thus, these dailies are produced for review or approval in step  102   a . The daily segments can be provided via Windows Media Video (WMV) and/or QuickTime (QT), although they may be recorded to, e.g., DigiBeta and converted later.  
         [0020]     A Master can be copied to a WMV/QT file for executive review, where executives (or directors, editors, assistants, advertisers, promoters, etc., herein collectively referred to as “executives”) may pass the same file along to other executives to gain approval or exposure as shown in step  104 . The executives will play this content in a player  108 , such as Windows Media Player, QuickTime player, or other suitable player. In step  105  and  106 , after approval by required movie executives, the DigitBeta Master is converted to film for theatrical release and to a DVD disk image file (e.g., MPEG2 disk image file), respectively. (As will be appreciated, the disk image file is the file that is used to direct the burning of DVDs.). In step  107  and  108 , the disk image is distributed, e.g., using a content distribution service like WamNet, to a duplication house for DVD disk creation and release.  
         [0021]     In addition, a DigiBeta Master or editor version may be converted to MPEG-2 (8-10 Mbps) and this file is converted to WMV and/or QuickTime using, e.g., Telestream Flip Factory™.  
         [0022]     Instead of film as in step  105 , a final movie may be digitally stored as a master copy for transmission in digital cinema applications.  
         [0023]     Automatic Logging  
         [0024]     The first improvement in such a workflow involves automatic logging, and can occur, e.g., at step  102   a  and/or  104  in  FIG. 1 .  
         [0025]     One preferred logging system is illustrated in  FIG. 2 . A personal computer (PC) or video editing terminal includes a PC controller  200  (e.g., software executing on the personal computer or video editing terminal). PC controller  200  control a video router  220  as video copies are made or transmitted. A watermark embedder  230  embeds an embedder ID via digital watermarking and, preferably, embeds a time stamp into video (or audio) copies  240 . The embedded ID may include, e.g., a number, text, code or plural-bit identifier that is preferably uniquely associated with the watermark embedder, PC, PC controller  200  and/or a database or data record. Ideally, the time stamp includes or indexes enough data to resolve the time, month, day and year. Many methods can be used, such as including minutes from a fixed date (e.g., how many minutes from Jan. 1, 2004). This example requires 27 bits for a 50 year range. The watermarking is preferably automatically (and unavoidably) activated as copies are generated or routed by video router. (In fact, the watermark embedder can be viewed as a continuous process, which embeds all content from a video router.) Usually, the video communications channel is a Serial Digital Interface (SDI), but other channels can be suitably interchanged herewith.  
         [0026]     The PC controller  200  helps manage or track user activity. For example, an active user  201  logs on to the PC. The PC controller  200  communicates which user accesses the PC to a database or record  210 . Thus, the data record includes a record of which users accessed the PC.  
         [0027]     When content  240  is found (especially if in an illegitimate or unexpected location), the watermark is read with digital watermark (DWM) detector  250 . The embedder ID and timestamp are obtained from the watermark by detector  250 . The embedder ID is used to locate a corresponding record in database or log  210 . (Alternatively, the embedder ID identifies a particular PC, which includes the data record itself.). The corresponding record indicates that the embedder ID is associated with the PC or PC controller  200 . A particular user is determined from the timestamp (e.g., a user is identified as using the PC during a time represented by the watermark date/time stamp). The log  210  may also include times representing when a user logged-on and log off of the PC, or which current users were logged on at time intervals, like every 15 minutes. Thus, the timestamp and recorded user access can be used to determine who is responsible for copying or transmitting the video in which the embedder ID was found.  
         [0028]     User identity can be authenticated to avoid spoofing the PC with a username and password. For example, the PC may use biometrics, such as a fingerprint, eye scan or face recognition; thus, further increasing authentication of the user. A digital image of the user&#39;s face or eye (e.g., iris or retina scan) can be saved in the log  210  to verify the user. Secure IDs can be used, such that the PC is secured by what you know (e.g., secure ID), what you are (e.g., employee level  10 , which may or may not be authorized to copy video), and what you have (e.g., perhaps a user is required to present a feature for biometric sampling, or even present a watermarked object for verification as discussed, e.g., in assignee&#39;s U.S. patent application Ser. No. 09/790,322, published as U.S. 2001-0037313 A1).  
         [0029]     In an alternative implementation, the controller PC  200  itself includes one or more computers and includes or cooperates with one or more digital watermark embedders.  
         [0030]     As an additional alternative, PC controller  200  does not “control” the video router. Instead, the PC controller  200  monitors and records user access on the PC. A watermark embedder automatically embeds the Embedder ID in any content routed by the router as discussed above.  
         [0031]     As mentioned above, an embedder ID is preferably uniquely associated with a PC, PC controller or particular data record. Thus, the embedder ID can always be traced to the PC or PC controller via the data repository  210 . In a multiple computer—but only one watermark embedder—scenario, unique association between a PC controller and embedder ID is preserved by connecting the watermark embedder  230  to the PC controllers and changing the embedder ID based upon which PC controller is active. (Of course, this interconnection is avoided by providing a separate watermark embedder for each PC controller, and ensuring the watermark embedder is embedding a video stream controlled by the PC controller.).  
         [0032]     Automatic Embedding Based Upon Rights Command  
         [0033]     A second improvement to the  FIG. 1  work flow provides automatic watermark embedding during video rendering, as shown in  FIG. 3 . The term “render” includes its familiar meaning of playing, providing, displaying or performing, etc. Specifically, a rights command  300 , which states that watermark embedding is required, is used to force a video player  108  to embed data with embedder  320 .  
         [0034]     A novel twist is that embedder  320 , and not rights command  300 , controls what is embedded in content. The rights command  300  may be part of a rights language, such as MPEG-21 REL, mentioned above. The embedder  320  may be a software plug-in that must be installed on a computer to play the video, such as a DirectX Media Object (DMO), as used with Windows Media Player. The embedder embeds a unique forensic ID related to the user, user&#39;s computer, video player or user&#39;s license  310 . The embedding occurs during (or just prior to) rendering of the video. User license  310  is optional (as shown by the dotted box) as the other forensic ID options could be used.  
         [0035]     The forensic ID is used for tracking purposes.  
         [0036]     Thus, a database is preferably maintained to link unique forensic IDs to users or players. (The association can be through predetermined identifiers or through physical addresses or computer serial numbers, etc.). This database is preferably protected for privacy.  
         [0037]     With reference to  FIGS. 1 and 3 , content is optimally encrypted. Player  108  then must search for rights commands  300  for decryption permissions (or instructions) and preferably enables embedder  320  prior to decrypting the content for rendering. A trusted player  108  could work with non-encrypted content, as it is “trusted” (or programmed) to look for rights commands  300  and enable embedder  320 . In either case, the player  108  preferably does not play content if the embedder  320  is not available or working.  
         [0038]     If illegitimate video is found, the forensic ID is read from the watermark with a watermark detector. If the detector cannot read the watermark in the illegitimate video sample, the original can be used to separate the DWM from the video. (For example, the illegitimate sample and/or the original is converted to similar formats, aspect ratio, size, etc., and then the difference between the sample and original which will typically yield at least some of the watermark signal.).  
         [0039]     Automatic Embedding Using Encrypted and DWM Segments  
         [0040]     A third improvement to the  FIG. 1  workflow is related to the second improvement, above, but uses a different player side approach.  
         [0041]     Protection is provided in player  108 . An encrypted file with certain segments duplicated with different watermark payloads is used. When the encrypted file is rendered, the rendered content contains a digital watermark with a forensic ID in its payload, which may include the active user ID, the player ID, or user&#39;s license ID. The digital watermark payload is created from decrypting different watermarked segments. The player is usually located, e.g., on a PC of a movie studio executive. The executive passes video along to another executive after she approves the version.  
         [0042]     In other words, this approach identifies the rendering device in a peer-to-peer (P2P) environment, where the P2P environment consists of movie executive&#39;s PC and email or ftp.  
         [0043]     A forensic ID, e.g. player or user license ID, cooperates with a database that records a corresponding executive&#39;s ID or rendering device ID. This database is preferably protected for privacy.  
         [0044]     DWM in DVD Disk Image During Transit  
         [0045]     A fourth improvement embeds one or more digital watermarks in a DVD disk image during transport of the disk image to a duplication house in block  107  ( FIG. 1 ). At least one watermark includes a payload that identifies the duplication house, e.g. a duplication ID. This system includes an embedder sitting on a server (or router) that is in the path of video distribution. The server may embed the video before distributing it based upon the request of the duplication house  107 . The embedder may alternatively reside at the duplication house  107  receiving server and embed video with a duplication ID upon receipt, or reside in a predictable location between the studio and distribution house  107 .  
         [0046]     If content is found in an unauthorized or unexpected location, such as on a public P2P network, a watermark detector can read the forensic ID, e.g. duplication ID, and link it to the duplication house  107 . (The duplication ID can be used to query a data record to identify a corresponding duplication house  107 . Protecting this data record for privacy is not a large issue since consumers are not involved, and the watermark detector is usually held by the content owner in secrecy.)  
         [0047]     Some combinations of the above disclosure are provided below. Of course, these are not the only combinations that are possible for the above disclosure, and are provided by way of example only. Other combinations are provided in the claims and still further combinations can be achieved based on the above disclosure.  
         [0048]     A. An apparatus comprising: 
        a media player; and     a digital watermark embedder,     wherein said media player searches content or data associated with the content for a rights command, wherein the right command requires said digital watermark embedder to embed an identifier into content rendered by said media player and, absent such embedding, the media player will not render the content, and     wherein the identifier is generated by said digital watermark embedder.     A1. The apparatus of A wherein the identifier is related to a user license.     A2. The apparatus of A wherein the identifier comprises a media player identifier.     A3. The apparatus of A wherein the identifier is related to a user via a restricted database.     A4. The apparatus of A wherein said content comprises video.     A5. The apparatus of A3 wherein the identifier is related to a user license.     A6. The apparatus of A3 wherein the identifier comprises a player identifier.     A7. The apparatus of A3 wherein the identifier is determined solely by the watermark embedder.        
 
         [0060]     B. A method comprising: 
        tracking which users access a computer system according to time;     communicating the users and related times for storage in a data record;     causing, without user input, unmarked video to be steganographically marked upon a copy or transmission of the video, wherein the steganographic marking embeds an identifier that is uniquely associated with the computer system or data record.     B1. The method of B wherein upon receipt of a timestamp, said method further comprises indexing the data record with at least a portion of the time stamp to identify a user.        
 
         [0065]     C. A method to forensically identify content leaked from a video workflow comprising: 
        upon rendering of content by a movie executive, steganographically embedding a device specific identifier in the content, wherein the device specific identifier is determined based on decryption keys to decrypt duplicate encrypted segments in the content; and     rendering the content. 
 
 Concluding Remarks 
       
 
         [0068]     Having described and illustrated the principles of the technology with reference to specific implementations, it will be recognized that the technology can be implemented in many other, different, forms.  
         [0069]     To provide a comprehensive disclosure without unduly lengthening the specification, applicants incorporate by reference the U.S. patent documents referenced above.  
         [0070]     The methods, processes, and systems described above may be implemented in hardware, software or a combination of hardware and software. For example, the auxiliary data encoding processes may be implemented in a programmable computer or a special purpose digital circuit. Similarly, auxiliary data decoding may be implemented in software, firmware, hardware, or combinations of software, firmware and hardware. The methods and processes described above may be implemented in programs executed from a system&#39;s memory (a computer readable medium, such as an electronic, optical or magnetic storage device).  
         [0071]     While  FIG. 1  illustrates a particular conventional workflow, my inventive techniques can be applied to other chain of custody and forensic tracking workflows as well.  
         [0072]     The particular combinations of elements and features in the above-detailed embodiments are exemplary only; the interchanging and substitution of these teachings with other teachings in this and the incorporated-by-reference patents/applications are also contemplated.