Patent Publication Number: US-2005144454-A1

Title: Video/image communication with watermarking

Description:
FIELD OF THE INVENTION  
      This invention relates to video transmission systems and related video encoding/decoding techniques. The invention is applicable to, but not limited to, a video compression system employing video watermarking where any tampering of a video image or portion of video image is to be detected.  
     BACKGROUND OF THE INVENTION  
      The ability to transmit real-time video and/or image data is a desirable characteristic of many current wireline and wireless communication systems. However, it is known that individual images/pictures, or a series of images say, in a transmitted video stream, may be subjected to ‘attacks’, i.e. the images may have been tampered with. Therefore, a need exists to protect image or video transmissions from such undesirable tampering. One known technique employed to protect still/video images or documents is by the use of “watermarks”.  
      In the context of the present invention, the terms ‘video’ and ‘image’ are used interchangeably, with the term ‘video’ generally used to represent one or more still images.  
      Wolfgang R, Podilchuk C, Delp E “Perceptual watermarks for digital images and video”, SPIE Conference on Security and Watermarking of Multimedia Content, January  1999 , describes some state of the art watermarking methods for use with video and images.  
      Protection of digital media (including image and video) has also become a key standardisation topic within the multimedia industry over the last year. Police users have formally stated that they do not envisage using digitally transmitted and processed images for evidential purposes without the existence of reliable tamper detection methods.  
      The European Broadcasting Union has issued a second call for systems that offer watermarking of multimedia transmissions for entertainment applications. In addition, the International Standards Organisation (ISO) has set up a working group known as MPEG-21, whose essential function is to investigate digital rights management including the authentication of multimedia data.  
      In image watermarking, a known binary pattern or signature is embedded into an image at the moment of image acquisition. Such watermarks are termed “robust”, because they are designed to remain intact regardless of any post-processing of the image such as filtering, cropping, etc.  
      While such watermarks do provide a useful degree of protection, they cannot at present be wholly relied on in a court of law. The purpose of these watermarking methods is such that they are not designed to possess the required degree of surety that an image has not been tampered with, in order for the image to be used as evidence.  
      Thus, there exists a need in the field of the present invention to provide a video communication unit and methods, based on a watermarking system, that can be used for testing a video sequence for evidence of tampering, wherein the abovementioned disadvantages may be alleviated. Furthermore, there exists a need for a labelling method to highlight areas of a video sequence that are detected as having been tampered with. Additionally, it would be beneficial to visually label tampered video sequences such that they are rendered unusable, or valueless, to the attacker.  
      Published prior art documents include: 
          (i) U.S. Pat. No. 5,875,249 (Mintzer et al.);     (ii) ‘Digital watermarking through quasi m-arrays’, YEH et al., IEEE conference proceedings 29 Nov. 1999, pages 459-461;     (iii) ‘A digital watermark’, OSBORNE et al., IEEE conference proceedings 13-16 Nov. 1994, pages 86-90. 
 
 Statement of Invention 
       

      The present invention provides video communication units, a video transmission system adapted to use one of the video communication units, a mobile radio device, a method of watermarking a video signal transmission in a video transmission system, a method of detecting tampering of a watermarked digital image, a method of visually labelling a video sequence that contains an attacked watermark, a storage medium storing processor-implementable instructions for controlling a processor to carry out any of the methods of the invention, a video communication unit adapted to perform any of the methods of the invention, a mobile radio device, all as claimed in the appended independent claims.  
    
    
     BRIEF DESCRIPTION OF THE DRAWINGS  
      Exemplary embodiments of the present invention will now be described, with reference to the accompanying drawings, in which:  
       FIG. 1  shows a watermark embedding method, in accordance with the preferred embodiment of the invention.  
       FIG. 2  shows a method of detecting frames that have been tampered with, in accordance with the preferred embodiment of the invention.  
       FIG. 3  shows a flowchart of a decision process for determining whether tampering has occurred, in accordance with the preferred embodiment of the invention.  
       FIG. 4  shows a block diagram of a video communication system incorporating a communication unit embedding a watermark, and a communication unit detecting a watermark, in accordance with the preferred embodiment of the invention. 
    
    
     DESCRIPTION OF PREFERRED EMBODIMENTS  
      The inventive concepts described herein find particular application in the current MPEG Standards activities, where a standard watermarking system for video use is to be defined. The detection of tampering, and the ability to determine what type of tampering has taken place, are necessary steps in ensuring user confidence in the images and video sequences a user is viewing in a potentially hostile multimedia communication environment.  
      In summary, the preferred embodiment of this invention aims to pre-process video material such that detection of tampering can take place.  
      Most current image and video watermark methods focus on pre-processing video and images such that any embedded watermark can be recovered, regardless of tampering, for example in copyright applications. The method described below, however, provides for ‘fragile’ watermarks that are destroyed when the images are altered.  
      Furthermore, the method makes clear that content has been tampered with such that the person carrying out the unauthorised processing realises that their actions are evident. As a consequence, they cannot re-sell or distribute the video as an “original”.  
      As an example, let us consider a person making “pirated” video files for unauthorised distribution via an internet web site. The preferred method, as described below, identifies where the video had been tampered with (e.g. in reformatting the video for web distribution) and applies a label to the tampered area, thereby rendering the video unsuitable for onward distribution.  
      The preferred embodiment of the present invention can be applied to video sequences consisting of at least two image frames. Furthermore, the preferred embodiment of the present invention can be applied to image formats including YC b C r  (a standard representation of a colour image as specified in ITU Rec  601 ), red/green/blue (RGB), or any single component (e.g. Y only) of an image format consisting of more than one component. Advantageously, the preferred embodiment of the present invention can also be applied in a restricted area or region of an image, or throughout the entire image.  
      The watermark arrangement of the preferred embodiment of the invention clearly shows when tampering has occurred, as tamper evident information is embedded into the video data stream. In summary, the method facilitates replication of bit planes in consecutive frames.  
      Furthermore, the method allows the video player to locate the exact spatial and temporal position of the tampering as the video is being played.  
      Referring first to  FIG. 1 , a watermarking method  100  is shown, in accordance with the preferred embodiment of the invention. The watermarking method  100  includes a sequence of video/image frames—frame ‘N’  110  to frame ‘N+5’  120 . Six frames are shown for clarity purposes only. Each video/image frame includes a number of data bits, ranging from a most significant bit plane (MSB)-‘bit  0 ’  130  to a least significant bit plane (LSB)-‘bit  7 ’  140 . Again, each frame is shown as having eight bit planes for clarity purposes only.  
      The usual representation of image data is as a series of pixels, located as rows and columns of an image. Common image formats include representation of each pixel in a number of bits, from 6 to 12 bits per component of the image. As an example, each single component (R, G or B) of a colour image may have 8 bits per pixel, an infrared image may have 12 bits per pixel represented as a luminance component.  
      A “bit plane” representation is the term given to the collection of individual bits at any one-bit position of a pixel across the entire image. As an example, consider an image or image region of size k columns and j rows, with ‘N’ bits per pixel. Each of the (k*j) pixels has ‘N’ bits, which are ordered from a least significant bit (LSB) usually termed bit ‘N−1’, up to a most significant bit (MSB) termed bit  0 .  
      A bit plane is a representation of the collection of (k*j) bits considered at one bit position P, where 0&lt;=P&lt;=(N−1). Thus the expression “the most significant bit plane” means that we consider the collection of all bit  0 &#39;s of all (k*j) pixels in an image or image region. For an image comprising ‘N’ bits per pixel, there will be ‘N’ bit planes.  
      It is within the contemplation of the invention that a video/image transmission system having any number of frames in a sequence of video/image frames, including any number of bit planes more than two, would benefit from the inventive concepts described herein.  
      In the preferred embodiment of the invention, a 7th bit plane (the LSB)  140  of a previous frame is moved into a 6th bit plane  150  of a current frame, assuming an 8 bit per pixel image component, which is a common image format. Clearly, it is not essential that the bit plane be moved to the next least important bit-plane. However, the more important the bit plane that is replaced, the larger the adverse affect on the quality of the video transmission.  
      The 7th bit plane (the least significant bit-plane)  140  from the previous frame is placed in the 6th bit plane  150  of the current frame on a pixel-by-pixel basis. This process repeats throughout the sequence, as illustrated in  FIG. 1 .  
      As the 6th and 7th bit planes contain the least significant portions of the video/image data, they can be viewed as essentially noise. Such noise is imperceptible to the human visual system. Thus, the inventors of the present invention have recognised the benefits of using such ‘noise’ as a form of tamper detection, in utilising the replication of bit planes without producing any noticeable artefacts in the image. As the 7th (LSB) bit plane  140  is used, very small alterations in the pixel values of the frame (±1) allow any tampering to be detected.  
      It is noteworthy that for improved robustness to attack and reduced disturbance to the original image, a subset of pixels in a frame may be chosen for this process, in contrast to using the entire video/image frame.  
      Referring next to  FIG. 2 , a method  200  of detecting which frame has been tampered with is shown, in accordance with the preferred embodiment of the invention.  
       FIG. 2  shows two video/image sequences: 
          (i) an un-tampered sequence of frames  250 , from a frame ‘N−2’  255  to a frame ‘N+2’  260 ; and     (ii) a tampered sequence of frames  210 , from a frame ‘N−2’  215  to a frame ‘N+2’  235 .        
      Frame ‘N’  225  is shown as having being tampered with. In order for the video player to test the video sequence, it extracts the 7th bit plane from the previous frame ‘N−1’  220  and compares it on a pixel-by-pixel basis with the 6th bit plane from the current frame ‘N’  225 . Any points within the two bit planes that do not match up indicate tampering at those pixels within the tampered frame ‘N’  225 . The location of the tampered frame, i.e. whether the tamper occurred in the current frame or previous frame, can also be found using the method illustrated in  FIG. 3 .  
      As an example, consider the un-tampered sequence  250 . Comparison of the 7th bit plane of the previous frame with the 6th bit plane of the current frame would reveal no differences, to the bit plane content, on a pixel-by-pixel basis. However, in the tampered sequence  210 , if the 6th bit plane of the current frame ‘N’  225  does not exactly match the 7th bit plane of the previous frame ‘N−1’  220 , for some of the pixels in frame ‘N’  225  of a sequence, then we know tampering has occurred. This could be in frame ‘N’  225  or frame ‘N−-1’  220 . In this case, a further test is performed.  
      The 7th bit plane of frame ‘N’  225  is compared with the 6th bit plane of frame N+ 1 ′  230  for those pixels believed to have been tampered. If the bit planes for those pixels between those frames are equal, then the tampering is known to have occurred in frame ‘N−1’  220 . If the bit planes for those pixels between those frames are not equal, then the tampering is known to have occurred in frame ‘N’  225 .  
      By utilising the least significant bits or bit planes of a video/image transmission in this manner, an effective means of watermark embedding and tamper detection has been provided.  
      Referring now to  FIG. 3 , a flowchart  300  of a decision process for determining whether tampering has occurred is illustrated. As mentioned, a bit-wise comparison of the 6 th  bit plane of the current frame is made with the 7 th  bit plane of the previous frame, as in step  302 . If the comparison yields a match, namely the bit planes are equal in step  304 , the next pixel is selected, as shown in step  306 .  
      If the comparison does not yield a match, namely the bit planes are not equal in step  304 , a second bit-wise comparison is made, of the 7 th  bit plane of the current frame with the 6 th  bit plane of the next frame, as in step  308 .  
      If the second comparison yields a match, namely the bit planes are equal in step  310 , a decision is made that tampering of this pixel occurred in the previous ‘N−1’ frame, as shown in step  314 . If the comparison does not yield a match, namely the bit planes are not equal in step  310 , a decision is made that tampering of this pixel occurred in the current ‘N’ frame, as shown in step  312 .  
      Furthermore, if an area is detected as having been altered, for example by intentional tampering, it is within the contemplation of the invention that the area is visually labelled, as in step  316 , to inform a user viewing the video of such tampering. The visual labelling may take any form appropriate to make clear that tampering has occurred, which may include one or any combination of the following techniques: 
          (i) replacing any or all of the tampered image pixels with a known value. The known value is preferably selected to be sufficiently different from the source image content such that the tampering is clearly visible, for example black, white, any saturated colour, any non-natural colour;     (ii) altering only the coloured appearance of a tampered pixel such that the underlying image content remains visible but the tampering is clearly marked;     (iii) replacing only one component of a tampered pixel with a known value, for example 0 or 255, in an image format comprising more than one component;     (iv) visually labelling (using one or more of (i) or (ii) or (iii) above) the complete image frame within which any of the pixels are detected as having been tampered with and/or;     (v) visually labelling (using one or more of (i) or (ii) or (iii) above) the complete image frame, and all subsequent images in the video sequence, within and following an image frame in which any of the pixels are detected as having been tampered with.        

      Examples of ‘watermarking’ communication units suitable for incorporating the aforementioned inventive concepts are described in filed UK patent applications GB9914384.4 and GB0031085.4, to the present applicant, whose contents are contained herein by reference.  
      However,  FIG. 4  describes a preferred configuration of video/image communication units to implement the preferred embodiment of the present invention. Referring now to  FIG. 4 , a video communication system  400  is shown, in accordance with the preferred embodiment of the invention. The video communication system  400  includes a transmitting video/image communication unit  405  for embedding a watermark, and a receiving video/image communication unit  450  for detecting a watermark.  
      The transmitting video/image communication unit  405 , includes a video/image input port  415  for receiving video/image signals. A video/image signal is passed to processor  410 , which includes three watermark-embedding processes.  
      A first selection function/algorithm  420  selects the component in which to embed the tamper evidence. A second selection function/algorithm  425  then selects the region of the video/image in which to embed the tamper evidence. The tamper evidence is then applied in function/algorithm  430 , in accordance with the method described with respect to  FIG. 1  and  FIG. 2 . The video signal is then transmitted from transmitter  435  to the receiving video/image communication unit  450 .  
      It is within the contemplation of the invention that alternative forms of moving video or image data may be used, for example ‘transmission’ may take the form of copying onto video tape, sending as an internet file, copying onto a floppy disk etc.  
      The receiving video/image communication unit  450 , includes a receiver  455  for receiving video/image signals. A received watermarked video/image signal is passed to processor  460 , which includes three watermark processes.  
      A first function/algorithm  465  applies the tamper evidence, in accordance with the method described with respect to  FIG. 1  and  FIG. 2 . A second tamper detection function/algorithm  470  then detects whether tampering occurred, in accordance with the method described with respect to  FIG. 3 . If tampering is detected, the video/image signal is passed to a third visually labelling function/algorithm  475 , to label the tampered areas, prior to passing the tampered signal to a display  480 .  
      A benefit of the aforementioned inventive concepts is that they can be readily implemented in existing video communication units. More generally, the set of algorithms used to effect the image frame/5 th  bit plane manipulation and processing may be implemented in a respective communication unit in any suitable manner. For example, new apparatus may be added to a conventional communication unit.  
      Alternatively existing parts of a conventional communication unit may be adapted, for example by reprogramming one or more processors  410 ,  460  therein. As such the required adaptation may be implemented in the form of processor-implementable instructions stored on a storage medium, such as a floppy disk, hard disk, programmable read only memory (PROM), random access memory (RAM) or any combination of these or other storage multimedia.  
      It will be understood that the video transmission and watermarking arrangements described above provide at least the following advantages: 
          (i) means for submission of video evidence to a court of law for which it may be shown that the video has not been tampered with since initial acquisition and storage;     (ii) means for purchasers of video and image material to authenticate the material such that it has not been altered since initial acquisition and storage;     (iii) means for distributors of video and image material to verify that material passed to them for distribution is genuine and has not been tampered with since initial acquisition and storage; and     (iv) means for fraudulent tampering of images and video to be detected, and the location of such tampering to be revealed to users of the material.        

      In summary, a video communication unit comprising a video input for receiving a video or image signal transmission has been described. The video or image signal includes a number of video or image frames, wherein each video or image frame includes a number of bit planes. The video input is operably coupled to a processor that replicates at least one bit plane in at least two received video or image frames to provide a means of tamper detection of the video or image signal transmission.  
      In addition, or in the alternative, a video communication unit has been described, for example the above video communication unit, that includes a video receiver for receiving, from a transmitting video communication unit, a watermarked video signal transmission. The watermarked video signal transmission has a number of video or image frames, wherein each video or image frame includes a number of bit planes. The receiver is operably coupled to a processor that compares at least one bit plane of at least two subsequent video or image frames to detect any tampering of the watermark.  
      In addition, or in the alternative, a video communication unit, for example either of the above video communication units, has been described that includes a processor that detects tampering of an area of an image of a received video or image transmission. The processor visually labels, upon detection of said tampering, said area to inform a user viewing the image or video of said tampering of said video or image transmission. In such a case, it is not essential that the aforementioned method of tamper detection be used.  
      In the preferred embodiment of the invention, a mobile radio device may incorporate any of the above video communication units. The mobile radio device may be a mobile phone, a portable or mobile PMR radio, a personal digital assistant, a laptop computer or a wirelessly networked PC. Further, a video transmission system adapted to use any of the above video communication units has been provided.  
      A method of watermarking a video signal transmission in a video transmission system has also been described. The method includes the steps of receiving a video or image signal transmission that has a number of video or image frames, wherein each video or image frame includes a number of bit planes. The method also includes the step of replicating at least one bit plane in a video or image frame to provide a means of tamper detection of the video or image signal transmission.  
      Thus, a method of embedding a watermark in a video sequence has been described that should be sufficient for evidentiary purposes of tampering, thereby improving upon the disadvantages with prior art arrangements.  
      In addition, or in the alternative, a method of detecting tampering of a watermarked image, has been described. The method includes the steps of receiving a watermarked image that has a number of video or image frames, wherein each video or image frame includes a number of bit planes, at least one of which is watermarked. The method also includes the steps of extracting said watermarked bit plane from a previous frame; and comparing said at least one watermarked bit plane in at least two subsequent video or image frames to detect any tampering.  
      In addition, or in the alternative, a method of visually labelling a video or image transmission has also been described. The method includes the step of altering a coloured appearance of a tampered pixel to inform a user viewing the video or image transmission of said tampering of said video or image transmission. Again, in such a case, it is not essential that the aforementioned method of tamper detection be used.  
      In such a manner, the labelling method allows areas of a video sequence to be highlighted that are detected as having been tampered with.