Source: https://patents.google.com/patent/US6574350B1/en
Timestamp: 2019-04-24 12:31:16+00:00

Document:
2000-05-30 Assigned to DIGIMARC CORPORATION reassignment DIGIMARC CORPORATION ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: GUSTAFSON, AMMON E., RHOADS, GEOFFREY B.
Image, video, or audio data is encoded with both a frail and a robust watermark. The two watermarks respond differently to different forms of processing (e.g., copying the object may render the frail watermark unreadable), permitting an original object to be distinguished from a processed object. Appropriate action can then taken in response thereto.
This application is a continuation-in-part of application Ser. No. 09/287,940, filed Apr. 7, 1999, which claims priority to abandoned application No. 60/082,228, filed Apr. 16, 1998. This application is also a continuation of application Ser. No. 09/433,104, filed Nov. 3, 1999, which is a continuation-in-part of application Ser. No. 09/234,780, filed Jan. 20, 1999, which claims priority to abandoned application No. 60/071,983, filed Jan. 20, 1998. These applications are incorporated herein by reference.
This application is also a continuation-in-part of application Ser. No. 09/442,440, filed Nov. 17, 1999, which is a continuation of application Ser. No. 08/951,858, filed Oct. 16, 1997 (now U.S. Pat. No. 6,026,193), which is a continuation of application Ser. No. 08/436,134, filed May 8, 1995 (now U.S. Pat. No. 5,748,763).
The present application relates to digital watermarking, and particularly relates to digital watermarking techniques employing both frail and robust watermarks.
For expository convenience, the following discussion focuses on an exemplary application of the disclosed technology—encoding the images printed on banknotes with both frail and robust watermarks. As noted later, however, the technology also finds application beyond image watermarking, including in video and audio watermarking.
The problem of casual counterfeiting of banknotes first arose two decades ago, with the introduction of color photocopiers. A number of techniques were proposed to address the problem.
U.S. Pat. No. 5,659,628 (assigned to Ricoh) is one of several patents noting that photocopiers can be equipped to recognize banknotes and prevent their photocopying. The Ricoh patent particularly proposed that the red seal printed on Japanese yen notes is a pattern well-suited for machine recognition. U.S. Pat. No. 5,845,008 (assigned to Omron), and U.S. Pat. Nos. 5,724,154 and 5,731,880 (both assigned to Canon) show other photocopiers that sense the presence of the seal emblem on banknotes, and disable a photocopier in response.
Recent advances in color printing technology have greatly increased the level of casual counterfeiting. High quality scanners are now readily available to many computer users, with 300 dpi scanners available for under $100, and 600 dpi scanners available for marginally more. Similarly, photographic quality color ink-jet printers are commonly available from Hewlett-Packard Co., Epson, etc. for under $300.
These tools pose new threats. For example, a banknote can be doctored (e.g. by white-out, scissors, or less crude techniques) to remove/obliterate the visible patterns on which prior art banknote detection techniques relied to prevent counterfeiting. Such a doctored document can then be freely scanned or copied, even on photocopiers designed to prevent processing of banknote images. The removed pattern(s) can then be added back in, e.g. by use of digital image editing tools, permitting free reproduction of the banknote.
In accordance with aspects of the present invention, these and other current threats are addressed by digitally watermarking banknotes, and equipping devices to sense such watermarks and respond accordingly.
(Watermarking is a quickly growing field of endeavor, with several different approaches. The present assignee's work is reflected in the earlier-cited related applications, as well as in U.S. Pat. Nos. 5,841,978, 5,748,783, 5,710,834, 5,636,292, 5,721,788, and laid-open PCT application WO97/43736. Other work is illustrated by U.S. Pat. Nos. 5,734,752, 5,646,997, 5,659,726, 5,664,018, 5,671,277, 5,687,191, 5,687,236, 5,689,587, 5,568,570, 5,572,247, 5,574,962, 5,579,124, 5,581,500, 5,613,004, 5,629,770, 5,461,426, 5,743,631, 5,488,664, 5,530,759, 5,539,735, 4,943,973, 5,337,361, 5,404,160, 5,404,377, 5,315,098, 5,319,735, 5,337,362, 4,972,471, 5,161,210, 5,243,423, 5,091,966, 5,113,437, 4,939,515, 5,374,976, 4,855,827, 4,876,617, 4,939,515, 4,963,998, 4,969,041, and published foreign applications WO 98/02864, EP 822,550, WO 97/39410, WO 96/36163, GB 2,196,167, EP 777,197, EP 736,860, EP 705,025, EP 766,468, EP 782,322, WO 95/20291, WO 96/26494, WO 96/36935, WO 96/42151, WO 97/22206, WO 97/26733. Some of the foregoing patents relate to visible watermarking techniques. Other visible watermarking techniques (e.g. data glyphs) are described in U.S. Pat. Nos. 5,706,364, 5,689,620, 5,684,885, 5,680,223, 5,668,636, 5,640,647, 5,594,809.
Most of the work in watermarking, however, is not in the patent literature but rather in published research. In addition to the patentees of the foregoing patents, some of the other workers in this field (whose watermark-related writings can by found by an author search in the INSPEC database) include I. Pitas, Eckhard Koch, Jian Zhao, Norishige Morimoto, Laurence Boney, Kineo Matsui, A. Z. Tirkel, Fred Mintzer, B. Macq, Ahmed H. Tewfik, Frederic Jordan, Naohisa Komatsu, and Lawrence O'Gorman.
In the present disclosure it should be understood that references to watermarking encompass not only the assignee's watermarking technology, but can likewise be practiced with any other watermarking technology, such as those indicated above.
In accordance with an exemplary embodiment of the present invention, an object—such as a banknote image—is encoded with two watermarks. One is relatively robust, and withstands various types of corruption, and is detectable in the object even after multiple generations of intervening distortion. The other is relatively frail, so that it fails with the first distortion. If a version of the object is encountered having the robust watermark but not the frail watermark, the object can be inferred to have been processed, and thus not an original.
The foregoing and other features and advantages of the present invention will be more readily apparent from the following Detailed Description, which proceeds with reference to the accompanying drawings.
FIG. 1 shows part of an automatic teller machine employing principles of the present invention.
FIG. 2 shows part of a device (e.g. a photocopier, scanner, or printer) employing principles of the present invention.
FIG. 3 shows part of another device employing principles of the present invention.
Watermarks in banknotes and other security documents (passports, stock certificates, checks, etc.—all collectively referred to as banknotes herein) offer great promise to reduce such counterfeiting, as discussed more fully below. Additionally, watermarks provide a high-confidence technique for banknote authentication.
As noted, banknotes presently include certain visible structures, or markings (e.g., the seal emblem noted in the earlier-cited patents), which can be used as aids to note authentication (either by visual inspection or by machine detection). Desirably, a note is examined by an integrated detection system (24), for both such visible structures (22), as well as the present watermark-embedded data, to determine authenticity.
The visible structures can be sensed using known pattern recognition techniques. Examples of such techniques are disclosed in U.S. Pat. Nos. 5,321,773, 5,390,259, 5,533,144, 5,539,841, 5,583,614, 5,633,952, 4,723,149 and 5,424,807 and laid-open foreign application EP 766,449. The embedded watermark data can be recovered using the scanning/analysis techniques disclosed in the cited patents and publications.
To reduce counterfeiting, it is desirable that document-reproducing technologies recognize banknotes and refuse to reproduce same. Referring to FIG. 2, a photocopier (30), for example, can sense the presence of either a visible structure (32) or embedded banknote watermark data (34), and disable copying if either is present (36). Scanners and printers can be equipped with a similar capability—analyzing the data scanned or to be printed for either of these banknote hallmarks. If either is detected, the software (or hardware) disables further operation.
The watermark detection criteria provides an important advantage not otherwise available. As noted, an original bill can be doctored (e.g. by white-out, scissors, or less crude techniques) to remove/obliterate the visible structures. Such a document can then be freely copied on either a visible structure-sensing photocopier or scanner/printer installation. The removed visible structure can then be added in via a second printing/photocopying operation. If the printer is not equipped with banknote-disabling capabilities, image-editing tools can be used to insert visible structures back into image data sets scanned from such doctored bills, and the complete bill freely printed. By additionally including embedded watermark data in the banknote, and sensing same, such ruses will not succeed.
Desirably, the visible structure detector and the watermark detector are integrated together as a single hardware and/or software tool. This arrangement provides various economies, e.g., in interfacing with the scanner, manipulating pixel data sets for pattern recognition and watermark extraction, electronically re-registering the image to facilitate pattern recognition/watermark extraction, issuing control signals (e.g. disabling) signals to the photocopier/scanner, etc.
While the foregoing applications disabled potential counterfeiting operations upon the detection of either a visible structure or watermarked data, in other applications, both criteria must be met before a banknote is recognized as genuine. Such applications typically involve the receipt or acceptance of banknotes, e.g. by ATMs as discussed above and illustrated in FIG. 1.
The reader may first assume that banknote watermarking is effected by slight alterations to the ink color/density/distribution, etc. on the paper. This is one approach. Another is to watermark the underlying medium (whether paper, polymer, etc.) with a watermark. This can be done by changing the microtopology of the medium (a la mini-Braille) to manifest the watermark data. Another option is to employ a laminate on or within the banknote, where the laminate has the watermarking manifested thereon/therein. The laminate can be textured (as above), or its optical transmissivity can vary in accordance with a noise-like pattern that is the watermark, or a chemical property can similarly vary.
An improvement to existing encoding techniques is to add an iterative assessment of the robustness of the mark, with a corresponding adjustment in a re-watermarking operation. Especially when encoding multiple bit watermarks, the characteristics of the underlying content may result in some bits being more robustly (e.g. strongly) encoded than others. In an illustrative technique employing this improvement, a watermark is first embedded in an object. Next, a trial decoding operation is performed. A confidence measure (e.g. signal-to-noise ratio) associated with each bit detected in the decoding operation is then assessed. The bits that appear weakly encoded are identified, and corresponding changes are made to the watermarking parameters to bring up the relative strengths of these bits. The object is then watermarked anew, with the changed parameters. This process can be repeated, as needed, until all of the bits comprising the encoded data are approximately equally detectable from the encoded object, or meet some predetermined signal-to-noise ratio threshold.
Another option is to mark an object with watermarks of two different levels of robustness, or strength. The more robust watermark withstands various types of corruption, and is detectable in the object even after multiple generations of intervening distortion. The less robust watermark can be made frail enough to fail with the first distortion of the object. In a banknote, for example, the less robust watermark serves as an authentication mark. Any scanning and reprinting operation will cause it to become unreadable. Both the robust and the frail watermarks should be present in an authentic banknote; only the former watermark will be present in a counterfeit.
Still another form of multiple-watermarking is with content that is compressed. The content can be watermarked once (or more) in an uncompressed state. Then, after compression, a further watermark (or watermarks) can be applied.
Still another advantage from multiple watermarks is protection against sleuthing. If one of the watermarks is found and cracked, the other watermark(s) will still be present and serve to identify the object.
The foregoing discussion has addressed various technological fixes to many different problems. Exemplary solutions have been detailed above. Others will be apparent to the artisan by applying common knowledge to extrapolate from the solutions provided above.
For example, the technology and solutions disclosed herein have made use of elements and techniques known from the cited references. Other elements and techniques from the cited references can similarly be combined to yield further implementations within the scope of the present invention. Thus, for example, holograms with watermark data can be employed in banknotes, single-bit watermarking can commonly be substituted for multibit watermarking, technology described as using imperceptible watermarks can alternatively be practiced using visible watermarks (glyphs, etc.), techniques described as applied to images can likewise be applied to video and audio, local scaling of watermark energy can be provided to enhance watermark signal-to-noise ratio without increasing human perceptibility, various filtering operations can be employed to serve the functions explained in the prior art, watermarks can include subliminal graticules to aid in image re-registration, encoding may proceed at the granularity of a single pixel (or DCT coefficient), or may similarly treat adjoining groups of pixels (or DCT coefficients), the encoding can be optimized to withstand expected forms of content corruption. Etc., etc., etc. Thus, the exemplary embodiments are only selected samples of the solutions available by combining the teachings referenced above. The other solutions necessarily are not exhaustively described herein, but are fairly within the understanding of an artisan given the foregoing disclosure and familiarity with the cited art.
(To provide a comprehensive disclosure without unduly lengthening the following.
based on a result of said analysis, determining whether the set of sampled data corresponds to an original, or a degraded reproduction of an original, wherein the presence of the first but not the second watermark information indicates a conversion that has impaired the second watermark information.
2. The method of claim 1, wherein at least the first watermark is effected by slight alterations to the sampled data's color.
3. The method of claim 2, wherein the conversion comprises at least scanning and reprinting.
controlling operation of equipment in accordance with said determination.
5. The method of claim 4 that includes disabling an operation of said equipment in accordance with said determination.
6. The method of claim 5 that includes disabling said operation if one of said watermarks is detected and the other is not.
7. A storage medium having computer instructions stored thereon causing a computer programed thereby to perform the method of claim 4.
determining, based on a result of said analysis, whether the set of sampled data corresponds to an original, or a degraded reproduction of an original, wherein the set of sampled data corresponds to an original only if both of said watermarks are detected.
9. The method of claim 8, wherein at least the first watermark is effected by slight alterations to the data sample's color.
10. The method of claim 9, wherein a degraded original results from at least scanning and reprinting of an original.
based on the presence of the first digital watermark and the condition of the second digital watermark, determining whether the data set corresponds to an original, or to a degraded reproduction of an original.
12. The method of claim 11, wherein the sampled data corresponds to a degraded reproduction of an original when the first digital watermark is present in the data set, and the condition of the second digital watermark is at least one of degraded and undetectable.
analyzing differences between the first and second sets of results to differentiate the original document from the copy of the original document.
"High Water FBI Limited Presentation Image Copyright Protection Software," FBI Ltd brochure, Jul., 1995, 17 pages.
60/000442, Hudetz, Jun. 20, 1995.
60/082228, Rhoads, Apr. 16, 1998.
60/141763, Davis, Jun. 30, 1999.
60/158015, Davis et al., Oct. 6, 1999.
Audio Watermarking Architectures for Persistent Protection, Presentation to SDMI PDWG, Mar. 29, 1999, J. Winograd, Aris Technologies, pp. 1-16.
Audio Watermarking Architectures for Secure Digital Music Distribution, A Proposal to the SDMI Portable Devices Working Group by ARIS Technologies, Inc, Mar. 26, 1999, pp. 1-11.
Audio Watermarking System to Screen Digital Audio Content for LCM Acceptance, A Proposal Submitted in Response to PDWG99050504-Transitio CƒP by ARIS Technologies, Inc., May 23, 1999, Document Version 1.0, 15 pages.
Boland et al., "Watermarking Digital Images for Copyright Protection", Fifth Int'l Conference on Image Processing and it's Application, Jul. 1995, pp. 326-330.
Brassil et al., Electronic Marking and Identification Techniques to Discourage Document Copying, Proceedings of INFOCOM '94 Conference on Computer, IEEE Commun. Soc Conference, Jun. 12-16, 1994, 1278-1287.
Bruckstein, A.M.; Richardson, T.J., A holographic transform domain image watermarking method, Circuits, Systems, and Signal Processing vol. 17, No. 3, p. 361-89, 1998. This paper includes an appendix containing an internal memo of Bell Labs, which according to the authors of the paper, was dated Sep. 1994.
Cookson, Chris, General Principles of Music Uses on Portable Devices, presented to SDMI, Mar. 5, 1999.
Dautzenberg, "Watermarking Images," Department of Microelectronics and Electrical Engineering, Trinity College Dublin, 47 pages, Oct. 1994.
Koch et al., "Copyright Protection for Multimedia Data," Fraunhofer Institute for Computer Graphics, Dec. 16, 1994, 15 pages.
Koch et al., "Towards Robust and Hidden Image Copyright Labeling," Proc. of 1995 IEEE Workshop on Nonlinear Signal and Image Processing, Jun. 20-22, 1995, 4 pages.
Kurak et al., "A Cautionary Note On Image Downgrading," 1992 IEEE, pp. 153-159.
Levy, "AIPL's Proposal for SDMI: An Underlying Security System" (slide presentation), Mar. 29, 1999, 23 slides..
Microsoft Response to CƒP for Technology Solutions to Screen Digital Audio Content for LCM Acceptance, SDMI, PDWG Tokyo, May 23, 1999, 9 pages.
Mintzer et al., "Safeguarding Digital library Contents and Users" Digital Watermarking, D-Lib Magazine, Dec. 1997: ISSN 1082-9873.
Mintzer et al., "Safeguarding Digital Library Contents and Users: Digital Watermarking," D-Lib Magazine, Dec. 1997, 12 pages.
Response to CƒP for Technology Solutions to Screen Digital Audio Content for LCM Acceptance, NTT Waveless Radio Consotium, May 23, 1999, 9 pages.
Rindfrey, "Towards and Equitable System for Access Control and Copyright Protection in Broadcast Image Services: The Equicrypt Approach," Intellectual Property Rights and New Technologies, Proc. of the Conference, R. Oldenbourg Verlag Wien Munchen 1995, 12 pages.
Sandford II et al., "The Data Embedding Method", Proceedings of the SPIE vol. 2615, pp. 226-259, 1996.
Schreiber et al., "A Compatible High-Definition Television System Using the Noise-Margin Method of Hiding Enhancement Information," SMPTE Journal, Dec. 1989, pp. 873-879.
SDMI Example Use Scenarios (Non-Exhaustive), Version 1.2, Jun. 16, 1999.
Szepanski, "A Signal Theoretic Method for Creating Forgery-Proof Documents for Automatic Verifications," Proceedings 1979 Carnahan Conference on Crime Countermeasures, May 16, 1979, pp. 101-109.
Szepanski, "Additive Binary Data Transmission for Video Signals," Conference of the Communications Engineering Society, 1980, NTG Technical Reports, vol. 74, pp. 343-351. (German text and English translation enclosed).
Szepanski, "Additive Binary Data Transmission for Video Signals," Papers Presented at Conf. Of Comm. Engineering Soc. Sep. 30-Oct. 3, 1980, Technical Reports vol. 74, pp. 342-352.
Tanaka et al., "A Visual Retrieval Sysem with Private Information for Image Database," Proceeding International Conference on DSP Applications and Technology, Oct. 1991, pp. 415-421.
Thomas, Keith, Screening Technology for Content from Compact Discs, May 24, 1999, 11 pages.
Tirkel et al, "Electronic Water Mark," DICTA-93, Macquarie University, Sydney, Australia, Dec., 1993, pp. 666-673.
Tirkel et al., "Electronic Water Mark," Dicta-93, Marquarie University, Sydney, Australia, Dec., 1993, pp. 666-672.
U.S. Application No. 09/314648, Rodriguez et al., filed May 19, 1999.
U.S. Pat. application No. 09/074,034, filed May 6, 1998, Rhoads, 382/100.
U.S. Pat. application No. 09/127,502, filed Jul. 31, 1998, Rhoads, 382/100.
U.S. Pat. application No. 09/185,380, filed Nov. 3, 1998, Davis et al., 382/100.
U.S. Pat. application No. 09/287,940, filed Apr. 7, 1999, Rhoads, 382/100.
U.S. Pat. application No. 09/293,601, filed Apr. 15, 1999, Rhoads, 382/135.
U.S. Pat. application No. 09/293,602, filed Apr. 15, 1999, Rhoads, 382/100.
U.S. Pat. application No. 09/342,972, filed Jun. 29, 1999, Rhoads, 382/100.
U.S. Pat. application No. 09/428,359, filed Oct. 28, 2000, Davis et al., 382/100.
U.S. Pat. application No. 09/431,990, filed Nov. 3, 1999, Rhoads, 380/54.
U.S. Pat. application No. 09/465,418, filed Dec. 16, 1999, Rhoads et al., 382/100.
U.S. Pat. application No. 09/498,223, filed Feb. 3, 2000, Rhoads et al., 382/100.
U.S. Pat. application No. 09/562,524, filed May 1, 2000, Carr et al., 382/100.
U.S. Pat. application No. 09/574,726, filed May 18, 2000, Rhoads et al., 382/100.
U.S. Pat. application No. 09/625,577, filed Jul. 25, 2000, Carr et al., 382/100.
U.S. Pat. application No. 09/645,779, filed Aug. 24, 2000, Tian et al., 382/100.
U.S. Pat. application No. 09/689,226, filed Oct. 11, 2000, Brunk, 382/100.
U.S. Pat. application No. 09/689,250, filed Oct. 11, 2000, Ahmed, 382/100.
U.S. Pat. application No. 09/689,293, filed Oct. 11, 2000, Tian et al., 382/100.
U.S. Pat. application No. 09/761,280, filed Jan. 16, 2001, Rhoads, 382/100.
U.S. Pat. application No. 09/761,349, filed Jan. 16, 2001, Rhoads, 382/100.
U.S. Pat. application No. 09/765,102, filed Jan. 17, 2001, Shaw, 382/199.
U.S. Pat. application No. 60/082,228, filed Apr. 16, 1998, Rhaods.
U.S. Pat. application No. 60/198,138, filed Apr. 17, 2000, Alattar.
U.S. Pat. No. 09/234,780, filed Jan. 20, 1999, Rhoads et al., 382/100.
U.S. Pat. No. 09/404,291, filed Sep. 23, 1999, Levy, 713/176.
U.S. Pat. No. 09/478,713, filed Jan. 6, 2000, Cookson, 706/057.
U.S. Pat. No. 60/071,983, filed Jan. 20, 1998, Levy.
U.S. Pat. No. 60/114,725, filed Dec. 31, 1998, Levy.
U.S. Pat. No. 60/116,641, filed Jan. 21, 1999, Cookson.
Vidal et al., "Non-Noticeable Information Embedding in Color Images: Marking and Detection", IEEE 1999, pp. 293-297.
Weber et al., "Correlative Image Registration," Seminars in Nuclear Magazine, vol. XXIV, No. 4, Oct., 1994, pp. 311-323.
Winograd, J.M., "Audio Watermarking Architecture for Secure Digital Music Distribution," a Proposal to the SDMI Portable Devices Working Group, by Aris Technologies, Inc., Mar. 26, 1999.
Wolfgang et al., "A Watermark for Digital Images," Computer Vision and Image Processing Laboratory, Purdue University, Sep. 1996, pp. 219-222.

References: application No. 60
 application No. 60
 Application No. 09
 application No. 09
 application No. 09
 application No. 09
 application No. 09
 application No. 09
 application No. 09
 application No. 09
 application No. 09
 application No. 09
 application No. 09
 application No. 09
 application No. 09
 application No. 09
 application No. 09
 application No. 09
 application No. 09
 application No. 09
 application No. 09
 application No. 09
 application No. 09
 application No. 09
 application No. 60
 application No. 60