Abstract:
The present disclosure relates generally to managing models representing different expected distortions associated with a plurality of data captures. One claim recites a method comprising: obtaining a plurality of models each representing a different expected distortion associated with a plurality of data captures, the plurality of data captures each resulting in distortion of a machine-readable signal; indexing the plurality of models; upon receiving a request, selecting a model associated with the request; and providing the selected model. Of course, other methods and combinations are provided as well.

Description:
RELATED APPLICATION DATA 
       [0001]    This application is a division of U.S. patent application Ser. No. 12/109,573, filed Apr. 25, 2008 (published as US 2008-0298632 A1), which claims the benefit of U.S. Patent Application No. 60/913,987, filed Apr. 25, 2007. Each of the above applications is hereby incorporated by reference in its entirety. 
     
    
     TECHNICAL FIELD 
       [0002]    The present disclosure relates generally to data hiding and digital watermarking. In some implementations the present disclosure relates to correcting optical scan or other capture distortion, e.g., such as lens blurring, focus errors and other scanning anomalies. In other implementations the present disclosure relates to managing models representing different expected distortions associated with a plurality of data captures. 
       BACKGROUND AND SUMMARY 
       [0003]    Digital watermarking—a form of steganography—is a process for modifying media content to embed a machine-readable code into the content. The content may be modified such that the embedded code is imperceptible or nearly imperceptible to the user, yet may be detected through an automated detection process. Most commonly, digital watermarking is applied to media such as images, audio signals, and video signals. However, it may also be applied to other types of data, including text documents (e.g., through line, word or character shifting, background texturing, etc.), software, multi-dimensional graphics models, and surface textures of objects. 
         [0004]    Digital watermarking systems typically have two primary components: an embedding component (or encoder) that embeds a watermark in media content, and a reading component (or detector) that detects and reads the embedded watermark. The embedding component embeds a watermark by altering data samples of the media content in the spatial, temporal or some other domain (e.g., Fourier, Discrete Cosine or Wavelet transform domains). The reading component analyzes target content to detect whether a watermark is present. In applications where the watermark encodes information (e.g., a plural-bit message), the reader extracts this information from the detected watermark. 
         [0005]    The present assignee&#39;s work in steganography, data hiding and digital watermarking is reflected, e.g., in U.S. Pat. Nos. 5,862,260, 6,408,082, 6,614,914 and 7,027,614, which are each hereby incorporated by reference. A great many other approaches are familiar to those skilled in the art. The artisan is presumed to be familiar with the full range of literature concerning steganography, data hiding and digital watermarking. 
         [0006]    In some cases digital watermarking and other machine-readable indicia (e.g., barcodes, data glyphs, etc.) may be detected from optical scan data, examples of which are disclosed in, e.g., U.S. Pat. Nos. 5,978,773, 6,522,770, 6,681,028, 6,947,571 and 7,174,031, which are each hereby incorporated by reference. Today&#39;s camera phones and other handheld readers present expanded decoding opportunities—and challenges. 
         [0007]    One challenge is providing handheld cameras (e.g., in a cell phone or other mobile device) to an army of users, with nearly each user having a different idea on proper focal length for image and video capture. 
         [0008]    In some cases a user may want to be close to a marked image or video to, e.g., capture high spatial frequency content (see  FIG. 1 ); but the close positioning often results in a captured image that is slightly out of focus (or blurred)—which may hamper detection of a machine-readable code contained or represented in the image. 
         [0009]    Thus, one inventive combination provides a method including: obtaining input data; altering a digital watermark or a digital watermarking process to pre-distort a digital watermark signal, wherein the altering is intended to counteract or compensate for expected distortion enabling machine-based detection of an embedded, pre-distorted digital watermark signal despite the expected distortion; and embedding the pre-distorted digital watermark signal in the input data. 
         [0010]    Another inventive combination provides a method including: obtaining a plurality of different models representing different expected distortion associated with a plurality of different data captures, the different data captures each resulting in distortion of a machine-readable signal; indexing the different models; upon receiving a request, selecting a model associated with the request; and providing the selected model. 
         [0011]    Still another inventive combination provides a method including: obtaining input data; altering a digital watermark or a digital watermarking process to pre-distort a digital watermark signal, wherein the altering is intended to counteract or compensate for expected distortion due to image capture or an image capture device, the altering enabling machine-based detection of an embedded, pre-distorted digital watermark signal despite the expected distortion; and embedding the pre-distorted digital watermark signal in the input data. 
         [0012]    Yet another inventive combination provides a method including: obtaining input data, the input data representing imagery captured with at least an optical lens, the input data comprising test data and a machine-readable signal; evaluating characteristics associated with the test data to determine information regarding lens blurring of the input data associated with the optical lens; adjusting the input data to compensate for or to correct the lens blurring based at least in part on the information; and analyzing the compensated for or corrected input data to obtain the machine-readable signal. 
         [0013]    Another inventive combination includes: obtaining input data, the input data representing imagery or video, the input data comprising test data and a machine-readable signal; determining characteristics associated with the test data to determine information regarding signal capture distortion of the input data; based on at least the characteristics, determining an amount of correction or counteracting to be applied to the input data; applying a determined amount of correction or counteracting to the input data; and analyzing corrected or counteracted input data to obtain the machine-readable signal. 
         [0014]    Further combinations, aspects, implementations, features, embodiments and advantages will become even more apparent with reference to the following detailed description, the accompanying drawings and claims. 
     
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         [0015]      FIG. 1  illustrates image capture with a cell phone including an optical sensor. 
           [0016]      FIG. 2A  illustrates an example target image. 
           [0017]      FIG. 2B  illustrates examples of adverse effects on focus (e.g., blurring) of the  FIG. 2A  example target image. 
           [0018]      FIG. 3  illustrates a plot diagram. 
           [0019]      FIG. 4  is a block diagram illustrating an embedding process. 
           [0020]      FIG. 5  is a diagram showing an approximation or estimate of lens blurring. 
           [0021]      FIG. 6  is a correction template or filter to compensate for the blurring shown in  FIG. 5 . 
           [0022]      FIG. 7  is a blurred signal ( FIG. 5 ) after compensation by the  FIG. 6  correction template or filter. 
           [0023]      FIG. 8A  is a diagram showing test data in a spatial frequency domain (with acceptable lens blur);  FIG. 8B  is another diagram showing test data in a spatial frequency domain (with unacceptable lens blur). 
           [0024]      FIG. 9  shows a digital watermarked gray patch. 
           [0025]      FIG. 10A  shows the watermarked gray patch of  FIG. 9  after capture distortion (e.g., lens blurring). 
           [0026]      FIG. 10B  shows the  FIG. 10A  patch after compensation or correction (e.g., using the  FIG. 6  correction). 
       
    
    
     DETAILED DESCRIPTION 
       [0027]    When an image is distorted (e.g., blurred, out of focus, etc.) by an optical capture device (e.g., a cell phone camera, PDA, digital camera, etc.) the loss of information from a target image ( FIG. 2A ) can be in the form shown in  FIG. 2B . For example, in some cases there may be a loss of information in some high spatial frequencies. 
         [0028]    The distortion may result in some high spatial frequencies of the target image being “out of phase” or otherwise distorted. 
         [0029]    In the case of a target image including or representing machine-readable information, e.g., a barcode, data glyph or digital watermark, some higher spatial frequency information corresponding to at least some of the machine-readable information is also out of phase, making reliable detection of the machine-readable information more difficult. 
         [0030]    Distortion such as blurring can be modeled or approximated by a convolution of a target image with, e.g., a Bessel function, as postulated by J. I. Yellott et al., “Correcting spurious resolution in defocused images,” SPIE 6492, pp. 649200-1-64920O-12 (2007), hereby incorporated by reference.  FIG. 3  illustrates a portion of a Bessel function, where the negative areas (shown with dotted boxes) correspond to a 90 degree phase shift. 
         [0031]    Some examples are now provided illustrating and describing compensation for distortion. 
         [0032]    In a first implementation a target image includes a digital watermark signal. But prior to being embedded in the target image, the digital watermark signal is pre-distorted with a phase shift, e.g., shifted 90 degrees relative to expected distortion. For example, if the distortion is likely to occur in high frequency areas such as discussed above with respect to convolving with the function shown in  FIG. 3 , high frequency components of the digital watermark are shifted 90 degrees. (An appropriate shift can be achieved by, e.g., convolving a digital watermark signal with an appropriate or corresponding Bessel function.) The pre-distorted watermark is then embedded into an image and printed, engraved or otherwise reproduced. Then, when expected distortion occurs (e.g., blurring due to image capture or focus errors), the high frequency watermark components are distorted (e.g., convolved) into a more readable form, enabling better watermark detection. 
         [0033]    As another example, if a digital watermark includes signal elements with values corresponding to [+1, 0, −1] at positions in a high frequency area, the signal is preferably phase shifted to compensate for excepted distortion. In the case of distortion modeled by a convolution with a target image and the function shown in  FIG. 3 , the signal is shifted 90 degrees resulting in signal elements with values corresponding to [−1, 0, +1] at the positions in the high frequency area. When this pre-distorted digital watermark signal is subjected to blurring during image capture, the pre-distorted signal elements are distorted again—but in an expected manner—resulting in signal elements with values corresponding to [+1, 0 −1], which correspond to the original signal elements. (The blurring shifts the signal essentially back to its original, pre-distorted form.) 
         [0034]      FIG. 4  illustrates a block diagram corresponding to one implementation of pre-distortion. 
         [0035]    An image  10  is obtained to be watermarked. The image  10  can be of any form, e.g., a color image, grayscale, a photograph, a graphic or artwork, video representation, etc. If the image  10  is in analog form, e.g., a printed image, it can be optically scanned or captured (e.g., with a digital camera or optical sensor) or otherwise converted into a digital image. 
         [0036]    A message  12  is provided (e.g., 1 or more bits). The message  12  is intended to be hidden within the image  10  with digital watermarking or steganography. A digital watermarking signal is generated by a digital watermark encoder  20  to represent or otherwise carry the message  12 . The digital watermark signal may be an additive signal, e.g., one that is added (or subtracted, multiplied, etc.) to the image  10 , may represent instructions or changes that should be made to the image  10  to carry the message, e.g., based on a key, and/or may include changes or modifications to frequency domain coefficients, and/or may include a random or pseudo-random component. Of course, other digital watermarking techniques may be used as well. 
         [0037]    The digital watermark signal is pre-distorted to compensate for expected distortion, e.g., by a distortion module  22 . Distortion module  22  may be incorporated into encoder  20  or encoder  20  may other wise communicate or cooperate with a distortion module  22 . Distortion module  22  accesses or otherwise determines a distortion model  24 . Distortion model  24  provides a template, mask or other instructions to be used by distortion module  22  when pre-distorting the digital watermark signal. The distortion model  24  can be based on or tailored to, e.g., the type of image  10 , the watermark message  12 , the expected distribution channel through which image  10  will travel, the expected type of image capture, optical lens system or a model of distortion introduced by such, digital watermark encoder  20 , human visual system (HVS), etc. In one implementation, the distortion module  22  (or watermark encoder  20 ) communicates with a database, model library or a network remote resource to access an appropriate model  24 . The term “appropriate” in this context implies that a model  24  is selected or obtained to compensate for expected distortion. Of course, the distortion module  22  may be pre-programmed with a default distortion model  24 , e.g., based on or tailored to expected distortion that a digital watermark will most likely encounter. 
         [0038]    Prior to embedding the digital watermark signal in the image  10 , the distortion module  22  accesses a distortion model  24  and pre-distorts a digital watermark signal to counteract or compensate for expected distortion associated with optical scanning or capture. Returning to an example discussed above, e.g., blurring or other distortion that can be modeled with convolving image  10  with a Bessel Function, the digital watermark is phase shifted in some or all higher frequencies, e.g., 90 degrees, to compensate for the expected distortion. 
         [0039]    The pre-distorted digital watermark—representing or carrying message  12 —is embedded in image  10  to yield a watermarked image  14 . The watermarked image  14  can then be printed or otherwise reproduced in an analog form. 
         [0040]    A printed (or otherwise fixed) image  16  that includes a pre-distorted digital watermark signal can then be optically scanned or captured—which introduces expected distortion. But since the digital watermark signal has been pre-distorted, the expected distortion transforms (e.g., modeled as a convolution with a determined function) the pre-distorted digital watermark signal into a more readable form. 
         [0041]    Optical data corresponding to the printed (or otherwise fixed), watermarked image  16  is provided to a digital watermark decoder to analyze the scan data and recover the message  12 . Of course, the digital watermark decoder may be co-located with an image sensor or otherwise located on a handheld device (e.g., a cell phone or personal digital assistant (PDA)). Still further, the digital watermark decoder may be remotely located from an image capture device. 
       Post-Capture Correction 
       [0042]    In other embodiments, a digital watermark is provided without the pre-distortion discussed above. Correction for lens blur is applied after image capture. These post-capture correction embodiments may allow for an even more imperceptible watermark, relative to a pre-distortion watermark. 
         [0043]    While post-capture creation may include some post-processing, it may be more flexible relative to the pre-distortion methods and systems discussed above, as many types or amounts of capture distortion can be corrected. 
         [0044]    A digital watermark is embedded in a host signal (e.g., an image or video). The embedded host signal is provided onto a surface (projected or rendered if video) or object (e.g., printed, engraved, etc.) and then an optical scan or digital image is captured of the surface or object. 
         [0045]    Recall, here, the digital watermark has not been pre-distorted as discussed above with respect to  FIG. 4 . So prior to watermark detection (or as an initial stage of the watermark detection) the captured image data is filtered or corrected. The filter adjusts the captured image data to compensate for the capture distortion (e.g., lens blur). 
         [0046]    To determine an amount or level of compensation or correction, one process quantifies or otherwise evaluates image blur in the image or video data. Such an evaluation or quantification helps to determine an appropriate amount of correction to be applied to captured image and video. 
         [0047]    In one example, test data can be introduced into an image or video. Characteristics associated with the test data can be evaluated in captured imagery to help determine an amount or level of lens blur. 
         [0048]    One example of test data is white noise (or other type of noise, e.g., a pseudo-random pattern, noise in some predetermined frequencies, etc.), which can be added to an image or video, e.g., as a part of or before/after digital watermark embedding. The white noise is preferably imperceptible in the image or video, but of course, there may be some applications where some perceptibility is allowed. After image or video capture, the white noise can be evaluated, e.g., in a spatial frequency domain as shown in  FIGS. 8A and 8B . 
         [0049]      FIG. 8A  illustrates a spatial frequency response for captured imagery with acceptable capture distortion. The term “acceptable” can be quantified by determining white noise signal magnitude at a predetermined spatial frequency or range of frequencies. If the signal magnitude is at or above a predetermined magnitude for the predetermined spatial frequency (or range of frequencies) the capture distortion can be deemed acceptable. 
         [0050]      FIG. 8B  illustrates a spatial frequency response for captured imagery with unacceptable capture distortion. The term “unacceptable” can be quantified by determining white noise signal magnitude at a predetermined spatial frequency or range of frequencies. If the signal magnitude is below a predetermined magnitude for the predetermined spatial frequency (or range of frequencies) the capture distortion can be deemed unacceptable. 
         [0051]    The signal magnitude level can also be used to help determine or estimate an amount or level of correction or adjustment that is needed as well. For example, if the magnitude is below (or between) a first pre-determined amount, a first level is determined; and if the magnitude is below (or between) a second pre-determined amount, a second level is determined. An amount or level of correction or adjustment corresponding to the first pre-determined amount or the second pre-determined amount is then selected. 
         [0052]    Another example of test data are predetermined structures. For example, a dark spot surrounded by a lighter (e.g., white) color is provided in the spatial domain in an image or video. The dark spot and surrounding lighter color are preferably subtle so as not to detract from the image or video. A spatial frequency plot of captured imagery including such test data is useful in evaluating capture distortion such as lens blur. The dot structure is approximated by a delta function in a spatial frequency domain. The magnitude and frequency location of the delta function can be used to determine an amount, type or level of capture distortion. Appropriate correction can be determined and applied to captured imagery if needed. 
         [0053]    Once an estimate of the lens blur is determined, it can be used to help correct capture distortion in captured imagery. For example, the estimate may be mapped against pre-determined levels of lens blurring, e.g., stored in a data structure associated with or cooperating with a digital watermark detector. In one implementation, estimated distortion is used to determine a level, type or degree of lens blurring or other distortion. A corresponding scale factor or other corresponding modification can then be obtained, e.g., from a data structure. An obtained scale factor or other modification can then be applied to a master correction template or filter. 
         [0054]    An approximation of one example lens blur is shown in  FIG. 5 , where pixel magnitude of the blur is illustrated. The illustrated blur spans more than 15 pixels, which would severely disrupt an image or video in the pixel area. A master correction template or filter is shown in  FIG. 6 , e.g., for a 1-dimensional signal. (Of course, other types of signals, e.g., 2-dimensional signal can be similarly corrected.) The master correction template or filter is adjusted, scaled or modified by a determined scale factor or other corresponding modification, if needed. The adjusted, scaled or modified template or filter is then applied to the captured imagery to counteract, adjust or correct image blur ( FIG. 7 ). A digital watermark detector (or other machine-readable symbology detector) analyzes the counteracted, adjusted or corrected captured imagery to read or detect a digital watermark (or other machine-readable code). 
         [0055]    In another implementation, a correction template or filter is generated upon evaluation of image blur. For example, based on an evaluation of test data in a spatial frequency domain, a correction filter is generated to counteract lens blur. A generated correction template or filter is applied to captured imagery to counteract, adjust or correct image blur. A digital watermark detector (or other machine-readable symbology detector) analyzes the counteracted, adjusted or corrected captured imagery to read or detect a digital watermark (or other machine-readable code). 
         [0056]    To further illustrate some of these processes, an original watermark was created by embedding a Digimarc Mobile digital watermark signal (provided by Digimarc Corporation headquartered in Beaverton, Oreg.) into a gray image patch. The digital watermark signal is readily detectable from the watermarked gray image patch ( FIG. 9 ). Lens blur (e.g., approximated by  FIG. 5 ) is applied to the watermarked gray patch ( FIG. 9 ), resulting in a blurred image patch shown in  FIG. 10A . The digital watermark is difficult (and perhaps impossible) to read from the distorted  FIG. 10A  patch. But, after compensation (e.g., as approximated in  FIG. 6 ) is applied to the distorted  FIG. 10A  image, e.g., as shown in  FIG. 6 , the watermark is readily detected from the resulting image shown in  FIG. 10B . My tests show that the compensated for gray patch ( FIG. 10B ) reads about 5 dB better than the  FIG. 10A  patch and approaches the detection results of a digital watermark embedded in the original gray image patch ( FIG. 9 ). 
       CONCLUDING REMARKS 
       [0057]    Having described and illustrated the principles of the technology with reference to specific implementations, it will be recognized that the technology can be implemented in many other, different, forms. 
         [0058]    The methods, processes, components, modules, generators and systems described above may be implemented in hardware, software or a combination of hardware and software. For example, digital watermark embedding or decoding processes may be implemented in a programmable computer or a special purpose digital circuit. Similarly, these watermarking processes may be implemented in software, firmware, hardware, or combinations of software, firmware and hardware. 
         [0059]    The methods, components and processes described above may be implemented in software programs (e.g., C, C++, Visual Basic, Java, executable binary files, etc.) executed from a system&#39;s memory (e.g., a computer readable medium, such as an electronic circuitry and/or an optical or magnetic storage device). 
         [0060]    The section headings are provided for the reader&#39;s convenience. Features found under one heading can be combined with features found under another heading. Of course, many other combinations are possible given the above detailed and enabling disclosure. 
         [0061]    We have used the terms “image” and “imagery” above. Both of these terms should be construed broadly enough to include images or video. 
         [0062]    The particular combinations of elements and features in the above-detailed embodiments are exemplary only; the interchanging and substitution of these teachings with other teachings in this and the above-mentioned U.S. patent documents are also contemplated.