Source: https://patents.google.com/patent/US9947327B2/en
Timestamp: 2018-09-18 22:28:31
Document Index: 629300147

Matched Legal Cases: ['Application No. 61', 'Application No. 2008321318', 'Application No. 2012241085', 'Application No. 2008321318', 'application No. 2008321318', 'Application No. 2', 'application No. 2858944', 'Application No. 2', 'Application No. 2', 'Application No. 2', 'application No. 2', 'Application No. 200880120098', 'Application No. 200880120098', 'Application No. 200880120098', 'Application No. 200880120098', 'application No. 08850597', 'Application No. 08850597', 'application No. 08850597', 'Application No. 2012241085', 'Application No. 2010', 'Application No. 2010']

US9947327B2 - Methods and apparatus for performing variable block length watermarking of media - Google Patents
Methods and apparatus for performing variable block length watermarking of media Download PDF
US9947327B2
US9947327B2 US13907286 US201313907286A US9947327B2 US 9947327 B2 US9947327 B2 US 9947327B2 US 13907286 US13907286 US 13907286 US 201313907286 A US201313907286 A US 201313907286A US 9947327 B2 US9947327 B2 US 9947327B2
US13907286
US20130268279A1 (en )
This patent arises from a continuation of U.S. patent application Ser. No. 12/361,991, filed Jan. 29, 2009 (now U.S. Pat. No. 8,457,951), and claims the benefit of U.S. Provisional Application No. 61/024,443, filed Jan. 29, 2008, the entireties of which are incorporated by reference.
FIG. 3A is a lookup table representing example block sizes representative of different information symbols for a given frequency index, wherein such a lookup table may be used by the block and index selector of FIG. 2.
In one example, the masking evaluator 204 conducts the masking evaluation by determining a maximum change in energy Eb or a masking energy level that can occur at any critical frequency band without making the change perceptible to a listener. The masking evaluation carried out by the masking evaluator 204 may be carried out as outlined in the Moving Pictures Experts Group-Advanced Audio Encoding (MPEG-AAC) audio compression standard ISO/IEC 13818-7:1997, for example. The acoustic energy in each critical band influences the masking energy of its neighbors and algorithms for computing the masking effect are described in the standards document such as ISO/IEC 13818-7:1997. These analyses may be used to determine for each audio block the masking contribution due to tonality (e.g., how much the audio being evaluated is like a tone) as well as noise like (i.e., how much the audio being evaluated is like noise) features in each critical band. The resulting analysis by the masking evaluator 204 provides a determination, on a per critical band basis, the amplitude of a code frequency that can be added to the audio 104 without producing any noticeable audio degradation (e.g., without being audible).
X ⁡ ( k ) = ∑ n = 0 n = N - 1 ⁢ ⁢ x ⁡ ( n ) ⁢ exp ⁡ ( - j ⁢ 2 ⁢ π ⁢ ⁢ kn N ) Equation ⁢ ⁢ 1
where x(n), n=0, 1, . . . N−1 are the time domain values of audio samples taken at sampling frequency Fs, X(k) is the complex spectral Fourier coefficient with frequency index k and 0≤k<N. Frequency index k can be converted into a frequency according to Equation 2.
0 ≤ k < N 2 - 1
where n=0 . . . 2015 is the time domain sample index within the block and Aw is the amplitude computed provided from a psycho-acoustic masking model of the masking evaluator. If the masking evaluation is performed using consecutive 512-sample overlapping sub-blocks, with a 256-sample overlap, Aw is varied from sub-block to sub-block and the code signal is multiplied by an appropriate window function to prevent edge effects. In such an arrangement, this synthesized sinusoid will only be fully observable when performing a spectral analysis using a block size of 2016 or, considering an 8 KHz sampling rate at the decoder 116, a block size of 336. However, the watermark signal can be chosen to be of arbitrary duration. In one example implementation, this watermark signal may be repeated in 9 consecutive blocks each the block size dictated by the block length and index selector 206. Note that the processing block size is chosen to support the use of commonly used psycho-acoustic models such as MPEG-AAC. For the example given here the signal will be embedded in 9 blocks of 2016 samples followed by an additional 288 samples to include all the 9 blocks of 2048 samples.
The example process 400 then determines the masking energy provided by the audio block (e.g., the block of 2016 samples) and, therefore, the corresponding ability to hide additional information inserted into the audio at the selected block size and frequency index (block 408). As explained above, the masking evaluation may include conversion of the audio block to the frequency domain and consideration of the tonal or noise-like properties of the audio block, as well as the amplitudes at various frequencies in the block. Alternatively, the evaluation may be carried out in the time domain. Additionally, the masking may also include consideration of audio that was in a previous audio block. As noted above, the masking evaluation may be carried out in accordance with the MPEG-AAC audio compression standard ISO/IEC 13818-7:1997, for example. The result of the masking evaluation is a determination of the amplitudes or energies of the code frequencies inserted at the specified block size and frequency index that are to be added to the audio block, while such code frequencies remain inaudible or substantially inaudible to human hearing.
However, when the requisite iterations to redundantly encode the code frequencies into audio blocks have completed (block 414), pads the samples if such padding is required (block 416). As explained above, the processing block size is chosen to support the use of commonly used psycho-acoustic models such as MPEG-AAC. For example, the code signal will be added into 9 blocks of 2016 samples that will be followed by an additional 288 samples of padding to include all 18,432 samples. Padding will effectively leave these 288 samples of the host audio unchanged.
θ = 2 ⁢ π ⁢ ⁢ k m N m ,
An alternate representation of the mathematics underlying Equations 5-10 is provided below in conjunction with Equations 11-18. Equation 11 shows a standard representation of a DFT, wherein xn are the time-domain real-valued samples, N is the DFT size, Yk,N (t) is a complex-valued Fourier coefficient calculated at time t from N previous samples {xn}, and k is the frequency (bin) index.
Y k , N ⁡ ( t ) = ∑ n = 0 N - 1 ⁢ ⁢ x n ⁢ e - 2 ⁢ π ⁢ ⁢ j ⁢ k N ⁢ n Equation ⁢ ⁢ 11
Y k , N ⁡ ( t ) = ∑ n = 0 M - 1 ⁢ ⁢ x n ⁢ e - 2 ⁢ π ⁢ ⁢ j ⁢ k N ⁢ n Equation ⁢ ⁢ 12
( e - 2 ⁢ π ⁢ ⁢ j ⁢ k N ⁢ 0 , e - 2 ⁢ π ⁢ ⁢ j ⁢ k N ⁢ 1 , … ⁢ , e - 2 ⁢ π ⁢ ⁢ j ⁢ k N ⁢ ( M - 1 ) ) .
To obtain a recursive expression for computing the value Yk,N (t) given in Equation 12, assuming that x0 is the oldest sample and xM is the newest incoming sample we find the result as shown in Equation 13 for the next discrete time instant t+1.
Y k , N ⁡ ( t + 1 ) = ∑ n = 0 M - 1 ⁢ ⁢ x n + 1 ⁢ e - 2 ⁢ π ⁢ ⁢ j ⁢ k N ⁢ n = ∑ m = 1 M ⁢ ⁢ x m ⁢ e - 2 ⁢ π ⁢ ⁢ j ⁢ k N ⁢ m ⁢ e 2 ⁢ π ⁢ ⁢ j ⁢ k N Equation ⁢ ⁢ 13
Y k , N ⁡ ( t + 1 ) = e 2 ⁢ π ⁢ ⁢ j ⁢ k N ⁡ [ ∑ m = 1 M ⁢ ⁢ x m ⁢ e - 2 ⁢ π ⁢ ⁢ j ⁢ k N ⁢ m + x 0 - x 0 ] = Equation ⁢ ⁢ 14 = e 2 ⁢ π ⁢ ⁢ j ⁢ k N ⁡ [ ∑ m = 0 M - 1 ⁢ ⁢ x m ⁢ e - 2 ⁢ π ⁢ ⁢ j ⁢ k N ⁢ m - x 0 + e - 2 ⁢ π ⁢ ⁢ j ⁢ k N ⁢ M ⁢ x M ] = Equation ⁢ ⁢ 15 = e 2 ⁢ π ⁢ ⁢ j ⁢ k N ⁡ [ Y k , N ⁡ ( t ) - x 0 + e - 2 ⁢ π ⁢ ⁢ j ⁢ k N ⁢ M ⁢ x M ] Equation ⁢ ⁢ 16
Re ⁢ ⁢ Y k , N ⁡ ( t + 1 ) = cos ⁡ ( 2 ⁢ π ⁢ k N ) ⁢ Re ⁢ ⁢ Y k , N ⁡ ( t + 1 ) - sin ⁡ ( 2 ⁢ π ⁢ k N ) ⁢ Im ⁢ ⁢ Y k , N ⁡ ( t + 1 ) - cos ⁡ ( 2 ⁢ π ⁢ k N ) ⁢ x 0 + cos ⁡ ( 2 ⁢ π ⁢ k N ⁢ ( M - 1 ⁢ ⁢ mod ⁢ ⁢ N ) ) ⁢ x M Equation ⁢ ⁢ 17 Im ⁢ ⁢ Y k , N ⁡ ( t + 1 ) = sin ⁡ ( 2 ⁢ π ⁢ k N ) ⁢ Re ⁢ ⁢ Y k , N ⁡ ( t + 1 ) + cos ⁡ ( 2 ⁢ π ⁢ k N ) ⁢ Im ⁢ ⁢ Y k , N ⁡ ( t + 1 ) - sin ⁡ ( 2 ⁢ π ⁢ k N ) ⁢ x 0 + sin ⁡ ( 2 ⁢ π ⁢ k N ⁢ ( M - 1 ⁢ ⁢ mod ⁢ ⁢ N ) ) ⁢ x M Equation ⁢ ⁢ 18
Consider for example the symbol S2 that may be encoded using any one of the tables 300, 330, or 360 of FIG. 3A, 3B, or 3C. If a symbol were encoded using the table 3A, the analyzer 514 would perceive a boost in the energy in the table of FIG. 8 in the cell corresponding to frequency index 40 and the symbol would be dictated by the block size having the maximum amplitude. Thus, the analyzer 514 would process the table of FIG. 8 to determine the maximum energy in the row corresponding to the frequency index 40. This may be carried out by normalizing the row in proportion to the maximum amplitude in the table row corresponding to frequency index 40. If, for example, the normalization reveals that the row entry corresponding to block size 336 (presuming the sampling rate at the decoder is 8 kHz, or one-sixth of the sampling frequency of the encoder) is the maximum, then the analyzer determines that the symbol S2 was encoded.
1. A method to encode auxiliary data in audio, the method comprising:
selecting, by executing an instruction with a processor and based on a first symbol in a code, a first frequency from a set of frequencies;
selecting a first block size by executing an instruction with the processor, the selection of the first block size based on the first symbol and the code, a combination of the first block size and the first frequency to represent the first symbol;
synthesizing a code frequency according to the first block size and the first frequency by executing an instruction with the processor;
combining, by executing an instruction with the processor, the code frequency with a first block of input audio samples of the audio having the first block size to form a block of encoded audio samples encoded with the first symbol, the code frequency and the first block of input audio samples overlapping in time; and
outputting the encoded audio samples to a device that produces an audio signal from the encoded audio samples.
2. The method of claim 1, further including padding audio samples adjacent the block of encoded audio samples with a number of unmodified samples corresponding to a difference between the first block size and a predetermined block size.
3. The method of claim 1, wherein the first symbol encoded in the block of encoded audio samples is detectable at the first frequency when the block of encoded audio samples is decoded according to the first block size and the first symbol is not detectable at the first frequency when the block of encoded audio samples is decoded according to a different block size.
4. The method of claim 1, further including accessing a lookup table based on the first symbol to select the first frequency and the first block size.
5. An apparatus to encode auxiliary data in audio, the apparatus comprising:
a selector to select, based on a first symbol in a code, a first frequency from a set of frequencies, and to select a first block size based on the first symbol and the code, a combination of the first block size and the first frequency to represent the first symbol; and
a combiner to:
synthesize a code frequency according to the first block size and the first frequency;
combine the code frequency with a first block of input audio samples of the audio having the first block size to form a block of encoded audio samples encoded with the first symbol, the code frequency and the first block of input audio samples overlapping in time; and
output the encoded audio samples to a device that produces an audio signal from the encoded audio samples.
6. The apparatus of claim 5, wherein the selector is to pad audio samples adjacent the block of encoded audio samples with a number unmodified samples corresponding to a difference between the first block size and a predetermined block size.
7. The apparatus of claim 5, wherein the first block size includes a number of samples of the audio.
8. The apparatus of claim 5, wherein the first symbol encoded in the block of encoded audio samples is detectable at the first frequency when the block of encoded audio samples is decoded using the first block size and the first symbol is not detectable at the first frequency when the block of encoded audio samples is decoded using a second block size different than the first block size.
9. The apparatus of claim 5, wherein the selector is to access a lookup table based on the first symbol to select the first frequency and the first block size.
10. An article of manufacture comprising machine readable instructions which, when executed, cause a processor to at least:
select, based on a first symbol in a code, a first frequency from a set of frequencies;
select a first block size based on the first symbol and the code, a combination of the first block size and the frequency to represent the first symbol;
11. The article of manufacture of claim 10, wherein the instructions are further to cause the machine to pad audio samples adjacent the block of encoded audio samples with a number of unmodified samples corresponding to a difference between the first block size and a predetermined block size.
12. The article of manufacture of claim 10, wherein the first symbol encoded in the block of encoded audio samples is detectable at the first frequency when the block of encoded audio samples is decoded according to the first block size and the first symbol is not detectable at the first frequency when the block of encoded audio samples is decoded according to a different block size.
13. The article of manufacture of claim 10, wherein the instructions are further to cause the machine to access a lookup table based on the first symbol to select the first frequency and the first block size.
14. The method of claim 1, further including converting the encoded audio samples into an analog form prior to being output.
sampling the audio to determine the input audio samples; and
converting the input audio samples to a frequency domain, the combining of the code frequency with the block of input audio samples being done in the frequency domain.
16. The apparatus of claim 5, wherein the combiner is to convert the encoded audio samples into an analog form prior to being output.
a sampler to sample the audio to determine the input audio samples; and
the combiner is to convert the input audio samples to a frequency domain, the combining of the code frequency with the block of input audio samples being done in the frequency domain.
18. The article of manufacture of claim 10, wherein the instructions are further to cause the machine to convert the encoded audio samples into an analog form prior to being output.
19. The article of manufacture of claim 10, wherein the instructions are further to cause the machine to:
sample the audio to determine the input audio samples; and
convert the input audio samples to a frequency domain, the combining of the code frequency with the block of input audio samples being done in the frequency domain.
US13907286 2008-01-29 2013-05-31 Methods and apparatus for performing variable block length watermarking of media Active 2029-10-03 US9947327B2 (en)
US12361991 Continuation US8457951B2 (en) 2008-01-29 2009-01-29 Methods and apparatus for performing variable black length watermarking of media
US15908457 Continuation US20180190301A1 (en) 2008-01-29 2018-02-28 Methods and apparatus for performing variable block length watermarking of media
US20130268279A1 true US20130268279A1 (en) 2013-10-10
US9947327B2 true US9947327B2 (en) 2018-04-17
WO2001078271A2 (en) 2000-04-06 2001-10-18 Nielsen Media Research, Inc. System and method for adding an inaudible code to an audio signal and method and apparatus for reading a code signal from an audio signal
JP2003053076A (en) 2001-08-13 2003-02-25 Brother Ind Ltd Sewing machine program preparing device
JPH10500263A (en) 1994-03-31 1998-01-06 セリディアン コーポレイション Apparatus and method for decoding with include code to an audio signal
JP2003500702A (en) 1999-05-25 2003-01-07 アービトロン インコーポレイテッド Decoding of information in the audio signal
Australian Government, "Examiner's First Report," issued in connection with Australian Patent Application No. 2008321318, dated Feb. 15, 2011, (2 pages).
Australian Government, "Examiner's Report No. 1", issued in connection with Australian Patent Application No. 2012241085, dated Sep. 16, 2013, (3 pages).
Australian Government, "Examiner's Report No. 2", issued in connection with Australian Patent Application No. 2008321318, dated Feb. 21, 2012, (2 pages).
Australian Government, "Notice of Acceptance", issued in connection with Australian patent application No. 2008321318, dated Oct. 18, 2012, (3 pages).
Canadian Intellectual Property Office, "First Office action", issued in connection with Canadian Patent Application No. 2,705,549, dated Aug. 29, 2012, (2 pages).
Canadian Intellectual Property Office, "Notice of Allowance," isssued in connection with application No. 2858944, on dated Feb. 1, 2017, 1 page.
Canadian Intellectual Property Office, "Office action", issued in connection with Canadian Patent Application No. 2,705,549, dated Jul. 25, 2014, (2 pages).
Canadian Intellectual Property Office, "Office action", issued in connection with Canadian Patent Application No. 2,705,549, dated Oct. 22, 2013, (2 pages).
Canadian Intellectual Property Office, "Office Action," issued in connection with Application No. 2,858,944, dated Feb. 16, 2015, 5 pages.
Canadian Patent Office, "Office action", issued in connection with Canadian patent application No. 2,858,944, dated Jan. 19, 2016, 4 pages.
Chinese Patent Office, "First Office action", issued in connection with Chinese Patent Application No. 200880120098.7, dated Aug. 29, 2011, (9 pages).
Chinese Patent Office, "Notice of Allowance," issued in connection with Chinese Patent Application No. 200880120098.7, dated Aug. 1, 2013, (4 pages).
Chinese Patent Office, "Second Office action," issued in connection with Chinese Patent Application No. 200880120098.7, dated Jun. 20, 2012, (4 pages).
Chinese Patent Office, "Third Office action," issued in connection with Chinese Patent Application No. 200880120098.7, dated Feb. 28, 2013, (7 pages).
European Patent Office, "Examination Report", issued in connection with European patent application No. 08850597.7-2225, dated Jun. 6, 2012, (9 pages).
European Patent Office, "Examination Report," issued in connection with Application No. 08850597.9, dated Feb. 25, 2015, 5 pages.
European Patent Office, "Intention to Grant," issued in connection with application No. 08850597.9, dated Dec. 5, 2016, 42 pages.
International Searching Authority, "International Search Report" issued in connection with International application No. PCT/US2008/079595, dated Jan. 19, 2009, (4 pages).
International Searching Authority, "Written Opinion" issued in connection with International application No. PCT/US2008/079595, dated Jan. 19, 2009, (11 pages).
IP Australia, "Notice of Acceptance," issued in connection with Application No. 2012241085, dated Oct. 8, 2014, 2 pages.
Japanese Patent Office, "Notice of Allowance" and Text of Office action, issued in connection with Japancese Patent Application No. 2010-534075, dated Oct. 9, 2013, (5 pages).
Japanese Patent Office, "Summary of Office Action" and Text of Office action, issued in connection with Japanese Patent Application No. 2010-534075, dated Jul. 3, 2012, (4 pages).
The International Bureau, "International Preliminary Report on Patentability," issued in connection with International application No. PCT/US2008/079595, dated May 18, 2010, (12 pages).
United Kingdom Radio Academy, Techcon Conference, http://www.ibs.org.uk/public/lineuparchive/2003/Aug-Sep/TECHCON_2003.pdf, 2 pages, Jul. 2003.
United States Patent and Trademark Office, "Final Office Action," issued in connection with U.S. Appl. No. 13/730,271, dated Mar. 4, 2016, 10 pages.
United States Patent and Trademark Office, "Final Office Action," issued in connection with U.S. Appl. No. 13/730,271, dated May 13, 2015, 29 pages.
United States Patent and Trademark Office, "Non-Final Office Action," issued in connection with U.S. Appl. No. 13/730,271, dated Dec. 16, 2014, 49 pages.
United States Patent and Trademark Office, "Non-Final Office Action," issued in connection with U.S. Appl. No. 13/730,271, dated Sep. 17, 2015, 10 pages.
United States Patent and Trademark Office, "Notice of Allowance", issued in connection with U.S. Appl. No. 12/249,619, dated Aug. 29, 2012, (10 pages).
United States Patent and Trademark Office, "Notice of Allowance", issued in connection with U.S. Appl. No. 12/249,619, dated Dec. 26, 2012, (7 pages).
United States Patent and Trademark Office, "Notice of Allowance", issued in connection with U.S. Appl. No. 12/361,991, dated Aug. 21, 2012, (7 pages).
United States Patent and Trademark Office, "Notice of Allowance", issued in connection with U.S. Appl. No. 12/361,991, dated Feb. 21, 2012, (8 pages).
United States Patent and Trademark Office, "Notice of Allowance", issued in connection with U.S. Appl. No. 12/361,991, dated Feb. 7, 2013, (5 pages).
United States Patent and Trademark Office, "Notice of Allowance," issued in connection with U.S. Appl. No. 13/730,271, dated May 27, 2016, 12 pages.
United States Patent and Trademark Office, "Office Action", issued in connection with U.S. Appl. No. 12/249,619, dated Mar. 21, 2012, (21 pages).
United States Patent and Trademark Office, "Supplemental Notice of Allowance", issued in connection with U.S. Appl. No. 12/249,619, dated Sep. 21, 2012, (4 pages).
United States Patent and Trademark Office, "Supplemental Notice of Allowance", issued in connection with U.S. Appl. No. 12/361,991, dated Feb. 27, 2013, (2 pages).
United States Patent and Trademark Office, "Supplemental Notice of Allowance", issued in connection with U.S. Appl. No. 12/361,991, dated May 24, 2012, (2 pages).
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:SRINIVASAN, VENUGOPAL;TOPCHY, ALEXANDER;SIGNING DATES FROM 20090527 TO 20090528;REEL/FRAME:030903/0189