AUDIO WATERMARKING BASED ON SINUSOIDAL AMPLITUDE MODULATION
Akira Nishimura
Tokyo University of Information Sciences,
Department of Media and Cultural Studies, Faculty of Informatics,
1200–1, Yatoh-cho, Wakaba-ku, Chiba-city, Chiba 265-8501, Japan
ABSTRACTA new watermarking system based on amplitude modulation is proposed. Sinusoidal amplitude modulationsat relatively low modulation frequencies applied to theneighboring subband signals in opposite phase are usedas the carrier of embedded information. The embedded information is encoded in the form of the relative
phase diﬀerences between the amplitude modulations applied to several groups of subband signals. Extraction of
the amplitude modulations from the watermarked signal is performed by calculating the logarithmic ratio of
the amplitude envelopes extracted from the neighboring subband signals. Deterioration of the sound quality
resulting from the watermarking is barely detectable because modulation detection interference in the human
auditory system and masking in the frequency-domain
disturb the perceptual detectability of the additional amplitude modulations.
1. INTRODUCTION
Recent studies on audio watermarking have resulted insigniﬁcant progress in inaudibility and reliability. Audio watermarking techniques have achieved robustness
against, for example, MPEG compression, additive noise,
low pass ﬁltering, pitch change and time scale modiﬁcation [1, 2]. The inaudibility and reliability of most techniques depend on the acoustic characteristics of the hostsignal. However, few studies have conﬁrmed the reliability of the watermarking techniques using a number ofmusical samples or actual sample sounds. In addition,robustness against emission in closed spaces, i.e., reverberation and reﬂection disturbance, has rarely been considered. Robustness against reverberations and reﬂections are important for audio watermarking techniques
for live performances.
In the present paper, a new watermarking system
based on subband coding using amplitude modulation
is proposed. The system is robust for perceptual audio codings such as MP3 or RealAudio, reﬂections andreverberations, additive Gaussian or colored noise and1­4244­0469­X/06/$20.00 ©2006 IEEE

spectral modiﬁcations. A key is required for embedding
and decoding, and no host signal is required for decoding.
Robustness testing has been conducted for 100 pieces ofmusic of various genres.
2. AUDIO WATERMARKING SYSTEM BASED ON
AMPLITUDE MODULATION
2.1. Embedding method
At the beginning of the embedding process, a host signal
H(t) of the length of a data frame period is separated
into 2n subband signals hm (t) by a ﬁlterbank with equal
bandwidth:2n
H(t) =hm (t).
(1)
m=1Sinusoidal amplitude modulations (SAMs) at relatively
low modulation frequency (f -Hz) are applied to the neighboring subband signals h2m (t) and h2m+1 (t) in opposite
phase. An embedding key deﬁned by a known pseudorandom number generator arbitrary classiﬁes n subband
pairs into k subband groups. It also deﬁnes random initial phase angles r(m) of SAM for each subband pair.
The output of an amplitude modulated subband pairxim (t) which belongs to the i-th subband group is givenbyxim (t)= h2m (t)(1 + A(m) sin(2πf t + r(m) + p(i))) +h2m+1 (t)(1 − A(m) sin(2πf t + r(m) + p(i))), (2)where A(m) is the depth of SAM of the m-th subband
pair. Embedded information is encoded by Phase Shift
Keying (PSK), deﬁned as the diﬀerences between phaseangles of SAM of the ﬁrst subband group and that of thei-th subband group, p(1) and p(i) (i = 2, ..., k). 4-PSK
encodes 2-bit information (Di = 0, 1, 2, 3) to every π/2phase angle of p(i).

IV ­ 797

0p(i) =πDi2i = 1;
i = 2, ..., k.

(3)

ICASSP 2006

As a result, 2(k − 1) bits information is embedded perdata frame period. Multiplex watermarking can be applied using diﬀerent modulation frequencies simultaneously. Finally, a watermarked signal X(t) is obtained bysumming up all amplitude modulated signals xm (t).
nX(t) =xm (t).

(4)

m=1

The embedding key determines the initial phases of
the SAMs applied to all pairs of the neighboring subbands and also determines which neighboring subbands
belong to the same subband group. Combined with thesmall SAM depth applied to each subband, the embedded watermark oﬀers high concealment.
Synchronization of the data frames is achieved by inverting the relative phase of the SAMs between successive frames for the ﬁrst subband group.
2.2. Extraction method
The amplitude envelopes of the subbands Em (t)(m =1, ..., 2n) of the watermarked frame signal can be derived
from the amplitude spectrum of the half-overlapped running FFT, where t is the unit of time deﬁned by a halfof the FFT length. Extraction of the embedded modulation waveform Gm (t) is performed by calculating thelogarithmic ratio of the amplitude envelopes extracted
from the neighboring subband signals.
Gm (t) = log

E2m (t).
E2m+1 (t)(5)

Synchronized addition of the AM waveforms for the ith subband group is conducted after compensation of theinitial phase diﬀerences r(m) in order to emphasizes themodulation waveform (Fig.1). Consequently, the modulation depth A(m) in the embedding process can bekept small. Initial phase diﬀerences between the ﬁrst
and the i-th subband, that is embedded information, areobtained by comparing phase angles of the FFT spectra
calculated from the extracted AM waveforms.
phase shift keying

synchronized additioncompensation of
initial phase difference
modulation detection

2.3. Finding a starting point of the data frame
Before decoding PSK data, a starting point of the embedded data frame must be detected. A rectangular temporal window W (t) of the data frame length T is iteratively applied to the modulation waveform G1 (t) extracted from the ﬁrst subband group. A starting pointof the windowing is denoted u in Eq. 6. Then, F (u) is
derived by subtracting the synchronized addition of themodulation waveforms in the odd order windows fromthe synchronized addition of the modulation waveforms
in the even order windows.
Hu={G1 (u), G1 (u + 1), ..., G1 (u + T − 1)} (6)

Hu+2vT −v=0

Hu+(2v+1)T(7)v=0The Fourier amplitude of F (u) which corresponds to themodulation frequency f , referred as AMPf (F (u)), exhibits a maximum when the positions of the window
completely overlap with the positions of the frame. Consequently, the starting point y of the data frame is givenby(8)y = argmax AMPf F (u).
u2.4. Method used to decide the depth of the SAMs
The depth of SAM A(m) for the m-th subband pair is determined relative to the weighted sum of the magnitudeof the amplitude ﬂuctuations around the modulation frequency (MF) observed in the amplitude envelope of thesubband signal h2m + h2m+1 . In the present study, theweighting function of the amplitude ﬂuctuation, that is,the characteristics of the modulation ﬁlter, is –10 dB at
MF/2 and 2MF, and 0 dB at MF. The intensity of thewatermarking is deﬁned as the modulation depth relative
to the amplitude at the output of the modulation ﬁlter.
A(m) is re-calculated for every two to four modulation
periods.
This decision process takes into account modulation
masking in the human auditory system [3]. Since theperceptual detectability of amplitude modulation is disturbed by the inherent amplitude ﬂuctuations containedin the host signal, and the sidebands induced by the SAM
are apt to be masked by the musical signal, deteriorationof the sound quality induced by the watermarking is considered to be barely detectable.
3. COMPUTER SIMULATION OF
WATERMARKING AND EXTRACTION...

Frequency

W (t)G1 (t)F (u) =

subband grouping
based on a key

Fig. 1. Overview of watermark extraction.

=A computer simulation was conducted to conﬁrm thatthe watermarking was successful for 100 pieces of musicIV ­ 798

in a music database containing various types of music(RWC-MDB-2001-G)[4]. Amplitude modulated watermarks of 3 Hz or 12 Hz are embedded in the initial 60second left-channel signal of each piece of music. The
details of the simulated conditions and parameters areshown in Table 1.
Modiﬁcations to the watermarked music included MP3
encoding and decoding, RealAudio8, reverberation, additive Gaussian noise and amplitude quantization conversion. Reverberation was applied by convolving a random Gaussian sequence with exponential decay. The
level of additive Gaussian noise was determined relative
to the level of the sinusoid of maximum amplitude in16-bit quantization.
The bit detection rate, deﬁned as the ratio of thenumber of correct bits to the total number of embedded
bits, is shown in Table 2. The results revealed that thewatermarking was robust for the perceptual audio codings, such as MP3 or RealAudio. More than half of thepieces of music retained 100% correct watermarks after
MP3 encoding and decoding at greater than 48 kbps foran embedding intensity of –10 dB. The same was true
for reverberations, except for the 12-Hz modulation frequency. Watermarks of low SAM frequency are robust
for reverberations because valleys of SAM are diﬃcult toﬁll by reverberation.
Table 1. Embedding conditions and parameters.
parameters values
sampling freq. 44100 Hzembedding region < 11025 Hzsubband pairs (n) 32
subband groups (k) 3frame period 5 s (3-Hz), 2 s (12-Hz)
bit rate 0.8 bps (3-Hz), 2 bps (12-Hz)
4. PERCEPTUAL DETECTABILITY
Perceptual detection thresholds of the intensity of thewatermarking were obtained by the transformed up-down
method with the AXB discrimination task (70.7% threshold) for three 5-seconds musical signals, which were relatively discriminable among the other pieces of music in
the music genre database. Watermarks of 3-Hz and 12Hz amplitude modulations were simultaneously appliedto the host signal. The other embedding conditions andparameters are the same as the computer simulation andare shown in Table 1. Three trained listeners participated in the experiment. All stimuli were presented diotically through headphones (STAX Lambda Nova Classic). The average detection thresholds obtained from atleast three measurements are shown in Table 3.
Computer simulation of watermark extraction for thesethree musical signals revealed that bit correction rates

were more than 95% at an embedding intensity of –20
dB after the modiﬁcations tested in the previous section,
except for 32 kbps perceptual codecs and reverberation
at 12-Hz modulation embedding.
These results conﬁrmed that the minimal required
magnitude for extraction is lower than the perceptual
detection thresholds of the watermark. In other words,
the eﬀective embedded watermarks were barely perceptible.
Table 3. Detection thresholds of the watermark in ﬁve
seconds of music from RWC-MDB-G-2001. (dB)
Track no. subj. 1 subj. 2 subj. 3
No. 45
-21.7
-11.9
-14.7
No. 69-10.5
-11.1
-16.4
No. 87
-18.6
-18.1
-15.7
5. DISCUSSION
Since the watermarking method is still relatively weakwith respect to malicious attacks such as pitch change
and time scale modiﬁcation, there is room for improvement. Theoretically, pitch change is more serious thantime scale modiﬁcation, because pitch change shifts frequency boundaries between neighboring subbands andcauses negation of amplitude modulation in the subbands. The amount of pitch change can be estimatedby the decrease of the intensity of extracted modulationin high frequency region. Eﬃcient step-by-step searchingby expansion and compression along both the frequency
and time axes to extract salient amplitude modulations
should be introduced in the future.
Although the robustness for spectral modiﬁcations,additive colored noise and lowpass ﬁltering are not practically conﬁrmed in the present paper, these modiﬁcations will not seriously damage the present watermarking method. Spectral modiﬁcation may change relative
levels between neighboring subbands, however, the levelchange only aﬀects a direct current component in theextracted SAM waveform after calculation of the logarithmic ratio between amplitude envelops of neighboring subbands. Since embedded amplitude modulations
are equally applied to the wide frequency range of thehost signal, watermarks survive after lowpass ﬁltering or
additive colored noise as long as the eﬀective frequency
regions for sound quality are not disturbed.
From non-oﬃcial listening tests on several pieces ofwatermarked music, perceptual quality degradation ofmusical signals that contain sounds of several musicalinstruments at the same time were diﬃcult to detect.
Perceptual quality degradation was not detectable evenwhen the intensity of watermarking was greater than 0dB. The present decision scheme of the modulation depth
is based on amplitude ﬂuctuations within the channel.

IV ­ 799

Table 2. Bit detection rate after various modiﬁcations to the watermarked music. Numbers in parentheses indicate
the number of pieces of music having a detection rate of 100% out of 100 pieces of music.
Mod. freq. and Mod. ratio
bit rate
32 kbps
48 kbps
64 kbpsbit rate
32 kbps
44 kbps
reverberation time
1 sec.
0.3 sec.
noise level (overall)–50 dB–70 dB
resolution
8 bit

3 Hz, –10 dB

3 Hz, –20 dB 12 Hz, –10 dB 12 Hz, –20 dB
MP3 encoding and decoding
94.9% (29)
60.3% (0)90.8% (1)
70.5% (0)99.2% (86)83.3% (20)
99.0% (47)90.2% (9)99.8% (96)93.1% (54)
99.9% (92)96.2% (36)
RealAudio 8 encoding and decoding
96.0% (6)90.6% (2)98.0% (13)89.7% (0)95.9% (15)
93.4% (5)
98.5% (17)
94.7% (0)reverberation
99.1% (76)
89.0% (22)71.7%(0)64.0% (0)99.9% (97)96.5% (52)
97.2%(18)
86.8% (10)addition of white noise
98.9% (88)79.3% (2)
95.9% (41)87.8% (1)
99.5% (86)93.8% (49)99.8% (84)
97.3% (33)
sampling bit conversion
97.0% (59)
91.0% (5)96.3% (51)
94.5% (1)

However, it is well known that amplitude ﬂuctuationsof remote channels can interfere with modulation detection, which is called modulation detection interference(MDI)[5]. Therefore, MDI makes perceptual detection ofembedded amplitude modulation diﬃcult, and wide variation in the detection thresholds observed among various
kinds of music may be partly due to MDI. Considering
the results of the perceptual detectability test, detailed
modeling of the MDI in the human auditory system [6]will enable more information embedding without perceptual deterioration of the sound quality.
The present watermarking method does not include
a scheme for error correction to embedded data. The
present paper reports the ﬁrst proposal of a new methodfor data hiding and embedding. Practical implementation should adopt eﬀective block or convolutional coding
in order to achieve high reliability.
6. SUMMARY
A new watermarking system based on amplitude modulation is proposed. The system is robust against perceptual audio codings, reverberations and additive Gaussian
noise. Detecting the existence of the watermarking isdiﬃcult without knowing the key used in the encoding
process. No host signal is required for the decoding process. Perceptual detection of the quality degradation of
the watermarked music is quite diﬃcult if the watermark
is embedded with an appropriate intensity.
Acknowledgments: This work is partly supported by agrant from the Okawa Foundation for Information andTelecommunications.

7. REFERENCES[1] Ryuki Tachibana, Shuichi Shimizu, Seiji Kobayashi,
and Taiga Nakamura,“Au audio watermarking method robust against time- and frequencyﬂuctuation,” in Proc. of Security and Watermarkingof Multimedia Contents III, SPIE, 2001, vol. 4314,
pp. 104—115.
[2] Wei Li and Xiangyang Xue, “Audio watermarking basd on music content analysis: Robust against
time scale modiﬁcation,” in Digital Watermarking:
Second International Workshop, IWDW 2003, LNCS
2939, 2004, pp. 289—300, Springer-Verlag.
[3] T. Houtgast, “Frequency selectivity in amplitudemodulation detection,” J. Acoust. Soc. Am., vol. 85,pp. 1676—1680, 1989.
[4] Masataka Goto, Hiroki Hashiguchi, Takuichi
Nishimura, and Ryuichi Oka, “Rwc music database:Music genre database and musical instrument sounddatabase,” in Proceedings of the 4th International
Conference on Music Information Retrieval (ISMIR2003), 2003, pp. 229—230.
[5] William A. Yost and Stanley Sheft, “Modulation detection interference: Across-frequency processing andauditory grouping,” Hearing Research, vol. 79, pp.
48—58, 1994.
[6] Torsten Dau, Birger Kollmeier, and Armin
Kohlrausch,“Modeling auditory processing ofamplitude modulation. ii. spectral and temporalintegration,” J. Acoust. Soc. Am., vol. 102, no. 5,
pp. 2906–2919, 1997.

IV ­ 800