# Robust and Blind Audio Watermarking Algorithm in Dual Domain for Overcoming Synchronization Attacks

Authors: Qiuling Wu; Aiyan Qu; Dandan Huang
Journal: Mathematical Problems in Engineering
 (2020)

---

## Abstract

How to effectively resist synchronization attacks is the most challenging topic in the research of robust watermarking algorithms. A robust and blind audio watermarking algorithm for overcoming synchronization attacks is proposed in dual domain by considering time domain and transform domain. Based on analysing the characteristics of synchronization attacks, an implicit synchronization mechanism (ISM) is developed in the time domain, which can effectively track the appropriate region for embedding and extracting watermarks. The data in this region will be subjected to discrete cosine transform (DCT) and singular value decomposition (SVD) in turn to obtain the eigenvalue that can be utilized to carry watermarks. In order to extract the watermark blindly, the eigenvalue will be quantized. Genetic algorithm (GA) is utilized to optimize the quantization step to balance both transparency and robustness. The experimental results confirm that the proposed algorithm not only withstands various conventional signal processing operations but also resists malicious synchronization attacks, such as time scale modification (TSM), pitch-shifting modification (PSM), jittering, and random cropping. Especially, it can overcome TSM with strength from −30% to +30%, which is much higher than the standard of the International Federation of the Phonographic Industry (IFPI) and far superior to the other algorithms in related papers.


---

## Body

## 1. Introduction

### 1.1. Related Works

With the rapid development of network and computer technology, people edit, modify, store, and disseminate audio media easily by using various audio editing software [1–3]. While the editing software brings us convenience, it also makes unauthorized users perform a variety of infringements on the audio media, such as malicious tampering, forgery, deletion, and unauthorized distribution. Sometimes, these infringements not only jeopardize the safety of personal property and the credibility of the audio media but also may even endanger national public safety in acute cases [4–6]. How to effectively protect the security of those audio media has become a research hotspot in information security, communication, and some related fields. Robust audio watermarking algorithm pays much attention to improve its ability for preventing watermarks hidden in the audio from being destroyed under complex environments [7, 8], so it must not only be able to withstand the conventional signal processing operations encountered when using those audio media normally but also need to be extremely resistant to many malicious synchronous attacks that may cause the structure of the audio media to change.Synchronization attacks may cause serious damage to the structure of the audio, resulting in the extraction failure due to the inaccuracy of the embedding region [9–12], so they have become the most challenging attacks in the research of audio watermarking algorithms [13–15]. Hu et al. [16] proposed an audio watermarking algorithm based on lifting wavelet transform. The authors claimed that the algorithm had good robustness to some conventional signal processing attacks and synchronization attacks, and its payload capacity reached 43.07 bps when SNR was over 21 dB. However, it can be seen from the experimental results that the algorithm robustness still needs to be improved when resisting TSM. Xiang and Huang [17] designed an audio watermarking algorithm with a constant watermark synchronization mechanism according to the insensitivity of the histogram shape of audio media. Hu and Chang [18] proposed a self-synchronous audio watermarking algorithm based on discrete wavelet transformation (DWT) and DCT. This algorithm concealed the synchronous signal in the first approximation sub-band and recalibrated the embedding position by extracting the zero-crossing point of the synchronous signal. The experimental results showed that this algorithm was effective for some synchronization attacks but poor for some signal processing attacks. Wang et al. [19] proposed a robust audio watermarking algorithm which utilized the invariance of exponential moment to enhance its robustness. However, it was poor for amplitude scaling and MP3 compression as shown in experimental results. Yuan et al. [20] put forward an audio watermarking algorithm that detected the mel-cepstrum coefficient as a synchronous signal when extracting the watermark in the DWT domain. Wang et al. [21] proposed a robust audio watermarking algorithm based on empirical mode decomposition. In this algorithm, the audio was evenly segmented into numerous fragments, and then each audio fragment was separated into two parts. One part was utilized to embed the synchronization code, and the other part was used to embed the watermark in the residue of higher-order statistics after empirical mode decomposition. If synchronization codes could not be accurately acquired, watermark extraction would fail, which was a fatal shortcoming of this algorithm. Chen et al. [22] proposed an audio watermarking algorithm that embedded the watermark into the low-frequency coefficients of the audio in the DWT domain. This algorithm enhanced its robustness by increasing the embedding depth, but this behaviour also led to the low transparency. In general, audio watermarking algorithms with the ability to resist synchronization attacks must have an effective synchronization mechanism, which can be used to track the embedding position [23, 24]. However, most existing algorithms are usually robust to only one or two of these attacks, and some algorithms even lose robustness to conventional signal processing operations due to their excessive pursuit of robustness to some synchronization attacks. In addition, how to balance the overall performance of the algorithm by optimizing the parameters of the designed algorithm is also an issue with research significance.


### 1.2. Contributions

Based on the above introduction, we can see that there are still many problems to be solved in antisynchronization attacks. Our contributions in this paper are as follows.(1)
An ISM is developed to effectively search for the appropriate embedding region when embedding watermarks and to automatically track the region where the watermark is located when extracting watermarks. Based on analysing the characteristics of synchronization attacks, it is found that the shape of the voiced frame almost has not changed after being subjected to TSM, so the proposed ISM takes the sample point with the largest amplitude in the voiced frame as the synchronization mark to identify the embedding region and extracting region. When embedding watermarks, the appropriate region will be searched out from the voiced frames by using ISM, and then the data in the chosen embedding region will be further operated to carry watermarks. When extracting watermarks, ISM can automatically track the region where the watermark is located.(2)
GA is utilized to optimize the key algorithm parameter to balance both transparency and robustness. The data in the embedding region will be processed by DCT and SVD in turn to obtain the eigenvalue that can be used to carry watermarks. In order to extract the watermark blindly, the eigenvalue is quantized when embedding or extracting the watermark, so the quantization step is an important parameter, which directly affects the transparency and robustness of the algorithm. We propose an optimal audio watermarking algorithm using GA to further enhance the overall performance of this algorithm.Besides, this algorithm adopts several additional measures to improve the robustness, such as twice even segmentation to the audio, and the operation that embeds the same watermark into three voiced frames.The remainder of this paper is organized as follows. In Section1, we review some related works about the existing audio watermarking algorithms which can overcome synchronization attacks and then introduce our contributions in this proposed algorithm. Section 2 describes the proposed ISM and shows the implementation flow chart in detail. The principle of the proposed audio watermarking algorithm will be elaborated in Section 3, and this section will be divided into four subjects, including the embedding principle, the extracting principle, optimization of the quantization step, and the measure to further improve robustness. Section 4 evaluates the performance of this proposed algorithm and compares their performance with other algorithms in recent years. Finally, Section 5 draws up the conclusion and gives the possible future research task.


## 1.1. Related Works

With the rapid development of network and computer technology, people edit, modify, store, and disseminate audio media easily by using various audio editing software [1–3]. While the editing software brings us convenience, it also makes unauthorized users perform a variety of infringements on the audio media, such as malicious tampering, forgery, deletion, and unauthorized distribution. Sometimes, these infringements not only jeopardize the safety of personal property and the credibility of the audio media but also may even endanger national public safety in acute cases [4–6]. How to effectively protect the security of those audio media has become a research hotspot in information security, communication, and some related fields. Robust audio watermarking algorithm pays much attention to improve its ability for preventing watermarks hidden in the audio from being destroyed under complex environments [7, 8], so it must not only be able to withstand the conventional signal processing operations encountered when using those audio media normally but also need to be extremely resistant to many malicious synchronous attacks that may cause the structure of the audio media to change.Synchronization attacks may cause serious damage to the structure of the audio, resulting in the extraction failure due to the inaccuracy of the embedding region [9–12], so they have become the most challenging attacks in the research of audio watermarking algorithms [13–15]. Hu et al. [16] proposed an audio watermarking algorithm based on lifting wavelet transform. The authors claimed that the algorithm had good robustness to some conventional signal processing attacks and synchronization attacks, and its payload capacity reached 43.07 bps when SNR was over 21 dB. However, it can be seen from the experimental results that the algorithm robustness still needs to be improved when resisting TSM. Xiang and Huang [17] designed an audio watermarking algorithm with a constant watermark synchronization mechanism according to the insensitivity of the histogram shape of audio media. Hu and Chang [18] proposed a self-synchronous audio watermarking algorithm based on discrete wavelet transformation (DWT) and DCT. This algorithm concealed the synchronous signal in the first approximation sub-band and recalibrated the embedding position by extracting the zero-crossing point of the synchronous signal. The experimental results showed that this algorithm was effective for some synchronization attacks but poor for some signal processing attacks. Wang et al. [19] proposed a robust audio watermarking algorithm which utilized the invariance of exponential moment to enhance its robustness. However, it was poor for amplitude scaling and MP3 compression as shown in experimental results. Yuan et al. [20] put forward an audio watermarking algorithm that detected the mel-cepstrum coefficient as a synchronous signal when extracting the watermark in the DWT domain. Wang et al. [21] proposed a robust audio watermarking algorithm based on empirical mode decomposition. In this algorithm, the audio was evenly segmented into numerous fragments, and then each audio fragment was separated into two parts. One part was utilized to embed the synchronization code, and the other part was used to embed the watermark in the residue of higher-order statistics after empirical mode decomposition. If synchronization codes could not be accurately acquired, watermark extraction would fail, which was a fatal shortcoming of this algorithm. Chen et al. [22] proposed an audio watermarking algorithm that embedded the watermark into the low-frequency coefficients of the audio in the DWT domain. This algorithm enhanced its robustness by increasing the embedding depth, but this behaviour also led to the low transparency. In general, audio watermarking algorithms with the ability to resist synchronization attacks must have an effective synchronization mechanism, which can be used to track the embedding position [23, 24]. However, most existing algorithms are usually robust to only one or two of these attacks, and some algorithms even lose robustness to conventional signal processing operations due to their excessive pursuit of robustness to some synchronization attacks. In addition, how to balance the overall performance of the algorithm by optimizing the parameters of the designed algorithm is also an issue with research significance.


## 1.2. Contributions

Based on the above introduction, we can see that there are still many problems to be solved in antisynchronization attacks. Our contributions in this paper are as follows.(1)
An ISM is developed to effectively search for the appropriate embedding region when embedding watermarks and to automatically track the region where the watermark is located when extracting watermarks. Based on analysing the characteristics of synchronization attacks, it is found that the shape of the voiced frame almost has not changed after being subjected to TSM, so the proposed ISM takes the sample point with the largest amplitude in the voiced frame as the synchronization mark to identify the embedding region and extracting region. When embedding watermarks, the appropriate region will be searched out from the voiced frames by using ISM, and then the data in the chosen embedding region will be further operated to carry watermarks. When extracting watermarks, ISM can automatically track the region where the watermark is located.(2)
GA is utilized to optimize the key algorithm parameter to balance both transparency and robustness. The data in the embedding region will be processed by DCT and SVD in turn to obtain the eigenvalue that can be used to carry watermarks. In order to extract the watermark blindly, the eigenvalue is quantized when embedding or extracting the watermark, so the quantization step is an important parameter, which directly affects the transparency and robustness of the algorithm. We propose an optimal audio watermarking algorithm using GA to further enhance the overall performance of this algorithm.Besides, this algorithm adopts several additional measures to improve the robustness, such as twice even segmentation to the audio, and the operation that embeds the same watermark into three voiced frames.The remainder of this paper is organized as follows. In Section1, we review some related works about the existing audio watermarking algorithms which can overcome synchronization attacks and then introduce our contributions in this proposed algorithm. Section 2 describes the proposed ISM and shows the implementation flow chart in detail. The principle of the proposed audio watermarking algorithm will be elaborated in Section 3, and this section will be divided into four subjects, including the embedding principle, the extracting principle, optimization of the quantization step, and the measure to further improve robustness. Section 4 evaluates the performance of this proposed algorithm and compares their performance with other algorithms in recent years. Finally, Section 5 draws up the conclusion and gives the possible future research task.


## 2. ISM for Tracking Embedding Region

Synchronization attacks may cause the position of the data in the audio to shift, which may lead to extraction failure because the location of the watermark cannot be obtained accurately [25]. Therefore, it is very important to design an effective synchronization mechanism for tracking the embedding region. If the data in the voiced frames are modified too much, the audio may not be used normally because of the obvious degradation of audio quality, so synchronization attack usually only modifies the data in redundant frames, but not in voiced frames. TSM attacks by 10% and −10% are applied to an audio clip, respectively, and the waveform comparison is illustrated in Figure 1. It can be seen from the pictures that the absolute positions of the two voiced frames in this audio clip all have shifted on the time axis, but their shapes do not change much in the process of being stretched in Figure 1(b) or compressed in Figure 1(c), so it is relatively safe to conceal watermarks into these voiced frames.Figure 1
Audio waveform with the searched embedding region under TSM. (a) The embedding regions without attack. (b) The embedding regions under −10% TSM. (c) The embedding regions under +10% TSM.
(a)(b)(c)If the watermark is only embedded in voiced frames and the embedding region in the audio is independent of the absolute position of the audio data on the time axis, it will greatly improve the algorithm’s ability to withstand synchronization attacks. As long as the embedding region can be effectively tracked, the watermark will be accurately extracted. Based on the above analysis, an ISM is developed in our study, which can search out the appropriate embedding region when embedding watermarks and can effectively track the extracting region where the watermark is located when extracting watermarks. As shown in Figure1, the regions between the two red dashed vertical lines are the embedding regions in the two voiced frames, and “∗” indicates the synchronization mark which is the position of the tracked sample point with the largest amplitude. It can be observed in Figure 1 that the proposed ISM can more accurately track the appropriate embedding region under TSM.Figure2 shows the implementation flowchart of the ISM. Assuming that the length of the voiced frame is N, the length of the region for embedding watermarks is Ne and N≥Ne. The specific implementation process can be described as follows.(i)
Step 1: extract all the voiced frames with the length ofN from the audio.(ii)
Step 2: search for the sample point with the largest amplitude in each voiced frame and record its position asp.(iii)
Step 3:Ne sample points in the surrounding region of p, which are in the range of [p1,p2], can be used to carry watermarks, where p1=p−floorNe/3 is the starting position in this embedding region, p2=p+floor2Ne/3 is the ending position, and floor indicates that the data in brackets should be rounded down.Figure 2
Flowchart of the proposed ISM.The embedding region [p1,p2] has the following three conditions.(1)
Ifp<floorNe/3, it indicates that the position p of the sample point with the largest amplitude is closer to the head of this voiced frame, then the starting position of the embedding region should be set as p1=1, and the ending position is p2=Ne.(2)
Ifp>floor2Ne/3, this condition shows that the position p is closer to the end of this voiced frame, then the starting position should be set as p1=N−Ne+1, and the ending position is p2=N.(3)
IffloorNe/3≤p≤floor2Ne/3, it indicates that the position p is in the middle part of this voiced frame, then the starting position of the region should be set as p1=p−floorNe/3, and the ending position is p2=p+floor2Ne/3.


## 3. Principle of the Watermarking Algorithm

### 3.1. Principle of Embedding Watermarks

The embedding algorithm mainly includes the following several parts. Firstly, the proposed ISM is used to search for the best embedding region. Then, DCT is performed on the data in the embedding region to determine the frequency range for carrying the watermark. Finally, the DCT coefficients in the frequency range are processed by SVD to conceal the watermark by the quantization method. Figure3 shows the principle diagram of the embedding algorithm.Figure 3
The principle diagram of the embedding algorithm.Suppose that the binary watermark can be expressed in the following formula:(1)W0w0i,1≤i≤Lw,where w0i∈0,1 and Lw is the length of W0. In order to improve the security, W0 should be encrypted before it is concealed into the audio.Apply logistic mapping formula to generate a chaotic sequenceci with the same size as W0, as shown in the formulas.(2)xi+1=a0xi1−xi,ci=1,xi≥δ,0,xi<δ,where 0<xi<1, x1∈0,1 is the initial value when i=1, and δ⊂0,1 is a threshold to obtain ci. The logistic system will be in chaos when 3.5699456≤α0≤4.Exclusive OR operation is performed onW0 and ci to obtain the encrypted information W1, as shown in formula (3), where ⊕ stands for the exclusive OR operator. Triple key Chx1,α0,δ is the unique correct key to decrypt W1.(3)W1=w1i=w0i⊕ci,1≤i≤Lw.ConvertW1 into a two-dimensional array W2 shown in formula (4), where w2u,v∈0,1 and Lw=L1×L2. Suppose that A is the original audio with K sample points, as expressed in the formula.(4)W2=w2u,v,1≤u≤L1,1≤v≤L2,(5)A=aj,1≤j≤K,where aj is the amplitude of the jth sample point. Divide A into L1 audio fragments, namely, Al (1≤l≤L1), and each audio fragment has K1 sample points, K1=floorK/L1.Then,A can be divided into two parts, namely, Ar and As, where Ar will be used for carrying the watermark, and As does not participate in the embedding process. Ar can be expressed in formula (6), and its size is L1×K1.(6)Ar=A1⋮Al⋮AL1L1×K1=a1a2⋯aK1⋮⋮al−1K1+1al−1K1+2⋯alK1⋮⋮aL1−1K1+1aL1−1K1+2⋯aL1K1L1×K1.The watermarkW2 will be embedded into Ar, that is to say, each audio fragment Al needs to carry L2 bits watermark. To prevent the audio quality from decreasing too much, Al will be divided into several audio frames with the length of N, and only the voiced frame Amax with the largest energy is used to carry the watermark. The proposed ISM is used to track the appropriate embedding region [p1,p2] in Amax.We will take the embedding process that embedL2 bits binary watermark into Amax as an example to illustrate the core embedding scheme. Figures 4 and 5 show the main data and the flowchart of this core embedding scheme.Figure 4
Data in the core embedding algorithm.Figure 5
Flowchart of the core embedding algorithm.In our study, DCT is used to determine the frequency range where the watermark is located. Apply DCT on the data between [p1,p2] to obtain the DCT coefficient Adct and then get the intermediate frequency coefficient Aif from Adct in the range [b0+1, b0+Lif], where b0 is the starting position and Lif is the length of Aif. The frequency range [f1,f2] for embedding the watermark can be calculated according to the formulas.(7)f1=b0+1×fhNekHz,f2=b0+Lif×fhNekHz,where fh is the maximum cutoff frequency of the audio, and its value is usually half of the sampling rate. Divide Aif into L2 data blocks, namely, Blockv (1≤v≤L2), and the length of the data blocks can be calculated as Lsvd=floorLif/L2. Apply SVD on Blockv to obtain the eigenvalue λv, as shown in formula (8), Uv=uv where is a single element matrix, Vv is an orthogonal matrix with the dimension of Lsvd×Lsvd, and Sv is a row matrix with the dimension of 1×Lsvd in which only the first element λv≠0 and all other elements are equal to 0.(8)Blockv=USVT=uvλv0...0v11...v1Lsvd⋮⋱⋮vLsvd1⋯vLsvdLsvd.According to the stability characteristics of SVD, the eigenvalueλv usually does not change greatly when Blockv changes slightly, so one bit binary watermark can be hidden into one eigenvalue. In order to realize blind extraction, cc=floorλv/a will be obtained by quantization, where a is the quantization step. If the binary watermark is “0,” cc will be modified to be an even number; otherwise, cc will be set as an odd number. The embedding rule is described in Table 1.Table 1
Embedding rule.
Watermark bitOriginal parity ofccEmbedding ruleModified parity ofcc0Evenλy′=a×cc±0.5aEven0Oddλv=a×cc−0.5aEven1Evenλv′=a×cc−0.5aOdd1Oddλv′=a×cc+0.5aOddThen, the modified eigenvalueλv′ is shown in the following formula:(9)λv′=a×cc±a2.The modified data blockBlock′v can be reconstructed according to the following formula:(10)Block′v=US′VT.Repeat the above process to modify all eigenvalues, andL2 bits binary watermark can be concealed into all Blockv (1≤v≤L2). According to the process described above, each row of the binary data in W2 can be concealed into each Al (1≤l≤L1); finally, W2 will be completely concealed into Ar. The embedding process can be described as follows.(i)
Step 1: convertW1 into W2 with the size of L1×L2.(ii)
Step 2: divide the original audioA into L1 audio fragments Al with the same length.(iii)
Step 3: divideAl into audio frames with the length of N and find out the voiced frame Amax.(iv)
Step 4: search for the embedding region [p1,p2] by using ISM in Amax.(v)
Step 5: apply DCT on the data in the embedding region to obtainAif.(vi)
Step 6: divideAif into L2 data blocks Blockv and perform SVD on Blockv to obtain λv.(vii)
Step 7: embedL2 bits binary watermark into Amax according to the embedding rule in Table 1.(viii)
Step 8: repeat Step 4 to Step 7 until all watermark bits are concealed.(ix)
Step 9: recombine all audio fragments to recover the carried audioA′.


### 3.2. Principle of Extracting Watermarks

The extracting algorithm is the inverse process of the embedding algorithm, and its principle is shown in Figure6, in which the “core extracting scheme” is the most important part of the whole extracting algorithm. The process of extracting the watermark can be described as follows.Figure 6
The principle diagram of the extracting algorithm.(i)
Step 1: divide the carried audioA′ into L1 audio fragments Al′ with the same length.(ii)
Step 2: divideAl′ into audio frames with the length of N and find out the voiced frame Amax′.(iii)
Step 3: track the region containing the watermark by using ISM inAmax′.(iv)
Step 4: apply DCT on the data in the tracked region to obtainAif′.(v)
Step 5: divideAif′ into L2 data blocks Block′v and perform SVD on Block′v to obtain λv′.(vi)
Step 6: quantify all eigenvalues and judge their parity to obtainL2 bits binary watermark. The extracting rule can be expressed in formula (11), where cc′=floorλv′/a, 1≤u≤L1, 1≤v≤L2.(vii)
Step 7: repeat Step 2 to Step 6 until all watermark bits are extracted.(viii)
Step 8: decrypt and recover the watermarkw1′ from w2′.(11)w2′u,v=0,cc′is even,1,cc′is odd.Figure7 shows the flowchart of the core extracting scheme. In particular, the key parameters in the extracting algorithm should be consistent with the corresponding parameters in the embedding algorithm, including N, Ne, L1, L2, b0, Lif, and a.Figure 7
Flowchart of the core extracting scheme.


### 3.3. Optimization of the Quantization Step

From the watermark embedding principle mentioned above, quantization step is an important parameter, which directly affects the transparency and robustness of the algorithm. In order to balance the algorithm performance, GA is used to search the optimal quantization step intelligently. The fitness functionFitness is constructed with SNR and BER as shown in the formula.(12)SNR>SNR0,Fitness=1BER,where SNR0 is the lower threshold of transparency. The selected quantization step should not make the algorithm transparency lower than SNR0 dB. SNR and BER can be defined in formulas.(13)SNRA,A′=10lg∑j=1KA2∑j=1KA′−A2,BER=∑i=1Lwwi⊕w′iLw×100%,where A and A′ represent the original audio and the carried audio, respectively, and wi and wi′ denote the original watermark and the extracted watermark. The population consists of C1 chromosomes, and each chromosome with the length of C2, which will be encoded by using a binary encoding approach, can be converted into the quantization step. Formula (14) can be used to describe the transformation relationship between each chromosome CHr and each quantization step ar (1≤r≤C1).(14)ar=B2DCHr100,where B2DCHr means converting CHr from binary to decimal. The detailed process can be described as follows.(i)
Step 1: set the parameters, including the crossover probabilitypc and the mutation probability pm, and then generate an initial population POP0.(ii)
Step 2: calculate the quantization stepar according to formula (14) and then execute the embedding algorithm proposed in Section 3.1 after the carried audio is subjected to some attacks.(iii)
Step 3: pick out all qualified chromosomes whenSNR>SNR0 and then execute the extracting algorithm proposed in Section 3.2.(iv)
Step 4: calculate the fitness value according to formula (12) and obtain the best chromosome with the largest fitness value.(v)
Step 5: perform selection operation by roulette to get the transition populationPOP0′.(vi)
Step 6: perform crossover operation on each pair of chromosomes except for the best chromosome to obtain the next transition populationPOP0″.(vii)
Step 7: perform mutation operation on each chromosome except for the best chromosome to obtain the next generation populationPOP1.(viii)
Step 8: repeat Step 2 to Step 7 until the global optimal chromosome appears.


### 3.4. Measure to Further Improve Robustness

In order to improve the robustness of the algorithm, the same row of the binary watermark can be repeatedly embedded into three voiced frames with the highest energy inAl. When extracting watermarks, three groups of binary watermarks are extracted from the three voiced frames, respectively, and compared bit by bit to obtain a more accurate group of binary watermarks according to formula (15), where w21′u,v, w22′u,v, and w23′u,v are the three groups of binary watermarks extracted from three voiced frames, respectively.(15)w2′u,v=1,∑s=13w2s′u,v>3/2,0,∑s=13w2s′u,v≤3/2


## 3.1. Principle of Embedding Watermarks

The embedding algorithm mainly includes the following several parts. Firstly, the proposed ISM is used to search for the best embedding region. Then, DCT is performed on the data in the embedding region to determine the frequency range for carrying the watermark. Finally, the DCT coefficients in the frequency range are processed by SVD to conceal the watermark by the quantization method. Figure3 shows the principle diagram of the embedding algorithm.Figure 3
The principle diagram of the embedding algorithm.Suppose that the binary watermark can be expressed in the following formula:(1)W0w0i,1≤i≤Lw,where w0i∈0,1 and Lw is the length of W0. In order to improve the security, W0 should be encrypted before it is concealed into the audio.Apply logistic mapping formula to generate a chaotic sequenceci with the same size as W0, as shown in the formulas.(2)xi+1=a0xi1−xi,ci=1,xi≥δ,0,xi<δ,where 0<xi<1, x1∈0,1 is the initial value when i=1, and δ⊂0,1 is a threshold to obtain ci. The logistic system will be in chaos when 3.5699456≤α0≤4.Exclusive OR operation is performed onW0 and ci to obtain the encrypted information W1, as shown in formula (3), where ⊕ stands for the exclusive OR operator. Triple key Chx1,α0,δ is the unique correct key to decrypt W1.(3)W1=w1i=w0i⊕ci,1≤i≤Lw.ConvertW1 into a two-dimensional array W2 shown in formula (4), where w2u,v∈0,1 and Lw=L1×L2. Suppose that A is the original audio with K sample points, as expressed in the formula.(4)W2=w2u,v,1≤u≤L1,1≤v≤L2,(5)A=aj,1≤j≤K,where aj is the amplitude of the jth sample point. Divide A into L1 audio fragments, namely, Al (1≤l≤L1), and each audio fragment has K1 sample points, K1=floorK/L1.Then,A can be divided into two parts, namely, Ar and As, where Ar will be used for carrying the watermark, and As does not participate in the embedding process. Ar can be expressed in formula (6), and its size is L1×K1.(6)Ar=A1⋮Al⋮AL1L1×K1=a1a2⋯aK1⋮⋮al−1K1+1al−1K1+2⋯alK1⋮⋮aL1−1K1+1aL1−1K1+2⋯aL1K1L1×K1.The watermarkW2 will be embedded into Ar, that is to say, each audio fragment Al needs to carry L2 bits watermark. To prevent the audio quality from decreasing too much, Al will be divided into several audio frames with the length of N, and only the voiced frame Amax with the largest energy is used to carry the watermark. The proposed ISM is used to track the appropriate embedding region [p1,p2] in Amax.We will take the embedding process that embedL2 bits binary watermark into Amax as an example to illustrate the core embedding scheme. Figures 4 and 5 show the main data and the flowchart of this core embedding scheme.Figure 4
Data in the core embedding algorithm.Figure 5
Flowchart of the core embedding algorithm.In our study, DCT is used to determine the frequency range where the watermark is located. Apply DCT on the data between [p1,p2] to obtain the DCT coefficient Adct and then get the intermediate frequency coefficient Aif from Adct in the range [b0+1, b0+Lif], where b0 is the starting position and Lif is the length of Aif. The frequency range [f1,f2] for embedding the watermark can be calculated according to the formulas.(7)f1=b0+1×fhNekHz,f2=b0+Lif×fhNekHz,where fh is the maximum cutoff frequency of the audio, and its value is usually half of the sampling rate. Divide Aif into L2 data blocks, namely, Blockv (1≤v≤L2), and the length of the data blocks can be calculated as Lsvd=floorLif/L2. Apply SVD on Blockv to obtain the eigenvalue λv, as shown in formula (8), Uv=uv where is a single element matrix, Vv is an orthogonal matrix with the dimension of Lsvd×Lsvd, and Sv is a row matrix with the dimension of 1×Lsvd in which only the first element λv≠0 and all other elements are equal to 0.(8)Blockv=USVT=uvλv0...0v11...v1Lsvd⋮⋱⋮vLsvd1⋯vLsvdLsvd.According to the stability characteristics of SVD, the eigenvalueλv usually does not change greatly when Blockv changes slightly, so one bit binary watermark can be hidden into one eigenvalue. In order to realize blind extraction, cc=floorλv/a will be obtained by quantization, where a is the quantization step. If the binary watermark is “0,” cc will be modified to be an even number; otherwise, cc will be set as an odd number. The embedding rule is described in Table 1.Table 1
Embedding rule.
Watermark bitOriginal parity ofccEmbedding ruleModified parity ofcc0Evenλy′=a×cc±0.5aEven0Oddλv=a×cc−0.5aEven1Evenλv′=a×cc−0.5aOdd1Oddλv′=a×cc+0.5aOddThen, the modified eigenvalueλv′ is shown in the following formula:(9)λv′=a×cc±a2.The modified data blockBlock′v can be reconstructed according to the following formula:(10)Block′v=US′VT.Repeat the above process to modify all eigenvalues, andL2 bits binary watermark can be concealed into all Blockv (1≤v≤L2). According to the process described above, each row of the binary data in W2 can be concealed into each Al (1≤l≤L1); finally, W2 will be completely concealed into Ar. The embedding process can be described as follows.(i)
Step 1: convertW1 into W2 with the size of L1×L2.(ii)
Step 2: divide the original audioA into L1 audio fragments Al with the same length.(iii)
Step 3: divideAl into audio frames with the length of N and find out the voiced frame Amax.(iv)
Step 4: search for the embedding region [p1,p2] by using ISM in Amax.(v)
Step 5: apply DCT on the data in the embedding region to obtainAif.(vi)
Step 6: divideAif into L2 data blocks Blockv and perform SVD on Blockv to obtain λv.(vii)
Step 7: embedL2 bits binary watermark into Amax according to the embedding rule in Table 1.(viii)
Step 8: repeat Step 4 to Step 7 until all watermark bits are concealed.(ix)
Step 9: recombine all audio fragments to recover the carried audioA′.


## 3.2. Principle of Extracting Watermarks

The extracting algorithm is the inverse process of the embedding algorithm, and its principle is shown in Figure6, in which the “core extracting scheme” is the most important part of the whole extracting algorithm. The process of extracting the watermark can be described as follows.Figure 6
The principle diagram of the extracting algorithm.(i)
Step 1: divide the carried audioA′ into L1 audio fragments Al′ with the same length.(ii)
Step 2: divideAl′ into audio frames with the length of N and find out the voiced frame Amax′.(iii)
Step 3: track the region containing the watermark by using ISM inAmax′.(iv)
Step 4: apply DCT on the data in the tracked region to obtainAif′.(v)
Step 5: divideAif′ into L2 data blocks Block′v and perform SVD on Block′v to obtain λv′.(vi)
Step 6: quantify all eigenvalues and judge their parity to obtainL2 bits binary watermark. The extracting rule can be expressed in formula (11), where cc′=floorλv′/a, 1≤u≤L1, 1≤v≤L2.(vii)
Step 7: repeat Step 2 to Step 6 until all watermark bits are extracted.(viii)
Step 8: decrypt and recover the watermarkw1′ from w2′.(11)w2′u,v=0,cc′is even,1,cc′is odd.Figure7 shows the flowchart of the core extracting scheme. In particular, the key parameters in the extracting algorithm should be consistent with the corresponding parameters in the embedding algorithm, including N, Ne, L1, L2, b0, Lif, and a.Figure 7
Flowchart of the core extracting scheme.


## 3.3. Optimization of the Quantization Step

From the watermark embedding principle mentioned above, quantization step is an important parameter, which directly affects the transparency and robustness of the algorithm. In order to balance the algorithm performance, GA is used to search the optimal quantization step intelligently. The fitness functionFitness is constructed with SNR and BER as shown in the formula.(12)SNR>SNR0,Fitness=1BER,where SNR0 is the lower threshold of transparency. The selected quantization step should not make the algorithm transparency lower than SNR0 dB. SNR and BER can be defined in formulas.(13)SNRA,A′=10lg∑j=1KA2∑j=1KA′−A2,BER=∑i=1Lwwi⊕w′iLw×100%,where A and A′ represent the original audio and the carried audio, respectively, and wi and wi′ denote the original watermark and the extracted watermark. The population consists of C1 chromosomes, and each chromosome with the length of C2, which will be encoded by using a binary encoding approach, can be converted into the quantization step. Formula (14) can be used to describe the transformation relationship between each chromosome CHr and each quantization step ar (1≤r≤C1).(14)ar=B2DCHr100,where B2DCHr means converting CHr from binary to decimal. The detailed process can be described as follows.(i)
Step 1: set the parameters, including the crossover probabilitypc and the mutation probability pm, and then generate an initial population POP0.(ii)
Step 2: calculate the quantization stepar according to formula (14) and then execute the embedding algorithm proposed in Section 3.1 after the carried audio is subjected to some attacks.(iii)
Step 3: pick out all qualified chromosomes whenSNR>SNR0 and then execute the extracting algorithm proposed in Section 3.2.(iv)
Step 4: calculate the fitness value according to formula (12) and obtain the best chromosome with the largest fitness value.(v)
Step 5: perform selection operation by roulette to get the transition populationPOP0′.(vi)
Step 6: perform crossover operation on each pair of chromosomes except for the best chromosome to obtain the next transition populationPOP0″.(vii)
Step 7: perform mutation operation on each chromosome except for the best chromosome to obtain the next generation populationPOP1.(viii)
Step 8: repeat Step 2 to Step 7 until the global optimal chromosome appears.


## 3.4. Measure to Further Improve Robustness

In order to improve the robustness of the algorithm, the same row of the binary watermark can be repeatedly embedded into three voiced frames with the highest energy inAl. When extracting watermarks, three groups of binary watermarks are extracted from the three voiced frames, respectively, and compared bit by bit to obtain a more accurate group of binary watermarks according to formula (15), where w21′u,v, w22′u,v, and w23′u,v are the three groups of binary watermarks extracted from three voiced frames, respectively.(15)w2′u,v=1,∑s=13w2s′u,v>3/2,0,∑s=13w2s′u,v≤3/2


## 4. Performance Evaluation

In this section, the performance of the proposed algorithm will be tested. In order to evaluate the performance of this proposed algorithm, the quality of the audio can be evaluated by three ways, including SNR, the object difference grade (ODG) which is one of the output values obtained from the perceptual evaluation of audio quality (PEAQ), and the mean opinion score (MOS). According to the standard of IFPI, SNR should be greater than 20 dB to make the audio have good transparency. BER can be used to evaluate the algorithm robustness. Generally, small BER means that the algorithm has strong robustness to various attacks. According to the standard of IFPI, the BER of the extracted watermark is no less than 20% when the carried audio is attacked. NC can be used to compare the similarity between the original watermark and the extracted watermark, as shown in formula (16). When NC is close to 1, the original watermark is very similar to the extracted watermark.(16)NC=∑i=1Lwwi×w′i∑i=1Lwwi2∑i=1Lww′i2.The experimental parameters are as follows. (1) Algorithm parameters:C1=10, C2=8, pc=0.8, pm=0.1, SNR0=25, N=4096, Ne=1024, L1=64, L2=64, Lif=1024, and b0=300. (2) The watermark is a binary image as shown in Figure 8(p), and its size is 128×32. (3) The PEAQ metric was the basic standard which was released by the TSP Lab at McGill University. (4) The tested audio comprises twenty 64-second audio signals, formatted by WAV, sampled at 44100 Hz with 16-bit resolution, including popular songs, classical songs, rock songs, and dialogues.Figure 8
The extracted images under different processing operations. (a) No attack. (b) MP3 compression with 64 kbps. (c) MP3 compression with 128 kbps. (d) Noise corruption with 20 dB. (e) Noise corruption with 30 dB. (f) Noise corruption with 40 dB. (g) Requantization. (h) Resampling. (i) Echo addition with 50 s. (j) Echo addition with 100 s. (k) Low-pass filtering (4 kHz). (l) Low-pass filtering (8 kHz). (m) Low-pass filtering (12 kHz). (n) Amplitude scaling by 0.8. (o) Amplitude scaling by 1.2. (p) The original image.
(a)(b)(c)(d)(e)(f)(g)(h)(i)(j)(k)(l)(m)(n)(o)(p)The detailed experimental environment and software are described as follows: (1) computer system—64-bit Microsoft Windows 10; (2) programming language—Matlab 2016R; (3) software for processing audio signals—Cool Edit Pro V2.1.


### 4.1. Transparency and Capacity

The payload capacity of this algorithm can be calculated according to the following formula:(17)Cap=LwT=L1×L2T,where T is the time length that the audio carries the watermark. In our study, T is equal to 64 seconds, so the payload capacity is 128×32bit/64s=64 bps. The average values about the payload capacity (bps), SNR (dB), ODG, MOS of the audio, BER (%), and NC of the extracted watermark are listed in Table 2.Table 2
Experimental results (no attack).
ItemsSNRODGMOSBER (%)NCCap (bps)Proposed25.96−0.994.50.00164Paper [16]21.08−0.394.60.00143.07Paper [22]21.20−1.224.10.00186.13Paper [23]24.63−2.833.81.280.973243.07In our test, all the audio signals are processed with the four algorithms mentioned in Table2, respectively, to obtain four groups of carried audio signals which will be provided to 20 listeners (10 males and 10 females, aged between 18 and 60 years old) in order to get MOS scores. Table 2 shows that this algorithm has good transparency because the average SNR is up to 25.96 dB, ODG is −0.99, and MOS is 4.5 while the payload capacity is 64 bps, which is higher than the standard of IFPI. Most importantly, BER is equal to 0, and NC is equal to 1, which indicates that this algorithm has good robustness when there is no attack, so the extracted watermark image in Figure 8(a) is the same as the original image shown in Figure 8(p). Compared with the algorithms in paper [16] and paper [23], this proposed algorithm has a larger payload capacity and better transparency, and the robustness is stronger than that in paper [23]. Although the payload capacity of this algorithm is not as high as the algorithm in the paper [22], the transparency is more superior. Besides, this proposed algorithm is more robust than other three algorithms, which will be discussed in Section 4.2.Figure9 shows the waveform pictures of the carried audio without attack before and after embedding the watermark (we only display an audio clip lasting about 3 seconds to clearly show the details), and their spectrogram pictures are shown in Figure 10. It can be seen that there is no obvious change in the waveform and spectrogram of the audio before and after embedding the watermark, which indicates that this algorithm’s transparency is nice.Figure 9
Waveform comparison before and after embedding the watermark. (a) The original audio. (b) The carried audio.
(a)(b)Figure 10
Spectrogram comparison before and after embedding the watermark. (a) The original audio. (b) The carried audio.
(a)(b)


### 4.2. Robustness

This section will evaluate the algorithm robustness by BER and NC when resisting against various conventional signal processing operations and synchronous attacks and compare the experimental results with other algorithms in three related papers.


#### 4.2.1. Conventional Signal Processing Operations

Conventional signal processing operations are the most common attacks encountered by audio in the process of being used and spread, and they may cause damage or even loss of the watermark hidden in the audio, so the watermarking algorithm must have strong robustness to withstand these attacks. These operations mainly include the following types in Table3.Table 3
Attack types and specifications.
ItemAttack typeDescriptionAMP3 compressionApplying MP3 compression with 64 kbps and 128 kbps.BNoise corruptionAdding Gaussian noise with 20 dB, 30 dB, and 40 dB.CRequantizationQuantizing the carried audio with 16 bit-8 bit-16 bit per sample.DResamplingDropping the sampling rate with 44.1 kHz-22.05 kHz-44.1 kHz.EEcho additionAdding an echo with a delay of 50 ms and 100 ms.FLow-pass filteringApplying low-pass filter with 4 kHz, 8 kHz, and 12 kHz.GAmplitude scalingScaling the carried audio’s amplitude by 0.8 and 1.2.BER (%) and NC of the extracted watermark are averaged and listed in Table4 under these signal processing operations. The extracted images whose NC values are closest to the average value are shown in Figure 8. According to the experimental results in Figure 8 and Table 4, this algorithm has strong robustness against conventional signal processing operations, which can be summarized as follows.Table 4
Average BER (%) and NC of the extracted watermark under different processing operations.
ItemAttackBERNCItemAttackBERNCA64 kbps1.280.9740E50 ms0.001128 kbps0.790.9854100 ms0.080.9985B20 dB0.070.9987F4 kHz25.640.745830 dB0.0118 kHz0.050.999040 dB0.00112 kHz0.001CDown0.001GReduce0.720.9932DDown0.700.9872Enlarge0.910.9876When resisting noise corruption with 30 dB and 40 dB, requantization, low-pass filtering with cutoff frequency of 12 kHz, and echo addition with 50 ms, the extracted images are almost the same with the original image. BER values are equal to 0, and NC values are equal to 1, which indicates that the proposed algorithm has excellent robustness against these attacks.When resisting MP3 compression, noise corruption with 20 dB, low-pass filtering with cutoff frequency of 8 kHz, resampling, echo addition with 50 ms, and amplitude scaling, the extracted images are similar to the original image. BER values are below 1.28%, and NC values are above 0.9740, so the proposed algorithm has good robustness against these attacks.When resisting low-pass filtering with cutoff frequency of 4 kHz, the former half of the extracted watermark image is very clear, while another half is completely blurred, NC is 0.7458, and BER is 25.64%, as shown in Figure8(k). The reason for this phenomenon is mainly related to the algorithm parameters, including the length Ne of the data by DCT, the region [b0+1,b0+Lif] for embedding the watermark, and the sampling rate fs of the audio. In our experiment, sampling rate is 44.1 kHz, Ne=4096, b0=300, and Lif=1024, so the embedding frequency range [f1,f2] can be calculated as follows.(18)f1=b0+1×fhNe=300+1×22.054096=1.62kHz,(19)f2=b0+Lif×fhNe=300+1024×22.054096=7.13kHz.It can be seen from formulas (18) and (19) that the watermark can be concealed in the frequency range of [1.62, 7.13] kHz in the audio, so low-pass filtering with a cutoff frequency higher than 7.13 kHz or lower than 1.62 kHz almost has no effect on the watermark.When resisting low-pass filtering with cutoff frequency of 8 kHz, the extracted watermark is relatively clear, NC is 0.9990, and BER is 0.05%. However, because the upper limit of the embedding region is 7.13 kHz, which is very close to the cutoff frequency (8 kHz) of the filter, and the low-pass filter has 3 dB amplitude attenuation near the cutoff frequency, there are still a few noise points in the extracted watermark image shown in Figure8(l). When resisting low-pass filtering with cutoff frequency of 12 kHz, the extracted watermark is the same as the original image, as shown in Figure 8(m), NC is equal to 1, and BER is equal to 0. When the audio is subjected to low-pass filtering with cutoff frequency of 4 kHz, the frequency components above 4 kHz in the audio will be removed, so the watermark in the frequency range of [1.62, 4] kHz can be extracted (the former half of the image in Figure 8(k) is very clear), while the watermark in the frequency range of [4, 7.13] kHz cannot be extracted (another half in Figure 8(k) is completely blurred). In practical application, the cutoff frequency of the low-pass filter and the frequency range of the embedding region should be staggered by adjusting the algorithm parameters to prevent watermarks from being damaged.


#### 4.2.2. Synchronous Attack

Synchronization attack is the most challenging type in the research of robust watermarking algorithm.In Table5, there are four kinds of synchronization attacks with different strengths for testing the robustness, including TSM, PSM, jittering, and random cropping. After the audio is subjected to the above synchronization attacks, BER (%) and NC values of the extracted watermark are, respectively, averaged and listed in Tables 6–9. The extracted images whose NC values are closest to the average value are shown in Figures 11–14.Table 5
Attack types and specifications.
ItemAttack typeDescriptionHTSMApply TSM from −10% to 10% on the audio, respectively.IPSMApply PSM from −5% to 5% on the audio, respectively.JJitteringDelete or add one sample every some samples in the audio.KRandom croppingRandomly cut out several samples in the different parts.Table 6
Average BER (%) and NC values of the extracted watermark under TSM.
Strength (%)BERNC−308.790.9232−207.640.9088−153.740.9225−93.610.9531−74.050.9842−53.340.9901−42.910.9878−33.080.9866−22.510.9853−12.730.9876+12.340.9867+22.560.9821+32.710.9821+43.610.9790+52.690.9911+73.470.9759+93.320.9794+159.130.8851+2012.040.8490+3015.310.8267Table 7
Average BER (%) and NC of the extracted information under PSM.
Strength (%)BERNC−520.580.8680−415.650.8923−314.140.8988−212.400.9333−17.740.9112+18.920.9156+212.600.9297+314.940.8977+415.820.9037+519.650.8772Table 8
Average BER (%) and NC of the extracted information under jittering.
StrengthBERNC1/5003.680.93031/10002.450.96281/15002.420.95681/20001.830.97121/30001.590.97181/40000.970.98171/50000.890.98431/100000.830.98771/500000.670.98841/1000000.340.9933Table 9
Average BER (%) and NC of the extracted information under random cropping.
StrengthBERNC100 points (front)0.610.9903100 points (middle)0.580.9912100 points (back)0.650.9898200 points (front)0.810.9873200 points (middle)0.800.9893200 points (back)0.840.9898500 points (front)1.290.9798500 points (middle)1.050.9889500 points (back)0.980.9897800 points (front)1.900.9754800 points (middle)1.010.9863800 points (back)1.780.9799Figure 11
The extracted images under TSM. (a) TSM (−30%). (b) TSM (−20%). (c) TSM (−15%. (d) TSM (−9%). (e) TSM (−7%). (f) TSM (−5%). (g) TSM (−4%). (h) TSM (−3%). (i) TSM (−2%). (j) TSM (−1%). (k) TSM (1%). (l) TSM (2%). (m) TSM (3%). (n) TSM (4%). (o) TSM (5%). (p) TSM (7%). (q) TSM (9%). (r) TSM (15%). (s) TSM (20%). (t) TSM (+30%).
(a)(b)(c)(d)(e)(f)(g)(h)(i)(j)(k)(l)(m)(n)(o)(p)(q)(r)(s)(t)Figure 12
The extracted images under PSM. (a) PSM (−5%). (b) PSM (−4%). (c) PSM (−3%). (d) PSM (−2%). (e) PSM (−1%). (f) PSM (+1%). (g) PSM (+2%). (h) PSM (+3%). (i) PSM (+4%). (j) PSM (+5%).
(a)(b)(c)(d)(e)(f)(g)(h)(i)(j)Figure 13
The extracted images under jittering. (a) Jittering (1/500). (b) Jittering (1/1000). (c) Jittering (1/1500). (d) Jittering (1/2000). (e) Jittering (1/3000). (f) Jittering (1/4000). (g) Jittering (1/5000). (h) Jittering (1/10000). (i) Jittering (1/50000). (j) Jittering (1/100000).
(a)(b)(c)(d)(e)(f)(g)(h)(i)(j)Figure 14
The extracted images under random cropping. (a) 100 points (front). (b) 100 points (middle). (c) 100 points (back). (d) 200 points (front). (e) 200 points (middle). (f) 200 points (back). (g) 500 points (front). (h) 500 points (middle). (i) 500 points (back). (j) 800 points (front). (k) 800 points (middle). (l) 800 points (back).
(a)(b)(c)(d)(e)(f)(g)(h)(i)(j)(k)(l)(1) TSM. Table 6 shows the average BER (%) and NC of the extracted watermark under TSM with different strengths from −30% to +30%. It can be seen from the experimental results in Table 6 and Figure 11 that this algorithm has excellent robustness for overcoming TSM attacks with different strengths. BER values all are below 15.31%, which is far superior to the standard of IFPI. NC values all are above 0.8267, so the extracted images all can distinguish its content in them.(2) PSM. When the audio is subjected to PSM, its playing time will not change, but the position and shape of the voiced frame will change slightly. Table 7 shows the average BER (%) and NC of the extracted watermark under PSM with different strengths from −5% to 5%. Although the extracted images are not very clear in Figure 12, their content can still be distinguished.(3) Jittering. Table 8 shows the average BER (%) and NC of the extracted watermark under jittering with different strengths from 1/100000 to 1/500. The extracted images are shown in Figure 13. As shown in Figure 13(a), under the maximum attack strength (1/500), the extracted image contains more noise points, but its main feature can still be identified, in which NC is 0.9303 and BER is 3.68%. As the attack strength weakens, the extracted images become more and more similar to the original image in Figure 8(p), and BER values become smaller and smaller, so this proposed algorithm has strong robustness against jittering.(4) Random Cropping. The average BER (%) and NC of the extracted watermark under random cropping with different strengths are shown in Table 9. From the experimental results, the extracted images in Figure 14 all are relatively clear, NC values are above 0.9754, and BER values are below 1.90%, which all show that this proposed algorithm is robust against random cropping.


#### 4.2.3. Comparative Analysis of Robustness

In order to compare the algorithm robustness with the related algorithms in papers [16], [22], and [23], Table 10 lists BER (%) values of these algorithms when resisting signal processing operations and synchronization attacks. From the experimental results in Table 2 and Table 10, the comparative analysis is discussed as follows about these four algorithms. Compared with the algorithm in paper [16], this proposed algorithm has larger payload capacity, higher transparency, and better robustness against synchronization attacks and conventional signal processing operations except for some attacks, such as MP3 compression with 64 kbps, low-pass filtering with cutoff frequency of 4 kHz, and PSM. According to the embedding principle of this algorithm, the frequency band where the watermark is located can be changed by modifying the algorithm parameters in practical application, so this proposed algorithm can overcome the attack from low-pass filtering with cutoff frequency of 4 kHz in fact. In the following comparative analysis with the other two algorithms, this viewpoint will not be reiterated. The robustness of this proposed algorithm is far superior to the algorithms in paper [22] and paper [23], although the payload capacity is slightly lower than that in paper [22]. Synchronization attacks may change the overall structure of the audio, but it has little effect on the shape of the voiced frame. The proposed ISM in this algorithm can accurately track the position of the largest amplitude in the voiced frame to determine the embedding region where the watermark is located. Therefore, this algorithm has strong robustness against various malicious attacks.Table 10
Robustness comparison with other algorithms (BER, %).
AttacksProposedPaper [16]Paper [22]Paper [23]A64 kbps1.281.932.1926.48128 kbps0.790.010.0410.08B20 dB0.070.4416.4111.4730 dB0.010.010.003.2240 dB0.000.000.001.83CDown0.000.010.002.17DDown0.700.000.002.35E50 ms0.000.257.031.32100 ms0.080.1911.520.97F4 kHz25.640.006.5550.258 kHz0.050.002.0234.1112 kHz0.000.000.6721.16GReduce0.720.0031.301.28Enlarge0.910.010.391.28H−5%3.346.4541.76.19+5%2.696.5142.26.52I−1%7.741.2438.266.75+1%8.921.8437.5414.59J1/5003.680.5928.1546.621/10002.450.3325.8337.18K200 points0.821.347.3639.28100 points0.620.816.6236.73


### 4.3. Security

The watermark hidden in audio is protected by encryption technology and information hiding technology, so it is necessary to analyse the security of this algorithm from the key space constructed by encryption technology and information hiding technology.The proposed algorithm uses a triple keyChx1,α0,δ to encrypt the watermark and seven key parameters (N, Ne, L1, L2, b0, Lif, a) to conceal the watermark. Chx1,α0,δ and a are taken in the real field, so this algorithm has infinite key space in theory. In fact, they are affected by the word length, so their key space is limited. In our test, the computer system is 64-bit, N, Ne, and Lif all are 16-bit, and L1, L2, and b0 are 10-bit, so the key space to encrypt the watermark can be calculated as 2192, and the key space to conceal the watermark is 2142. From the above analysis, even if the attacker obtains the principle of the algorithm, as long as these key parameters are not known, it is difficult for the attacker to obtain the watermark.


### 4.4. Complexity

The complexity of the algorithm is an important index to evaluate the performance of the algorithm. It is usually measured by the computational cost when embedding and extracting the watermark. In our experiment, the average time for extracting the watermark is 0.8544 s and that for embedding the watermark without GA is 1.6246 s. When GA is used to search for the best algorithm parameter, the embedding time is related to the evolution time of GA. It can be seen that GA enables our proposed algorithm to achieve a good balance between transparency and robustness, but it also brings a large computational cost. When embedding the watermark, the average time for searching the embedding region in a voiced frame is 0.037 ms. When extracting the watermark, the average time for tracking the extracting region in a voiced frame is 0.036 ms. It can be seen that our proposed ISM takes up very little computational cost to find the synchronization marks.


## 4.1. Transparency and Capacity

The payload capacity of this algorithm can be calculated according to the following formula:(17)Cap=LwT=L1×L2T,where T is the time length that the audio carries the watermark. In our study, T is equal to 64 seconds, so the payload capacity is 128×32bit/64s=64 bps. The average values about the payload capacity (bps), SNR (dB), ODG, MOS of the audio, BER (%), and NC of the extracted watermark are listed in Table 2.Table 2
Experimental results (no attack).
ItemsSNRODGMOSBER (%)NCCap (bps)Proposed25.96−0.994.50.00164Paper [16]21.08−0.394.60.00143.07Paper [22]21.20−1.224.10.00186.13Paper [23]24.63−2.833.81.280.973243.07In our test, all the audio signals are processed with the four algorithms mentioned in Table2, respectively, to obtain four groups of carried audio signals which will be provided to 20 listeners (10 males and 10 females, aged between 18 and 60 years old) in order to get MOS scores. Table 2 shows that this algorithm has good transparency because the average SNR is up to 25.96 dB, ODG is −0.99, and MOS is 4.5 while the payload capacity is 64 bps, which is higher than the standard of IFPI. Most importantly, BER is equal to 0, and NC is equal to 1, which indicates that this algorithm has good robustness when there is no attack, so the extracted watermark image in Figure 8(a) is the same as the original image shown in Figure 8(p). Compared with the algorithms in paper [16] and paper [23], this proposed algorithm has a larger payload capacity and better transparency, and the robustness is stronger than that in paper [23]. Although the payload capacity of this algorithm is not as high as the algorithm in the paper [22], the transparency is more superior. Besides, this proposed algorithm is more robust than other three algorithms, which will be discussed in Section 4.2.Figure9 shows the waveform pictures of the carried audio without attack before and after embedding the watermark (we only display an audio clip lasting about 3 seconds to clearly show the details), and their spectrogram pictures are shown in Figure 10. It can be seen that there is no obvious change in the waveform and spectrogram of the audio before and after embedding the watermark, which indicates that this algorithm’s transparency is nice.Figure 9
Waveform comparison before and after embedding the watermark. (a) The original audio. (b) The carried audio.
(a)(b)Figure 10
Spectrogram comparison before and after embedding the watermark. (a) The original audio. (b) The carried audio.
(a)(b)


## 4.2. Robustness

This section will evaluate the algorithm robustness by BER and NC when resisting against various conventional signal processing operations and synchronous attacks and compare the experimental results with other algorithms in three related papers.


### 4.2.1. Conventional Signal Processing Operations

Conventional signal processing operations are the most common attacks encountered by audio in the process of being used and spread, and they may cause damage or even loss of the watermark hidden in the audio, so the watermarking algorithm must have strong robustness to withstand these attacks. These operations mainly include the following types in Table3.Table 3
Attack types and specifications.
ItemAttack typeDescriptionAMP3 compressionApplying MP3 compression with 64 kbps and 128 kbps.BNoise corruptionAdding Gaussian noise with 20 dB, 30 dB, and 40 dB.CRequantizationQuantizing the carried audio with 16 bit-8 bit-16 bit per sample.DResamplingDropping the sampling rate with 44.1 kHz-22.05 kHz-44.1 kHz.EEcho additionAdding an echo with a delay of 50 ms and 100 ms.FLow-pass filteringApplying low-pass filter with 4 kHz, 8 kHz, and 12 kHz.GAmplitude scalingScaling the carried audio’s amplitude by 0.8 and 1.2.BER (%) and NC of the extracted watermark are averaged and listed in Table4 under these signal processing operations. The extracted images whose NC values are closest to the average value are shown in Figure 8. According to the experimental results in Figure 8 and Table 4, this algorithm has strong robustness against conventional signal processing operations, which can be summarized as follows.Table 4
Average BER (%) and NC of the extracted watermark under different processing operations.
ItemAttackBERNCItemAttackBERNCA64 kbps1.280.9740E50 ms0.001128 kbps0.790.9854100 ms0.080.9985B20 dB0.070.9987F4 kHz25.640.745830 dB0.0118 kHz0.050.999040 dB0.00112 kHz0.001CDown0.001GReduce0.720.9932DDown0.700.9872Enlarge0.910.9876When resisting noise corruption with 30 dB and 40 dB, requantization, low-pass filtering with cutoff frequency of 12 kHz, and echo addition with 50 ms, the extracted images are almost the same with the original image. BER values are equal to 0, and NC values are equal to 1, which indicates that the proposed algorithm has excellent robustness against these attacks.When resisting MP3 compression, noise corruption with 20 dB, low-pass filtering with cutoff frequency of 8 kHz, resampling, echo addition with 50 ms, and amplitude scaling, the extracted images are similar to the original image. BER values are below 1.28%, and NC values are above 0.9740, so the proposed algorithm has good robustness against these attacks.When resisting low-pass filtering with cutoff frequency of 4 kHz, the former half of the extracted watermark image is very clear, while another half is completely blurred, NC is 0.7458, and BER is 25.64%, as shown in Figure8(k). The reason for this phenomenon is mainly related to the algorithm parameters, including the length Ne of the data by DCT, the region [b0+1,b0+Lif] for embedding the watermark, and the sampling rate fs of the audio. In our experiment, sampling rate is 44.1 kHz, Ne=4096, b0=300, and Lif=1024, so the embedding frequency range [f1,f2] can be calculated as follows.(18)f1=b0+1×fhNe=300+1×22.054096=1.62kHz,(19)f2=b0+Lif×fhNe=300+1024×22.054096=7.13kHz.It can be seen from formulas (18) and (19) that the watermark can be concealed in the frequency range of [1.62, 7.13] kHz in the audio, so low-pass filtering with a cutoff frequency higher than 7.13 kHz or lower than 1.62 kHz almost has no effect on the watermark.When resisting low-pass filtering with cutoff frequency of 8 kHz, the extracted watermark is relatively clear, NC is 0.9990, and BER is 0.05%. However, because the upper limit of the embedding region is 7.13 kHz, which is very close to the cutoff frequency (8 kHz) of the filter, and the low-pass filter has 3 dB amplitude attenuation near the cutoff frequency, there are still a few noise points in the extracted watermark image shown in Figure8(l). When resisting low-pass filtering with cutoff frequency of 12 kHz, the extracted watermark is the same as the original image, as shown in Figure 8(m), NC is equal to 1, and BER is equal to 0. When the audio is subjected to low-pass filtering with cutoff frequency of 4 kHz, the frequency components above 4 kHz in the audio will be removed, so the watermark in the frequency range of [1.62, 4] kHz can be extracted (the former half of the image in Figure 8(k) is very clear), while the watermark in the frequency range of [4, 7.13] kHz cannot be extracted (another half in Figure 8(k) is completely blurred). In practical application, the cutoff frequency of the low-pass filter and the frequency range of the embedding region should be staggered by adjusting the algorithm parameters to prevent watermarks from being damaged.


### 4.2.2. Synchronous Attack

Synchronization attack is the most challenging type in the research of robust watermarking algorithm.In Table5, there are four kinds of synchronization attacks with different strengths for testing the robustness, including TSM, PSM, jittering, and random cropping. After the audio is subjected to the above synchronization attacks, BER (%) and NC values of the extracted watermark are, respectively, averaged and listed in Tables 6–9. The extracted images whose NC values are closest to the average value are shown in Figures 11–14.Table 5
Attack types and specifications.
ItemAttack typeDescriptionHTSMApply TSM from −10% to 10% on the audio, respectively.IPSMApply PSM from −5% to 5% on the audio, respectively.JJitteringDelete or add one sample every some samples in the audio.KRandom croppingRandomly cut out several samples in the different parts.Table 6
Average BER (%) and NC values of the extracted watermark under TSM.
Strength (%)BERNC−308.790.9232−207.640.9088−153.740.9225−93.610.9531−74.050.9842−53.340.9901−42.910.9878−33.080.9866−22.510.9853−12.730.9876+12.340.9867+22.560.9821+32.710.9821+43.610.9790+52.690.9911+73.470.9759+93.320.9794+159.130.8851+2012.040.8490+3015.310.8267Table 7
Average BER (%) and NC of the extracted information under PSM.
Strength (%)BERNC−520.580.8680−415.650.8923−314.140.8988−212.400.9333−17.740.9112+18.920.9156+212.600.9297+314.940.8977+415.820.9037+519.650.8772Table 8
Average BER (%) and NC of the extracted information under jittering.
StrengthBERNC1/5003.680.93031/10002.450.96281/15002.420.95681/20001.830.97121/30001.590.97181/40000.970.98171/50000.890.98431/100000.830.98771/500000.670.98841/1000000.340.9933Table 9
Average BER (%) and NC of the extracted information under random cropping.
StrengthBERNC100 points (front)0.610.9903100 points (middle)0.580.9912100 points (back)0.650.9898200 points (front)0.810.9873200 points (middle)0.800.9893200 points (back)0.840.9898500 points (front)1.290.9798500 points (middle)1.050.9889500 points (back)0.980.9897800 points (front)1.900.9754800 points (middle)1.010.9863800 points (back)1.780.9799Figure 11
The extracted images under TSM. (a) TSM (−30%). (b) TSM (−20%). (c) TSM (−15%. (d) TSM (−9%). (e) TSM (−7%). (f) TSM (−5%). (g) TSM (−4%). (h) TSM (−3%). (i) TSM (−2%). (j) TSM (−1%). (k) TSM (1%). (l) TSM (2%). (m) TSM (3%). (n) TSM (4%). (o) TSM (5%). (p) TSM (7%). (q) TSM (9%). (r) TSM (15%). (s) TSM (20%). (t) TSM (+30%).
(a)(b)(c)(d)(e)(f)(g)(h)(i)(j)(k)(l)(m)(n)(o)(p)(q)(r)(s)(t)Figure 12
The extracted images under PSM. (a) PSM (−5%). (b) PSM (−4%). (c) PSM (−3%). (d) PSM (−2%). (e) PSM (−1%). (f) PSM (+1%). (g) PSM (+2%). (h) PSM (+3%). (i) PSM (+4%). (j) PSM (+5%).
(a)(b)(c)(d)(e)(f)(g)(h)(i)(j)Figure 13
The extracted images under jittering. (a) Jittering (1/500). (b) Jittering (1/1000). (c) Jittering (1/1500). (d) Jittering (1/2000). (e) Jittering (1/3000). (f) Jittering (1/4000). (g) Jittering (1/5000). (h) Jittering (1/10000). (i) Jittering (1/50000). (j) Jittering (1/100000).
(a)(b)(c)(d)(e)(f)(g)(h)(i)(j)Figure 14
The extracted images under random cropping. (a) 100 points (front). (b) 100 points (middle). (c) 100 points (back). (d) 200 points (front). (e) 200 points (middle). (f) 200 points (back). (g) 500 points (front). (h) 500 points (middle). (i) 500 points (back). (j) 800 points (front). (k) 800 points (middle). (l) 800 points (back).
(a)(b)(c)(d)(e)(f)(g)(h)(i)(j)(k)(l)(1) TSM. Table 6 shows the average BER (%) and NC of the extracted watermark under TSM with different strengths from −30% to +30%. It can be seen from the experimental results in Table 6 and Figure 11 that this algorithm has excellent robustness for overcoming TSM attacks with different strengths. BER values all are below 15.31%, which is far superior to the standard of IFPI. NC values all are above 0.8267, so the extracted images all can distinguish its content in them.(2) PSM. When the audio is subjected to PSM, its playing time will not change, but the position and shape of the voiced frame will change slightly. Table 7 shows the average BER (%) and NC of the extracted watermark under PSM with different strengths from −5% to 5%. Although the extracted images are not very clear in Figure 12, their content can still be distinguished.(3) Jittering. Table 8 shows the average BER (%) and NC of the extracted watermark under jittering with different strengths from 1/100000 to 1/500. The extracted images are shown in Figure 13. As shown in Figure 13(a), under the maximum attack strength (1/500), the extracted image contains more noise points, but its main feature can still be identified, in which NC is 0.9303 and BER is 3.68%. As the attack strength weakens, the extracted images become more and more similar to the original image in Figure 8(p), and BER values become smaller and smaller, so this proposed algorithm has strong robustness against jittering.(4) Random Cropping. The average BER (%) and NC of the extracted watermark under random cropping with different strengths are shown in Table 9. From the experimental results, the extracted images in Figure 14 all are relatively clear, NC values are above 0.9754, and BER values are below 1.90%, which all show that this proposed algorithm is robust against random cropping.


### 4.2.3. Comparative Analysis of Robustness

In order to compare the algorithm robustness with the related algorithms in papers [16], [22], and [23], Table 10 lists BER (%) values of these algorithms when resisting signal processing operations and synchronization attacks. From the experimental results in Table 2 and Table 10, the comparative analysis is discussed as follows about these four algorithms. Compared with the algorithm in paper [16], this proposed algorithm has larger payload capacity, higher transparency, and better robustness against synchronization attacks and conventional signal processing operations except for some attacks, such as MP3 compression with 64 kbps, low-pass filtering with cutoff frequency of 4 kHz, and PSM. According to the embedding principle of this algorithm, the frequency band where the watermark is located can be changed by modifying the algorithm parameters in practical application, so this proposed algorithm can overcome the attack from low-pass filtering with cutoff frequency of 4 kHz in fact. In the following comparative analysis with the other two algorithms, this viewpoint will not be reiterated. The robustness of this proposed algorithm is far superior to the algorithms in paper [22] and paper [23], although the payload capacity is slightly lower than that in paper [22]. Synchronization attacks may change the overall structure of the audio, but it has little effect on the shape of the voiced frame. The proposed ISM in this algorithm can accurately track the position of the largest amplitude in the voiced frame to determine the embedding region where the watermark is located. Therefore, this algorithm has strong robustness against various malicious attacks.Table 10
Robustness comparison with other algorithms (BER, %).
AttacksProposedPaper [16]Paper [22]Paper [23]A64 kbps1.281.932.1926.48128 kbps0.790.010.0410.08B20 dB0.070.4416.4111.4730 dB0.010.010.003.2240 dB0.000.000.001.83CDown0.000.010.002.17DDown0.700.000.002.35E50 ms0.000.257.031.32100 ms0.080.1911.520.97F4 kHz25.640.006.5550.258 kHz0.050.002.0234.1112 kHz0.000.000.6721.16GReduce0.720.0031.301.28Enlarge0.910.010.391.28H−5%3.346.4541.76.19+5%2.696.5142.26.52I−1%7.741.2438.266.75+1%8.921.8437.5414.59J1/5003.680.5928.1546.621/10002.450.3325.8337.18K200 points0.821.347.3639.28100 points0.620.816.6236.73


## 4.2.1. Conventional Signal Processing Operations

Conventional signal processing operations are the most common attacks encountered by audio in the process of being used and spread, and they may cause damage or even loss of the watermark hidden in the audio, so the watermarking algorithm must have strong robustness to withstand these attacks. These operations mainly include the following types in Table3.Table 3
Attack types and specifications.
ItemAttack typeDescriptionAMP3 compressionApplying MP3 compression with 64 kbps and 128 kbps.BNoise corruptionAdding Gaussian noise with 20 dB, 30 dB, and 40 dB.CRequantizationQuantizing the carried audio with 16 bit-8 bit-16 bit per sample.DResamplingDropping the sampling rate with 44.1 kHz-22.05 kHz-44.1 kHz.EEcho additionAdding an echo with a delay of 50 ms and 100 ms.FLow-pass filteringApplying low-pass filter with 4 kHz, 8 kHz, and 12 kHz.GAmplitude scalingScaling the carried audio’s amplitude by 0.8 and 1.2.BER (%) and NC of the extracted watermark are averaged and listed in Table4 under these signal processing operations. The extracted images whose NC values are closest to the average value are shown in Figure 8. According to the experimental results in Figure 8 and Table 4, this algorithm has strong robustness against conventional signal processing operations, which can be summarized as follows.Table 4
Average BER (%) and NC of the extracted watermark under different processing operations.
ItemAttackBERNCItemAttackBERNCA64 kbps1.280.9740E50 ms0.001128 kbps0.790.9854100 ms0.080.9985B20 dB0.070.9987F4 kHz25.640.745830 dB0.0118 kHz0.050.999040 dB0.00112 kHz0.001CDown0.001GReduce0.720.9932DDown0.700.9872Enlarge0.910.9876When resisting noise corruption with 30 dB and 40 dB, requantization, low-pass filtering with cutoff frequency of 12 kHz, and echo addition with 50 ms, the extracted images are almost the same with the original image. BER values are equal to 0, and NC values are equal to 1, which indicates that the proposed algorithm has excellent robustness against these attacks.When resisting MP3 compression, noise corruption with 20 dB, low-pass filtering with cutoff frequency of 8 kHz, resampling, echo addition with 50 ms, and amplitude scaling, the extracted images are similar to the original image. BER values are below 1.28%, and NC values are above 0.9740, so the proposed algorithm has good robustness against these attacks.When resisting low-pass filtering with cutoff frequency of 4 kHz, the former half of the extracted watermark image is very clear, while another half is completely blurred, NC is 0.7458, and BER is 25.64%, as shown in Figure8(k). The reason for this phenomenon is mainly related to the algorithm parameters, including the length Ne of the data by DCT, the region [b0+1,b0+Lif] for embedding the watermark, and the sampling rate fs of the audio. In our experiment, sampling rate is 44.1 kHz, Ne=4096, b0=300, and Lif=1024, so the embedding frequency range [f1,f2] can be calculated as follows.(18)f1=b0+1×fhNe=300+1×22.054096=1.62kHz,(19)f2=b0+Lif×fhNe=300+1024×22.054096=7.13kHz.It can be seen from formulas (18) and (19) that the watermark can be concealed in the frequency range of [1.62, 7.13] kHz in the audio, so low-pass filtering with a cutoff frequency higher than 7.13 kHz or lower than 1.62 kHz almost has no effect on the watermark.When resisting low-pass filtering with cutoff frequency of 8 kHz, the extracted watermark is relatively clear, NC is 0.9990, and BER is 0.05%. However, because the upper limit of the embedding region is 7.13 kHz, which is very close to the cutoff frequency (8 kHz) of the filter, and the low-pass filter has 3 dB amplitude attenuation near the cutoff frequency, there are still a few noise points in the extracted watermark image shown in Figure8(l). When resisting low-pass filtering with cutoff frequency of 12 kHz, the extracted watermark is the same as the original image, as shown in Figure 8(m), NC is equal to 1, and BER is equal to 0. When the audio is subjected to low-pass filtering with cutoff frequency of 4 kHz, the frequency components above 4 kHz in the audio will be removed, so the watermark in the frequency range of [1.62, 4] kHz can be extracted (the former half of the image in Figure 8(k) is very clear), while the watermark in the frequency range of [4, 7.13] kHz cannot be extracted (another half in Figure 8(k) is completely blurred). In practical application, the cutoff frequency of the low-pass filter and the frequency range of the embedding region should be staggered by adjusting the algorithm parameters to prevent watermarks from being damaged.


## 4.2.2. Synchronous Attack

Synchronization attack is the most challenging type in the research of robust watermarking algorithm.In Table5, there are four kinds of synchronization attacks with different strengths for testing the robustness, including TSM, PSM, jittering, and random cropping. After the audio is subjected to the above synchronization attacks, BER (%) and NC values of the extracted watermark are, respectively, averaged and listed in Tables 6–9. The extracted images whose NC values are closest to the average value are shown in Figures 11–14.Table 5
Attack types and specifications.
ItemAttack typeDescriptionHTSMApply TSM from −10% to 10% on the audio, respectively.IPSMApply PSM from −5% to 5% on the audio, respectively.JJitteringDelete or add one sample every some samples in the audio.KRandom croppingRandomly cut out several samples in the different parts.Table 6
Average BER (%) and NC values of the extracted watermark under TSM.
Strength (%)BERNC−308.790.9232−207.640.9088−153.740.9225−93.610.9531−74.050.9842−53.340.9901−42.910.9878−33.080.9866−22.510.9853−12.730.9876+12.340.9867+22.560.9821+32.710.9821+43.610.9790+52.690.9911+73.470.9759+93.320.9794+159.130.8851+2012.040.8490+3015.310.8267Table 7
Average BER (%) and NC of the extracted information under PSM.
Strength (%)BERNC−520.580.8680−415.650.8923−314.140.8988−212.400.9333−17.740.9112+18.920.9156+212.600.9297+314.940.8977+415.820.9037+519.650.8772Table 8
Average BER (%) and NC of the extracted information under jittering.
StrengthBERNC1/5003.680.93031/10002.450.96281/15002.420.95681/20001.830.97121/30001.590.97181/40000.970.98171/50000.890.98431/100000.830.98771/500000.670.98841/1000000.340.9933Table 9
Average BER (%) and NC of the extracted information under random cropping.
StrengthBERNC100 points (front)0.610.9903100 points (middle)0.580.9912100 points (back)0.650.9898200 points (front)0.810.9873200 points (middle)0.800.9893200 points (back)0.840.9898500 points (front)1.290.9798500 points (middle)1.050.9889500 points (back)0.980.9897800 points (front)1.900.9754800 points (middle)1.010.9863800 points (back)1.780.9799Figure 11
The extracted images under TSM. (a) TSM (−30%). (b) TSM (−20%). (c) TSM (−15%. (d) TSM (−9%). (e) TSM (−7%). (f) TSM (−5%). (g) TSM (−4%). (h) TSM (−3%). (i) TSM (−2%). (j) TSM (−1%). (k) TSM (1%). (l) TSM (2%). (m) TSM (3%). (n) TSM (4%). (o) TSM (5%). (p) TSM (7%). (q) TSM (9%). (r) TSM (15%). (s) TSM (20%). (t) TSM (+30%).
(a)(b)(c)(d)(e)(f)(g)(h)(i)(j)(k)(l)(m)(n)(o)(p)(q)(r)(s)(t)Figure 12
The extracted images under PSM. (a) PSM (−5%). (b) PSM (−4%). (c) PSM (−3%). (d) PSM (−2%). (e) PSM (−1%). (f) PSM (+1%). (g) PSM (+2%). (h) PSM (+3%). (i) PSM (+4%). (j) PSM (+5%).
(a)(b)(c)(d)(e)(f)(g)(h)(i)(j)Figure 13
The extracted images under jittering. (a) Jittering (1/500). (b) Jittering (1/1000). (c) Jittering (1/1500). (d) Jittering (1/2000). (e) Jittering (1/3000). (f) Jittering (1/4000). (g) Jittering (1/5000). (h) Jittering (1/10000). (i) Jittering (1/50000). (j) Jittering (1/100000).
(a)(b)(c)(d)(e)(f)(g)(h)(i)(j)Figure 14
The extracted images under random cropping. (a) 100 points (front). (b) 100 points (middle). (c) 100 points (back). (d) 200 points (front). (e) 200 points (middle). (f) 200 points (back). (g) 500 points (front). (h) 500 points (middle). (i) 500 points (back). (j) 800 points (front). (k) 800 points (middle). (l) 800 points (back).
(a)(b)(c)(d)(e)(f)(g)(h)(i)(j)(k)(l)(1) TSM. Table 6 shows the average BER (%) and NC of the extracted watermark under TSM with different strengths from −30% to +30%. It can be seen from the experimental results in Table 6 and Figure 11 that this algorithm has excellent robustness for overcoming TSM attacks with different strengths. BER values all are below 15.31%, which is far superior to the standard of IFPI. NC values all are above 0.8267, so the extracted images all can distinguish its content in them.(2) PSM. When the audio is subjected to PSM, its playing time will not change, but the position and shape of the voiced frame will change slightly. Table 7 shows the average BER (%) and NC of the extracted watermark under PSM with different strengths from −5% to 5%. Although the extracted images are not very clear in Figure 12, their content can still be distinguished.(3) Jittering. Table 8 shows the average BER (%) and NC of the extracted watermark under jittering with different strengths from 1/100000 to 1/500. The extracted images are shown in Figure 13. As shown in Figure 13(a), under the maximum attack strength (1/500), the extracted image contains more noise points, but its main feature can still be identified, in which NC is 0.9303 and BER is 3.68%. As the attack strength weakens, the extracted images become more and more similar to the original image in Figure 8(p), and BER values become smaller and smaller, so this proposed algorithm has strong robustness against jittering.(4) Random Cropping. The average BER (%) and NC of the extracted watermark under random cropping with different strengths are shown in Table 9. From the experimental results, the extracted images in Figure 14 all are relatively clear, NC values are above 0.9754, and BER values are below 1.90%, which all show that this proposed algorithm is robust against random cropping.


## 4.2.3. Comparative Analysis of Robustness

In order to compare the algorithm robustness with the related algorithms in papers [16], [22], and [23], Table 10 lists BER (%) values of these algorithms when resisting signal processing operations and synchronization attacks. From the experimental results in Table 2 and Table 10, the comparative analysis is discussed as follows about these four algorithms. Compared with the algorithm in paper [16], this proposed algorithm has larger payload capacity, higher transparency, and better robustness against synchronization attacks and conventional signal processing operations except for some attacks, such as MP3 compression with 64 kbps, low-pass filtering with cutoff frequency of 4 kHz, and PSM. According to the embedding principle of this algorithm, the frequency band where the watermark is located can be changed by modifying the algorithm parameters in practical application, so this proposed algorithm can overcome the attack from low-pass filtering with cutoff frequency of 4 kHz in fact. In the following comparative analysis with the other two algorithms, this viewpoint will not be reiterated. The robustness of this proposed algorithm is far superior to the algorithms in paper [22] and paper [23], although the payload capacity is slightly lower than that in paper [22]. Synchronization attacks may change the overall structure of the audio, but it has little effect on the shape of the voiced frame. The proposed ISM in this algorithm can accurately track the position of the largest amplitude in the voiced frame to determine the embedding region where the watermark is located. Therefore, this algorithm has strong robustness against various malicious attacks.Table 10
Robustness comparison with other algorithms (BER, %).
AttacksProposedPaper [16]Paper [22]Paper [23]A64 kbps1.281.932.1926.48128 kbps0.790.010.0410.08B20 dB0.070.4416.4111.4730 dB0.010.010.003.2240 dB0.000.000.001.83CDown0.000.010.002.17DDown0.700.000.002.35E50 ms0.000.257.031.32100 ms0.080.1911.520.97F4 kHz25.640.006.5550.258 kHz0.050.002.0234.1112 kHz0.000.000.6721.16GReduce0.720.0031.301.28Enlarge0.910.010.391.28H−5%3.346.4541.76.19+5%2.696.5142.26.52I−1%7.741.2438.266.75+1%8.921.8437.5414.59J1/5003.680.5928.1546.621/10002.450.3325.8337.18K200 points0.821.347.3639.28100 points0.620.816.6236.73


## 4.3. Security

The watermark hidden in audio is protected by encryption technology and information hiding technology, so it is necessary to analyse the security of this algorithm from the key space constructed by encryption technology and information hiding technology.The proposed algorithm uses a triple keyChx1,α0,δ to encrypt the watermark and seven key parameters (N, Ne, L1, L2, b0, Lif, a) to conceal the watermark. Chx1,α0,δ and a are taken in the real field, so this algorithm has infinite key space in theory. In fact, they are affected by the word length, so their key space is limited. In our test, the computer system is 64-bit, N, Ne, and Lif all are 16-bit, and L1, L2, and b0 are 10-bit, so the key space to encrypt the watermark can be calculated as 2192, and the key space to conceal the watermark is 2142. From the above analysis, even if the attacker obtains the principle of the algorithm, as long as these key parameters are not known, it is difficult for the attacker to obtain the watermark.


## 4.4. Complexity

The complexity of the algorithm is an important index to evaluate the performance of the algorithm. It is usually measured by the computational cost when embedding and extracting the watermark. In our experiment, the average time for extracting the watermark is 0.8544 s and that for embedding the watermark without GA is 1.6246 s. When GA is used to search for the best algorithm parameter, the embedding time is related to the evolution time of GA. It can be seen that GA enables our proposed algorithm to achieve a good balance between transparency and robustness, but it also brings a large computational cost. When embedding the watermark, the average time for searching the embedding region in a voiced frame is 0.037 ms. When extracting the watermark, the average time for tracking the extracting region in a voiced frame is 0.036 ms. It can be seen that our proposed ISM takes up very little computational cost to find the synchronization marks.


## 5. Conclusions

In our study, it is found that the playing time of the audio will be longer or shorter after being attacked by TSM, but the shape of the voiced frame will not change basically. Therefore, an ISM which can search for the embedding region where the watermark is located is developed, in which it takes the sample point with the largest amplitude in the voiced frame as the synchronization mark. GA is utilized to optimize the key algorithm parameter to balance both transparency and robustness. Combining the “energy concentration” characteristic of DCT and the stability characteristic of SVD, a robust and blind audio watermarking algorithm with ISM and GA is proposed for overcoming malicious synchronization attacks and conventional signal processing operations.The following measures are taken to improve the algorithm robustness. Firstly, the proposed ISM can accurately track the region where the watermark is located. Even if the structure of the audio changes slightly, the ISM can accurately search this synchronization mark in the voiced frame to track the region that can be used to embed and extract the watermark. Secondly, GA is utilized to optimize the key algorithm parameter to balance both transparency and robustness. Thirdly, the audio will be divided evenly twice to avoid the drift of the embedding region caused by the change of the audio structure. At last, the watermark is repeatedly embedded in three voiced frames to improve the algorithm robustness. Embedding the same watermark in the three voiced frames is equivalent to embedding the watermark with a triple repetition code. The experimental results confirm that this proposed algorithm has excellent robustness in the case that the payload capacity is 64 bps; it can not only withstand conventional signal processing operations but also resist TSM, PSM, jittering, and random cropping. Especially, this algorithm even stands up to TSM with strength from −30% to +30%.Although the proposed algorithm has excellent robustness when overcoming TSM, jittering, random cropping, and various conventional signal processing operations, the experimental results of this algorithm under PSM are not good enough, mainly because PSM makes the synchronization mark in the voiced frame shift, which leads to the error bits in the extracted watermark. Therefore, the performance of this algorithm is not enough when withstanding some attacks, such as deliberately distorting the peak amplitude points to remove synchronization mark, which will be further studied in our future work. In addition, GA is used to optimize the key algorithm parameter, which is very helpful to balance the transparency and robustness. However, GA needs a long evolution time to search for the optimal algorithm parameters, which greatly increases the computational cost. Based on this, this algorithm is not suitable for the application with strict time requirements. In future research, we will strive to enhance the security and robustness against more types of synchronization attacks.



---

*Source: 2035747-2020-11-21.xml*