Secure echo‐hiding audio watermarking method based on improved PN sequence and robust principal component analysis

IET Signal ProcessingVolume 14, Issue 4 p. 229-242 Research ArticleFree Access Secure echo-hiding audio watermarking method based on improved PN sequence and robust principal component analysis Shengbei Wang, Shengbei Wang orcid.org/0000-0002-4713-5012 Tianjin Key Laboratory of Autonomous Intelligence Technology and Systems, Tianjin Polytechnic University, 300387 Tianjin, People's Republic of ChinaSearch for more papers by this authorChao Wang, Chao Wang Tianjin Key Laboratory of Autonomous Intelligence Technology and Systems, Tianjin Polytechnic University, 300387 Tianjin, People's Republic of ChinaSearch for more papers by this authorWeitao Yuan, Weitao Yuan Tianjin Key Laboratory of Autonomous Intelligence Technology and Systems, Tianjin Polytechnic University, 300387 Tianjin, People's Republic of ChinaSearch for more papers by this authorLin Wang, Lin Wang Techfantasy. Co. Ltd., 300387 Tianjin, People's Republic of ChinaSearch for more papers by this authorJianming Wang, Corresponding Author Jianming Wang wangjianming@tjpu.edu.cn Tianjin Key Laboratory of Autonomous Intelligence Technology and Systems, Tianjin Polytechnic University, 300387 Tianjin, People's Republic of ChinaSearch for more papers by this author Shengbei Wang, Shengbei Wang orcid.org/0000-0002-4713-5012 Tianjin Key Laboratory of Autonomous Intelligence Technology and Systems, Tianjin Polytechnic University, 300387 Tianjin, People's Republic of ChinaSearch for more papers by this authorChao Wang, Chao Wang Tianjin Key Laboratory of Autonomous Intelligence Technology and Systems, Tianjin Polytechnic University, 300387 Tianjin, People's Republic of ChinaSearch for more papers by this authorWeitao Yuan, Weitao Yuan Tianjin Key Laboratory of Autonomous Intelligence Technology and Systems, Tianjin Polytechnic University, 300387 Tianjin, People's Republic of ChinaSearch for more papers by this authorLin Wang, Lin Wang Techfantasy. Co. Ltd., 300387 Tianjin, People's Republic of ChinaSearch for more papers by this authorJianming Wang, Corresponding Author Jianming Wang wangjianming@tjpu.edu.cn Tianjin Key Laboratory of Autonomous Intelligence Technology and Systems, Tianjin Polytechnic University, 300387 Tianjin, People's Republic of ChinaSearch for more papers by this author First published: 01 June 2020 https://doi.org/10.1049/iet-spr.2019.0376Citations: 2AboutSectionsPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onFacebookTwitterLinkedInRedditWechat Abstract Echo-hiding has been widely studied for audio watermarking. This study proposes a more secure echo-hiding method based on modified pseudo-noise (PN) sequence and robust principal component analysis (RPCA). In the proposed method, the RPCA is used to decompose the original audio signal into low-rank and sparse parts and then a pair of opposite modified PN sequences is employed to embed watermarks. The modified PN sequence improves the robustness of watermark detection by providing additional correlation peaks. Meanwhile, benefit from the RPCA and the opposite PN sequences, the security of the proposed method is improved since watermarks cannot be detected from the whole signal even if the PN sequence is known, which is an obvious improvement compared with the previous PN-based echo-hiding methods. In the watermark detection process, the authors make use of the low-rank and sparse characteristics of the watermarked signal to detect watermarks from the low-rank and sparse parts, respectively. Based on this basic framework, they also propose a multi-bit embedding scheme, which obtains a doubled embedding capacity compared with the previous PN-based echo-hiding methods. The proposed method was evaluated with respect to inaudibility, security, and robustness. The experiment results verified the effectiveness of the proposed method. 1 Introduction With the rapid development of digital media and network technology, the audio transmission has become more and more convenient. However, illegal dissemination behaviours that ignore copyright seriously harm the interests of audio authors. As a result, copyright protection has lodged itself in the public mind. In order to solve this problem, scholars want to add tags to the audio to prove the ownership and therefore the audio watermarking technology has been proposed [1, 2]. After so many years of exploration, the achievements of watermarking technology have been significantly improved. Audio watermarking [3] has been considered as an effective technique to prevent audio from unauthorised operations. In general, there are several requirements for audio watermarking, e.g. inaudibility, blindness, robustness, capacity, and security [3-5]. Inaudibility requires that the embedded watermarks should not degrade the audio quality [6]. Blindness suggests that the watermarks can be detected without the original audio. At present, more and more scholars pay attention to blind watermarking. Robustness guarantees that the embedded watermarks cannot be destroyed by allowable audio operations such as compression. Embedding capacity is also necessary to evaluate the watermarking method since the copyright of the audio signal can be better protected when more watermarks are embedded. The last and most recent concern on watermarking is that the copyright information contained in the watermarked signal should not be easily discovered by the attackers, which is called security. In general, the five requirements described above are the main criteria for evaluating a watermarking method [7]. In the past, many audio watermarking methods have been proposed, e.g. support vector [8, 9], phase coding [10], spread spectrum [11, 12], techniques based on masking [13, 14], patchwork [15, 16], echo-hiding [17-23], and so on. As one typical audio watermarking method, echo-hiding has a simple and easy-to-operate embedding and detection process. Besides, the watermark detection of echo-hiding does not need the original signal (i.e. it is a blind method). The echo-hiding method was first proposed by Gruhl et al. [17], who described how to embed the watermarks using one backward echo kernel and how to detect it using cepstrum operations. As a critical factor for echo-hiding, echo kernel plays a vital role which greatly affects the performance of the watermarking method. Therefore, many echo-hiding algorithms have been proposed to design more effective kernels, such as the dual-kernel [24-26], backward–forward echo kernels [27] etc. Although the above echo-hiding methods have a simple embedding and detection process, they have a fatal security flaw, since the watermarks can be easily obtained with a cepstrum analysis of the watermarked signal. In order to overcome this drawback, the security algorithm [18, 19] using pseudo-noise (PN) sequence has been proposed. The PN sequence is used as a secret key to embed multiple echoes into an original signal. Owing to very small amplitude of each echo, there will be no obvious peak in the power spectrum after cepstrum analysis, that is, its power spectrum becomes nearly smooth in the mean time sense. Therefore, it is impossible to detect the watermarks directly through cepstrum analysis in these watermarking schemes. In order to obtain the watermarks, the corresponding PN sequence must be used for correlation operation after the cepstrum analysis, which greatly increases the security of the basic echo-hiding methods [17, 24-27]. In [19], the PN sequence-based echo-hiding method was further improved. During the watermark detection process, the authors used real cepstrum instead of the complex cepstrum to make the cepstrum peak more obvious. In recent years, some modified PN sequences have been proposed for echo-hiding methods to enhance robustness and inaudibility performance. In [28], the PN sequence was modified to a new sequence denoted by q (n) and three peaks were obtained after correlation, which improved the accuracy of watermark detection. In [21], the PN sequence was improved and more large peaks were produced after correlation. These two schemes improved the performance of watermarking methods by modifying the PN sequence and taking advantage of the correlation operation. However, the algorithms mentioned above still have the following shortcomings: (i) Low security: In these methods, the PN sequence is used as a security key for watermark detection. If the PN sequence is leaked, the watermarks will be easily detected by the attackers. Furthermore, many methods, e.g. [21, 28], try to obtain better robustness by modifying the PN sequence, however, such modifications also make the PN sequence more regular, thus the complexity of the PN sequence is reduced which makes it easier to crack. (ii) Low capacity: For most PN sequence-based echo-hiding methods, the PN sequence should be relatively long to achieve good inaudibility and robustness performance. However, if the length of the PN sequence is guaranteed to be long enough, the number of watermarks that can be embedded will be reduced. To solve the above issues, this paper proposes a more secure echo-hiding method based on improved PN sequence and the robust principal component analysis (RPCA) [29]. As we considered, the root cause of the security problem in PN-based echo-hiding methods is that these methods directly add the echoes on the original whole signal. Therefore, watermarks can be easily detected after cepstrum and correlation analysis of the watermarked signal when the PN sequence is known. In this paper, the RPCA is employed to improve the PN-based echo-hiding method. The original signal is first decomposed into two parts, i.e. low-rank and sparse parts, using RPCA. Watermarks are separately embedded into them using a pair of opposite PN sequences. Accordingly, the cepstrum and correlation analysis of whole watermarked signal can no longer produce any obvious peaks for watermark detection even using the correct PN sequences, since there is an opposite effect between the correlation results of the two parts in the watermarked signal (low-rank and sparse parts). To correctly detect the watermarks, the proposed method takes advantage of the low-rank and sparse characteristics of the embedded echoes. Watermarks can be separately detected from the low-rank and sparse parts using the correct decomposition parameter of RPCA and the corresponding PN sequence. In particular, the PN sequence employed above is designed by considering the correlation property of the PN sequence and it provides two extra correlation peaks to effectively improve the robustness of watermark detection. Based on the above framework, this paper also implements a multi-bit embedding scheme which achieves a doubled embedding capacity compared with the previous PN-based echo-hiding methods. This paper is organised as follows. Section 2 reviews typical echo-hiding methods. Section 3 describes our proposed method based on the improved PN sequence and the RPCA. Section 4 evaluates the inaudibility, security, and robustness of the proposed method with a series of experiments and compares it with the previous echo-hiding method and the PN sequence-based echo-hiding method. In the last section, we give a summary of our work. 2 Review of typical echo-hiding The widely accepted model of echo-hiding is (1)where is the original audio signal, is the echo kernel, is the watermarked signal, and the operation symbol stands for convolution. The backward echo kernel is defined as (2)where is a Dirac delta function, denotes the attenuation amplitude of the echo, and d is the delay of the echo. To improve the security, the PN sequence-based echo kernel [18, 19] was proposed, which is given by (3)where , is the PN sequence of length L. The pulse representation of PN-based echo kernel is shown in Fig. 1. In the watermark detection process, the watermarked signal is analysed by real cepstrum analysis of (1), i.e. (4)where , , is the absolute value operation, and and are Fourier transform and inverse Fourier transform, respectively. For PN-based echo-hiding, the can be calculated in more detail [19] (5)The PN sequence is a necessary condition for obtaining the watermarks. The cross-correlation operation of (4) is carried out with the PN sequence (6)where calculates the mathematical expectation. We know from (6) that is small and negligible, and when , the term of has a maximum value of . Hence, we can detect the watermarks by detecting the maximum value (peak value). Fig. 1Open in figure viewerPowerPoint Echo kernels based on PN sequence In order to further improve the above algorithms, some modified PN sequences were proposed. In [28], the modified PN sequence was proposed, which is defined by (7)where (8)and the function is used to take an integer in the direction nearest to zero. The correlation operation of q (i) is (9)The calculation result is shown in the top two panels of Fig. 2. We can find that there are three peaks at , , and . By detecting these three peaks, the robustness of watermark detection is effectively improved. Fig. 2Open in figure viewerPowerPoint Top two panels: left panel: the result of and right panel: the zoom-in observation of the left panel, where the length of is and . Bottom two panels: left panel: the result of and right panel: the zoom-in observation of the left panel, where the length of is , , and In [21], the PN sequence is modified to (10)where is defined as the number of repetitions. The correlation result of this sequence can be seen in the bottom two panels of Fig. 2. We can obtain peaks for watermark detection. These peaks are located at . The above two methods indeed improve the robustness of watermark detection compared with the original PN sequence. However, such modifications to the PN sequence reduce its complexity and therefore make it easier to crack, i.e. the watermarking method will be less secure. In addition, to ensure the performance of the watermarking method, the PN sequence should be as long as possible. However, this will reduce the embedding capacity of the echo-hiding method. In the next section, we will introduce the proposed echo-hiding method based on the improved PN sequence and RPCA. 3 Proposed watermarking scheme 3.1 RPCA decomposition Although the PN sequence increases the security of the echo-hiding method, the watermarks can be easily obtained by the attackers when the PN sequence is known. The key reason for this is that most of the previous echo-hiding methods (including the PN sequence-based methods) directly add the echoes to the original signal. Accordingly, the cepstrum of the watermarked signal can be regarded as the addition of the cepstrum of the original signal and that of the echo kernel, which is conducive for the attackers to watermark detection. We considered if the echoes were not added directly to the original signal, the watermarks will not be easily obtained even if the PN sequence is known. As a result, the security of the typical echo-hiding methods and the PN-based echo-hiding methods will be further improved. This paper introduces RPCA [29, 30] to the original PN-based echo-hiding methods. In the proposed method, the original audio signal is first decomposed into two parts, i.e. low-rank and sparse parts, and then watermarks are separately embedded into them using the proposed kernels. That is, the echoes are not added directly to the original signal but to its sub-signals. The process of audio decomposition based on RPCA is as follows. Since the audio signal (denoted as ) is the one-dimensional signal, we first compute the short-time Fourier transform (STFT) representation of in time–frequency (T –F) domain. The obtained T –F representation is denoted by , where F is the number of frequency bins and T is the number of time bins. Note that to obtain a good decomposition effect, the window size and hop size (half of the window size) change with frame lengths, to ensure the obtained T –F representation a square matrix (approximately) (For example, for frame length of 5512 (i.e. 8 bps for 44.1 kHz sampled signal), we use window size of 144 and hop size of 72. The obtained is nearly a square matrix, where and . The window size and hop size for other frame length can be calculated the same way.). The magnitude and phase spectrograms of are and , respectively. The decomposition operation of the RPCA is performed on by solving the following convex optimisation problem using principal component pursuit: (11)where is a low-rank matrix, is a sparse matrix, and the () is a positive parameter that controls the decomposition balance between the low-rank and the sparse parts. The is the nuclear norm operation and the is the -norm. In (11), the and are separately defined as (12) (13)where is the i th singular value of , f is the index of frequency bin (), and t is the index of time bins (). When considering the constraint condition , the objective function in (11) can be set as follows (14)where is a positive scalar, the is the Frobenius norm. The and can be obtained by solving (14) with proximal gradient method. The obtained and are then synthesised with the previously obtained phase spectrogram P and we can obtain the time-domain low-rank signal and sparse signal , respectively, using inverse STFT. Then the is decomposed as (15)Watermarks are embedded into and , respectively. Note that, since different will produce different decomposition results, the can be also used as a secret key for the proposed method. In the next section, we will describe how the proposed method improves security during the watermark embedding and detection process and how to increase the embedding capacity of the proposed method with a multi-bit embedding scheme. 3.2 Watermark embedding process When and are obtained, the one-bit watermark w is duplicated as and and then separately embedded into them. The embedding process is shown in the top panel of Fig. 3. Fig. 3Open in figure viewerPowerPoint Process of watermark embedding and detection 3.2.1 Design of the echo kernel Before watermark embedding, the echo kernels for low-rank and sparse parts need to be designed. As to ensure the robustness of the proposed method, a new echo kernel is designed. In general, the cross-correlation describes the degree which two functions (sequences) match each other at different relative positions. This degree can be calculated by mathematical expectation, i.e. . The cross-correlation turns into the autocorrelation when these two functions are identical and when they are completely matched, a positive peak will occur, i.e. . Conversely, when they are completely mismatched, a negative peak will occur, i.e. . Inspired by [21, 28], we propose a new modified PN sequence (16)where , can be any PN sequence, and is determined by the frame length and less than the frame length. For example, when then . When calculating , there will be two negative peaks appearing at and with peak value of 0.5 (i.e. ) except for the positive peak at , since . As shown in Fig. 4, we can clearly find three peaks in the correlation result. Therefore, detection of the watermarks using these three peaks can considerably improve the robustness. Fig. 4Open in figure viewerPowerPoint Results of , where the length of is and It should be noted that the proposed PN sequence in (16) can also be designed as (17)Similar to the above analysis, there will be two positive peaks appearing at and with a peak value of 0.5 (i.e. ) except for the positive peak at . The result of is shown in Fig. 5. By comparing Figs. 4 and 5, we can know that the detection performance of and are the same. In the experiment, we use to evaluate the proposed method. Fig. 5Open in figure viewerPowerPoint Results of , where the length of is and Compared with the previous PN sequences, the proposed PN sequence is also regularised after modification. However, as the proposed PN sequence is not modified in adjacent (two or three adjacent) positions [21, 28], but over a wide range (i.e. L), it is less regular than the previous modified PN sequences. 3.2.2 Secure embedding with opposite kernels To improve the security of the proposed method, we use and to design the echo kernel for the low-rank and sparse parts. In addition, we use the forward and backward echo kernels [27] to strengthen the kernels and increase the robustness of watermark detection. The kernels for the low-rank part and sparse part are defined as (18) (19)where is the kernel for the low-rank part, is the kernel for the sparse part, the j is the index of the proposed PN sequence, in (18) and (19) are set the same for the low-rank part and the sparse part. These two echo kernels are separately performed on the low-rank signal and sparse signal using convolution function to obtain the watermarked low-rank signal and the watermarked sparse signal . The final watermarked signal is obtained by (20) 3.3 Watermark detection process 3.3.1 Detection from the low-rank part and the sparse part The watermark detection process is shown in the bottom panel of Fig. 3. Similar to the embedding process, the watermarked signal is first decomposed into the low-rank part and sparse part using RPCA (21)where we use the same to decompose the watermarked signal. Recall the watermark embedding process, since the embedded echoes are separately generated by the low-rank part and the sparse part , they should have similar low-rank and sparse properties as and . As a result, the echoes generated by will be assigned to and the echoes generated by will be assigned to in watermark detection process if we use the same to decompose . Accordingly, we will have (22) (23)The real cepstrum analysis can be separately performed on the low-rank part and the sparse part for watermark detection, i.e. (24) (25)Since the forward and backward echo kernels are employed in the proposed method, by referring to [27] and the derivation of (5), the following results can be obtained (26) (27)The cross-correlation is then performed on and using the proposed sequence and , respectively (see (6)) (28) (29)According to the analysis in Section 3.2, there are one positive peak at and two negative peaks at and in the correlation result of the low-rank part and the sparse part, so we use the following equations to detect the watermarks from them (30) (31)Since the Delta function of different delays ( and ) are used to embed '0' and '1', the watermark bit can be detected by comparing the peak values at these two positions. For low-rank part, if , the watermark bit for the low-rank part is detected as , otherwise as . The detection process of the sparse part is performed the same way, i.e. if , the watermark bit for sparse part is detected as , otherwise as . In addition, we use a small trick to improve watermark detection performance. Here, we set the value before the delay d ( or ) of the watermarked signal to 0 to reduce the impact on the correlation operation. This will make the peaks in the delay position more pronounced and will improve the detection rate (DR). 3.3.2 Final decision on the watermark bit We can separately detect one-bit watermark from the low-rank part and the sparse part of one audio frame. In a general case, if the detected two watermark bits are the same, then the final watermark bit of the current frame is determined as the same bit, i.e. (32)However, if the detected two bits are different from each other, i.e. one of the following two cases happens: (i) & (i.e. and ) or (ii) & (i.e. and ), then the final watermark bit of current frame is determined as follows: (i) when & : (33) (ii) when & : (34) That is, we take into account the reliabilities of the detected watermark bits from two parts to determine the final watermark bit. Note that, similar to other previous PN sequence-based echo-hiding methods, the proposed is also required to be as long as possible to increase the inaudibility and robustness of the proposed method, which will reduce the embedding capacity. To address this problem, we make use of the obtained two sub-signals, i.e. the low-rank and sparse parts, of each audio frame, to embed more bits. We propose a multi-bit embedding scheme based on the above basic framework. This scheme is covered in more detail in the last part of this section. 3.4 Security analysis of the proposed method The security of the proposed method is ensured in a way that the watermarks cannot be detected from the whole watermarked signal. According to (20), the cepstrum analysis of the whole watermarked signal can be written as (35)or (36)Then the can be expressed as half of the sum of (35) and (36) (37)Since each audio frame has its own low-rank and sparse characteristics, we cannot confirm the analytical expressions of the low-rank part and the sparse part for each frame, and consequently, the last two terms in (37) cannot be theoretically analysed. Therefore, we experimentally investigated whether the last two terms can produce correlation peaks for watermark detection when applying the correlation operation to them. Here, we tested two cases, i.e. watermarks were separately embedded into the low-rank and sparse parts with: (i) different delays ( and ); and (ii) the same delay (). The length of the PN sequence (or ) was set as 60% of the frame length, the embedding capacity was 16 bps, and . We calculated the correlation between the last two terms of (37) and the PN sequence. To obtain the statistical results, we randomly selected 35 frames and performed correlation on each of them. Fig. 6 shows the averaged correlation result calculated on all 35 frames. It can be seen that there was no peak at the delay positions no matter for different delays or the same delay. Therefore, the last two terms of (37) have almost negligible effect on the final correlation results. Fig. 6Open in figure viewerPowerPoint Cross-correlation results between the last two terms of (37) and the PN sequence (a) The cross-correlation results when embedding watermarks using different delays (the delays are 10 and 15, respectively), (b) The cross-correlation results when embedding watermarks using the same delay (delay is 10) According to the above analysis, the in (37) can be approximately written as (38)We calculated the cross-correlation between (38) and the PN sequences ( and ) to observe if there are correlation peaks. The cross-correlation between (38) and is (39)We can see from the above equation that the first two terms in (37) always produce opposite peaks at the same delay positions when they are correlated with . These peaks are effectively compensated in the correlation result. When using for correlation, we can get the same result. Therefore, the correlation peaks cannot be discovered from the whole watermarked signal even the PN sequences of low-rank and sparse parts are known, which means the watermarks cannot be detected from the whole watermarked signal. The above analysis suggests that the proposed method is more secure compared with the previous PN sequence-based echo-hiding methods. 3.5 Multi-bit embedding scheme In the above framework, the same watermark is embedded into low-rank and sparse parts, so this does not really increase the embedding capacity. This section describes how to embed two bits into one frame under the proposed framework. In general, when we embed two bits into one frame, it turns out as the following two cases: Case 1: Two bits are the same. In this case, the two watermark bits can be '0' & '0' or '1' & '1'. In the embedding process, we use opposite PN sequences to separately embed them ('0' & '0' or '1' & '1') into the low-rank part and the sparse part. In particular, the delay for both the low-rank and the sparse will be set the same, that is, we set the delay as for '0' & '0' case and for '1' & '1' case, respectively. An example is shown in Figs. 7 a and b. As explained in Section 3.4, it would be unable to observe the correlation peaks when the PN sequences (positive or negative ) are correlated with the whole watermarked signal since we use the same delay but opposite PN sequences. Therefore, this watermark embedding method is also secure. To correctly detect the watermark bits, we apply RPCA decomposition to the watermarked signal and watermarks ('0' & '0' or '1' & '1') can be detected by finding the correlation peaks from low-rank and sparse parts, respectively. Fig. 7Open in figure viewerPowerPoint Illustration of the cross-correlation results between low-rank and sparse parts with PN sequence in multi-bit embedding scheme (a), (b) The same two watermark bits are embedded into low-rank and sparse parts, (c), (d) The different watermark bits are embedded into low-rank and sparse parts Case 2: Two bits are different. In this case, the two watermark bit