# An Audio Watermarking Algorithm Based on Adversarial Perturbation +
## 3 The Proposed Methodology
### Using Pre-Trained Module for Transformations
#### 3.2.2 Contrastive Learning Model for Audio Features  
Although a supervised classification pre-trained model can improve feature invariance by data augmentation, it also suffers from semantic collapse, in that the supervised model will tend to learn only the information needed for classification [54]. Therefore, we used contrastive learning [55] to obtain another self-supervised PTM.  
The structure of the PTM is the same as the feature extractor in Section 3.2.1; however, we used a different training scheme which consists of four main components:  
1. A randomized data augmentation produces two augmented counterparts of the same audio track; the data augmentation is shown in Table 1.
2. An audio feature extractor maps the augmented audio to the feature domain, which will also function as the PTM in our watermarking system.
3. A projection operator \(f_{p}(\cdot)\) obtains audio representations based on audio features, and the representations are used to compute the contrastive loss. The projection operator consists of two linear layers and an activation layer as \[\mathbf{z}=f_{p}(\mathbf{o}_{ptm})=\mathbf{W}_{p,2}\text{ReLU}(\mathbf{W}_{p,1 }\mathbf{o}_{ptm})\] (2) where \(\mathbf{W}_{p,1}\) and \(\mathbf{W}_{p,2}\) are the parameters of linear layers.
4. A contrastive loss function is used to compute the similarity of audio representations, where different augmented counterparts of the same audio have high similarity and different audio have low similarity. Herein, normalized temperature-scaled cross entropy (NT-Xent) [56] is adopted as the loss function.  
\begin{table}
\begin{tabular}{c c c} \hline \hline
**Abbreviation** & **Attack** & **Description** \\ \hline CLP & Closed loop & Audio is not attacked. \\ WGN & White Gaussian noise & Gaussian noise with SNR of 20 dB is added. \\ LPF & Low-pass filter & Audio is filtered, with 8 kHz cutoff. \\ HPF & High-pass filter & Audio is filtered, with 100 Hz cutoff. \\ CRP & Random cropping & Audio is cropped randomly, retaining 80\% of duration. \\ RVB & Reverberation & Simulating audio propagation in a closed room. \\ DEN \({}^{\star}\) & Simulated denoising & A combination of WGN and LPF. \\ REC \({}^{\star}\) & Simulated recording & A combination of RVB, WGN, LPF, and HPF. \\ \hline \hline \end{tabular}
\end{table}
Table 1: The transformations or attacks used in this study. The transformations labeled \({}^{\star}\) are not applied to the training of the PTM.  
Figure 4: The structure of the residual block. The \(\sigma\) means the sigmoid function.  
Figure 5 illustrates the complete training scheme. In contrastive learning, augmented counterparts of one audio sample are mutually positive samples, e.g., \((\mathbf{x}_{1}(n),\mathbf{x}_{2}(n))\) and \((\mathbf{y}_{1}(n),\mathbf{y}_{2}(n))\) are two pairs of positive samples. The other audio samples are referred to as negative samples. The NT-Xent loss for a positive pair of audio \((i,j)\) is defined as  
\[\ell_{i,j}=-\log\frac{\exp(\sin(\mathbf{z}_{i},\mathbf{z}_{j})/\tau)}{\sum_{k= 1}^{2N}\mathbb{I}_{k\
eq i}\exp(\sin(\mathbf{z}_{i},\mathbf{z}_{k})/\tau)} \tag{3}\]  
where \(\mathbb{I}_{k\
eq i}\in\{0,1\}\) is an indicator evaluating to \(1\) iff \(k\
eq i\) and \(\tau\) denotes a temperature parameter that helps the loss to control the sensitivity to negative samples. The loss is computed for all positive pairs in a mini-batch, both \((i,j)\) and \((j,i)\). More details on training for contrastive learning can be found in the literature [55; 57].