# Review of Image Forensic Techniques Based on Deep Learning
## 4 Robust Image Watermarking Based on Deep Learning
### Robust Image Watermark against Non-Differentiable Attack
#### 4.2.1 Robust Image Watermark against JPEG Attack
This was related to its use of random initialization of parameters in the maximum pooling, convolutional layer, and 3D noise layer during the simulation, which resulted in a poor simulation effect for JPEG compression. Due to the introduction of the non-differentiable nature of real JPEG compression, it cannot be introduced into the end-to-end network to implement back-propagation updates of model parameters directly. To address this problem, [PERSON] et al. [125] used a mini-batchof real and simulated (MBRS) JPEG compression to improve the JPEG compression attack robustness. In the attack layer, one of several small mini-batches attacks was selected randomly from the real, simulated, and equivalent sound layers as the noise layer. Please note that the attacks polled by the model in the first iteration are simulated to facilitate cumulative updates based on the differentiable gradient of the first iteration. This was performed thanks to the Adam momentum optimizer with its historical gradient update, which is expressed as Equations (12)-(16). \[m_{t}=\beta_{1}\cdot m_{t-1}+(1-\beta_{1})\cdot g_{t} \tag{12}\]  
\[v_{t}=\beta_{2}\cdot v_{t-1}+(1-\beta_{2})\cdot g_{t}^{2} \tag{13}\]  
\[\hat{m}_{t}=\frac{m_{t}}{1-\beta_{1}^{t}} \tag{14}\]  
\[\hat{v}_{t}=\frac{v_{t}}{1-\beta_{2}^{t}} \tag{15}\]  
\[\theta_{t}=\theta_{t-1}-\frac{\eta}{\sqrt{\hat{v}_{t}+\varepsilon}}\cdot\hat{ m}_{t} \tag{16}\]  
where \(m_{t}\) and \(v_{t}\) denote the first and second moment estimates at the time step \(t\), i.e., the exponential moving average of the gradient and the gradient squared. Where \(\beta_{1}\) and \(\beta_{2}\) denote the average coefficient, usually set to a value close to \(1\). \(g_{t}\) denotes the gradient in time step. \(\theta_{t}\) denotes the model parameter in time step \(t\). \(\eta\) denotes the learning rate. \(\varepsilon\) denotes a small constant added for numerical stability to prevent the denominator from being zero. \(\hat{m}_{t}\) and \(\hat{v}_{t}\) denote unbiased estimation modified to the first and second moment estimation, respectively. Thanks to the Adam momentum optimizer with its historical gradient update, even if the attack layer rotated to the non-differentiable real JPEG compression attack, the internal parameters of the codec network could still be updated by the accumulation of historical differentiable gradients, which avoided the problem of non-differentiable real JPEG compression and achieved a better simulation quality of JPEG compression effectively. However, the model ignored the feature tensor of the image in the spatial and channel directions for image feature extraction, which led to poor robustness in the face of high-intensity salt and pepper noise. [PERSON] et al. [126] proposed a pseudo-differentiable JPEG method. JPEG pseudo-noise was the difference between the compressed processed image and the original image. Since its backpropagation without going through the pseudo-noise sound path, there was no problem of non-differentiable. However, its robustness to conventional noise was poor due to the lack of noise prior when back propagating. [PERSON] et al. [127] proposed a robust watermarking framework by combining reversible and irreversible mechanisms. In the reversible part, a diffusion extraction module (DEM) (consisting of a series of fully connected layers and a Haar transform) and a fusion separation module (FSM) were designed to implement watermark embedding and extraction in a reversible manner. For the irreversible part, the irreversible attention model which was composed of a series of convolution layers including full-connected layer, squeeze and excitation block, and a dedicated noise selection module (NSM) were introduced to improve the JPEG compression robustness.