## 2 Related Work and Background  
### Background: Deep Neural Networks  
In this work, we focus on classical DNN architectures for image classification, as the vast majority of DNN watermarking schemes have been developed and evaluated in this context. As depicted in Figure 1, a DNN model \(M\) is a stack of layers that transforms an input layer holding data \(X\) to classify into an output layer holding the label scores \(Y\). Different types of layers are commonly used:  
**Fully connected (FC) layer--**It consists of a set of neurons such that each neuron of layer FC\({}^{m}\) is connected to all neurons of the previous layer FC\({}^{m-1}\). More precisely, an FC layer of \(I_{m}\) neurons connected to a previous layer of \(I_{m-1}\) neurons receives as input a vector \(a^{(m-1)}\) of size \(I_{m-1}\) and provides as output a vector \(a^{(m)}\) with \(I_{m}\) components that serves as input to the next layer. This computation can be expressed as  
\[a^{(m)}=g_{m}(W^{(m)}a^{(m-1)}+b^{(m)}) \tag{1}\]  
where \(W^{(m)}\) is the matrix of neuron weights of shape \(I_{m}\times I_{m-1}\), \(b^{(m)}\in\mathbb{R}^{I_{m}}\) is the bias vector, and \(g_{m}(\cdot)\) is a nonlinear activation function applied component-wise (e.g., ReLU, sigmoid, and Tanh...). 2. **Convolutional layer** Conv\((O_{m},P_{m},Q_{m})\)--As illustrated in Figure 1, such a layer receives as input a feature map \(A^{(m-1)}\) of \(O_{m-1}\) channels and computes as output a new feature map \(A^{(m)}\) composed of \(O_{m}\) channels. The output at each channel is called an activation map and is computed as \[A^{(m)}_{\circ}=g_{m}\left(\sum_{k}W^{(m)}_{\circ k}*A^{(m-1)}_{k}+b^{(m)}_{ \circ}\right)\] (2) where \(*\) denotes the 2D convolution operation, \(W^{(m)}_{\circ k}\) is the filter weight matrix of shape \(P_{m}\times Q_{m}\), and \(b^{(m)}_{\circ}\in\mathbb{R}\) is the bias term. The matrix \(W^{(m)}_{\circ k}\) parameterizes a spatial filter that is automatically tuned during training to detect or enhance specific features from \(A^{(m-1)}\). 3. **Pooling (PL) layers--**These implement spatial dimensionality reduction operations in DNNs. The most common pooling layers are max pooling and average pooling. MaxPooling\((n,n)\) is a patch kernel operator of size \(n\times n\) applied to \(A^{(m)}\) that outputs the maximum value found in each \(n\times n\) patch of \(A^{(m)}\). For average pooling, the average of the values in a patch is computed. The appropriate choice of pooling function can make the model more robust to input distortions. In any case, a PL layer reduces the number of trainable parameters for subsequent layers. 4. **Flatten layer--**This corresponds to a special reshaping operation required to transition from bidimensional (or multidimensional) layers to one-dimensional, fully connected layers. 5. **Batch normalization (BN) layers--**These layers normalize inputs so as to have zero mean and unit variance, which helps stabilize and accelerate the training process. BN layers contain learnable parameters (scale and shift) that have been utilized in some watermarking schemes, particularly in passport-based methods [20], where watermarks are embedded in these normalization parameters. Architecting a DNN involves developing a suitable sequence of convolutional, pooling, batch normalization, and fully connected layers, along with their hyperparameters (e.g., learning rate, momentum, and weight decay). Typical architectures, as shown in Figure 1, introduce a pooling layer after one or more convolutional layers, defining a convolutional block that is repeated until the activation map size is small enough to feed fully connected layers. A typical DNN training is based on two iterative phases: feed-forward and back-propagation. Before the first feed-forward phase begins, all model parameters are initialized (e.g., with random values). The training data are then fed into the DNN in batches, where the batch size is the number of samples used to perform a single forward and backward pass.