# Hybrid CNN-NLP Model for Detecting LSB Steganography in Digital Images
## 3 CNN-NLP Architecture Design for LSB Steganography Detection
### CNN Architecture  
The three-dimensional tensor resulting from the image preprocessing stage serves as the input to the proposed CNN architecture. The network consists of three convolutional blocks with residual connections, followed by a global average pooling (GAP) layer, a fully connected dense layer with 128 units (using 50% dropout) with the \(LeakyReLU\) activation function, and, finally, a \"Softmax\" layer utilized for binary classification. Each residual convolutional block receives as input a three-dimensional tensor \(X\in\mathbb{R}^{150\times 150\times 5}\) (see Equation (4)), which, as described above, is composed of a normalized residual image and four bit planes corresponding to the extracted LSBs. The residually connected convolutional blocks \(B_{i}\) are defined as follows:  
\[Z_{1}^{(i)}=LeakyReLU\Big{(}BN\Big{(}W_{1}^{(i)}*X^{(i)}+b_{1}^{(i)}\Big{)} \Big{)} \tag{5}\]  
where a series of sequential operations is applied to \(X^{(i)}\), and \(*\) corresponds to the convolution operator. First, a two-dimensional convolution operation is performed with a set of filters \(W_{1}^{(i)}\) and a bias \(b_{1}^{(i)}\). Next, the result of this operation is normalized using BN, which is batch normalization. Then, the \(LeakyReLU\) activation function is applied, which allows the passage of negative values, improving the gradient flow and preventing the deactivation of neurons. The intermediate output \(Z_{1}^{(i)}\) is then processed by a second convolutional layer, which applies a new set of filters \(W_{2}^{(i)}\) and a bias \(b_{2}^{(i)}\), followed again by batch normalization and a \(LeakyReLU\) activation, resulting in \(Z_{2}^{(i)}\), as expressed in Equation (6):  
\[Z_{2}^{(i)}=LeakyReLU\Big{(}BN\Big{(}W_{2}^{(i)}*Z_{1}^{(i)}+b_{2}^{(i)}\Big{)} \Big{)} \tag{6}\]Subsequently, as seen in Equation (7), a residual connection \(Y^{(i)}\) is implemented summing the output \(Z^{(i)}_{2}\) with the original input of the block \(X^{(i)}\), which allows the input information to be preserved throughout the block, facilitating the training of deep networks. Finally, a spatial reduction with a window of \(2\times 2\) and stride 2 is applied, which reduces the spatial dimensions of the activation map. \[Y^{(i)}=MaxPool\Big{(}Z^{(i)}_{2}+X^{(i)}\Big{)} \tag{7}\]  
On the other hand, it is worth noting that _LeakyReLU_ is the activation function used in all convolutional blocks, which allows the passage of gradients even when the activation is negative, thereby mitigating the problem of gradient flow and preventing neurons from becoming inactive during training, a common issue with _ReLU_. The activation function used for this case is defined in Equation (8):  
\[f(x)=\left\{ \begin{array}{ll}x,&\text{if }x>0;\\ 0.2x,&\text{if }x\leq 0.\end{array} \right. \tag{8}\]  
All of the above makes the block residual and allows some of the original information to flow unaffected by the convolutions, improving learning and preventing the gradient from fading. Once the three residual convolutional blocks have processed the input tensor, the resulting activation map passes through a GAP layer, which allows all the spatial information learned by the filters to be condensed into a feature vector \(g\in\mathbb{R}^{C}\) by calculating the average of each channel (or filter) over the entire resulting image. The mathematical operation to generate this vector is shown in Equation (9):  
\[g=\frac{1}{HW}\sum_{x=1}^{H}\sum_{y=1}^{W}Y^{(3)}(x,y,:) \tag{9}\]  
The dimensions \(H\) and \(W\) correspond to the spatial height and width of the output of the last convolutional block. In the case of the proposed model, after applying three \"Max-Pooling\" layers with stride 2, the final dimensions are \(H=18\) and \(W=18\), resulting in a global feature vector of a size equal to the number of filters (\(C=256\)). The resulting feature vector \(g\in\mathbb{R}^{C}\), obtained from the GAP layer as described in Equation (9), is then passed through a fully connected layer with 128 neurons.