# Spatial Domain-Based Nonlinear Residual Feature Extraction for Identification of Image Operations
## 2 Proposed Approach for Identification of Image Operations
### Employed Convolutional Neural Network Model for Classification  
With the extracted SDNR feature, we next propose applying the five-layer CNN, as shown in Figure 3, to detect and identify a variety of image operations. The applied five-layer CNN includes two convolutional layers, two pooling layers, and one fully connected layer, followed by one softmax classifier. Compared with the conventional methods, which take pixel values as input, the proposed method improves the generalization ability and accelerates the network convergence by replacing the pixel values with extracted features. To match the feature size of \(1\times 325\) as explained above, we set the size of the input layer to be \(20\times 20\) and employ a padding operation to the extracted features. The convolutional layers aim to extract the feature maps, and each of their neurons is connected with the neighboring neuron in its former layer. As shown in Figure 3, our employed CNN model involves two convolutional layers and two pooling layers. Regarding the convolutional layers, convolutional layer 1 has six kernels with a receptive field size of \(5\times 5\), and each feature map is \(16\times 16\) in size; while convolutional layer 2 has twelve kernels with a size of \(5\times 5\), and each feature map has a size of \(4\times 4\). The subsampling layer, which is an important component of the CNN model, is put between the two convolutional layers. Through reducing the connections between the two convolutional layers, the subsampling layer helps to reduce the calculation complexity. Regarding the pooling layers, both of these two layers have one kernel with a size of \(2\times 2\) that resamples the input spatially and reduces 75% of the activations. The down pooling methods that are frequently used are mean pooling and max pooling. In addition to the two convolutional layers and the two subsampling layers described above, the employed CNN model involves one full connection layer that is connected with each neuron in the former layer and is followed by a softmax classifier. The full connection layer transforms the feature map extracted from the former layer into a vector with a size of \(1\times 48\) and feeds this vector into the softmax classifier to perform identification. Softmax regression is the promotion of a logistic regression model in problems with multiple classifications. The estimation of probability values is usually achieved through the hypothetical function \(h_{\theta}(x)\) in Equation (9). Given \(x\) as the input and \(y\) as the class label, we assume that the output of the function is a \(k\)-dimensional vector, which means  
Figure 3: Architecture of the employed convolutional neural network (CNN). that there are \(k\) numbers for the value of the class label \(y\), and the sum of the vector factors in the representation of the \(k\) estimated probability values is 1. \[h_{\theta}(x^{i})=\left( \begin{array}{c}p\Big{(}y^{(i)}=1\Big{|}x^{i};\theta \Big{)}\\ p\Big{(}y^{(i)}=2\Big{|}x^{i};\theta\Big{)}\\ \vdots\\ p\Big{(}y^{(i)}=k\Big{|}x^{i};\theta\Big{)}\end{array} \right)=\frac{1}{\sum \limits_{j=1}^{k}e^{\theta_{j}^{\tau}x^{(i)}}}\left( \begin{array}{c}e^{ \theta_{1}^{\tau}x^{(i)}}\\ e^{\theta_{2}^{\tau}x^{(i)}}\\ \vdots\\ e^{\theta_{k}^{\tau}x^{(i)}}\end{array} \right) \tag{9}\]  
where \(\theta_{1},\theta_{2}\cdots,\theta_{k}\in\Re^{n+1}\); \(\frac{1}{\sum\limits_{j=1}^{k}e^{\theta_{j}^{\tau}x^{(i)}}}\) represents the normalization of the probability distribution. Usually, the softmax regression algorithm can be solved by minimizing the cost function; however, it has been proven that there is more than one minimization solution of the cost function of a softmax regression algorithm. To solve the multisolution phenomenon, we propose employing the method that adds a weight attenuation term into the cost function. The cost function after adding the weight attenuation term is shown in Equation (10).