# Improved Neural Differential Distinguisher Model for Lightweight Cipher Speck
## 3 Improved Neural Distinguishers Model for Speck32
### Design the Network Structure  
In this paper, the model of [PERSON] [14] is improved in order to smoothly converge the residual neural network to an optimal solution. A network model with higher accuracy is proposed, which significantly reduces the training time and makes it more efficient to attack the Speck. The framework of the neural network is shown in Figure 5.  
The neural network is divided into four parts: an input layer consisting of multiple ciphertext pairs, an initial convolutional layer made of a one-layer convolutional neural network, a residual tower consisting of three layers of convolutional neural network optimized by the Inception module, and a prediction head consisting of multiple fully connected layers (distinguished by the colors in Figure 5). **Initial convolution. ** After transforming the one-dimensional initial ciphertext data from Section 4.2 into \([s,w,\frac{2L}{w}]\) three-dimensional input data, the train data enters the initial convolution layer, where L represents the block size of the target cipher, \(w\) is the size of a basic unit, and \(s=16\), \(w=16\) for Speck32. The number of channels in the initial convolutional layer is \(3N_{f}\), where \(3N_{f}\) is the number of filters in the convolutional layer and \(N_{f}=32\). The convolutional layer is first convolved by a convolutional kernel of size 1, and then the convolved results are batch normalized. Finally, rectifier nonlinearity is applied to the output of batch normalization, and the resulting \([s,w,3N_{f}]\) matrix is passed to the residual block. **Residual block. ** The residual neural network model constructed in this paper contains five residual blocks. Compared with the neural network model of [PERSON] [14], the residual block in this paper contains three convolutional blocks of \(3N_{f}\) channels. The first convolutional block is convolved by a convolutional kernel of size one and then the output data is directly transferred to the second convolutional block. The second convolutional block consists of a \(2\times 2\) convolutional layer, a \(4\times 4\) convolutional layer, and an \(8\times 8\) convolutional layer; each convolutional layer has \(N_{f}\) channels. The three convolutional layers are concatenated in the channel dimension, similar to the Inception module in GoogLeNet. Batch normalization is applied to the output of the concatenated layers. The nonlinearity of the rectifier is applied to the output of batch normalization. The output from the second  
Figure 5: The framework of the neural network. convolution block is applied to the convolution of kernel size one, then a batch normalization layer, and finally a rectifier layer. At the end of the convolution block, the output of the last rectification layer in that block is added to the input of the convolution block, and the result is transferred to the next block. **Prediction head. ** The prediction head consists of a GlobalAveragePooling layer, two hidden layers, and one output cell. The three fully connected layers consist of 64, 64, and 1 neural unit, followed by batch normalization and rectifier layers. Finally, in order to constrain the final output neural unit between 0 and 1, the output neural unit is activated using the Sigmoid activation function. The Sigmoid activation function is a logistic function defined as \(f(x)=\frac{1}{1+e^{-x}}\). The overall structure of the neural network for training the differential-neural distinguisher is shown in Figure 6.  
**Basic training scheme. ** In this paper, the training is run for 20 epochs on the training dataset with a \(SN=10^{6}\) sample size. The batch size for dataset processing is adjusted according to parameter \(s\) to maximize GPU performance, where \(s\) is the number of ciphertext pairs in a single sample, so the number of ciphertext pairs in the training dataset is \(CN=SN\times s\). The \(SM=10^{5}\) samples will be used for validation, containing \(CM=SM\times s\) ciphertext pairs.