# Using Convolutional Neural Networks for Blocking Prediction in Elastic Optical Networks
## 4 CNN Structure
Usually, the output of the convolution and pooling layers passes through activation functions such as ReLU (rectified linear unit), Softmax, and Sigmoid. In this work, the activation function which is used for convolution and pooling is ReLU. Also, the two last layers of our employed CNN, which are fully connected layers, use the ReLU activation function. The Softmax activation function is employed in the output layer because this function provides a better result in classification. This activation function labels each input sequence of snapshots ranging from zero to one representing the probability of _leading-to-block_. In the design of a CNN model, the accuracy of models can be increased through calibration. Part of the calibration can be carried out by tuning the hyperparameters. Hyperparameters are the parameters used to tune the construction and consequently the performance and functionality of a neural network. The hyperparameters can also affect the computational efficiency of the neural network model. Examples of such parameters are the number of convolution layers, pooling layers, kernel size, learning rate, number of middle layers nodes, sample length, and number of training epochs. In this work, models are tuned with the hyperparameters to improve the accuracy of the prediction model. Different hyperparameters are evaluated for the proposed model's architecture through a trial-and-error sensitivity analysis. Accuracies are compared to different combinations of these parameters (detailed in Table 1), and the best combination is selected. GridSearchCV is a technique for finding the optimal parameter values from a given set of parameters in a grid. Accuracy is defined as the ratio of true results to the total number of cases investigated. With the help of the GridSearchCV technique, the best parameters for the models are explored and used. \begin{table}
\begin{tabular}{l l} \hline \hline
**Hyperparameter** & **Explored Range** \\ \hline Number of convolutional layers (stacks) & from 2 to 7 \\ \hline Number of FC layers & from 1 to 5 \\ \hline Filter size in the first and second convolutional layers & from 20 to 1500 \\ \hline Filter size in other convolutional layers (except the first 2) & 5, 20, 50, 120, and 700 \\ \hline Augmentation window samples size in different layers [25] & from 20 to 3000 \\ \hline Filter numbers [25] & from 8 to 256 \\ \hline Learning rate & from 0.01 to 0.0001 \\ \hline \multirow{2}{*}{Max-pooling patch size} & for CNN 1D: 2 to 4 \\  & for CNN 2D: 2 \(\times\) 2 to 4 \(\times\) 4 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Tuning hyperparameters. The size of the filters greatly affects the classification results. Kernels with a larger size in the first layer help the model to converge better and achieve higher classification accuracy. In this work, the topology of input data as a 1D and 2D input matrix of data is implemented. The performance of the 1D CNN being classified may be negatively impacted by the complexity of the data. On the other hand, the 2D CNN model performs much better in complex datasets. The following figures depict two different configurations of the proposed CNN models, which are used in this work. Figure 8 represents the proposed CNN 1D model, and Figure 9 represents the proposed CNN 2D model. In the 2D CNN, the layer structure is exactly the same as that of the 1D CNN, with the difference being that it uses convolutions in two dimensions and, therefore, max-pooling in 2D. The configuration of the feature map layers of the CNN models is detailed in the picture. The main difference between the 1D CNN and the 2D CNN is that the kernel moves in one direction and two dimensions, respectively. The computational complexity between 1D and 2D CNN modes differs significantly [24]. Compact 1D CNNs are highly suited for real-time and low-cost applications, particularly on mobile or handheld devices, due to their minimal processing requirements [24].