# AI-Based Approach to Firewall Rule Refinement on High-Performance Computing Service Network
## 3 Methodology
### Machine-Learning Algorithms
#### 3.2.3 OneR
SVM is a powerful and versatile algorithm that can be applied to a wide range of classification tasks, including both linearly and non-linearly separable data, and has the ability to control the trade-off between model complexity and generalization performance [8]. SVM supports various kernel functions, such as linear functions, polynomial functions, RBFs, and sigmoid functions, which can be selected based on the data characteristics and problem domain. In this experiment, we tested all types of kernels and found that SVM with a polynomial kernel showed the best score. The SVM algorithm identifies the optimal hyperplane that separates classes in the feature space. In the case of linearly separable classes, the hyperplane is represented by the equation of a hyperplane (for linearly separable data refer to Equation (4); for non-linearly separable data with kernel refer to Equation (5)):

\[\mathbf{w}\cdot\mathbf{x}+b=0 \tag{4}\]

\[\mathbf{w}\cdot\phi(\mathbf{x})+b=0 \tag{5}\]

where \(w\) is the weight vector (normal to the hyperplane), \(x\) is the input feature vector, \(b\) is the bias or intercept term, and \(\phi(\mathbf{x})\) is the feature-mapping function. The optimization problem for finding \(w\) and \(b\) can be represented as (in Equation (6)):

\[ \begin{split}\min_{\mathbf{w},b}\frac{1}{2}\|\mathbf{w}\|^{2}+C \sum_{i=1}^{N}\xi_{i}\\ \text{subject to:}& y_{i}(\mathbf{w}\cdot\mathbf{x} _{i}+b)\geq 1-\xi_{i}\\ &\xi_{i}\geq 0\end{split}  \tag{6}\]

In this experiment, we used the _svm.SVC()_ function with a polynomial kernel, _degree_ = 3, and \(C\) = 2.0 parameters. SVMs are powerful and versatile models that perform well in many classification tasks. However, they may not be suitable for every situation, especially when computational resources are limited or interpretability is crucial. #### 3.2.6. Logistic Regression

For logistic regression, we used _LogisticRegression()_ provided by scikit-learn 1.4.0. Logistic regression models the probability that a given input belongs to a particular class. It extends linear regression by using a logistic function (also known as the sigmoid function) to transform the output of a linear combination of input features into a probability value between 0 and 1 [36]. The equation of the sigmoid function is expressed as (in Equation 7):

\[\sigma(z)=\frac{1}{1+e^{-z}} \tag{7}\]

where \(\sigma(z)\) represents the sigmoid function, \(z\) is the input variable, and \(e\) is the base of the natural logarithm. The sigmoid function takes any real number \(z\) as the input and outputs a value between 0 and 1, representing the probability of occurrence of an event. It has an S-shaped curve with values close to 0 and 1 for large negative and positive inputs, respectively, and a value of approximately 0.5 is obtained at \(z=0\). Logistic regression is advantageous owing to its simplicity, interpretability, and efficiency with small datasets. However, it may struggle to capture complex non-linear relationships and is sensitive to outliers. ### Implemented NN Model

PyTorch was used to implement the multiclass-classification NN model. It comprises four fully connected linear layers and an output layer. The rectified linear unit (ReLU) was used as the activation function after each linear layer, except after the output layer. Dropout regularization with a probability of 0.5 was applied after the ReLU activation function in the first layer. Dropout helps prevent overfitting by randomly dropping a proportion of the units during training. Batch normalization was applied after each linear layer, except after the output layer. It normalized the activation of each layer, facilitating faster training and better generalization. The batch size of the model parameter value was set to 4096, and the learning rate was set to 0.007. The description of each layer is as follows:

* layer_1: A linear layer with 4 (and 10) input features, implying that the input to this layer has either 4 or 10 features.