# AI-Based Approach to Firewall Rule Refinement on High-Performance Computing Service Network
## 4 Result and Discussion  
Figure 8 compares the F1-score average accuracies of the different classification algorithms with four and 10 features. KNN (neighbors = 3) showed the best performance among the tested machine-learning classification algorithms, with an accuracy of 0.97. The implemented NN model and J48 had the highest accuracy of 0.98 using 10 features. Naive Bayes showed a decent performance, improving slightly from 0.79 with four features to 0.83 with 10 features. SVM with a kernel polynomial showed the highest accuracy compared to linear, RBF, and sigmoid, particularly with 10 features. OneR and logistic regression showed consistent performance for both feature sets. These results demonstrate that the algorithm and number of features used significantly affect the classification accuracy, with more complex models, such as KNN, J48, and NN, achieving higher accuracy when the feature set was expanded. Figure 9 compares the improvement of different classification algorithms using four and ten features. SVM (polynomial) showed the highest improvement in the F1-score average accuracy, with an 11.36% increase. Logistic regression also showed a notable improvement, with a 4.70% increase. Naive Bayes showed a moderate improvement of 4.59%, whereas J48 showed a significant improvement of 2.94%. The NN model maintained a consistent improvement of 1.02%. KNN (neighbors = 3) and OneR had relatively small improvements of 0.33% and 0.21%, respectively. These results demonstrate the various levels of accuracy improvement achieved by the various classification algorithms. Notably, the SVM with a polynomial kernel showed the highest improvement in classification performance. Because more features provide more decision paths for classifying the data, models such as KNN, NN, and J48 can make better use of the additional information provided, which improves the accuracy. With 10 features, J48 can create more complex decision boundaries, which increases the accuracy. An SVM captures complex relationships in data better and performs well with an increased feature space. Models such as linear decision boundaries and logistic regression showed improved accuracy, indicating that the data may have become more separable with more features. Naive Bayes and OneR showed mixed  
Figure 8: F1-score average accuracy: performance improvement comparison between machine-learning algorithms and the proposed deep-learning model with different feature selections. results, implying that the nature of the dataset and the assumptions they made regarding the data distribution might have impacted their performance. Ultimately, the optimal choice depends on various factors such as the specific requirements of the problem, computational resources, interpretability needs, and the balance between model complexity and accuracy. For a well-rounded option, KNN (neighbors = 3) or NN are recommended. Figure 10 shows the training and validation accuracies and losses when using four and ten features and training for over 400 epochs. Regarding the loss comparison in the upper image, the training loss (gray line) decreases rapidly during the initial epochs and then gradually decreases, reaching approximately 0.12. The validation loss (gray line) also follows a similar trend, beginning at 0.20 and decreasing steadily to approximately 0.10 at the end of the epochs. The training loss (blue line) begins at approximately 0.21 and steadily decreases to approximately 0.08. The validation loss (blue line) follows a similar trend, starting from approximately 0.14 and decreasing steadily to approximately 0.08 at the end. Regarding the accuracy comparison in the bottom image, the training accuracy (gray line) begins at 87.56% and increases steadily to approximately 96.1% at the end. The validation accuracy (gray line) begins at 92.92% and increases to approximately 96.8% at the end. The training accuracy (blue line) begins at 91.80% and increases steadily to approximately 97.7% at the end.