## 5 Experiment
### Filter Rule Mining  
In this section, we compare the system call log parsing part of our method with the state-of-the-art log parsing algorithms, SwissLog and Drain, to test the correctness and precision of template mining in our method. Lastly, we will test the automatically generated rules alongside Falco's default rules and manually written rules in real-world anomaly-detection scenarios to compare their anomaly-detection capabilities. **Exp1. Quality of System Call Template Mining. ** In this experiment, we will compare the system call template mining part of our method with the best-performing log parsing algorithms, SwissLog and Drain, to test the correctness and precision of template mining in our method. To avoid manual creation of log template expressions, we refer to unsupervised evaluation metrics as per the reference [35]. This measurement standard ensures that log templates should neither be overly generalized nor should they have undetected variables. Given a set of log templates \(T\), the following metrics are calculated:  
\[AvgTokenLost(t)=AvgMatchLength(t)-|t| \tag{5}\]  
\[LengthLoss(T)=(\log|T|)^{\theta} \tag{6}\]  
\[QualityLoss(T)=\sum_{i=0}^{|T|}(\frac{AvgTokenLost(t_{i})}{|t_{i}|})^{2} \tag{7}\]  
\[Loss=LengthLoss(T)+QualityLoss(T) \tag{8}\]  
* \(t_{i}\) is a template in \(T\). * \(AvgMatchLength(t)\) is the average match length of all original logs in the log collection evaluated against template \(t\). * LengthLoss reflects how many templates are generated in \(T\). * QualityLoss evaluates if the generated templates suffer from overgeneralization. * \(\theta\) is a hyper-parameter controlling the importance between the two factors. According to the original paper, \(\theta\) is set to 1.5. * Loss represents the total loss of the template set. In the experiment, we set additional URL and file path regular expressions for Drain, with values of 4 and 0.5 for depth and threshold, respectively. For SwissLog, we set extra delimiters while using its default configuration. Both methods use an online mode, processing system call logs line by line. As shown in Table 3, our algorithm has its own strengths and weaknesses in terms of the total number of generated templates compared to Drain and SwissLog. In most scenarios, our algorithm has the least total loss or is close to the best performer, indicating that our algorithm provides higher-quality log templates in terms of both accuracy in matching original logs and stability when the total number of templates is close. **Exp2. System Call Filtering Effectiveness. ** In this experiment, we compare the number of system calls after filtering using system call type sequences and rule-based filtering with the original number of system calls. When filtering, we use a window size of 128, which means that if a sequence of 128 events appears in the analyzed type sequence or if there is any system call behavior within this window that is not in the whitelist, then this window and the subsequent 128 events are considered for analysis. For each application, we collect 30 min of system call data and ultimately obtain the average number of system calls per second. As shown in Table 4, the primary system call type sequence filtering reduces the original number of system calls by an average of 40%. After the secondary rule-based filtering, we end up with only 13% of the original system call count, effectively filtering out a large number of normal system calls. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline \multirow{2}{*}{**Scenario**} & \multicolumn{3}{c}{**Total Templates**} & \multicolumn{3}{c}{**Total Loss**} \\ \cline{2-7}  & **Ours** & **Drain** & **SwissLog** & **Ours** & **Drain** & **SwissLog** \\ \hline Employee Service & 17 & 24 & 19 & 5.160 & 5.590 & 6.920 \\ Gateway Service & 11 & 22 & 14 & 6.060 & 9.110 & 6.010 \\ Redis Service & 15 & 38 & 27 & 13.780 & 66.370 & 23.330 \\ MySQL Service & 21 & 57 & 61 & 13.190 & 28.410 & 30.680 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Template mining results.