# CL-MMAD: A Contrastive Learning Based Multimodal Software Runtime Anomaly Detection Method
## 7 Discussion  
CL-MMAD, a software runtime anomaly detection framework for both functional failure and performance issues, is proposed in this work. To our knowledge, this is the first time multimodal datasets have been used for software anomaly detection. There are several main findings in this work. **Finding 1: Data consistency between different modalities can be used for anomaly detection. **  
CL-MMAD is based on the contrastive learning framework and uses the double tower structure to capture the features of trace indicators and performance indicators separately. MSE loss is used to guide the model to construct the projection from different source modal spaces into a uniform space. When inconsistent data are passed through the model, the differences are magnified, which is expressed in the final feature space as a larger Euclidean distance. **Finding 2: The commonly used InfoNCE loss in contrastive learning framework is not well-suited for software anomaly detection tasks; MSE is simpler and better. **  
In this study, we compared the performance of the CL-MMAD model trained using the InfoNCE and mean squared error (MSE) losses. Our findings indicate that training the model under the InfoNCE loss requires significantly more negative samples per batch, necessitating a larger batch size. However, due to device limitations, the maximum batch size was set to 20, which falls short of optimal requirements. In contrast, the MSE loss promotes faster model convergence, with performance surpassing that of the InfoNCE loss within a mere 10 epochs. As such, utilizing the MSE loss may prove to be a more cost-effective and practical approach to software anomaly detection. **Finding 3: The CL-MMAD model demonstrates greater sensitivity to changes in performance indicators, highlighting the need for closer consideration of the trace indicator embedding learning process. **  
Notably, while the current model employs an approximately symmetric architecture, our results indicate reduced sensitivity to trace changes compared with performance indicator changes. Consequently, the model may fail to detect faults that produce only minor \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Model** & **Precision** & **Recall** & **Accuracy** & **F1-Score** \\ \hline CL-MMAD & **0.9820** & **0.9708** & **0.9740** & **0.9764** \\ \hline OmniAnomaly & 0.8735 & 0.8745 & 0.9169 & 0.8739 \\ \hline DAEMON & 0.9059 & 0.9550 & 0.9632 & 0.9337 \\ \hline \hline \end{tabular}
\end{table} 
Table 7: Comparisons with baseline models. deviations in the trace data. Given that performance data typically changes relatively slowly over time, with a collection interval of 1 s, trace data may generate a vast amount of information during that time. Therefore, the use of more sophisticated representation learning structures for trace data may improve the detection performance of functional failure detection. Furthermore, we implemented timed sampling during trace data collection to minimize performance overhead on the original system, but this also resulted in a trade-off with prediction accuracy. Funding: Compared with DAEMON and OmniAnomaly, the combination of internal trace information and external performance information can achieve better results when performing anomaly detection on performance issues. Empirical results show that, compared with baseline models, CL-MMAD achieves better performance on performance issues detection, which may be due to the introduction of internal trace information. CL-MMAD can integrate more information from multi-sources, such as internal trace, external performance indicators, and client-side feedback information, which can lead to more accurate detection of anomalies. Multimodal detection systems can be much more robust than unimodal frameworks. By integrating multi-sources, they can reduce the impact of noise and errors.