## 3 Methods
### Industrial Control Device Anomaly Detection Model Based on Fine-Tuned Llama3 Model  
The specific process of fine-tuning large models is shown in Figure 7. There are performance differences between different foundational large language models, and their effects often vary greatly after fine-tuning with the exact same data. Additionally, different models require varying amounts of computational power for fine-tuning and are suitable for different applications. Therefore, selecting an appropriate foundational large language model is the first step in fine-tuning the model to accomplish tasks in a specific domain. First, the selected foundational large language model must be secure and compliant. Considering that the anomaly detection task must meet data security and privacy compli  
Figure 6: ICS device hardware feature diagram. ance requirements, it is advisable to choose an open-source foundational large language model that supports local deployment for fine-tuning. Second, while foundational large language models typically excel in handling natural language-related tasks, they generally have limited ability to learn discrete, purely numerical sample features in the context of industrial control device anomaly detection tasks. By preprocessing the training samples to convert the discrete, purely numerical sample features of the anomaly detection task into common question-and-answer pairs in natural language processing tasks, the fine-tuned model can better accomplish the industrial control device anomaly detection task. Finally, the training process of large language models often requires substantial computational resources and expensive hardware support. Even during the fine-tuning process, the hardware resources required and the training speed can vary significantly depending on the pre-model and the fine-tuning method. Therefore, it is also essential to consider how to reduce the demand for hardware resources and improve training speed while ensuring the accuracy of anomaly detection. This study, based on publicly available information and evaluation data from authoritative institutions, selected gamma-7b-bnb-4 bit, llama-3-8b-bnb-4 bit, and Phi-3-mini-4k-instruct for fine-tuning and testing. The selection reasons are as follows: (1) All three models are open-source foundational large language models that support local fine-tuning and deployment without internet connection, meeting security and compliance requirements. (2) They can all be fine-tuned using question-and-answer pairs to accomplish the  
Figure 7: Architecture of anomaly detection of ICS devices. industrial control device anomaly detection task. Finally, (3) they can all be fine-tuned using Unsloth, with GPU resource requirements not exceeding 8 GB after using Unsloth. Moreover, compared to not using Unsloth acceleration, fine-tuning speed with Unsloth is approximately doubled. Due to the specificity of the industrial control device anomaly detection task, the detection accuracy of anomalous devices is virtually zero for various foundational large language models before fine-tuning. After fine-tuning, different models exhibit varying abilities to detect anomalous devices. Experimental results identified llama-3-8b-bnb-4 bit as the most suitable foundational large language model for the industrial control device anomaly detection task after fine-tuning, with the highest detection accuracy for anomalous devices when fine-tuned using the same method. The llama-3-8b-bnb-4 bit model adopts the latest optimization algorithms, significantly enhancing training efficiency. It can rapidly converge to high-quality model parameters while reducing computational resources and time, markedly lowering training costs. The model exhibits substantial improvements in inference tasks (such as complex logical analysis and question-answering systems) and natural language generation tasks (such as text continuation and summarization). The generated text is more coherent and logically consistent, demonstrating stability and consistency across multiple tasks.