# WorkloadGPT: A Large Language Model Approach to Real-Time Detection of Pilot Workload
## 3 Methods
### LoRA Fine-Tuning  
This study employed the low-rank adaptation (LoRA) model as the fine-tuning method. LoRA has been demonstrated to effectively fine-tune large language models with minimal computational overhead while preserving the knowledge acquired during pre-training [62].  
\begin{table}
\begin{tabular}{c c c} \hline \hline  & **Method Type** & **Example** \\ \hline Input Formatting & List Template & “The number of gaze shifts: 4” \\ Construction Methods & Text Template & “The number of gaze shifts is 4” \\ \hline  & Without feature explanation & “The number of gaze shifts is 4” \\ Ways of & & “The number of gaze shifts \\ Representing Features & With feature explanation & (indicating the frequency of the driver’s eye movements away from the focus of attention) is 4” \\ \hline \hline \end{tabular}
\end{table}
Table 1: Examples of Formatting Construction Methods.  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Types of Formatted Instance Datasets** & **List** & **Text** & **With Feature** & **Without Feature** \\
**Instance Datasets** & **Template** & **Template** & **Explanation** & **Explanation** \\ \hline D1 & \(\surd\) & & \(\surd\) & \\ D2 & \(\surd\) & & & \(\surd\) \\ D3 & & \(\surd\) & \(\surd\) & \\ D4 & & \(\surd\) & & \(\surd\) \\ \hline \hline \end{tabular} \(\surd\) represents Types of Formatted Instance.  
\end{table}
Table 2: Types of Formatted Instance Datasets.  
\begin{table}
\begin{tabular}{c c} \hline \hline
**Types of Formatted Instance Datasets** & **Model Accuracy** \\ \hline D1 & 83.6\% \\ D2 & 82.5\% \\ D3 & 87.3\% \\ D4 & 85.1\% \\ \hline \hline \end{tabular}
\end{table}
Table 3: Comparison Results of the Four Formatted Instances.  
In workload recognition tasks, such as those explored in this study, the data predominantly consists of tabular data. The LoRA method offers distinct advantages for recognizing this type of data using large language models, which are not achievable with other fine-tuning approaches:  
1. Capturing structural relationships: In classifying tabular data, it is crucial to understand and interpret the intrinsic structural relationships within the table [1]. Although traditional large language models like GPT excel in handling textual data, they often struggle to adequately process the structured elements inherent in tabular data when directly applied. While this study formatted the tabular data into natural language, the formatted instances still retained certain structural relationships. To enhance the model's capabilities in this area, LoRA incorporates low-rank matrices into the transformer layers, thereby improving the model's ability to understand and process the data relationships within these formatted instances;
2. Reducing the need for parameter tuning: Classification tasks involving tabular data often require extensive parameter tuning and model fine-tuning, which are resource-intensive for traditional large pre-trained models. LoRA addresses this issue by freezing the original model weights and solely modifying the added low-rank matrices, significantly reducing the number of parameters that require adjustments [63]. This approach not only lowers computational demands but also accelerates the model iteration process, enabling efficient tuning even under resource constraints;
3. Suitability for resource-constrained environments: This method is particularly well-suited for use in environments with limited computational and storage resources, while maintaining the performance and flexibility of the model. Consequently, LoRA can operate efficiently under resource constraints, achieving effective fine-tuning without compromising model performance.  
After multiple rounds of experiments and parameter tuning, this study determined the key parameters as shown in Table 4.