## 4 Experiments  
To demonstrate the practicality of our constructed dataset, we proposed fine-tuning smaller parameter models (below 10 billion parameters) using our dataset. Our model foundations include Qwen2-7B [28], Gemma2-9B [29], GLM4-9B [30], Llama3-8B [31], and Mistral-8B [32]. These models offer comprehensive support for model parameters and training frameworks, enabling fine-grained parameter updates. The architectures encompass a variety of designs including autoregressive (Llama3), mixture of experts (Mistral), and bidirectional attention (GLM), thereby facilitating the verification of the method's adaptability across different model families. Due to memory and fine-tuning time constraints, we applied 4-bit QLoRA [33] to fine-tune the base models. For the dataset, we allocated 5% for testing and the remainder for training, employing a learning rate of \begin{table}
\begin{tabular}{l} \hline \hline
**Conceptual Q&A Examples** \\ \hline ID: 4724, \\ Category: “Persistence”, \\ Input: “How can adversaries use transport agents for persistent access to systems?” \\ Output: “Adversaries can abuse Microsoft transport agents by registering a malicious transport agent with the Exchange server. This malicious agent can then be triggered by specific email events to provide a persistence mechanism”. \\ \hline \hline \end{tabular}
\end{table} 
Table 1: _Cont._ \begin{table}
\begin{tabular}{l} \hline \hline
**Practical QA Examples** \\ \hline ID: 867, \\ Category: “Vulnerability Scanning”, \\ Input: “How can I scan for open ports that might lead to exposure of sensitive \\ data (CWE-200)?” \\ Output: “Use ’nmap’ to scan for open ports with the command ’nmap -p 1-65535 -sV example.com”. \\ \hline ID: 45706, \\ Category: “Metasploit”, \\ Input: “How can one discover the open port created by \\ payload/linux/x64/shell_bind_tcp_random_port module?” \\ Output: “One can discover the open port created by the payload /linux/x64/shell_bind_tcp_random_port module using the nmap command ’nmap -sS target -p-”. \\ \hline ID: 41854, \\ Category: “Metasploit”, \\ Input: “What command is used by default to fetch the payload on the target \\ system in payload/cmd/windows/http/x64/vncinject/reverse_tcp_rc4?” \\ Output: “The payload /cmd/windows/http/x64/vncinject/reverse_tcp_rc4 \\ module uses the CERTUTIL command by default to fetch the payload on the \\ target system”. \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Examples of Practical Q&A in the PenQA. The main focus revolves around the specific command usage for tools in particular scenarios, along with the assignment of parameters for Metasploit modules in targeted contexts. \(2\times 10^{-5}\). For LoRA, we set the rank to 128 and alpha to 256. We applid LoRA to all Query, Key, and Value metrics in multihead self-attention blocks alongside Linear layers. We utilized the AdamW optimizer with a batch size of 8. Given that these models are generative in nature, it is challenging to achieve perfect word-for-word consistency in responses to identical prompts, and they may not entirely match the reference answers. Consequently, we employed popular natural language generation (NLG) metrics for evaluation. These include word overlap-based metrics such as BLEU [34], ROUGE [35], and METEOR [36]; word embedding-based metrics [37] that include Greedy Matching, Embedding Average, and Vector Extrema; and language model-based metrics such as BERTScore [38]. The results of our experiments are presented in Tables 3 and 4. We conducted predictions using our experimental models both before and after fine-tuning, comparing the outputs with reference answers. Within the experimental context, there was a discernible enhancement observed across all metrics, with each exhibiting an increment in score to a diverse extent. Our findings indicate that fine-tuning with our proposed instruction dataset indeed enhances the model's application of cybersecurity knowledge and proficiency with certain cybersecurity tools.