## 3 Dataset Generation
### ATf&CK  
To acquire knowledge and experience pertinent to penetration testing, we propose leveraging the ATT&CK (Adversarial Tactics, Techniques, and Common Knowledge) platform. Developed by MITRE, ATT&CK is a knowledge base that delineates and categorizes cyber adversary behaviors. It provides security professionals with a standardized methodology for analyzing and responding to cyber threats. The platform encompasses tactics, techniques, defenses, Cyber Threat Intelligence (CTI), and additional resources. ATT&CK offers standardized terminology and frameworks, facilitating learning for novices and enhancing communication and collaboration among security teams. Moreover, it provides comprehensive descriptions of threat behaviors, encompassing the entire process from initial access to exfiltration, documenting genuine and effective techniques that can be utilized for learning, and identifying and remediating security vulnerabilities. Penetration testing techniques essentially employ an offensive approach to bolster defense, actively testing target systems or environments using various tools and techniques under authorized conditions. Through ATT&CK, security practitioners can better comprehend the principles and methodologies underlying cyber threats, thereby enhancing organizational defense capabilities. Analogously, the textual content extracted from the ATT&CK platform serves as a manifestation of penetration testing knowledge. In the data sanitization and categorization process, we initially segmented the data within the MITRE ATT&CK framework according to the official delineation of 14 phases. We prioritized the selection of enterprise-level techniques, as the enterprise environment typically encompasses a broad range of operating systems, applications, and network devices, thereby providing a more comprehensive depiction of the aspects considered during penetration testing. Each phase's techniques essentially encompass the following elements: the names and IDs of subtechniques within the technique, the definition of the technique along with potential implementation scenarios, Procedure Examples detailing specific cybersecurity incidents, Mitigations outlining related countermeasures, Detection detailing pertinent detection methods, and References. Subsequently, we excised the References and ID information, retaining only the natural language sentences that facilitate  
Figure 1: The pipeline of our method for dataset construction and model augmentation. We have sanitized and organized the usage manuals of open-source cybersecurity tools and open-source cybersecurity knowledge documents into coherent text segments. Subsequently, we have extracted specific question-and-answer pairs from several of these segments, which include the usage commands for specific cybersecurity tools and concepts related to network security. Following this, we have designed an enhancement project that enables more capable large language models to learn the correspondences between our crafted questions and answers and the original text segments, thereby generating question-and-answer data for each segmented piece. Finally, we have split the resulting question-and-answer dataset and fine-tuned it on large language models under 10 B parameters to enhance their performance in penetration testing Q&A tasks. the understanding by language models. We also downplayed the specifics of cybersecurity incidents, as such information does not directly contribute to the model's acquisition of knowledge regarding penetration testing. Ultimately, we extracted and segmented the definitions and potential application scenarios of each technique and subtechnique, as well as the corresponding detection and defensive strategies for subsequent utilization. We focused primarily on extracting information from the platform's Techniques section, treating each subtechnique as a source of data for extraction.