# Language Model-Based Text Augmentation System for Cerebrovascular Disease Related Medical Report
## 1 Introduction  
AI is providing great help to people in various fields, and the medical field is no exception. AI innovates various medical tasks, such as medical diagnosis, treatment planning, and patient monitoring [1]. However, textual data in the medical field, such as medical records, contain sensitive patient information; thus, their usage is restricted [2, 3]. This is essential to ensure patient confidentiality, yet it poses a substantial challenge to developing deep learning models. These models require large amounts of data to learn meaningful patterns and achieve high performance. Consequently, the limitation in data accessibility hinders AI research in natural language processing (NLP).  
AI research has primarily relied on image data, such as Computed Tomography (CT) and Magnetic Resonance Imaging (MRI) scans, to overcome these limitations in the medical field. Images present fewer patient privacy issues while playing a crucial role in diagnosing diseases and planning treatments. Nevertheless, the importance of textual data remains significant. Textual data such as medical records, doctor's notes, and patient interview transcripts are essential for understanding the context and details of diseases. Recently, there has been an increasing interest in leveraging textual data in medical research, particularly in generating synthetic text data [4, 5].  
Synthetic text data generation is gaining attention as a solution to the data scarcity problem, providing an effective way to obtain large amounts of training data. However, this process faces several critical issues. The generated texts lack annotated information, necessitating manual annotation. Additionally, there is a risk of hallucination, where the generatedtext contains incorrect information, making it unsuitable for use as training data. Therefore, human evaluation and correction of the generated text's appropriateness are required [6; 7; 8]. This process consumes significant time and resources during data augmentation.  
This paper proposes a cerebrovascular disease-specific text augmentation system using a DistilGPT2-based synthetic text generation model and a BioBERT-based text classification model [9; 10]. We randomly extracted a subset of the report data to create five training-test pairs (5000, 10,000, 15,000, and 20,000). The training data of each pair were then input into the synthetic text generation model to generate new texts. Subsequently, we used all report data to fine-tune a classification model. The classification model classified the generated texts as either intracerebral hemorrhage (ICH) or normal and filtered out samples likely to confuse the model. This process created an augmented dataset for each pair.  
Through this approach, we identified the optimal amount of augmented data that maximizes model performance based on the original data's size and the counterproductive amount of augmented data. The process is as follows: (1) Fine-tune BioBERT, ClinicalBERT, and BiomedBERT using the original data for each pair and incrementally adding augmented data for fine-tuning [11; 12]. (2) Calculate the accuracy of the fine-tuned models and evaluate whether the augmented data from our proposed system improves model performance.  
This paper is structured as follows: Section 2 introduces related work, Section 3 describes the models and methods used, Section 4 analyzes the experimental results, and Section 5 presents the discussion. Finally, Section 6 summarizes the study and discusses future research directions. This study proposes a novel method for effectively utilizing textual data in the medical field, contributing to expanding AI-driven medical data analysis capabilities.