# Language Model-Based Text Augmentation System for Cerebrovascular Disease Related Medical Report
## 5 Discussion  
### Discussion of Previous Research Results  
We propose a text augmentation system for medical reports related to cerebrovascular diseases using a DistilGPT2-based synthetic text generation model and a BioBERT-based classification model. Our method demonstrated that the augmented synthetic text contributes to improving model performance. However, since this study is limited to cerebrovascular diseases, we aim to explore whether we can effectively apply it in other medical domains. Therefore, in this discussion, we investigate the potential generalizability of our methodology through case studies from different medical fields. The scarcity of labeled data is a common challenge across various medical domains, leading to extensive research into numerous text data augmentation methods to address this issue. For instance, in drug identification and incident classification, ChatGPT has been used to generate synthetic text data, significantly improving pre-trained BERT models' performance [33]. This study demonstrated that synthetic data generated in diverse contexts helps the model better recognize patterns in drug-related text. Similarly, in predicting patient readmission, GPT-2 was utilized to augment text data, balancing the dataset and optimizing model performance through fine-tuning [34]. This study emphasized that the quantity and quality of augmented data critically impact model performance, a key consideration in our study. Additionally, the EDA technique was modified to perform effective text augmentation in a named entity recognition (NER) task on Chinese electronic medical records [35]. This \begin{table}
\begin{tabular}{c c c c c c c c c c c c c} \hline \hline
**Technique** & **Model** & **Metric** & **Baseline** & **+20\%** & **+40\%** & **+60\%** & **+80\%** & **+100\%** & **+120\%** & **+140\%** & **+160\%** & **+180\%** & **+200\%** \\ \hline \multirow{4}{*}{Ours} & BioBERT & Acc & 77.52 & 72.82 & 69.04 & 75.28 & 79.88 & 76.66 & 80.78 & 80.36 & 81.02 & 80.78 & 81.7 \\  & F1-score & 0.66 & 0.63 & 0.63 & 0.69 & 0.77 & 0.55 & 0.79 & 0.78 & 0.78 & 0.52 & 0.79 \\ \cline{2-13}  & Clinical & Acc & 80.12 & 80.16 & 79.42 & 81.16 & 80.86 & 81.68 & 81.46 & 81.82 & 82.56 & 82.48 & 82.8 \\  & BERT & F1-score & 0.8 & 0.8 & 0.74 & 0.81 & 0.79 & 0.81 & 0.8 & 0.8 & 0.68 & 0.69 & 0.75 \\ \cline{2-13}  & \multirow{2}{*}{BiomedBERT} & Acc & 61.16 & 68.08 & 66.38 & 71 & 79.2 & 75.58 & 80.02 & 80.92 & 81.44 & 81.4 & 79.48 \\  & F1-score & 0.64 & 0.61 & 0.67 & 0.75 & 0.76 & 0.76 & 0.78 & 0.78 & 0.76 & 0.65 & 0.71 \\ \hline \multirow{4}{*}{EDA} & BioBERT & Acc & 79.18 & 79.56 & 79.28 & 79.82 & 78.78 & 75.56 & 79.5 & 79.9 & 77.64 & 78.24 & 79.32 \\  & F1-score & 0.82 & 0.82 & 0.82 & 0.82 & 0.81 & 0.8 & 0.82 & 0.82 & 0.81 & 0.81 & 0.82 \\ \cline{1-1} \cline{2-13}  & Clinical & Acc & 82.52 & 81.98 & 82.32 & 81.48 & 82.14 & 82.1 & 81.12 & 81.84 & 82.46 & 80.86 & 82.0 \\ \cline{1-1}  & BERT & F1-score & 0.84 & 0.84 & 0.84 & 0.83 & 0.84 & 0.84 & 0.83 & 0.84 & 0.84 & 0.83 & 0.84 \\ \cline{1-1} \cline{2-13}  & \multirow{2}{*}{BiomedBERT} & Acc & 69.9 & 77.42 & 78.2 & 70.84 & 79.12 & 69.56 & 74.5 & 75.24 & 77.2 & 70.82 & 64.52 \\ \cline{1-1}  & & F1-score & 0.78 & 0.81 & 0.82 & 0.65 & 0.82 & 0.64 & 0.79 & 0.79 & 0.81 & 0.67 & 0.62 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: The accuracy and F1 scores of each model are fine-tuned with the augmented text generated by the proposed system and EDA. research highlighted how maintaining the semantic consistency of augmented text enhances the model's ability to generalize. These case studies suggest that the text augmentation techniques we employed in our research could also be effectively applied in other medical fields. However, they also underscore a significant difference: the absence of a filtering mechanism to ensure domain-specific data accuracy. This difference is crucial, particularly in medical domains where data accuracy and relevance are paramount, thus highlighting the value of our approach.