# Entity Extraction of Key Elements in 110 Police Reports Based on Large Language Models
## 4 Results and Analysis
### Extraction Experiment Based on Data Augmentation  
Subsequently, using the zero-shot police data augmentation method based on ChatGPT designed in this study, approximately 6000 simulated police reports were generated. The distribution of the simulated data was observed through dimensionality reduction. The t-SNE dimensionality reduction visualization of both the generated data and the original data is shown in Figure 4, where triangles represent the generated data and circles represent the original data. This distribution change can explain the observed performance gap between models fine-tuned on synthetic data and those fine-tuned on original data. The training data were expanded to four times the size of the original dataset, with experiments conducted using generated datasets of 1000, 2000, 3000, 4000, 5000, and  
Figure 4: t-SNE dimensionality reduction distribution. 6000 entries. These correspond to 1\(\times\), 1.5\(\times\), 2.5\(\times\), 3\(\times\), 3.5\(\times\), and 4\(\times\) of the original data size, respectively. The results are shown in Table 9, We have bolded the best results in the table. The comprehensive performance trends as observed in the curve chart are shown in Figure 5. To further elucidate the learning process and demonstrate the model's convergence, we have included learning curves depicting the evolution of loss during training in Appendix C. These curves provide valuable insights into the stabilization of the model's performance with respect to the number of training samples, complementing the analysis presented in Figure 5.  
It can be observed from the overall upward trend in the data augmentation curve (Figure 5) that the method for generating data is effective and improves model training. However, there is a noticeable decline in performance when the dataset is initially increased by 1.5 to 2 times the size. This may be due to the vector differences between the generated and original data distributions, which can disrupt the model's understanding of the original data, leading to a slight decrease in performance. As the dataset is further augmented to 2 to 3 times the original size, the model gradually adapts to the new data distribution, and the performance improves and stabilizes. Beyond this ratio, increasing the data augmentation further does not significantly enhance the model's performance due to the limited complexity of the generated data compared to real-world scenarios. For the models using the original prompt and the designed prompt, the ablation study results before and after fine-tuning are presented as shown in Table 10, we have bolded the best results in the table. The original prompt and the designed prompt are the prompts designed in the prompt engineering section of this paper (Section 3.1). Specific examples of these prompts can be found in Appendix B.  
Figure 5: Comprehensive effectiveness of augmentation multipliers. \begin{table}
\begin{tabular}{c c c c c c c c c c c c} \hline \hline  & \multicolumn{2}{c}{**Initial**} & \multicolumn{2}{c}{**1.5\(\times\)**} & \multicolumn{2}{c}{**2.5\(\times\)**} & \multicolumn{2}{c}{**3\(\times\)**} & \multicolumn{2}{c}{**3.5\(\times\)**} & \multicolumn{2}{c}{**4\(\times\)**} \\ \cline{2-13}
**Element** & **P** & **R** & **P** & **R** & **P** & **R** & **P** & **R** & **P** & **R** & **P** & **R** \\ \hline Location & 0.905 & 0.905 & 0.925 & 0.925 & 0.932 & 0.932 & 0.939 & 0.939 & 0.939 & 0.939 & 0.946 & 0.946 \\ Key behaviors & 0.908 & 0.937 & 0.898 & 0.922 & 0.911 & 0.931 & 0.932 & 0.956 & 0.912 & 0.943 & 0.917 & 0.947 \\ Key items & 0.874 & 0.853 & 0.907 & 0.894 & 0.917 & 0.901 & 0.924 & 0.908 & 0.918 & 0.911 & 0.910 & 0.901 \\ Organizations & 0.900 & 0.851 & 0.899 & 0.838 & 0.903 & 0.886 & 0.927 & 0.914 & 0.926 & 0.921 & 0.925 & 0.919 \\ Micro & 0.897 & 0.887 & 0.907 & 0.895 & 0.916 & 0.912 & **0.930** & **0.929** & 0.924 & 0.929 & 0.928 \\ \hline \hline \end{tabular}
\end{table} 
Table 9: Augmentation results comparison.