# Entity Extraction of Key Elements in 110 Police Reports Based on Large Language Models
## 4 Results and Analysis
### Base Model Fine-Tuning  
After constructing the original training set, the base model was tested for the extraction of the four key elements, yielding the following results: The best results obtained in the table have been highlighted in bold. As shown in Table 4, for the extraction of location elements, the best-performing base model was Qwen. After fine-tuning, the best-performing models were Baichuan and Yi. As shown in Table 5, for the extraction of key behavior elements, the best-performing base model was Baichuan. After fine-tuning, the best-performing model remained Baichuan. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline \multirow{2}{*}{**Model**} & \multicolumn{3}{c}{**Original**} & \multicolumn{3}{c}{**After Fine-Tuning**} \\ \cline{2-7}  & **P** & **R** & **F1** & **P** & **R** & **F1** \\ \hline ChatGLM & 0.737 & 0.837 & 0.783 & 0.837 \(\pm\) 0.002 & 0.837 \(\pm\) 0.002 & 0.837 \(\pm\) 0.002 \\ Qwen & 0.722 & 0.884 & **0.795** & 0.878 \(\pm\) 0.011 & 0.878 \(\pm\) 0.011 & 0.878 \(\pm\) 0.011 \\ Yi & 0.582 & 0.605 & 0.593 & 0.905 \(\pm\) 0.008 & 0.905 \(\pm\) 0.008 & **0.905 \(\pm\) 0.008** \\ Llama & 0.357 & 0.509 & 0.419 & 0.891 \(\pm\) 0.010 & 0.891 \(\pm\) 0.010 & 0.891 \(\pm\) 0.010 \\ Baichuan & 0.649 & 0.653 & 0.651 & 0.905 \(\pm\) 0.008 & 0.905 \(\pm\) 0.008 & **0.905 \(\pm\) 0.008** \\ Mistral & 0.517 & 0.728 & 0.605 & 0.857 \(\pm\) 0.014 & 0.857 \(\pm\) 0.014 & 0.857 \(\pm\) 0.014 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Location element extraction results. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Element** & **P** & **R** & **F1** \\ \hline Location & 0.940 \(\pm\) 0.012 & 0.790 \(\pm\) 0.010 & 0.859 \(\pm\) 0.002 \\ Key items & 0.817 \(\pm\) 0.010 & 0.683 \(\pm\) 0.008 & 0.741 \(\pm\) 0.006 \\ Key behaviors & 0.937 \(\pm\) 0.009 & 0.853 \(\pm\) 0.009 & 0.890 \(\pm\) 0.001 \\ Organizations & 0.577 \(\pm\) 0.028 & 0.443 \(\pm\) 0.026 & 0.503 \(\pm\) 0.007 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: UIE Results. As shown in Table 7, for the extraction of regularization elements, the best-performing base model was ChatGLM. After fine-tuning, the best-performing model remained ChatGLM. As shown in Table 8, considering the overall extraction performance for the aforementioned elements, the best-performing base model was ChatGLM, while after fine-tuning, the best-performing model was Baichuan. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline \multirow{2}{*}{**Model**} & \multicolumn{3}{c}{**Original**} & \multicolumn{3}{c}{**After Fine-Tuning**} \\ \cline{2-7}  & **P** & **R** & **F1** & **P** & **R** & **F1** \\ \hline ChatGLM & 0.662 & 0.655 & **0.659** & 0.854 \(\pm\) 0.003 & 0.832 \(\pm\) 0.004 & 0.843 \(\pm\) 0.003 \\ Qwen & 0.595 & 0.662 & 0.627 & 0.887 \(\pm\) 0.001 & 0.857 \(\pm\) 0.002 & **0.869 \(\pm\) 0.001** \\ Yi & 0.464 & 0.590 & 0.520 & 0.857 \(\pm\) 0.003 & 0.823 \(\pm\) 0.004 & 0.840 \(\pm\) 0.004 \\ Llama & 0.129 & 0.116 & 0.122 & 0.837 \(\pm\) 0.005 & 0.788 \(\pm\) 0.006 & 0.812 \(\pm\) 0.007 \\ Baichuan & 0.254 & 0.509 & 0.339 & 0.874 \(\pm\) 0.002 & 0.853 \(\pm\) 0.002 & 0.864 \(\pm\) 0.002 \\ Mistral & 0.413 & 0.468 & 0.438 & 0.869 \(\pm\) 0.004 & 0.853 \(\pm\) 0.004 & 0.861 \(\pm\) 0.004 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Key item element extraction results.