# Quantification and Analysis of Group Sentiment in Electromagnetic Radiation Public Opinion Events
## 3 EMR Public Sentiment Analysis System Design
### Data Collection Based on Topic Similarity
#### 3.1.2 Evaluation Experiment  
To verify the performance of the filtering system, this study designed an evaluation experiment, collecting data through two algorithms: one keywords-based and one semantic matching-based. The research used Recall and Precision: two important indicators used to evaluate the effectiveness of the retrieval. Recall is used to determine whether enough relevant posts have been collected during data collection, while Precision is used to evaluate the number of collected data that matches the keyword. The experiment selected \"electromagnetic radiation\" as the keyword for testing, collecting a total of 700 posts on Weibo. Then, the semantic similarity between each post's text and the keyword was calculated and normalized using MAX-MIN normalization. The criterion for judging the relevance of the collected data topics was as follows: the normalized similarity \(\geq\) threshold r = 0.1 (chosen as a compromise based on the accuracy requirement of the capture). The experiment recorded the number of collected posts and calculated the precision and recall. Precision refers to the proportion of correctly identified relevant items among all items identified as relevant by the system. Recall measures the proportion of correctly identified relevant items among all actual relevant items in the dataset. These metrics are calculated as follows:  
\[Precision=\frac{TP}{TP+FP}\]  
\[Recall=\frac{TP}{TP+FN}\]  
where _TP_ (True Positive) represents the number of correctly identified relevant items, _FP_ (False Positive) refers to the number of irrelevant items incorrectly identified as relevant, and _FN_ (False Negative) refers to the number of relevant items that were not identified by the system. High precision indicates a low false positive rate, while high recall indicates a low false negative rate. Together, these metrics provide a balanced view of the system's data extraction performance. The results are shown in Table 1. Before the filtering system was added, that is, when based on keyword search, all posts containing the keyword \"electromagnetic radiation\" within the set period were collected (with a Recall of 100%), but not all posts met the requirements (with a Precision of 41%). After the filtering system was added, many posts that only contained the keyword but were unrelated to the actual content of the keyword, were excluded, increasing the precision (with a Precision of 68%). At the same time, due to the exclusion of many posts, some posts containing relevant topics were also excluded, leading to a decrease in recall (with a Recall of 63%). Furthermore, to enhance the reliability of the dataset, we conducted manual annotation by inviting five annotators to label the data, which verified the accuracy and credibility of the collected information. Overall, the addition of the topic filter improved the usability of the data, but it also missed part of the data. Subsequently, the recall can be kept within an acceptable range by modifying the threshold, and the precision can be improved as much as possible. This study designed an experiment to perform sentiment analysis on data before and after filtering, analyzing the accuracy of the content of the top ten posts with positive \begin{table}
\begin{tabular}{c c c c} \hline \hline  & **Number of Collected Posts (Number)** & **Recall Rate (\%)** & **Accuracy (\%)** \\ \hline
**keyword-based** & 700 & 100 & 41 \\
**semantic-based** & 440 & 63 & 68 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Comparison of precision and recall. and negative sentiment values as shown in Table 2. We know that when conducting sentiment analysis without the filtering system, the results showed that most of the top ten posts with negative sentiment values were not related to the desired theme content. Furthermore, to enhance the reliability of the dataset, we conducted manual annotation by inviting two volunteers to label the data.