# An Automatic Participant Detection Framework for Event Tracking on Twitter
## 4 Results
### Discussion  
In this discussion, we analyse how people talk about events on Twitter before events start, and how our APD algorithms and off-the-shelf NER libraries perform in this environment. A glance at Figure 3 makes it immediately clear that the discussion starts well before the event itself. The figure shows a sharp spike in tweeting volume in our football datasets around the time when clubs release their line-ups for the upcoming match. This behaviour reflects the importance that users attribute to participants, as evidenced by how announcing the starting line-ups propels discussion. However, not all participants are equally important. The best way to understand how Twitter users talk about events is by examining the rankings produced by NLTK and TwitterNER. The low precision and recall results of these baselines stand out in Table 2, only around a quarter of NLTK's and TwitterNER's top 50 participants are valid. Intuition would have it that NER tools return many false positives because of Twitter's informal nature, but that is an inaccurate assessment. The poor results of the baselines do not reflect poorly on the quality of NLTK and TwitterNER, which are relatively capable of overcoming Twitter's challenges, but on the way Twitter users talk about events. In fact, discussions on Twitter go beyond the event and consider its broader context; the result of one football match may affect other teams, so Twitter users often talk about \begin{table}
\begin{tabular}{l c c c c c c c c} \hline \hline  & \multicolumn{4}{c}{**Baseline**} & \multicolumn{4}{c}{**Resolution**} \\ \cline{2-9}  & \multicolumn{2}{c}{**NLTK**} & \multicolumn{2}{c}{**TwitterNER**} & \multicolumn{2}{c}{**APD\({}_{\text{NLTK}}\)**} & \multicolumn{2}{c}{**APD\({}_{\text{TwitterNER}}\)**} \\ \cline{2-9}
**Match** & **Precision** & **Recall** & **Precision** & **Recall** & **Precision** & **Recall** & **Precision** & **Recall** \\ \hline Barcelona-Atletico Madrid & 0.16 & 0.16 & 0.14 & 0.14 & 0.73 & 0.16 & 0.70 & 0.14 \\ Wolves-Arsenal & 0.20 & 0.22 & 0.14 & 0.16 & 0.56 & 0.11 & 0.67 & 0.13 \\ Aston Villa-Manchester United & 0.32 & 0.36 & **0.42** & **0.47** & **0.93** & **0.31** & **0.95** & 0.40 \\ Liverpool-Burrley & **0.34** & **0.38** & 0.38 & 0.42 & 0.82 & **0.31** & 0.48 & 0.24 \\ Arsenal-Liverpool & **0.34** & **0.38** & 0.28 & 0.31 & 0.76 & 0.29 & 0.80 & 0.27 \\ Tottenham-Leicester & 0.30 & 0.33 & 0.36 & 0.40 & 0.71 & 0.22 & 0.56 & 0.20 \\ Liverpool-Chelsea & 0.20 & 0.22 & 0.26 & 0.29 & 0.50 & 0.20 & 0.53 & 0.22 \\ \hline Nets-Warriors & 0.18 & 0.27 & 0.28 & 0.42 & 0.69 & 0.27 & 0.78 & **0.42** \\ Lakers-Clippers & 0.16 & 0.24 & 0.20 & 0.30 & 0.62 & 0.24 & 0.67 & 0.30 \\ Celtics-Nets & 0.10 & 0.15 & 0.20 & 0.30 & 0.63 & 0.15 & 0.67 & 0.24 \\ Lakers-Mavericks & 0.16 & 0.24 & 0.18 & 0.27 & 0.71 & 0.15 & 0.78 & 0.21 \\ \hline Average & 0.22 & 0.27 & 0.26 & 0.32 & 0.70 & 0.22 & 0.69 & 0.25 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: The precision and recall of the rankings produced by the baselines and by our APD approaches after resolution. \begin{table}
\begin{tabular}{l c c c c} \hline \hline  & \multicolumn{2}{c}{**APD\({}_{\text{NLTK}}\)**} & \multicolumn{2}{c}{**APD\({}_{\text{TwitterNER}}\)**} \\ \cline{2-5}
**Match** & **Precision** & **Recall** & **Precision** & **Recall** \\ \hline Barcelona-Atletico Madrid & 0.46 & 0.45 & 0.43 & 0.35 \\ Wolves-Arsenal & **0.58** & 0.47 & 0.65 & 0.49 \\ Aston Villa-Manchester & 0.54 & 0.60 & **0.70** & 0.78 \\ Livinged & 0.48 & 0.53 & 0.48 & 0.53 \\ Arsenal-Liverpool & **0.58** & 0.64 & 0.58 & 0.64 \\ Tottenham-Leicester & 0.36 & 0.40 & 0.46 & 0.51 \\ Liverpool-Chelsea & **0.58** & 0.64 & 0.66 & 0.73 \\ \hline Nets-Warriors & 0.46 & **0.70** & 0.54 & **0.82** \\ Lakers-Clippers & 0.47 & 0.61 & 0.46 & 0.64 \\ Celtics-Nets & 0.17 & 0.24 & 0.40 & 0.61 \\ Lakers-Mavericks & 0.36 & 0.55 & 0.45 & 0.52 \\ \hline Average & 0.46 & 0.53 & 0.53 & 0.60 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: The precision and recall of our APD approaches’ final rankings after extrapolation. these effects.