# Recognizing Events in Spatiotemporal Soccer Data
## 5 Event Detection Quality Evaluation  
The most complicated part of event markup in our pipeline is the detection of passes and shots. Thus, we perform a brief evaluation to estimate its accuracy. We created a \"gold standard\" markup by watching the games that comprise our datasets in a 2D soccer simulator and manually annotating successful passes, unsuccessful passes, and shots on goal. In the case of the ST dataset, we analyzed the 40 longest episodes, corresponding to 60 min of playing time in total (see Table 3). In the case of the DS dataset, all five matches were annotated. As manual annotations are difficult to synchronize with events, we consider events as correctly recognized if they fall within a [\(-0.5\) s, \(+0.5\) s] interval of the corresponding \"gold standard\" event. Since the proposed algorithm relies on specific parameter values, listed in Table 2, fine-tuning them is necessary to achieve high event detection performance. The initial experimental setup is based on manually chosen values, reflecting our general understanding of soccer and observations of the gold standard markup process. To reveal the optimal combination of parameter values, we apply a greedy search routine. It evaluates the algorithm on the set of parameters \(\{p_{1},\ldots,p_{i},\ldots,p_{n}\}\), where the value of \(p_{i}\) is iterated over a predefined range, while the rest of the values remain constant. The same process is repeated for each parameter in the set. This approach is feasible in our case because most parameters are loosely related and affect different types of situations in the game. For example, both \(MinTrChangeAngle\) and \(MinSpeedChangeFactor\) values affect the ability of the system to recognize changes in ball possession. However, they are introduced to deal with different types of game episodes (see Listing 1), so an optimal value of one parameter should increase the overall performance of the whole event detection algorithm. It is also easy to choose reasonable ranges and loop steps due to physical constraints of soccer. The quality of event detection can be estimated with recall, precision, and \(F_{1}\) score values, calculated according to Equations (5)-(7). The search routine iterates over a range of permissible values of an individual parameter, while the rest are fixed at their initial values, specified in Table 2. \begin{table}
\begin{tabular}{l c c} \hline \hline \multirow{2}{*}{**Event**} & \multicolumn{2}{c}{**Quantity**} \\ \cline{2-3}  & **ST** & **DS** \\ \hline Successful pass & 1097 & 2910 \\ Unsuccessful pass & 95 & 331 \\ Shot on goal & 8 & 97 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Annotated events. Figure 2: A scheme of an unsuccessful pass and a deflected shot. We perform this routine for each parameter except \(Goal\,post\,Distance\), which affects shot on goal events only. Since the initial setup already provides optimal recognition of shots on goal, we keep the value of \(Goal\,post\,Distance\) unchanged. It should also be mentioned that we perform a search for optimal parameter values using the ST dataset only. The DS dataset contains significant player jitter, and thus cannot serve as a reliable ground for fine-tuning. In contrast, the ST dataset consists of short game fragments played by different teams in different matches, serving as a reliable sample of episodes found in a typical soccer game. In our experiments, 70% of the annotated ST dataset is used to determine the optimal parameter values, while the remaining 30% of data is reserved for the evaluation:  
\[Precision=\frac{TruePositives}{TruePositives+FalsePositives} \tag{5}\]  
\[Recall=\frac{TruePositives}{TruePositives+FalseNegatives} \tag{6}\]  
\[F_{1}score=2\cdot\frac{Precision\cdot Recall}{Precision+Recall}. \tag{7}\]  
The evaluation routine (the source code for event recognition and parameter tuning is available at github.com/vi3 itor/soccer-event-recognition) allows us to make the following observations (see Figure 3).