# Multimodal Shot Prediction Based on Spatial-Temporal Interaction between Players in Soccer Videos
###### Abstract  
Sports data analysis has significantly advanced and become an indispensable technology for planning strategy and enhancing competitiveness. In soccer, shot prediction has been realized on the basis of historical match situations, and its results contribute to the evaluation of plays and team tactics. However, traditional event prediction methods required tracking data acquired with expensive instrumentation and event stream data annotated by experts, and the benefits were limited to only some professional athletes. To tackle this problem, we propose a novel shot prediction method using soccer videos. Our method constructs a graph considering player relationships with audio and visual features as graph nodes. Specifically, by introducing players' importance into the graph edge based on their field positions and team information, our method enables the utilization of knowledge that reflects the detailed match situation. Next, we extract latent features considering spatial-temporal interactions from the graph and predict event occurrences with uncertainty based on the probabilistic deep learning method. In comparison with several baseline methods and ablation studies using professional soccer match data, our method was confirmed to be effective as it demonstrated the highest average precision of 0.948, surpassing other methods. event prediction; deep learning; sports video analysis; multimodal machine learning +
Footnote †: journal: Medical  
Many of these methods predict specific match events that will occur next or subsequently from tracking data and event stream data by applying machine learning techniques. In addition, it is possible to evaluate individual or series of events based on their predicted probabilities. For instance, the study in [12] performs the prediction of major attacking events (pass, dribble, cross, and shot) by using a recurrent neural network (RNN) or transformer [20]. This study also quantitatively evaluates each team's possession with the likelihood of a shot or cross at the occurrence time of an individual event. However, event prediction methods described above use tracking data and event stream data, which require costly equipment and expert intervention. Here, the use of video data that are widely available in sports can be considered as one of the alternative approaches. With the rapid development of computer vision [21] and the release of large video datasets such as SoccerNet [22], there has been widespread research conducted on video analysis techniques (multi-player tracking [23; 24; 25; 26], event detection/classification [27; 28; 29; 30], video summarization [31; 32], etc.) in the field of soccer analysis. A few proposals have also focused on predicting who receives the pass and the occurrence of fuels from videos and pose information of players [33; 34]. Based on a wealth of visual information in match videos, the above methods provide valuable insights without using expensive instruments. Additionally, since the direct use of video reduces reliance on manual annotation, it is expected to become a more accessible and efficient solution for tactical planning and improving player performance at various competition levels. However, it is crucial to predict the occurrence of major events such as passes and shots rather than who receives a pass or the foul occurrence for player evaluation, and thus technology is required to predict these events based on soccer videos. To tackle the aforementioned issue, we propose an event prediction method in soccer videos focusing on the shot, one of the primary events. In the proposed method, we employ unedited soccer videos, which are known as scouting videos, captured from a bird's-eye view with a single camera.