PassNet: Learning pass probability surfaces from single-location labels. An architecture for visually-interpretable soccer analytics
Authors: Javier Fernández
Luke Bornn
Venue:  (2020)
Keywords: fully convolutional neural networks
convolutional neural networks
sports analytics
interpretable machine learning
deep learning

Abstract: We propose a fully convolutional network architecture that is able to estimate a full surface of pass probabilities from single-location labels derived from high frequency spatio-temporal data of professional soccer matches. The network is able to perform remarkably well from low-level inputs by learning a feature hierarchy that produces predictions at different sampling levels that are merged together to preserve  both coarse and fine detail. Our approach presents an extreme case of weakly supervised learning where there is just a single pixel correspondence between ground-truth outcomes and the predicted probability map. By providing not just an accurate evaluation of observed events but also a visual interpretation of the results of other potential actions, our approach opens the door for spatio-temporal decision-making analysis, an as-yet little-explored area in sports. Our proposed deep learning architecture can be easily adapted to solve many other related problems in sports analytics; we demonstrate this by extending the network to learn to estimate pass-selection likelihood.

Under review as a conference paper at ICLR 2020
PASSNET: LEARNING PASS PROBABILITY SURFACES FROM SINGLE-LOCATION LABELS. AN ARCHITECTURE FOR VISUALLY-INTEEPRETAABL SOCCER ANALYTICS
Anonymous authors Paper under double-blind review
ABSTRACT
We propose fully convolutional network architecture that is able estimate a full surface of pass probabilities from single-location labels derived from high frequency spatio-temporal data of professional soccer matches. The network is able to perform remarkably well from low-level inputs by learning feature hierarchy that produces predictions at different sampling levels that are merged together preserve both coarse and fine detail. Our approach presents extreme case of weakly supervised learning where there is just single pixel correspondence between ground-truie truth outcomes and the predicted probability map. By providing not just an accurate evaluation of observed events but also a visual interpretation of the results of other potential actions, our approach opens the door for spatio- temporal decision making analysis, an ys- little-explored area in sports. Our proposed deep learning architecture can be easily adapted solve many other related problems in sports analytics; we demonstrate this by extending the network learn to estimate pass- selection likelihood.
INTRODUCTION
Sports analytics is fast-growing research field with strong focus on data-driver performance analysis from professional athletes and teams. Soccer, well as many other team-sports, has recently benefited from the availability of high-frequency tracking data of both player and ball locations, facilitating the development of fine-grained spatio-temporal performance metrics (Rein Memmert 2016, [Stein et al. 2017). One of the main goals of performance analysis is to answer specific questions from soccer coaches, but in order to do so we require models to be robust enough to capture the nuances of complex sport, and be highly interpretable so findings can be communicated effectively. In other words, we need models to be both accurate and also translatable to soccer coaches and game analysts in visual terms.
The majority of research in soccer analytics addresses questions related to either the most commonly observed events during a match, such as goals, shots and passes or the effects of players' movements and match dynamics (Gudmundsson Horton 2017). Most modeling approaches share one or more common issues, such as: heavy use of handcrafted features, no visual interpretability. and coarse representations that ignore meaningful spatial relationships. While many of these methods are able to provide valid insights into specific problems, in terms of model design we still lack comprehensive approach that can learn from lower-level input, exploit spatial relationships on any location (beyond location of the origin and destination of events), and provide not only accurate evaluations of observed events their given location but also an estimation of unobserved events at any other location on the field.
In this work we present a deep neural network architecture designed to estimate the probability of passing events in soccer from low-level tracking-data input and weak labeling of each event's s success. The network is able to estimate the probability of pass success at any location the field while providing a visual representation of these probabilities. Beyond pass probabilities, such an architecture can be adapted to many other problems such pass selection, pass risk and reward

---

Under review as a conference paper at ICLR 2020
evaluation, expected possession value estimation, and player movement evaluation, among many others.
The presented architecture was inspired by recently developed fully convolutional networks that have been proven be successful for image segmentation (Long al.. 2013 Yu & Koltun 2015 Pathak et al. 2015 Jing & Tian 2019). Our problem is similar in many ways to that of image segmentation, where the objective is provide a pixel-level classification of objects in the image. Image segmentation is usually addressed as a weakly supervised learning problem where only single labels of the objects in the image are provided, instead of pixel-wise labeling. In the case of pass probability, as well as in most of the other mentioned applications in soccer, only a single label is available for each pass to indicate whether it was successful or not, as well as the location of the origin and destination of passes. However, unlike image segmentation where an object label is typically associated with multiple pixels in the image, here the label only corresponds to single location in the original input. This set up presents an extreme case of weakly supervised learning. We use a deep jer-inspired nonlinear feature hierarchy (Long et al. 2015) to combine coarse and fine layers, through convolutional fusion layers. One of the main contributions of this paper is the computation of the network loss by single-pixel masking, according to the destination location associated each pass event. We show that from backpropagation of the loss between single-low single-location prediction of the full output matrix and the known outcome of the event we can learn complex spatial features and successfully estimate the pass probability map for any location on the field.
We compare our model with a baseline estimation of pass probability, as well as with linear and non-linear models built on top of handcrafted features, and achieve considerably better results for single pass probability estimation, while also properly estimating the complete probability map. In the following sections we describe related work, provide a detailed explanation of the architecture and design considerations of the model, and present experimental results on a broad set of 208, 489 passing events from professional soccer matches.
2 RELATED WORK
applied standpoint our work is related to several other approaches aimed at estimating pass probabilities and other performance metrics derived from spatio-temporal data in soccer. Regarding the technical approach we leverage recent findings on weakly supervised learning problems and the application of fully convolutional neural networks for image segmentation.
From an
Soccer analytics Pass probability estimation has been approached in several ways. A physics-based model of the time it takes each player to reach and control the ball has been used to derive pass probabilities on top of tracking data (Spearman et al. (017). Other approaches include the use of dominant regions, influenced by motion information, to determine which player is most likely to control the ball after a pass (Gudmundsson & Horton| 2017), or the building of linear models based on a carefully selected set of handcrafted features (Power et al. 2017). The related problem of pass selection has been approached by the application of convolutional neural networks that predict the likelihood of passing to a specific player on the attacking team (HumHHkčček et al. 2018). The estimation of pass value has been approached either by the aoppertddggideed development of algorithmic rules (Cakmak et al., [2018] Link et al.) al.| (016)). the application of standard machine learning algorithms on a set of handcrafted features (Gyarmati & Stanojevic [016)] Power et al., 2017), problem-specific deep learning models with dense layers and single output prediction (Wagenaar et al. [017]] Fernández) al. 2019). While some of the related work has estimated probability surfaces by inference on set
of discrete pass destination location (Spearman et al. 2017 Fernández et al. 2019). none has yet approached learning of probability surfaces directly.
Fully convolutional networks and weakly supervised learning Fully convolutional networks have been extensively applied to semantic image segmentation, specifically for the pixel-labeling problem to successfully detect wide pixel areas associated with object in images. The approach most related to our work builds a hierarchy of features at different sampling levels that are merged together to provide segmentation regions that preserve both fine and coarse details (Long al. 2015). From
2

---

Under review as a conference paper at ICLR 2020
learning perspective image segmentation has been approached either as fully supervised (Long et al.
2013 Noh et al.] | 2015), weakly supervised (Pathak et al. 2015 Pinheiro & Collobert 2015 Zhang
al.
2013] Durand et al. 2017). or semi-supervised learning problem (Papandreou
2013] Bai
et al.
2017, Souly et al. 2017). Commonly available labels are associated with many other pixels
the original image. However, in our case, labels only associated with single location in the
desired probability map, making our learning problem a rare and extreme case of weakly supervised learning.
3
A DEEP MODEL FOR INTERPRETABLE ANALYSIS IN SOCCER
104 68
x
B2 x4
20x17
S2
62 94
T04
TDKeeb
100x68
Condtt
Max pooling
Com220 prodiction
Upsamping
Fusion
Sigraed
sellector
Figure I: PassNet architecture for a coarse soccer field representation of 104 68 and 8 input channels.
We build our architecture top of tracking data extracted from videos of professional soccer matches, which consist of the 2D-location players and the ball at 10 frames per second, along with manually tagged passes. At every frame where pass has been labeled and create a representation of game situation consisting of a x h X matrix, where c channels of low-level information mapped a h coarse spatial representation of the locations the field. We seek an architecture that is able to learn both finer features at locations close to a possible passing destination and features considering information might be associated with the likelihood of nearby team-mates and opponents reaching the destination location and information about local spatial pressure. On the other hand, greater scale features might consider player's 's density and interceptability of the ball in its path from the location of origin. Finally, we seek estimate this passing probability any other location on the X spatial extent the field.
take snapshot of the tracking data
greater spatial scale. For passes, local features
This representation of the situation is processed by the deep neural network architecture presented in Figure [] The network creates feature hierarchy by learning convolutions at 1r, 1/2.r and 1/4x scales, while preserving the receptive field of the filters. Predictions are produced at each of these scales, and then upsampled nonlinearly and merged together through fusion layers. A sigmoid activation layer is applied to the latest prediction to produce pass probability estimations at every location, preserving the original input scale. During training, single-location prediction, associated with the destination sample pass selected to compute the log-loss that is backpropagated to adjust the network weights.

---

Under review as a conference paper at ICLR 2020
3.1 REASONING BEHIND THE CHOICE OF LAYERS
The network incorporates different types of layers: max pooling, linear, rectilinear linear unit (ReLu) and sigmoid activation layers, and 2D-convolutional filters (conv2d) for feature extraction, prediction, upsampling and fusion. In this section we present detailed explanation of the reasoning behind the choice of layers and the design of the architecture in general.
Convolutions for feature extraction each of the 1x, 1/2r and 1 /4x scales two layers of conv2d filters with 5 receptive field and stride of 1 are applied, each one followed by ReLu activation function layer, in order to extract spatial features at every scale. In order to keep the same dimensions after the convolutions we apply a symmetric padding the input matrix of the convolutional layer. We chose symmetric-paddinng to avoid border image artifacts that can hinder the predicting ability and visual representation of the model (Wu et al. 2019).
Fully convolutional network There are several conceptual and practical reasons for considering convolutional neural networks (convnets) for this problem. Convolutional filters are specifically designed to consider the relationships between nearby pixels so learned features are spatially aware. Convnets have proven to be successful in data sources with an Euclidean structure, such as images and videos, so a 2D-mapping of soccer field location-based information can be expected to be an ideal data structure for learning meaningful features. Also, these features are expected to be non-trivial and complex. Convnets have been proven to learn what are sometimes more powerful visual features than handcrafted ones, even given large receptive fields and weak label training (Long et al. 2014) Regarding the architecture design, we are interested in learning the full x h mapping of passing probabilities covering the whole of a soccer field, for which fully convolutional layers are more appropriate than classical neural networks built for classification, by subtracting dense prediction layers, and using 1x1 convolutions instead.
Pooling and upsampling The network applies downsampling twice through max pooling layers in order to obtain the 1/2z and 1/4z representations. Since activation field size is kept constant after every downsampling step the network is able to learn filters of a wider spatial extent, leading to the detection of coarse details. We learn non-linear upsampling functions at every upsampling step by first applying a 2x nearest neighbor upsampling and then two layers of convolutional filters. The first convolutional layer consists of 32 filters with
3 activation field and stride 1, followed by ReLu activation layer. The second layer consists of 1 layer with a 3 x 3 activation field and stride 1, followed by linear activation layer. This upsampling strategy has been shown to provide smoother outputs and to avoid artifacts that can be usually found in the application transposed convolutions (Odena et al. 2016).
Prediction and fusion layers Prediction layers consist of a stack of two convolutional layers, the first with 32 1 x convolutional filters followed by an ReLu activation layer, and the second consisting of 1 1 x 1 convolutional filter followed by a linear activation layer. Instead of reducing the output to single prediction value, we keep the spatial dimensions at each step and use 1 1 convolutions to produce predictions at each location. The stack learns a non-linear prediction on top of the output of convolutional layers. In order merge the outputs at different scales we concatenate the pair of matrices and pass them through convolutional layer of 1 X filter. In other approaches, where predictions at different scales are merged (Long et al. 2015 Wei al., 2018). usually matrixes are joined by simple addition. However, this addition would be highly dependent on the distribution of output values at each prediction step, and without proper normalization the differences numerical ranges might over-emphasize the ouiputs in one scale over the other. Or fusion layer learns simple weighting function to add predictions at every location.
3.2 LEARNING FROM SINGLE-LOCATION LABELS
We seek model that can produce accurate predictions of the pass probability to every location
1 h coarsed representation of a soccer field, given a 1 e representation of the game situation at the time pass is made. In training, only count with manually labeled location of the pass destination and a binary label the outcome of the pass.
4

---

Under review as a conference paper at ICLR 2020
Definition 3.1 (PassNet) Let X {x|x== x R be the of possible game state representations at any given time, where I, h E N2 are the height and length of a coarse representation of soccer field, and c E N2 the number of data channels, a PassNet is a function (x;0), f glxxxx + u..1] where I produces a pass probability map and 0 are the network parameters.
glxh
Definition 3.2 (Target-location loss) Given the sigmoid function o axx) - and let yu {0, 1}
be the outcome of the passing event at time t(xx) for game state Ihe dg the destination location of the passing event k, I a PassNet with parameters 0, and logloss the log-loss function, we define the carget-locatinn loss as
(1f 0), Uk: dx) logloss(f(1zz)
We approach the training of the model as weakly-supervised learning task. We consider this problem set to be an extreme case of weak learning where ground truth labels only correspond to a single location in the full mapping matrix that needs to be learned. The target-location loss presented in Definition 3.2 essentially shrinks the output of PassNet to single prediction value by selecting the prediction value at the destination of the pass, and then computes the log-loss between this single prediction and the ground-truth outcome value.
3.3 SPATIAL AND CONTEXTUAL CHANNELS FROM TRACKING DATA
Our architecture is designed to be built on top of two common sources of data for sports analyties: tracking data and event data. Tracking data consists of the location of the players and the ball at high frequency-rate, Event-data corresponds to manually labeled observed events, such as passes, shots and goals, including the location and time of each event. We normalize the location of players and the ball to ensure the team in possession of the ball is attacking from left to right, thus standardizing the representation of the game situation. On top of players' and ball location we derive low-level input channels including spatial (location and velocity) and contextual information (ball and goal distance). An ideal choice of (h, 1) is the maximum permitted size of soccer field (105, 68) in meters, so every location represents m of space the field. However, given that the architecture requires 1/2x pooling steps, we conveniently set h to 104.
Definition 3.3 (Tracking-data snapshot) Let (t). Zt) (t) E {z|= E R(xb) be the locations of the attacking team players, the location of the defending team players, the location of the ball, and the location of the opponent goal, respectively, at time t, then a tracking-date snapshot at time is defined as the 4-tuple (t) (t), Zt (t). (t).
In order create game state representation (t) X as described in Definition 3.1 we pro- duce 8 different channels on top of each tracking-date snapshot Z where pass has been ob- served. Let (i,j) E NIo.-11 Njo,--1j be a location in the coarse soccer field representation, Z(1) (t,, Za(t), (t)) the tracking-data snapshot fine the channel extracting functions C², C.V, W.v,Co CVv, (V,,CE E Rixh Rlxx, where (Zx pro- duces a sparse matrix setting for every (i.j) E Zr, CV+ and CV2, produce each one a sparse matrix setting the magniude of velocity in horizontal and vertical axis, respectively, for every (i,j) E ZR, and C produces a matrix with the Euclidean distance of each location to the first element (i,)) E 7z. Having this, we produce 8-channels game-state representation such that X (CL (Za), CV= (Zp), (%,(),), C²= (Za), CVv (Za), C (Zo), C (Zg)).
time t, Zx(()= E Zt),, we de-
4 EXPERIMENTS AND RESULTS
4.1 DATASET
We use tracking-data and event-data from 309 English Premier League matches from the 2014/2015 season, provided by STATS LLC. Each match contains the (z, v) location for every player and the ball sampled at 10Hz, The event-data provides the location, time, player, team and outcome for 208,419 passes. From this data we extract the channels described in Section 3.3 for coarse (104, 68) representation a soccer field to obtain a dataset of size 208624 x 104 x 68 x 8. There are 166,791 successful passes and 41, 1,628 missed passes, showing highly imbalanced distribution of outcomes.
5

---

Under review as a conference paper at ICLR 2020
4.2 BENCHMARK MODELS
We compare our results against a series of benchmark models of increasing levels of complexity. We define a baseline model Naive that for every pass outputs the known average pass completion in the full dataset (81.54%) following a similar definition in Power et al. (2017). We build two additional models Logistic Net and Dense2 Net based on set of handcrafted features built on top of tracking- Logistic Net is a network with a single sigmoid unit, and Dense2 Net is a neural network with two dense layers followed ReLu activations and sigmoid output unit.
Handcrafted features We build a set of spatial features on top of tracking-data based on location and motion information on players and the ball that is similar to most of the features calculated in previous work on pass probability estimation (Power et al. 2217 |Spearman et al. 2017 Gudmundsson Horton 2017: Hubáček et al., 2018). We define the following set of handcrafted features from each pass: origin and destination location, pass distance, attacking and defending team influence at both origin and destination, angle to goal at origin and destination, and max value of opponent influence in straight line between origin and destination. Team's 's spatial influence values are calculated following the model presented Fernández & Bornn (2018).
Pass selection model We provide an example of how the architecture can be extended to other similar problems by training the network to generate pass selection likelihood surfaces. This refers to the likelihood of a pass being made towards every other location on the field. To achieve this, we simply have to modify the sigmoid activation layer of the original architecture by a softmax activation layer. Softmax ensures that the sum of probabilities instead of pass success, our target output 1, indicating a pass has been made, while the loss is still computed by obtaining the log-loss between the prediction at the destination location and
the output surface add up to 1. For this case,
4.3 EXPERIMENTAL FRAMEWORK
Training, validation and test sets First the data is randomly shuffled and then split into a training, validation and test set with 60 20 20 distribution. We apply stratified split the success- ful/missed pass class ratio remains the same across datasets. The validation set is used for model selection during a grid-search process. The test set is left hold-out data, and results are reported on performance for this dataset. For the benchmark models, datasets are built by extracting the features described in Section 4.2] and an identical split performed. Features are standardized column- wise by subtracting by the mean and dividing by the standard deviation.
Approaching imbalance through -nowleeg We apply domain-knowledge of soccer in order to produce a balanced class outcome for the training set. We define a pass as unreachable if the closest player on the defending team to the destination location is 1.5 times closer than the closest player in the attacking team. This definition was empirically validated by a group of expert soccer analysts from F.C Barcelona. We randomly select successful passes without replacement and then draw random possible locations until the pass is classed as wireachable, and repeat this procedure until we reach a 50/50 balanced training dataset. The validation and test dataset are left as originally sampled. A class-weightin with no upsampling approach was tested, but better results where achieved by augmentation.
Optimization Both the PassNet network and the baseline models are trained using adaptive moment estimation (Adam). Model selection is achieved through grid-search on learning rates 10-3, 10-1 and 10 5, and batch sizes of 1, 16 and 32, while 31, B3 are set to 0.9 and 0.999 respectively. We use carly stopping with minimum delta rate of 0.001. Optimization is computed on a single Tesla M60 GPU and using Tensorflow 1.5.0. During the optimization the negative log-loss is minimized.
Let N be the number of examples in the dataset, y the ground-truth labels for a pass events and ý the model predictions. We report the negative log-loss
Metries
1 n Σ y Log(ū.) (1 ys) log(1 (.)
L(o v)
In order to validate the model calibration we use a variation of the expected calibration error (ECE) presented in Guo et al. (2017) which computes the expected difference between accuracy and

---

Under review as a conference paper at ICLR 2020
confidence of the model finite set of samples splitted into K bins of size 1/K, according to the predicted confidence or probability for every sample. Since our model is not designed for classification we use the count of number of examples of the positive class rather than accuracy for ECE.
Let B3 be bin where & [0, K]
then
K
Byl N
ECE-
Σ B
1(y = -1) -
Σ iE
01 B
BB
Bk
A perfectly calibrated model will have a ECE value of 0. Additionaly we provide calibration reliability plot (Guo et al. 2017) showing the mean confidence for every bin By.
4.4 RESULTS
Table ! prresents the results for the benchmark models and PassNet for the pass probability dataset. We can observe that PassNet achieves remarkably less error than the other models and produces calibrated estimation of pass probability. Despite the considerably large number of parameters in PassNet, the inference time for single sample is low enough to produce real time estimation for frame rates below 200Hz. The ECE results for both Logistic Net and PassNet represent a mean difference in probability estimation lower than 2.72% for both models, which make them suitable for use in practice. Figure presents a calibration reliability plot for each of the models. Consistently with the ECE results, both Logistic Net and PassNet produce well calibrated estimations of pass probabilities. Despite both models being calibrated, PassNet is proven to be considerable more precise when estimating individual sample probabilities, as shown the difference in log-loss between both.
Table 1: Results for the benchmark models and PassNet for the pass probability dataset.
Model
Log-loss
ECE
Inference time
Number parameters
Naive
0.418
0
Logistic Net
0.275
0.01854
0.00191s
11
Dense2 Net
0,255
0.0824
0.00204s
231
PassNet
0.182
0.0272
0.00457s
397, 259
1.0
0.8
Fraction positves
0.6
0.4
Perfectly calibrated
0.2
Naive (0.481)
Logistic (0.271)
Dense2 (0.252)
PassNett (0.182)
0.0
0.2
0.4 Mean predicted value by bin
0.6
0.8
1.0
Figure 2: the left calibration reliability plot, where for series of bins of equal sizes, the axis presents the mean predicted value for samples in each bin, and the Y axis the fraction of samples in each bin containing positive examples. On the right, a comparison of pass probability surfaces from Logistic Net, Dense2 Net and PassNet models, from top to bottom.
7

---

Under review as a conference paper at ICLR 2020
Figure presents the predicted pass probability surfaces and the pass selection surface for specific game situation during a professional soccer match (two videos showing the evolution of both surfaces in time can be obtained at atttt:: /ttp:y///inn.././ cc/pzcdzz and http::/ting cc hww.hhhchhaa eddz). We observe that the model is able to capture both fine-grained information, such as the influence of defending and attacking players on nearby locations, and coarse information such the probability of reaching wider spatial areas depending on the distance to the ball and the proximity of players. We can also observe that the model considers player' speed, so it can predict probabilities of passing to not-yet occupied spaces, critical aspect of practical soccer analysis. In the right column of Figure we we can also observe a comparison between predicted probabilities for the benchmark model and PassNet. The surfaces for the former are estimated by predicting the pass probability for every location individually. Although able to capture global information, both benchmark models fail to make more in-depth sense of local information, particularly when there is high density of players in local regions. They also over-estimate the probability of passing to regions that are barely reachable by any player.
0.75
0.5
0.25
-1.61
14.62
18.6
Figure 3: On the left column 2D representation of the pass probability surface (top) and the pass selection surface (bottom) for a given game situation. Yellow and blue circles represent locations of players the attacking and defending team, respectively, and the arrows represent the velocity vector for each player. The white circle represents the location of the ball. Pass selection likelihood is represented logarithmic scale. The right-hand column presents video capture from the associated soccer match with an overlay of the corresponding surface.
5 DISCUSSION AND FUTURE WORK
We have presented deep neural network architecture that is able to produce full pass probability surfaces from low level spatio-temporal soccer data. The network able perform remarkably well at estimating the probability of observed passes, while also being able to estimate the probability of any other potential pass. By merging features extracted at different sampling levels the network is able to extract both fine and coarse details, thereby managing to make sense of the complex spatial dynamics of soccer. The features are learned following an extreme case of weakly supervised learning where only single-location labels are available and the network is forced to indirectly provide accurate predictions of individual passes by directly learning full probability surface. The produced surfaces can provide an accurate visual representation of game situations that can be interpreted directly by soccer coaches. The network can also be adjusted to deal with many other related problems such the estimation of pass reward, the probability of actions leading goals, or the likelihood of ball turnover, among others. We show this by sucessfully training the same architecture to estimate surfaces of pass selection likelihood, with few simple adjustments.
8

---

Under review as a conference paper at ICLR 2020
REFERENCES
Wenjia Bai, Ozan Oktay. Matthew Sinclair, Hideaki Suzuki, Martin Rajchl, Giacomo Tarroni, Ben Glocker, Andrew King, Paul M Matthews, and Daniel Rueckert. Semi-supervised learning for network-based cardiac mr image segmentation. In International Conference on Medical Image Computing and Computer-Asiisted Intervention, pp. 253-260. Springer, 2017.
Ali Cakmak, Ali Uzun, and Emrullah Delibas. Computational modeling of pass effectiveness in soccer. Advances in Complex Systems, 21 21(03n04) 2018.
Thibaut Durand, Taylor Mordan, Nicolas Thome, and Matthieu Cord. Wildcat: Weakly supervised
learning of deep convnets for image classification,, pointwise localization and segmentation, In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 642-651, 2017.
Javier Fernández and Luke Bornn. Wide open spaces: A statistical technique for measuring space creation in professional soccer. In Proceedings of the 12th MIT Sloan Sports Analyties Conference. 2018.
Javier Fernández, Luke Bornn, and Dan Cervone. Decomposing the immeasurable sport: A deep learning expected possession value framework for soccer. In Proceedings of the 1 3th MIT Sloan Sports Analytics Conference, 2019.
Joachim Gudmundsson and Michael Horton. Spatio-temporal analysis of team sports. ACM Comput- ing Surveys (CSUR), 50(2):22, 2017.
Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger. On calibration of modern neural networks. In Proceedings of the 34th International Conference on Machine Learning-Voiie Volume 70, pp. 1321-1330. JMLR. org, 2017.
Laszlo Gyarmati and Rade Stanojevic. Qpass: merit-based evaluation of soccer passes. arXiv preprint arXiv:1608.0 03532, 2016.
Ondfej Hubáček, Gustav Sourek, and Filip Železný. Deep learning from spatial relations for soccer pass prediction. In International Workshop on Machine Learning and Data Mining for Sports Analytics, pp. 159-166. Springer, 2018.
Longlong Jing and Yingli Tian. Self-supervised visual feature learning with deep neural networks: A survey. arXiv preprint arXiv: 1921::1902.06162, 2019.
Daniel Link, Steffen Lang, and Philipp Seidenschwarz. Real time quantification of dangerousity in football using spatiotemporal tracking data. PloS one, 11(12):e018766, 2016.
Jonathan Long, Ning Zhang, and Trevor Darrell. Do convnets learn correspondence? In Advances in Neural Information Processing Systems, pp. 1601-1609, 2014.
Jonathan Long, Evan Shelhamer, and Trevor Darrell. Fully convolutional networks for semantic segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 3431-3440, 2015.
Hyeonwoo Noh, Seunghoon Hong, and Bohyung Han. Learning deconvolution network for semantic segmentation. In Proceedings of the IEEE international conference on computer vision, pp. 1520-1528, 2015.
Augustus Odena, Vincent Dumoulin, and Chris Olah. Deconvolution and checkerboard artifacts. Distill, 1(10);e3, 2016.
George Papandreou, Liang-Chies Chen, Kevin P Murphy, and Alan L Yuille. Weakly-and semi- supervised learning deep convolutional network for semantic image segmentation. Proceed- ings of the IEEE international conference computer vision, pp. 1742-1750, 2015.
Deepak Pathak, Philipp Krahenbuhl, and Trevor Darrell. Constrained convolutional neural networks for weakly supervised segmentation. Proceedings of the IEEE international conference on computer vision, pp. 1796-18800 1804, 2015.

---

Under review as a conference paper at ICLR 2020
Pedro O Pinheiro and Ronan Collobert. From image-level to pixel-level labeling with convolutional networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1713-1721, 2015.
Paul Power, Hector Ruiz, Xinyu Wei, and Patrick Lucey, Not all passes are created equal: Objectively measuring the risk and reward of passes in soccer from tracking data. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp- 1613, ACM, 2017.
Robert Rein and Daniel Memmert. Big data and tactical analysis in elite soccer: future challenges and opportunities for sports science. SpringerPlus, 5(1):1410, 2016.
Nasim Souly, Concetto Spampinato, and Mubarak Shah. Semi supervised semantic segmentation using generative adversarial network. In Proceedings of the IEEE International Conference
Computer Vision, pp. 5688-5696, 2017.
William Spearman, Austin Basye, Greg Dick, Ryan Hotovy, and Paul Pop. Physics-based modeling of pass probabilities soccer, In Proceeding of the 11th MIT Sloan Sports Analytics Conference, 2017.
Manuel Stein, Halldór Janetzko, Daniel Seebacher, Alexander Jäger, Manuel Nagel, Jürgen Hölsch, Sven Kosub, Tobias Schreck, Daniel Keim, and Michael Grossniklaus. How to make sense of team sport data: From acquisition to data modeling and research aspects. Data, 2(1):2, 2017.
Martijn Wagenaar, Emmanuel Okafor, Wouter Frencken, and Marco A Wiering. Using deep convolu- tional neural networks to predict goal-scoring opportunities in soccer. In ICPRAM, pp. 448-455, 2017.
Yunchao Wei, Huaxin Xiao, Honghui Shi, Zequn Jie, Jiashi Feng, and Thomas S Huang. Revisiting dilated convolution: A simple approach for weakly-and semi-supervised semantic segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7268-7277, 2018.
Shuang Wu, Guanrui Wang, Pei Tang, Feng Chen, and Luping Shi. Convolution with even-sized kernels and symmetric padding. arXiv preprint arXiv: 193x::9033.0885, 2019.
Fisher Yu and Vladlen Koltun. Multi-scale context aggregation by dilated convolutions. arXiv preprint arXiv:1511.0713 2015.
Luming Zhang, Mingli Song, Zicheng Liu, Xiao Liu, Jiajun Bu, and Chun Chen. Probabilistic graphlet cut: Exploiting spatial structure cue for weakly supervised image segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1915-1915, 2013.
10


Reviews:
[{"review_id": "Sye0HBDpFB", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The aim of the system presented in this paper is to produce a 2D map of probabilities showing the chance of successful completion of a soccer pass to all locations on the field, given coordinate locations of players and the ball sampled over time.  A network based on a fully-convolutional style semantic segmentation network is applied to a 2D, 8-channel game state representation, with final sigmoid layer to predict a pass success indicator at each location.  The system is trained using manually annotated pass success and destinations, which corresponds to a label on the destination point; locations other than the labeled point are not trained (treated as incomplete/\"don't-care\" training targets).  Evaluation is performed using both log loss and probability calibration measure, to measure effectiveness of predictions as well as how well they calibrate to correspond to probabilities in the sense defined by the measurement.\n\nThis is a fun application and it appears the system is effective at a basic level.  However, I think both the theory/explanation and experiments leave a fair bit of uncertainty as to what the probabilities mean and how to interpret them, including conflation of success probability and the model's certainty in its estimate.\n\nIn particular, it is unknown to what degree the values output by the model can be interpreted as a predictive probability of pass completion for anywhere on the field.  If one location says 0.8 and the other 0.5, does this mean that if the player were to actually pass to the 0.5 location, the chance of success is actually lower?  Or does it mean that the model is less confident or that this location was under-trained for this state?  The ECE measurement in conjunction with loss error doesn't quite address this:  although a good verification of calibration in its own sense, ECE simply confirms that for cases where the model predicts 0.5, there is success 0.5 of the time for locations that *exist in the test set*, which is sampled according to player action.  To verify the probability maps at all locations, one would need to be able to measure *any* point in the field, not just those already selected by the players' actions.  I think discussion and attempt to measure this is fairly important, as one of the intended applications in the motivation is analyzing what might have happened had a player selected a different destination point.\n\nUnfortunately, knowing the true outcomes at arbitrary locations the players didn't pass to is impossible, so addressing this issue is not straightforward, and unclear to me how it might be done.  A possible suggestion, is to use the destination selection predictions that the paper also mentions can be found using softmax instead of sigmoid.  Although this does not entirely eliminate the issue (these predictions themselves may conflate model confidence with actual selection probability), these maps would likely provide a good indication of player selections.  Thus they might be used to sample or reweight the test set, so that unlikely destination points are sampled more, to try to get a more uniform sample.\n\nEven with this issue, though, I feel that this is an interesting application and system that seems reasonable in its current state, if with important caveats.  Thus, I'd lean towards accepting.  However, I'd encourage the authors to discuss these differences and issues of output interpretation, and to try addressing if possible.\n\n\nAdditional questions and comments:\n\n* The train/val/test split appears to be uniform random by pass event.  It would be interesting to hold out all events for one or more teams, or holding out full games, to measure generalizability to these cases.\n\n* Are the destinations for unsuccessful passes the intercepted location, or an estimated intended location?  Does this difference affect the completion prediction at the intended location (beyond the interception location)?\n\n\n", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}, {"review_id": "HJeJc8nvtr", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper presents a deep convolutional neural network for estimating, at the given the moment, for any given position on the field, the probability that a pass to that position is successful. It also shows results regarding the probability that a pass to a particular position is attempted. The paper is well-written and the figures and videos do a very nice job of communicating the important ideas. The related work appears to be extensive and the description of the design choices of the architecture and the training procedure is clear and thorough. Based on the results in Table 1, the proposed network appears to compare very favorably against the baselines.\n\nOne thing I feel would make the paper stronger is the inclusion of more baselines and an ablation study. Both Logistic Net and Dense2 Net appear to be very lightweight. It is not obvious to me that all of the design choices made in the paper translated to gains in performance. Would a large network perform well without them? You mention that you tried a class-weighting with no sampling approach but that it did perform as well. Why not include these results in the paper?\n\nSmall typo: “where achieved by augmentation” -> “were achieved by augmentation”\n\nFrom the perspective of someone who is not an expert in this area, I think this is a nice paper. It appears to successfully address an interesting problem, it is well-organized, and its solution methodology may have applications to other related problems. My one major criticism is that I feel that there is insufficient information regarding how the design choices affected the performance.", "extra_scores": {"experience_assessment": "I do not know much about this area.", "review_assessment:_checking_correctness_of_derivations_and_theory": "N/A", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly."}, "extra_text": {}}, {"review_id": "B1eBEXSAtS", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Contribution:\nThis paper proposes PassNet an architecture designed for soccer pass analytics. PassNet approach is similar to UNet, having a downsampling and upsampling modules with a set of skip-connection between the two modules. To train their model, authors apply the log-loss at the location of the passing event.  Authors evaluate their approach on a soccer analytic dataset, where they demonstrate improvement over prior works relying on hand-crafted features.\n\nComment:\nContribution of the paper appears a bit incremental to me.  It seems that the paper is a direct application of Unet type of architecture on the specific problem of pass analytic. It would be nice to explicit what is the main contribution of the paper from the representation learning point of view.\n\nIn addition, some of the design choice of PassNet could be better justified. It would be nice to run an ablation study to show the importance of the different architectural component (conv2d prediction layers, backpropagating using only one outputs, skip-connection...)\n\nAuthors claim that their approach is an \"extreme case of weakly supervised learning\". I tend to disagree with the assessment. I understand that they only backprop the loss computed at one specific location of the output, but this a choice on the architectural part and not on the labelling. Training PassNet requires fine-grained label as it needs to know both label value and localization. Training the model without the use of label localization would be more akin to weakly-supervised learning.\n\nAuthors evaluate their approach on only one dataset.  It would be nice to extend the empirical results to other datasets/sports to ensure the robustness of the conclusions.  \n", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly.", "review_assessment:_checking_correctness_of_derivations_and_theory": "N/A"}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper proposes PassNet, which is an architecture that produces a 2D map of probability of successful completion of a soccer pass. The architecture has some similarities with UNet and has downsampling and upsampling modules with a set of skip-connections between them.

The reviewers raised several issues:
* Novelty compared to UNET
* Lack of ablation studies
* Uncertainty about what probabilities mean and issues regarding output interpretation.

The authors have tried to address these concerns in their rebuttal and provided additional experiments. They also argue that the application area (sport analytics) of the paper is novel. Even though the application area is interesting and might lead to new problems, this paper did not get enough support from reviewers to justify its acceptance.
Decision: Reject