Paper:
We propose a fully convolutional network architecture that is able to estimate a full surface of pass probabilities from single-location labels derived from high frequency spatio-temporal data of professional soccer matches. The network is able to perform remarkably well from low-level inputs by learning a feature hierarchy that produces predictions at different sampling levels that are merged together to preserve both coarse and fine detail. Our approach presents an extreme case of weakly supervised learning where there is just a single pixel correspondence between ground-truth outcomes and the predicted probability map. By providing not just an accurate evaluation of observed events but also a visual interpretation of the results of other potential actions, our approach opens the door for spatiotemporal decision-making analysis, an as-yet little-explored area in sports. Our proposed deep learning architecture can be easily adapted to solve many other related problems in sports analytics; we demonstrate this by extending the network to learn to estimate pass-selection likelihood.

Review:
The aim of the system presented in this paper is to produce a 2D map of probabilities showing the chance of successful completion of a soccer pass to all locations on the field, given coordinate locations of players and the ball sampled over time.  A network based on a fully-convolutional style semantic segmentation network is applied to a 2D, 8-channel game state representation, with final sigmoid layer to predict a pass success indicator at each location.  The system is trained using manually annotated pass success and destinations, which corresponds to a label on the destination point; locations other than the labeled point are not trained (treated as incomplete/"don't-care" training targets).  Evaluation is performed using both log loss and probability calibration measure, to measure effectiveness of predictions as well as how well they calibrate to correspond to probabilities in the sense defined by the measurement.
This is a fun application and it appears the system is effective at a basic level.  However, I think both the theory/explanation and experiments leave a fair bit of uncertainty as to what the probabilities mean and how to interpret them, including conflation of success probability and the model's certainty in its estimate.
In particular, it is unknown to what degree the values output by the model can be interpreted as a predictive probability of pass completion for anywhere on the field.  If one location says 0.8 and the other 0.5, does this mean that if the player were to actually pass to the 0.5 location, the chance of success is actually lower?  Or does it mean that the model is less confident or that this location was under-trained for this state?  The ECE measurement in conjunction with loss error doesn't quite address this:  although a good verification of calibration in its own sense, ECE simply confirms that for cases where the model predicts 0.5, there is success 0.5 of the time for locations that *exist in the test set*, which is sampled according to player action.  To verify the probability maps at all locations, one would need to be able to measure *any* point in the field, not just those already selected by the players' actions.  I think discussion and attempt to measure this is fairly important, as one of the intended applications in the motivation is analyzing what might have happened had a player selected a different destination point.
Unfortunately, knowing the true outcomes at arbitrary locations the players didn't pass to is impossible, so addressing this issue is not straightforward, and unclear to me how it might be done.  A possible suggestion, is to use the destination selection predictions that the paper also mentions can be found using softmax instead of sigmoid.  Although this does not entirely eliminate the issue (these predictions themselves may conflate model confidence with actual selection probability), these maps would likely provide a good indication of player selections.  Thus they might be used to sample or reweight the test set, so that unlikely destination points are sampled more, to try to get a more uniform sample.
Even with this issue, though, I feel that this is an interesting application and system that seems reasonable in its current state, if with important caveats.  Thus, I'd lean towards accepting.  However, I'd encourage the authors to discuss these differences and issues of output interpretation, and to try addressing if possible.
Additional questions and comments:
* The train/val/test split appears to be uniform random by pass event.  It would be interesting to hold out all events for one or more teams, or holding out full games, to measure generalizability to these cases.
* Are the destinations for unsuccessful passes the intercepted location, or an estimated intended location?  Does this difference affect the completion prediction at the intended location (beyond the interception location)?