Paper:
Reliable Decision Support using Counterfactual Models
Decision-makers are faced with the challenge of estimating what is likely to happen when they take an action. For instance, if I choose not to treat this patient, are they likely to die? Practitioners commonly use supervised learning algorithms to fit predictive models that help decision-makers reason about likely future outcomes, but we show that this approach is unreliable, and sometimes even dangerous. The key issue is that supervised learning algorithms are highly sensitive to the policy used to choose actions in the training data, which causes the model to capture relationships that do not generalize. We propose using a different learning objective that predicts counterfactuals instead of predicting outcomes under an existing action policy as in supervised learning. To support decision-making in temporal settings, we introduce the Counterfactual Gaussian Process (CGP) to predict the counterfactual future progression of continuous-time trajectories under sequences of future actions. We demonstrate the benefits of the CGP on two important decision-support tasks: risk prediction and "what if?" reasoning for individualized treatment planning.

Review:
Thanks to the authors for a very interesting paper. The main contribution that I see is the proposal of a counterfactual GP model, which can take into account discrete actions, events, and outcomes, and as the authors argue allows for a much better model of discrete event data, e.g. electronic medical records, by explicitly accounting for future actions. 
A minor point: the citation for Neyman shouldn't be  to 1990, as the paper is from 1923. Yes, the accessible translated version is from 1990, but the citation should include the fact that it's of an article from 1923, seeing as Neyman died in 1981. ;) 
I have some suggestions for strengthening the presentation and outlined some of what confused me. Hopefully addressing this will improve the paper.
The framework adopted, of counterfactual predictions, is a very reasonable way to think about causal inference. I believe that the two main assumptions are standard in some of the literature, but, following Pearl, I do not think that they are the best way to think about what is going on. Overall, I would very much like to see a graphical model in Section 2. 
Regarding Assumption 1, I would like an account following or at least engaging with Pearl's critique of VanderWeele's claim about the consistency rule as an assumption rather than a theorem. 
Regarding Assumption 2, you cite Pearl so I am sure you are aware of his viewpoint, but I think it is very important to make clear what the no unmeasured confounders assumption is, and I believe that a graphical model is the best way to do this. It is worth pointing out as well that, following Pearl, we can go beyond the (often unrealistic) assumption of no unmeasured confounders (see e.g. the back-door and front-door criteria) as long as we're willing to state clearly what our structural assumptions are. Far from a criticism of how you've set things up, I believe that this viewpoint will mean you can cover more cases, as long as there is a way to setup the appropriate regression using a GP (which there should be!)
Having I think understood Assumptions 1 and 2, it now seems like the claim is that we can estimate P(Y[a]) by marginalizing P(Y | A = a, X = x) with respect to X. But I'm having trouble connecting this with the claim that the CGP will do better than the RGP. The end of the "Model." section has the key insight, which I think you ought to expand a bit on and move earlier. It'd be helpful, I think, to write down the likelihood of the RGP so that it can be compared to Eq (3). It would also help to refer to Figure 1 (which is indeed best viewed in color, but could be made quite a bit more legible in grayscale!)---what would RGP do for Figure 1, for example? 
Now having read the experiments I'm confused again. In the "Experimental" scenario it is the case that after 12 hours there is a structural break, and all patients are assigned to a control condition, since no actions (treatments) are taken. But why are the RGP and CGP equivalent here?
The Observational scenario makes sense, and the fact that the CGP performs well is reassuring, but why does this support your claim that the CGP is able to "learn a counterfactual predictive model"? Shouldn't you ask the CGP to make predictions under different treatment regimens---to take the example from Figure 1, shouldn't you ask the CGP to predict what will happen if Drug A is administered at hour 13, if Drug B is administered at hour 13, or if no drug is administered? It seems to me that the CGP is a better predictive model than the RGP, and thus does a better job predicting. I think it's true that this means that under your assumptions it will also do a better job on counterfactuals, but you should evaluate this.
A question: how did you fit your models? I guess you maximized Equation 3, but there's some mixture modeling stuff going on here, and the model is highly non-convex. Were there issues with optimization? Did you need to do random restarts? How did you pick initial values? Where did the confidence intervals on parameters come from?