Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: This paper proposed a method for learning the optimal individualized treatment rule (ITR). It focuses on a situation of having extensive treatment options but limited observations for only a small number of specific treatments. This work designs an algorithm to merge similar treatments and provide optimal ITR in a reduced treatment space. The model is trained based on a designed convex optimization problem with an adaptive proximal gradient algorithm. The empirical study reveals its ability to detect the structure of the treatment space and merge similar ones to enable the optimal ITR from many treatment options. A theoretical analysis is also provided.
strengths_and_weaknesses: The algorithm focuses on a more practical situation of optimal ITR in precision medicine. The author models the problem as a convex minimization problem and solves it with an accelerated gradient method. This algorithm can provide a hierarchical structure of the relationship between treatments. By merging similar treatments, users can give optimal ITR on a reduced treatment space which is more feasible in practice.

I have the following concerns:
1. It looks like the final result automatically generates the hierarchical structure of the treatments. However, I didn't understand how such a hierarchical structure could happen by just solving the convex minimization problem. Your algorithm seems can only provide the proximity/dissimilarity matrix on treatments. Did you later apply any linkage-based hierarchical clustering algorithms on the proximity/dissimilarity matrix to generate the structure? Please specify.
2. Is there an analysis of your algorithm's time/space complexity?
3. Did you compare with other methods that can also provide a dendrogram? 
4. In lines 170-171, should it be if \hat{\xi}_k = 0, then Z_k is classified as V? 


questions: Referred to Strengths and Weaknesses
limitations: Will such a treatment algorithm introduce any bias to the patients?
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The authors consider the problem of finding optimal individualised treatment
rules – an important problem in precision medicine – that assigns treatments based
on covariates which may change over time. The particular setting of interest
is one where many treatments exist and where treatments have a structure induced
by shared mechanisms of action (e.g., different drugs that target the same pathway).

The proposed approach uses a fusion penalty term that encourages clustering
between treatments. This incorporates the structure assumption in a tunable way,
and allows sharing of information between infrequently observed treatments and
more frequently observed ones that are related.

strengths_and_weaknesses: - The literature is well covered in their citations, with the exception of
  confounding. There are works considering ITRs in the context of confounding
  through the use of instrumental variables. This is not strictly a problem
  since the authors aren't considering confounding.

- The work builds on existing with the novelty being the fusion term.


- The formulation is straightforward and easy to understand, as is the writing
  and presentation.

- There are two parts to the method, first there's the group lasso based
  classification into homogeneous and heterogenous variables, and second is the
  optimisation of the main objective function with the fusion penalty term. Most
  of the work concentrates on the second part: the theoretical properties assume
  the hetrogeneous variables are given and the experiments do not specifically
  investigate the identification of homo/hetrogenous variables. There is a short
  note on lines 150–152 stating it has been empirically observed to be more
  effective, but these are not shown anywhere. Furthermore, it's unclear if the
  group lasso classification was used in experiments or not, and if it was how
  the hyperparameter was chosen.

- The theoretical results are relevant and interesting but are limited to the
  fusion model only.

- The PDX study is not sufficiently described, it is unclear exactly which
  dataset is used as no reference is provided. There is a citation to another
  methods paper, cited as the authors have used their preprocessing steps, but
  no citation to the main data reference.

questions: - The PDX study employes a number of preprocessing steps, some of which are
  supervised. How was bias avoided with the supervised preprocessing? Was
  separate data used?

- The treatments employed are surely employing drugs with known targets. Do the
  targets/targeted pathways align with the inferred grouping structure?

limitations: - Given the strong patient focus in the intro, the confounding limitation should
  be discussed. The PDX study used in experiments is free of confounding, but
  typical observational patient data is not.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The work proposes a machine learning approach for precision medicine applications, where one needs to choose from a set of suitable treatments the best one for a given patient. Further, the proposed approach allows clustering similarly behaving treatments together, and the authors give theoretical guarantees that the underlying clustering will be recovered.

The proposed approach combines together a number of different approaches including:
- Basic regression model, where the outcome is modelled as a sum of a main effect M(z), and interaction effect T(z,a) 
- After solving a standard regression problem to recover M(z), group Lasso is used to find the subset of features that may contribute to the interaction effect
- Proposed adaptive proximal gradient algorithm is used to predict the remaining residual with T(z,a), and a pairwise regularizer is used to divide the treatments into clusters

Experiments on simulated data show that the approach works on finding the true underlying structure on simple simulated problems, and evaluation on a cancer treatment data set shows both competitive performance against basic ML regression approaches, as well as the treatment structure recovered by the method.


strengths_and_weaknesses: The paper addresses an important problem in precision medicine, and provides ideas that appear to be novel about how to, in addition to predicting, simultaneously cluster together similar treatments. The writing is fairly clear, the work and contributions well motivated, and I found the mathematical presentation to be rigorous yet accessible.

The proposed learning approach seems reasonable, though to me some of the choices made to make divide this into a sequence of solvable (convex) optimization problems feel a bit like not so well justified "hacks" (solving for M(z) and T(z,a) separately, the assumption that Z decomposes nicely to X and V that can be recovered with group Lasso...). Theoretical analysis seems convincing, but I cannot verify that all the details are correct.

The experimental analysis on the PDX study is interesting, though from results on a single data set it is difficult to ascertain yet how well the method can be in general expected to perform. The one thing that was left a bit unclear to me was does the proposed approach perform more accurate predictions, than the simple approach of doing non-personalized predictions (i.e. always predicting the one treatment that works best on training data, on the benchmark data most likely usually the BYL+BIN treatment). Based on zooming on Figures 3 and 4 this might be the case, but this could be more clearly discussed in the text. 

Minor comments:

Equation (1): I was a bit surprised there was not in addition to M(z) and T(z,a) a U(a) type of term that would depend only on the treatment, since in addition to personalized effects one would except some of the treatments to be overall superior to others. E.g. the BYL+BIN treatment in the PDX study. Though I suppose such effect can be encoded into T(z,a) with constant covariate for z, and identity encoding covariate for a...

"In practice, one may assume that only certain elements of Z...[divide into X and V]" - this is not intuitively obvious to me, as in many cases one could imagine same covariates contributing both to M and Z? Any plausible real-world examples of this?

I did wonder if some of the reinforcement learning type of terminology used in the beginning (rewards, value functions) were necessary, since in the end the considered setup is regression where training data has been gathered in advance, and gathering feedback and considering exploration/exploitation types of tradeoffs is not considered. The presentation could be simpler by leaving these connections out, but perhaps there is also value in making these connections.

questions: - How does the proposed method compare on real data to the baseline that always chooses the treatment that worked best on average on training data, and predicts the mean effect for that treatment? I find it a bit difficult to interpret from the metrics provided, how much is actually learned from the data compared to just predicting the mean.
- What does the "response scaled" -metric mean, what units, scaled how? Maybe something to add to the supplementary?
- Regarding comparison to the baseline methods, it would be good to tell in the supplementary materials the used parameter grids, and also what kind of feature representations these methods were supplied with. For example, if the goal is to learn both M(z) and T(z,a), in what form was the identity of the treatment a supplied to the baseline methods? The code is provided, but it would be good to be able to tell these details already from reading paper + supplementaries, in order to understand what the comparison means.
limitations: Limitations and negative societal impact not considered. The one possible limitation that comes to my mind, is that the method should be much more comprehensively validated on more realistic data sets and settings, before it could be ever considered for making treatment decisions for real-world patients.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This submission deals with treatment selection among a large collection of candidate ones for a specific individual or a group of individuals which share common response to the treatment. The authors propose to adopt a treatment clustering strategy by gathering treatments with similar effects. 

The proposed approach follows the line of thought of some prior arts in which a statistical regression model is defined for the reward viewed as a function comprising a general and common effect incurred from all treatments plus some treatment specific effects and an additive noise. The common and treatment specific parts of the reward are functions of the patient whose information is embedded in a fixed-size vector. The goal of the authors is to learn the regression coefficients based on supervision coming from data in the form triplets (patient, treatment, reward). They leverage a group promoting penalty to encourage treatments with similar effects to be clustered together. 
To rule out some of the covariates inside the vectors of patient features, the authors first use a group lasso (on the same reward regression) as a preliminary step. Finally, the main regression problem has properties allowing it to be solved by an accelerated proximal gradient algorithm. 

The paper is technically sound and quite well written. A theorem proves the consistency of the estimator under technical (but not unusual) assumptions.
strengths_and_weaknesses: 
Pros : 
- the paper incorporates a novel aspect (treatment grouping) in a standard model used for personalized treatment selection
- the proposed method has a theoretical group-consistency property
- the efficiency of the approach is experimentally validated on synthetic and real data

Cons : 
- the model is designed for continuous outputs whereas discrete ones may be observed for several treatments/pathologies
- the model backbone is a linear regression and may underfit the data in various situations
questions: Major : 

Have the authors considered investigating more flexible models by using for instance a kernelized version of linear regression ? 

I think that the interpretation of random variable $A$ should be clarified at the beginning of section 2. Indeed, should it be understood as the identity of the best treatment, or is it the result of some sampling distribution (e.g. uniform) from which the data is obtained ? Because it is difficult to imagine obtaining data from the former, I suspect the correct interpretation is the latter. The action sampling process in the first experiment of section 5 should be documented. 

How is the term $M_0(.)$ estimated in the real data experiment in section 5 ?

Minor : 

For better readability, I suggest not to use bold letters for scalar variables such as $\zeta_{a,k}$.
limitations: Not adapted to discrete rewards and limited learning capacity.
review_title: null

Meta-review: metareview: This paper proposes a method for learning the optimal individualized treatment rule (ITR). The proposed approach uses a fusion penalty term that encourages clustering between treatments. A dendrogram of the treatments is generated by running the proposed algorithm using different tuning parameters as a solution path. The effectiveness of the proposed approach is empirically validated on synthetic and real data. The paper is well written and technically sound. A thorough analysis/interpretation of the resulting model/results will further improve the paper.  
confidence: Less certain
Decision: Accept