Paper:
Joint Active Feature Acquisition and Classification with Variable-Size Set Encoding
We consider the problem of active feature acquisition where the goal is to sequentially select the subset of features in order to achieve the maximum prediction performance in the most cost-effective way at test time. In this work, we formulate this active feature acquisition as a joint learning problem of training both the classifier (environment) and the reinforcement learning (RL) agent that decides either to 'stop and predict' or 'collect a new feature' at test time, in a cost-sensitive manner. We also introduce a novel encoding scheme to represent acquired subsets of features by proposing an order-invariant set encoding at the feature level, which also significantly reduces the search space for our agent. We evaluate our model on a carefully designed synthetic dataset for the active feature acquisition as well as several medical datasets. Our framework shows meaningful feature acquisition process for diagnosis that complies with human knowledge, and outperforms all baselines in terms of prediction performance as well as feature acquisition cost.

Review:
The authors consider the problem of active feature acquisition (i.e., the setting where each feature includes a cost to see the value, often motivated by diagnostic tests in medical settings), where the goal is to maximize classifier performance with a minimum cost subset of features. Methodologically, they start by modifying the objective function to include empirical risk minimization and feature acquisition cost minimization. The acquisition mechanism is done via a RL formulation in a method similar to RL/imitation learning methods for feature selection (e.g., [15-18]), where the state is the observable features, the action is to acquire a new feature (or classify), and the reward is the (negative) feature acquisition cost. The interesting aspect of their approach is that the classifier and RL agent are jointly optimized (RL via deep Q-learning), requiring a clever set encoding method to manage the large search space (similar to [Vinyals, Bengio & Kudlur; ICLR 2016]). The joint optimization is completed via a joint encoding of the classification and policy with shared encoder layers. Experiments are first conducted on synthetic data sets to elucidate specific properties (e.g., the value of joint training vs. a pre-trained classifier, the benefits of set encoding). Secondly, experiments are conducted on PhysioNet (four classification tasks) and a private Medical check-up dataset. Evaluating using AUC, they show that they are able to generate competitive results while observing ~20% of the features.  In summary, this is an interesting approach to feature value acquisition that builds on recent work wrt meta-learning with RL-based approaches and adopting this work to this particular setting with the joint learning and set encoding methods that are entirely sensible.  Additionally, they provide sufficient training details where it seems likely that others would be able to produce this work. Experiments are conducted on synthetic sets, ‘real’ (bake-off) datasets, and an actual real-world problem. The only technical detail I find a bit surprising is that they are able to train these algorithms with 1000s of examples, when policy gradient is generally fairly data hungry, etc. Thus it would also be nice to see more ‘traditional’ baseline results (i.e., no deep learning, just an expected gain test) — particularly for the real-world problems. While I am not a true expert on this particular problem, this is ostensibly a novel approach to a potentially important problem with reasonably convincing results. However, the writing needs significant work (and I am not entirely certain it followed the latest style files) including structurally, grammatically, and even some of the precision of claims (covered in more detail below). Accordingly, I am ambivalent wrt accepting this work in its current form, learning toward rejection given the likely number of strong papers submitted.  [Quality]: Overall, the paper is well-motivated, specifies a sensible model, provides a reasonable level of implementation details, and generates promising experiments. While the writing needs work, I will leave this to ‘clarity’. I do have a few questions that are a bit nit-picky: (1) the authors state a ‘cost-sensitively (sic) regularized optimization problem’; however, it isn’t clear to me that this is a regularizer per se as I generally think of regularization as a variance reduction mechanism — not all penalty functions within the objective will do this (and I’m not certain this does), (2) the authors make a point that ‘To search this huge space efficiently and stably, we carefully devise the architecture with the recently proposed set encoding method’ — as best as I can tell, this is exactly the same method, correct?, (3) Lines 131-132: “the hyperparameter balancing the relative importances of two terms…is absorbed in the predefined cost” — isn’t this difficult to do while keeping the costs interpretable (or is this just notational)?.  Overall, precisely clarifying these differences wrt existing work would strengthen the paper.  [Clarity]: The paper needs significant writing work. I will give examples of each issue (in my opinion), but it is an overall issue. - Structure: The introduction starts with a discussion around deep learning and proceeds to feature acquisition — it isn’t entirely clear why deep learning is a significant aspect of this paper. I would think one would start with the feature acquisition aspect and then proceed to deep learning (although I think it really is secondary). The related work section is basically a ‘laundry list’ of papers. Without reading them, they are mostly just citations — while this is an opportunity to provide a general framework and demonstrate how this fits together. - Grammar: The paper is full of simple errors like missing determiners, syntactic errors, odd word choice, tc.  (i.e., Line 37: “Given a new dataset”; Line 36: “to be sufficient confident”; Line 47: “It is worth noting that we do not assume a fixed classifier that is pre-trained during a separate phase.; Line 57: “We formulate the feature acquisition problem as a cost-sensitive penalty in the objective function and optimize with a joint…”; Line: 69: “…we cannot guarantee that all of the features are available at the start”; Line 98: “Also, our”).  These are all over the place. Finally, I think there should be more discussion around the empirical results in general.  [Originality]: While I am not a true expert in this area, while it seems to draw significantly from related work, it also put together multiple relevant works and ostensibly thought through the details — showing promising results. While I think many could have come up with this formulation, I also don’t believe it trivial and is an advance over more directly formulations.  [Significance]: While I would like to see additional experiments contrasting with non-deep learning methods, this work does seems like a promising approach to active feature value acquisition. This is a practical problem (even if somewhat narrow in scope) — so I would say that it meets the bar in this regard.  Overall, this paper has good ideas, solid execution, and promising results. However, given the competitiveness of NIPS, I think the paper need significant writing work to be competitive in its current form.  Comments after author rebuttal: I do appreciate Table A and in reading the other reviewers comments, I may have over-indexed on the writing (which does need work, but it is difficult for me to validate) and under-appreciated the effort to get this validated in a practical setting. Thus, I am a bit more positive wrt this work, but still believe it should be somewhat borderline and wish we had more confident reviewers.