launch/ampere huggingface dataset:
The authors propose to speed up RL techniques, such as DQN, by utilizing expert demonstrations.  The  expert demonstrations are sequences of consecutive states that do not include actions, which is closer to a real setting of imitation learning.  The goal of this process is to extract a function that maps any given state to a subgoal.  Subgoals are then used to learn different Q-value functions, one per subgoal.  To learn the function that maps states into subgoals, the authors propose a surrogate reward model that corresponds to the angle between: the difference between two consecutive states (which captures velocity or direction) and a given subgoal.  A von Mises- Fisher distribution policy is then assumed to be used by the expert to generate actions that guide the agent toward the subgoal.  Finally, the mapping function state->subgoal is learned by performing a gradient descent on the expected total cost (based on the surrogate reward function, which also has free parameters that need to be learned). Finally, the authors use the DQN platform to learn a Q-value function using the learned  surrogate reward function that guides the agent to specific subgoals, depending on the situation. The paper is overall well-written,  and the proposed idea seems interesting.  However, there are rather little explanations provided to argue for the different modeling choices made, and the intuition behind them.  From my understanding, the idea of subgoal learning boils down to a non-parametric (or kernel) regression where each state is mapped to a subgoal based on its closeness to different states in the expert's demonstration.  It is not clear how this method would generalize to new situations.  There is also the issue of keeping tracking of a large number of demonstration states in memory.  This technique reminds me of some common methods in learning from demonstrations, such as those using GPs or GMMs,  but the novelty of this technique is the fact that the subgoal mapping function is learned in an IRL fashion, by tacking into account the sum of surrogate rewards in the expert's demonstration.  The architecture of the action value estimator does not seem novel,  it's basically just an extension of DQN with an extra parameter (subgoal g). The empirical evaluation seems rather mixed.  Figure 3 shows that the proposed method learns faster than DQN,  but Table I shows that the improvement is not statistically significant, except in two games, DefendCenter and PredictPosition.  Are these the results after all agents had converged?  Overall, this is a good paper,  but focusing on only a single game (Doom) is a weakness that needs to be addressed  because one cannot tell if the choices were tailored to make the method work well for this game.  Since the paper does not provide significant theoretical or algorithmic contribution,  at least more realistic and diverse experiments should be performed.
The authors propose a loss that is based on a RBF loss for metric learning and incorporates additional per exemplar weights in the index for classification.  Significant improvements over softmax are shown on several datasets. IMHO, this could be a worthwhile paper,  but the framing of the paper into existing literature is lacking  and thus it appears as if the authors are re-inventing the wheel (NCA loss) under a different name (RBF solver). The specific problems are: - The authors completely miss the connection to NCA loss (https://papers.nips.cc/paper/2566-neighbourhood-components-analysis.pdf)  and thus appear to be re-inventing the wheel.   - The proposed metric learning scenario is exactly as proposed in the NCA loss works,  while the classification approach adds an interesting twist by learning per exemplar weights.  I haven't encountered this before and it could make an interesting proposal.  Of course the benefit of this should be evaluated in ablation studies ( Tab 3 shows one experiment with marginal improvements). - The authors' use of 'solver' seems uncommon and confusing.  What is proposed is a loss in addition to building a weighted index in the case of classification. - In the metric learning comparison with softmax (end of page 9) the authors mentions that a Gaussian standard deviation for softmax is learned.  It appears as if the authors use the softmax logits as embedding  whereas the more common approach is to use the bottleneck layer.  This is also indicated by the discussion at the end of page 10 where the authors mention that softmax is restricted to axis aligned embeddings.  All softmax metric learning experiments should be carried out on appropriately sized bottleneck layers. - Some of the motivations of what the various methods learn seem flawed,  e.g. triplet loss CAN learn multiple modes per class and there is nothing in the Softmax loss that encourages the classes to fill a large region of the space. - Why don't the authors compare on ImageNet? Some positive points: - The authors mention in Sec 3.3 that updating the RBF centres is not required.  This is a crucial point that should be made a centerpiece of this work,  as there are many metric learning works that struggle with this.  Additional experiments that can investigate this point would greatly contribute to a well rounded paper. - The numbers reported in Tab 1 show very significant improvements If the paper was re-framed and builds on top of the already existing NCA loss, there could be valuable contributions in this paper.  The experimental comparisons are lacking in some respect,  as the comparison with Softmax as a metric learning method seems uncommon, i.e. using the logits instead of the bottleneck layer.  I encourage the authors to extend the paper and flesh out some of the experiments and then submit it again.
Although GAN recently has attracted so many attentions,  the theory of GAN is very poor.  This paper tried to make a new insight of GAN from theories  and I think their approach is a good first step to build theories for GAN.  However, I believe this paper is not enough to be accepted.  The main reason is that the main theorem (Theorem 4.1) is too restrictive. 1.	There is no theoretical result for failed conditions.  2.	To obtain the theorem, they assume the optimal discriminator.  However, most of failed scenarios come from the discriminator dynamics as in Figure 2.  3.	The authors could make more interesting results using the current ingredients.  For instance, I would like to check the conditions on eta and T to guarantee d_TV(G_mu*, G_hat{mu})<= delta_1 when |mu*_1 – mu*_2| >= delta_2 and |hat{mu}_1 – hat{mu}_2| >= delta_3.  In Theorem 4.1, the authors use the same delta for delta_1, delta_2, delta_3.  So, it is not clear which initial condition or target performance makes the eta and T.
[Summary] The paper is overall well written  and the literature review fairly up to date. The main issue is the lack of novelty. The proposed method is just a straightforward dimensionality reduction based on convolutional and max pooling layers. Using CNNs to handle variable length time series is hardly novel. In addition, as always with metric learning, why learning the metric if you can just learn the classifier? If the metric is not used in some compelling application, I am not convinced. [Detailed comments and suggestions] * Since "assumptions" is the only subsection in Section 2,  I would use \texbf{Assumptions.} rather than \subsection{Assumptions}. * Same remark for Section 4.1 "Complexity analysis". * Some missing relevant citations: Learning the Metric for Aligning Temporal Sequences. Damien Garreau, Rémi Lajugie, Sylvain Arlot, Francis Bach. In Proc. of NIPS 2014. Deep Convolutional Neural Networks On Multichannel Time Series For Human Activity Recognition. Jian Bo Yang, Minh Nhut Nguyen, Phyo Phyo San, Xiao Li Li, Shonali Krishnaswamy. In Proc.  of IJCAI 2015. Time Series Classification Using Multi-Channels Deep Convolutional Neural Networks Yi ZhengQi LiuEnhong ChenYong GeJ. Leon Zhao. In Proc. of International Conference on Web-Age Information Management. Soft-DTW: a Differentiable Loss Function for Time-Series. Marco Cuturi, Mathieu Blondel. In Proc. of ICML 2017.
The authors propose a scheme to generate questions based on some answer sentences, topics and question types.  Topics are extracted from questions using similar words in question-answer pairs.  It is similar to what we find in some Q&A systems (like lexical answer types in Watson).  A sequence classifier is also used to tag the presence of topic words.  Question types correspond mostly to salient questions words.  LSTMs are used to encode the various inputs and generate the questions.  The paper is well written and easy to follow.  I would expect more explanations why sentence classification and labeling results presented in Table 2 are so low.  Experimental results on question generation are convincing and clearly indicate that the approach is effective to generate relevant and well-structured short questions.  The main weakness of the paper is the selected set of question types that seems to be a fuzzy combination of answer types and question types (for ex. yes/no).  Some questions type can be highly ambiguous;  for instance “What” might lead to a definition, a quantity, some named entities...  Hence I suggest you revise your qt set.  I would also suggest, for your next experiments, that you try to generate questions leading to answers with list of values.
This paper offers a very promising approach to the processing of the type of sequences we find in dialogues, somewhat in between RNNs which have problem modeling memory, and memory networks whose explicit modeling of the memory is too rigid. To achieve that, the starting point seems to be a strength GRU that has the ability to dynamically add memory banks to the original dialogue and question sentence representations, thanks to the use of imperative DNN programming.  The use of the reparametrization trick to enable global differentiability is reminiscent of an ICLR'17 paper "Learning graphical state transitions".  Compared to the latter, the current paper seems to offer a more tractable architecture and optimization problem that does not require strong supervision and should be much faster to train. Unfortunately, this is the best understanding I got from this paper,  as it seems to be in such a preliminary stage that the exact operations of the SGRU are not parsable.  Maybe the authors have been taken off guard by the new review process where one can no longer improve the manuscript during this 2017 review  (something that had enabled a few paper to pass the 2016 review). After a nice introduction,  everything seems to fall apart in section 4, as if the authors did not have time to finish their write-up.  - N is both the number of sentences and number of word per sentence,  which does not make sense. - i iterates over both the sentences and the words.  The critical SGRU algorithm is impossible to parse - The hidden vector sigma, which is usually noted h in the GRU notation, is not even defined - The critical reset gate operation in Eq.(6) is not even explained, and modified in a way I do not understand compared to standard GRU. - What is t?  From algorithm 1 in Appendix A, it seems to correspond to looping over both sentences and words. - The most novel and critical operation of this SGRU, to process the entities of the memory bank, is not even explained.  All we get at the end of section 4.2 is " After these steps are finished, all entities are passed through the strength modified GRU (4.1) to recompute question relevance." The algorithm in Appendix A does  not help much.   With PyTorch being so readable, I wish some source code had been made available. Experiments reporting also contains unacceptable omissions and errors: - The definition of 'failed task', essential for understanding, is not stated (more than 5% error) - Reported numbers of failed tasks are erroneous:  it should be 1 for DMN+ and 3 for MemN2N. Page 3: dynanet -> dynet
Summary The authors argue that ensemble prediction takes too much computation time and resource, especially in the case of deep neural networks.  They then address the problem by proposing an adaptive prediction approach.  The approach is based on the observation that it is most important for ensemble approaches to focus on the "uncertain" examples.  The proposed approach thus conducts early-stopping prediction when the confidence (certainty) of the prediction is high enough, where the confidence is based on the confidence intervals of (multi-class) labels based on the student-t distribution.  Experiments on vision datasets demonstrate that the proposed approach is effective in reducing computation resources while maintaining sufficient accuracy. Comments * The experiments are limited in the scope of (image) multi-class classification.  It is not clear whether the proposed approach is effective for other classification tasks, or even more sophisticated tasks like multi-label classification or sequence tagging. * The idea appears elegant but rather straightforward.  One important baseline that is easy but not discussed is to set a static threshold on pairwise comparison (p_max - p_secondmax).  Would this baseline be competitive with the proposed approach?  Such a comparison is able to demonstrate the benefits of using confidence interval. * The overall improvement in computation time seems to be within a constant scale,  which can be easily achieved by doing ensemble prediction in parallel  (note that the proposed approach would require predicting sequentially).  So are there real applications that can benefit from the improvement? * typo: p4, line19, neural "netowkrs" -> neural "networks"