launch/ampere huggingface dataset:
The submission tackles an important problem of learning and transferring multiple motor skills.  The approach relies on using an embedding space defined by latent variables and entropy-regularized policy gradient / variational inference formulation that encourages diversity and identifiability in latent space. The exposition is clear and the method is well-motivated.  I see no issues with the mathematical correctness of the claims made in the paper.  The experimental results are both instructive of how the algorithm operates (in the particle example), and contain impressive robotic results.  I appreciated the experiments that investigated cases where true number of tasks and the parameter T differ, showing that the approach is robust to choice of T. The submission focuses particularly on discrete tasks and learning to sequence discrete tasks (as training requires a one-hot task ID input).  I would like a bit of discussion on whether parameterized skills (that have continuous space of target location, or environment parameters, for example) can be supported in the current formulation, and what would be necessary if not. Overall, I believe this is in interesting piece of work at a fruitful intersection of reinforcement learning and variational inference,  and I believe would be of interest to ICLR community.
This paper studies the critical points of shallow and deep linear networks. The authors give a (necessary and sufficient) characterization of the form of critical points and use this to derive necessary and sufficient conditions for which critical points are global optima. Essentially this paper revisits a classic paper by Baldi and Hornik (1989) and relaxes a few requires assumptions on the matrices. I have not checked the proofs in detail but the general strategy seems sound. While the exposition of the paper can be improved in my view this is a neat and concise result and merits publication in ICLR. The authors also study the analytic form of critical points of a single-hidden layer ReLU network. However, given the form of the necessary and sufficient conditions the usefulness of of these results is less clear. Detailed comments:- I think in the title/abstract/intro the use of Neural nets is somewhat misleading as neural nets are typically nonlinear. This paper is mostly about linear networks. While a result has been stated for single-hidden ReLU networks. In my view this particular result is an immediate corollary of the result for linear networks. As I explain further below given the combinatorial form of the result, the usefulness of this particular extension to ReLU network is not very clear. I would suggest rewording title/abstract/intro - Theorem 1 is neat, well done! - Page 4 p_i’s in proposition 1 From my understanding the p_i have been introduced in Theorem 1 but given their prominent role in this proposition they merit a separate definition (and ideally in terms of the A_i directly). - Theorems 1, prop 1, prop 2, prop 3, Theorem 3, prop 4 and 5 Are these characterizations computable i.e. given X and Y can one run an algorithm to find all the critical points or at least the parameters used in the characterization p_i, V_i etc? - Theorems 1, prop 1, prop 2, prop 3, Theorem 3, prop 4 and 5 Would recommend a better exposition why these theorems are useful. What insights do you gain by knowing these theorems etc. Are less sufficient conditions that is more intuitive or useful. (an insightful sufficient condition in some cases is much more valuable than an unintuitive necessary and sufficient one). - Page 5 Theorem 2 Does this theorem have any computational implications? Does it imply that the global optima can be found efficiently, e.g. are saddles strict with a quantifiable bound? - Page 7 proposition 6 seems like an immediate consequence of Theorem 1 however given the combinatorial nature of the K_{I,J} it is not clear why this theorem is useful. e.g . back to my earlier comment w.r.t. Linear networks given Y and X can you find the parameters of this characterization with a computationally efficient algorithm?
This paper studies a new architecture DualAC.  The author give strong and convincing justifications based on the Lagrangian dual of the Bellman equation  (although not new, introducing this as the justification for the architecture design is plausible). There are several drawbacks of the current format of the paper: 1. The algorithm is vague.  Alg 1 line 5: 'closed form':  there is no closed form in Eq(14).  It is just an MC approximation. line 6: Decay O(1/t^\beta).  This is indeed vague albeit easy to understand.  The algorithm requires that every step is crystal clear. 2. Also, there are several format error which may be due to compiling, e.g., line 2 of Abstract,'Dual-AC ' (an extra space).  There are many format errors like this throughout the paper.  The author is suggested to do a careful format check. 3. The author is suggested to explain more about the necessity of introducing path regularization and SDA.  The current justification is reasonable but too brief. 4. The experimental part is ok to me,  but not very impressive. Overall, this seems to be a nice paper to me.
The paper is easy to read  for a physicist,  but I am not sure how useful it would be for ICLR...  it is not clear for me it there is an interest for quantum problems in this conference.  This is something I will let to the Area Chair to deceede.  Other than this, the paper is interesting, certainly correct, and provides a nice perspective on the future of learning with quantum computers.  I like the  quantum "boltzmann machine" problems.  I feel, however, but it might be a bit far from the main interest of the conference. Comments: * What the authors called "Free energy-based reinforcement learning" seems to me just the minimization / maximiation of the free energy.  This is simply maximum likelihood applied to the free energy  and I think that calling it "reinforcement learning" is not only wrong, but also is very confusing, given this is usually reserved to an entirely different learning process. * While i liked the introduction of the quantum Boltzmann machine, I would be happy to learn what they can do?  Are these useful, for instance, to study correlated fermions/bosons?  The paper does not explain why one should be concerns with these devices. * The fact that the simulation on a classical computer agrees with the one on a quantum computer is promising,  but I would say that this shows that, so far, there is not yet a clear advantage in using a quantum computer.  This might change, but in the mean time, what is the benefits for the ICLR community?
The paper proposes a method  for learning object representations from pixels and then use such representations for doing reinforcement learning.   This method is based on convnets that map raw pixels to a mask and feature map.  The mask contains information about the presence/absence of objects in different pixel locations and the feature map contains information about object appearance.  I believe that the current method can only learn and track simple objects in a constant background, a problem which is  well-solved in computer vision.  Specifically, a simple method such as "background subtraction" can easily infer the mask (the outlying pixels which correspond to moving objects)  while simple tracking methods (see a huge literature over decades on computer vision) can allow to track these objects across frames.   The authors completely ignore all this previous work  and their "related work" section  starts citing papers from 2016 and onwards!   Is it any benefit of learning objects with the current (very expensive) method compared to simple methods such as  "background subtraction"?  Furthermore, the paper is very badly written  since it keeps postponing the actual explanations to later sections (while these  sections eventually refer to the appendices).   This makes reading the paper very hard.  For example, during the early sections you  keep referring to a loss function which will allow for learning the objects, but you never really give the form of this loss (which you should as soon as  you mentioning it)  and the reader needs to search into the appendices to find out what is happening.   Also, experimental results are very preliminary and not properly analyzed.   For example the results in Figure 3 are unclear and need to be discussed in detail in the main text.
This paper provides the analysis of empirical risk landscape for GENERAL deep neural networks (DNNs).  Assumptions are comparable to existing results for OVERSIMPLIFED shallow neural networks.  The main results analyzed: 1) Correspondence of non-degenerate stationary points between empirical risk and the population counterparts.  2) Uniform convergence of the empirical risk to population risk.  3) Generalization bound based on stability.  The theory is first developed for linear DNNs and then generalized to nonlinear DNNs with sigmoid activations. Here are two detailed comments: 1) For deep linear networks with squared loss, Kawaguchi 2016 has shown that the global optima are the only non-degerenate stationary points.  Thus, the obtained non-degerenate stationary deep linear network should be equivalent to the linear regression model Y=XW.  Should the risk bound only depends on the dimensions of the matrix W? 2) The comparison with Bartlett & Maass’s (BM) work is a bit unfair,  because their result holds for polynomial activations while this paper handles linear activations.  Thus, the authors need to refine BM's result for comparison.
The ms applies an LSTM on ECoG data and studies tranfer between subjects etc. The data includes only few samples per class. The validation procedure to obtain the model accuray is a bit iffy. The ms says: The test data contains 'at least 2 samples per class'. Data of the type analysed is highly dependend, so it is not unclear whether this validation procedure will not provide overoptimistic results. Currently, I do not see evidence for a stable training procedure in the ms. I would be curious also to see a comparison to a k-NN classifier using embedded data to gauge the problem difficulty. Also, the paper does not really decide whether it is a neuroscience contribution or an ML one. If it were a neuroscience contribution, then it would be important to analyse and understand the LSTM representation and to put it into a biological context fig 5B is a first step in this direction. If it where a ML contribution, then there should be a comprehensive analysis that indeed the proposed architecture using the 2 steps is actually doing the right thing, i.e. that the method converges to the truth if more and more data is available. There is also some initial experiments in fig 3A. Currently, I find the paper somewhat unsatisfactory and thus preliminary.
Summary: - This paper proposes a hand-designed network architecture on a graph of object proposals to perform soft non-maximum suppression to get object count. Contribution: - This paper proposes a new object counting module which operates on a graph of object proposals. Clarity: - The paper is well written and clarity is good.  Figure 2 & 3 helps the readers understand the core algorithm. Pros: - De-duplication modules of inter and intra object edges are interesting. - The proposed method improves the baseline by 5% on counting questions. Cons: - The proposed model is pretty hand-crafted.  I would recommend the authors to use something more general, like graph convolutional neural networks (Kipf & Welling, 2017) or graph gated neural networks (Li et al., 2016). - One major bottleneck of the model is that the proposals are not jointly finetuned.  So if the proposals are missing a single object, this cannot really be counted.  In short, if the proposals don’t have 100% recall, then the model is then trained with a biased loss function which asks it to count all the objects even if some are already missing from the proposals.  The paper didn’t study what is the recall of the proposals and how sensitive the threshold is. - The paper doesn’t study a simple baseline that just does NMS on the proposal domain. - The paper doesn’t compare experiment numbers with (Chattopadhyay et al., 2017). - The proposed algorithm doesn’t handle symmetry breaking when two edges are equally confident (in 4.2.2 it basically scales down both edges).  This is similar to a density map approach and the problem is that the model doesn’t develop a notion of instance. - Compared to (Zhou et al., 2017), the proposed model does not improve much on the counting questions. - Since the authors have mentioned in the related work, it would also be more convincing if they show experimental results on CL Conclusion: - I feel that the motivation is good, but the proposed model is too hand-crafted.  Also, key experiments are missing: 1) NMS baseline 2) Comparison with VQA counting work  (Chattopadhyay et al., 2017).  Therefore I recommend reject. References: - Kipf, T.N., Welling, M., Semi-Supervised Classification with Graph Convolutional Networks. ICLR 2017. - Li, Y., Tarlow, D., Brockschmidt, M., Zemel, R. Gated Graph Sequence Neural Networks. ICLR 2016.
In this paper authors are summarizing their work on building a framework for automated neural network (NN) construction across multiple tasks simultaneously. They present initial results on the performance of their framework called Multitask Neural Model Search (MNMS) controller. The idea behind building such a framework is motivated by the successes of recently proposed reinforcement based approaches for finding the best NN architecture across the space of all possible architectures. Authors cite the Neural Architecture Search (NAS) framework as an example of such a framework that yields better results compared to NN architectures configured by humans. Overall I think that the idea is interesting and the work presented in this paper is very promising. Given the depth of the empirical analysis presented the work still feels that it’s in its early stages. In its current state and format the major issue with this work is the lack of more in-depth performance analysis which would help the reader draw more solid conclusions about the generalization of the approach. Authors use two text classification tasks from the NLP domain to showcase the benefits of their proposed architecture. It would be good if they could expand and analyze how well does their framework generalizes across other non-binary tasks, tasks in other domains and different NNs. This is especially the case for the transfer learning task. In the NAS overview section, readers would benefit more if authors spend more time in outlining the RL detail used in the original NAS framework instead of Figure 1 which looks like a space filler. Across the two NLP tasks authors show that MNMS models trained simultaneously give better performance than hand tuned architectures. In addition, on the transfer learning evaluation approach they showcase the benefit of using the proposed framework in terms of the initially retrieved architecture and the number of iterations required to obtain the best performing one. For better clarity figures 3 and 5 should be made bigger. What is LSS in figure 4?
This is a dense, rich, and impressive paper on rapid meta-learning.  It is already highly polished,  so I have mostly minor comments. Related work: I think there is a distinction between continual and life-long learning,  and I think that your proposed setup is a form of continual learning (see Ring ‘94/‘97).  Given the proliferation of terminology for very related setups,  I’d encourage you to reuse the old term. Terminology: I find it confusing which bits are “meta” and which are not,  and the paper could gain clarity by making this consistent.  In particular, it would be good to explicitly name the “meta-loss” (currently the unnamed triple expectation in (3)).  By definition, then, the “meta-gradient” is the gradient of the meta-loss -- and not the one in (2), which is the gradient of the regular loss. Notation: there’s redundancy/inconsistency in the reward definition:  pick either R_T or \bold{r}, not both, and maybe include R_T in the task tuple definition?  It is also confusing that \mathcal{R} is a loss, not a reward (and is minimized)  -- maybe use another symbol? A question about the importance sampling correction: given that this spans multiple (long) trajectories, don’t the correction weights become really small in practice?  Do you have some ballpark numbers? Typos:- “event their learning” - “in such setting” - “experience to for”
The analyses of this paper (1) increasing the feature norm of correctly-classified examples induce smaller training loss, (2) increasing the feature norm of mis-classified examples upweight the contribution from hard examples, are interesting.  The reciprocal norm loss seems to be reasonable idea to improve the CNN learning based on the analyses.  However, the presentation of this paper need to be largely improved.  For example, Figure 3 seems to be not relevant to Property2  and may be show the feature norm is lower when the samples is hard example.  Therefore, the author used reciprocal norm loss which increases feature norm as shown in Figure 4.  However, both Figures are not explained in the main text,  and thus hard to understand the relation of Figure 3 and 4.  The author should refer all Figures and Tables.  Other issues are: -Large-margin Soft max in Figure 2 is not explained in the introduction section.  -In Eq.(7), P_j^I is not defined.  - In the Property3, The author wrote “ where r is lower bound of feature norm”.   However, r is not used. -In the experimental results, “RN” is not defined. -In the Table3, the order of \lambda should be increasing or decreasing order.  - Table 5 is not referred in the main text.
This work exploits the causality principle to quantify how the weights of successive layers adapt to each other. Some interesting results are obtained, such as "enforcing more independence between successive layers of generators may lead to better performance and modularity of these architectures" . Generally, the result is interesting and the presentation is easy to follow. However, the proposed approach and the experiments are not convincible enough. For example,  it is hard to obtain the conclusion "more independence lead to better performance" from the experimental results. Maybe more justifications are needed.
This paper proposes a novel image/album geolocation algorithm. This is based on the recent PlaNet approach, with several extensions, including a better mesh representation for discretization and the time-aware prediction. However, contributions are limited to me. Pros: - Clearly written - Good results - Interesting empirical analysis between UTC time and geo-location Contribution: - Triangulation based meshify vs quad-tree meshification was claimed to be one of the major contributions. It is an incremental improvement towards Wayand et al. However I don’t think this is significant enough for an ICLR paper. - Incorporating the time-ordering into album based geo-localization is considered a contribution on the application side. But in computer vision this is not the first one to exploit this information (see [a]) while the technical approach to encode this information is quite standard (LSTM). Improvement:- Compared against Wayand et al. and Vo et al. the quantitative results are not very impressive. Minor: Some choices in feature engineering are not explained well: e.g. In section 3.2, why you only choose top 10 maximum entries, why l1-norm is appended as a feature? Are those choice made because of validation performance? Minor writing: Fig. 1 “01:00 than” Fig. 1 “to make a prediction” “Fig. 3.1.1” Table 5. Why not bold the best performance? Acknowledgements shouldn’t be put in double-blind reviewing process [A] Evangelos Kalogerakis, Olga Vesselova, James Hays, Alexei A. Efros, Aaron Hertzmann, "Image Sequence Geolocation with Human Travel Priors", Proceedings of the IEEE Internaltional Conference on Computer Vision Recognition (ICCV), 2009.
The authors present a model for unsupervised NMT which requires no parallel corpora between the two languages of interest. While the results are interesting I find very few original ideas in this paper. Please find my comments/questions/suggestions below: 1) The authors mention that there are 3 important aspects in which their model differs from a standard NMT architecture. All the 3 differences have been adapted from existing works. The authors clearly acknowledge and cite the sources. Even sharing the encoder using cross lingual embeddings has been explored in the context of multilingual NER (please see https://arxiv.org/abs/1607.00198). Because of this I find the paper to be a bit lacking on the novelty quotient. Even backtranslation has been used successfully in the past (as acknowledged by the authors). Unsupervised MT in itself is not a new idea (again clearly acknowledged by the authors). 2) I am not very convinced about the idea of denoising. Specifically, I am not sure if it will work for arbitrary language pairs. In fact, I think there is a contradiction even in the way the authors write this. On one hand, they want to "learn the internal structure of the languages involved" and on the other hand they deliberately corrupt this structure by adding noise. This seems very counter-intuitive and in fact the results in Table 1 suggest that it leads to a drop in performance. I am not very sure that the analogy with autoencoders holds in this case. 3) Following up on the above question, the authors mention that "We emphasize, however, that it is not possible to use backtranslation alone without denoising". Again, if denoising itself leads to a drop in the performance as compared to the nearest neighbor baseline then why use backtranslation in conjunction with denoising and not in conjunction with the baseline itself. 4) This point is more of a clarification and perhaps due to my lack of understanding. Backtranslation to generate a pseudo corpus makes sense only after the model has achieved a certain (good) performance. Can you please provide details of how long did you train the model (with denoising?) before producing the backtranslations ? 5) The authors mention that 100K parallel sentences may be insufficient for training a NMT system. However, this size may be decent enough for  a PBSMT system. It would be interesting to see the performance of a PBSMT system trained on 100K parallel sentences. 6) How did you arrive at the beam size of 12 ? Was this a hyperparameter? Just curious. 7) The comparable NMT set up is not very clear. Can you please explain it in detail ? In the same paragraph, what exactly do you mean by "the supervised system in this paper is relatively small?"