launch/ampere huggingface dataset:
Active learning for deep learning is an interesting topic  and there is few useful tool available in the literature.  It is happy to see such paper in the field.  This paper proposes a batch mode active learning algorithm for CNN as a core-set problem.  The authors provide an upper bound of the core-set loss, which is the gap between the training loss on the whole set and the core-set.  By minimizing this upper bound, the problem becomes a K-center problem which can be solved by using a greedy approximation method, 2-OPT.  The experiments are performed on image classification problem (CIFAR, CALTECH, SVHN datasets), under either supervised setting or weakly-supervised setting.  Results show that the proposed method outperforms the random sampling and uncertainty sampling by a large margin.  Moreover, the authors show that 2-OPT can save tractable amount of time in practice with a small accuracy drop. The proposed algorithm is new  and writing is clear.  However, the paper is not flawless.  The proposed active learning framework is under ERM and cover-set, which are currently not supported by deep learning.  To validate such theoretical result, a non-deep-learning model should be adopted.  The ERM for active learning has been investigated in the literature, such as "Querying discriminative and representative samples for batch mode active learning" in KDD 2013, which also provided an upper bound loss of the batch mode active learning  and seems applicable for the problem in this paper.  Another interesting question is most of the competing algorithm is myoptic active learning algorithms.  The comparison is not fair enough.  The authors should provide more competing algorithms in batch mode active learning.
This paper is based on the theory of group equivariant CNNs (G-CNNs), proposed by Cohen and Welling ICML'16. Regular convolutions are translation-equivariant, meaning that if an image is translated, its convolution by any filter is also translated.  They are however not rotation-invariant for example.   G-CNN introduces G-convolutions, which are equivariant to a given transformation group G. This paper proposes an efficient implementation of G-convolutions for 6-fold rotations (rotations of multiple of 60 degrees), using a hexagonal lattice.  The approach is evaluated on CIFAR-10 and AID, a dataset of aerial scene classification.  The approach outperforms G-convolutions implemented on a squared lattice, which allows only 4-fold rotations on AID by a short margin.  On CIFAR-10, the difference does not seem significative (according to Tables 1 and 2). I guess this can be explained by the fact that rotation equivariance makes sense for aerial images, where the scene is mostly fronto-parallel, but less for CIFAR (especially in the upper layers), which exhibits 3D objects. I like the general approach of explicitly putting desired equivariance in the convolutional networks.  Using a hexagonal lattice is elegant, even if it is not new in computer vision  (as written in the paper).  However, as the transformation group is limited to rotations,  this is interesting in practice mostly for fronto-parallel scenes, as the experiences seem to show.  It is not clear how the method can be extended to other groups than 2D rotations. Moreover, I feel like the paper sometimes tries to mask the fact that the proposed method is limited to rotations.  It is admittedly clearly stated in the abstract and introduction, but much less in the rest of the paper. The second paragraph of Section 5.1 is difficult to keep in a paper.  It says that "From a qualitative inspection of these hexagonal interpolations we conclude that no information is lost during the sampling procedure."   "No information is lost" is a strong statement from a qualitative inspection, especially of a hexagonal image.   This statement should probably be removed.  One way to evaluate the information lost could be to iterate interpolation between hexagonal and squared lattices to see if the image starts degrading at some point.
This paper studies the adjustment of dropout rates  which is a useful tool to prevent the overfitting of deep neural networks.  The authors derive a generalization error bound in terms of dropout rates.  Based on this, the authors propose a regularization framework to adaptively select dropout rates.  Experimental results are also given to verify the theory. Major comments: (1) The Empirical Rademacher complexity is not defined.  For completeness, it would be better to define it at least in the appendix. (2) I can not follow the inequality (5).  Especially, according to the main text, f^L is a vector-valued function .  Therefore, it is not clear to me the meaning of \sum\sigma_if^L(x_i,w) in (5). (3) I can also not see clearly the third equality in (9).  Note that f^l is a vector-valued function.  It is not clear to me how it is related to a summation over j there. (4) There is a linear dependency on the number of classes in Theorem 3.1.  Is it possible to further improve this dependency? Minor comments:(1) Section 4: 1e-3,1e-4,1e-5 is not consistent with 1e^{-3}, 1e^{-4},1e^{-5} (2) Abstract: there should be a space before "Experiments". (3) It would be better to give more details (e.g., page, section) in citing a book in the proof of Theorem 3.1 Summary: The mathematical analysis in the present version is not rigorous.  The authors should improve the mathematical analysis.
I quite liked the revival of the dual memory system ideas and the cognitive (neuro) science inspiration.  The paper is overall well written and tackles serious modern datasets, which was impressive,  even though it relies on a pre-trained, fixed ResNet (see point below). My only complaint is that I felt I couldn’t understand why the model worked so well.  A better motivation for some of the modelling decisions would be helpful.  For instance, how much the existence (and training) of a BLA network really help  — which is a central new part of the paper, and wasn’t in my view well motivated.  It would be nice to compare with a simpler baseline, such as a HC classifier network with reject option.  I also don’t really understand why the proposed pseudorehearsal works so well.  Some formal reasoning, even if approximate, would be appreciated. Some additional comments below: - Although the paper is in general well written,  it falls on the lengthy side  and I found it difficult at first to understand the flow of the algorithm.  I think it would be helpful to have a high-level pseudocode presentation of the main steps. - It was somewhat buried in the details that the model actually starts with a fixed, advanced feature pre-processing stage (the ResNet, trained on a distinct dataset, as it should).  I’m fine with that,  but this should be discussed.  Note that there is evidence that the neuronal responses in areas as early as V1 change as monkeys learn to solve discrimination tasks.  It should be stressed that the model does not yet model end-to-end learning in the incremental setting. - p. 4, Eq. 4, is it really necessary to add a loss for the intermediate layers, and not only for the input layer?  I think it would be clearer to define the \mathcal{L} explictily somewhere.  Also, shouldn’t the sum start at j=0?
The authors present an estimator for the mutual information (MI) based on the Donsker-Varadhan representation for the KL divergence and its generalization to arbitrary f-divergences by Ruderman et al.  While that last work introduced an estimator based on optimization over the unit ball in an RKHS,  the current work propose to use a parametric function class given by a neural network  (I'd suggest that the authors make this point more explicit,  as currently it's not totally clear what their actual contribution is and how their work compares to the prior art they cite).  The authors show that such an estimator can be used to train models with less mode-dropping in adversarial models.  The work is quite straightforward,  but improves over similar work in the GAN space by Nowozin et al. by using Ruderman's tighter variational representation instead of Nguyen's one. The paper contains many typos and grammatical errors  and the authors should do an exhaustive proof-reading.  More problematic is that, right after eq. 10, the authors mention "We show in the Appendix that OMIE has the desirable strong consistency and convergence properties".  However, the appendix doesn't contain such a proof.  Is it missing from the submitted version?  I don't think that such a consistency proof is strictly necessary for a paper like this,  but for the review to be accurate I need to see the proof.  Since I can't find it,  I assume it does not exist.  In that case, the authors should give less emphasis to the MI estimator itself and more to the empirical properties and applications. The authors present some experiments comparing different estimators of MI applied to synthetic data.  Figure 1 is hard to read,  I suggest the authors try to come up with a more legible plot.  Figure 2 is also a bit surprising,  why show error for 50 dimensions but estimates for 2 dimensions?  Since these experiments are quick to run,  it would be helpful to get more information on how the gap between the methods change as the dimensionality increases (e.g. a surface plot with d and # of iterations on the x and y axes).  Also it would be highly beneficial to compare with the method in Ruderman at al.,  so that people interested in MI estimation but who don't plan on using the estimator as part of a neural net architecture can get some idea on how the inductive bias of NNs compare to RKHS. In the caption to Fig. 3 the authors state "The OMIEGAN generator learns a distribution with a high amount of structured noise", which I find hard to understand.  Probably the authors can be a bit more precise than saying "structured noise". I would recommend dropping the Information Bottleneck section to focus on showing more convincingly the impact of OMIE in GANs.  The experiments section currently looks rushed and lacking in depth. In summary, this work provides value by introducing a (previously known) superior f-divergence variational representation to the GAN community.  The mode-collapse prevention via MI maximisation is also interesting and deserves more experimental attention to make the paper stronger.
The authors study the problem of distributed routing in a network, where the goal is to minimize the maximal load (i.e. the load of the link with the highest utilization).  The authors advocate to use multi-agent reinforcement learning.  The main idea put forward by the authors is that by designing artificial rewards (to guide the agents), one can achieve faster exploration, in order to reduce convergence time. While the authors put forward several interesting ideas,  there are some shortcomings to the present version of the paper, including: - The design objective seems flawed from the networking point of view:  while minimizing the maximal load of a link is certainly a good starting point (to avoid instable queues)  one typically wants to minimize delay (or maximize flow throughput).  Indeed, it is possible to have a larger maximal load while reducing delay in many cases. - Furthermore, the authors do not provide a baseline to which the outcome of the learning algorithms they propose:  for instance how does their approach compare to simple policies (those are commonplace in networking) such as MaxWeight, Backpressure and so on ? - The authors argue that using multi-agent learning is more desirable than single agent (i.e. with a single reward signal which is common to all agents).  However, is multi-agent guaranteed to converge in such a setting ?  If some versions of the problem (for some particular reward signal) are not guaranteed to converge, it is difficult to understand whether "convergence"  is slow due to an inefficient exploration, or simply because convergence cannot occur in the first place. - The learning algorithms used are not clearly explained:  the authors simply state that they use "ACCNet" (from some unpublished prior work),  but to readers unfamiliar with this algorithm, it is difficult to judge the contents of the paper.  - In the numerical experiments, what is the "convergence rate" ?  is it the ratio between the mean reward of the learnt policy and that of the optimal ?  For how many time steps are the learning algorithm run before evaluating their outcome ?  What are the meaning of the various input parameter of ACCnet,  and is the performance sensitive to those parameters ?
This paper presents a new method for detecting hypernymy by extending the distributional inclusion hypothesis to low-rank embeddings. The first half of the paper is written superbly, providing a sober account of the state-of-the-art in hypernymy detection via distributional semantics, and a clear, well-motivated explanation of the main algorithm (DIVE). The reformulation of PMI is interesting; the authors essentially replace P(w) with the uniform distribution (this should be stated explicitly in the paper, BTW). They then augment SGNS to reflect this change by dynamically selecting the number of negatively-sampled contexts according to the target word (w). This is a clever trick. I was also very happy to see that the authors evaluated on virtually every lexical inference dataset available. However, the remainder of the paper describing the experiments and their results was extremely difficult to parse. First of all, the line between the authors' contribution and prior art is blurry, because they seem to be introducing new metrics for measuring hypernymy as part of the experiments. There are also too many moving parts: datasets, evaluation metrics, detection metrics, embedding methods, hyperparameters, etc. These need to be organized, controlled for, and clearly explained. I have two major concerns regarding the experiments, beyond intelligibility. First, the authors used a specific corpus (with specific preprocessing) to train their vectors, but compared their results to those reported in other papers, which are not based on the same data. This invalidates these comparisons. Instead, the other methods should be replicated and rerun on exactly the same corpus. I would also recommend using a much larger corpus, since 50M tokens is considered quite small when training word embeddings. My second concern is that summation (dS) is doing all the heavy lifting. In Table 2, we can see that the difference is only 0.1 between dS and W * dS (where W is also not trained using DIVE). Since dS is basically a proxy for difference in word frequency, could it be that the proposed method is just computing which word is more general? This looks awfully familiar to Levy et al's prototypical hypernym result. Miscellaneous Comments: - There's a very recent paper on using vector norms for detecting hypernymy that might be worth contrasting with: https://arxiv.org/pdf/1710.06371.pdf - Micro-averaging these datasets is problematic, because some of the datasets based on WordNet are much larger than the hand-annotated ones, and will likely drown them out. Because these datasets are so different, I think it is critical to look at the details and not only at the averages. - PMI filtering needs to be controlled/ablated in the experiments. - While explaining equation 6, the authors say that the gradients for x and y are similar; this is not true, because k is a function of x (or y), and if one appears more often in general (not necessarily with c), then the gradients will be different as well.
This paper leverages how deep Bayesian NNs, in the limit of infinite width, are Gaussian processes (GPs).  After characterizing the kernel function, this allows us to use the GP framework for prediction, model selection, uncertainty estimation, etc. - Pros of this work The paper provides a specific method to efficiently compute the covariance matrix of the equivalent GP and shows experimentally on CIFAR and MNIST the benefits of using the this GP as opposed to a finite-width non-Bayesian NN. The provided phase analysis and its relation to the depth of the network is also very interesting. Both are useful contributions as long as deep wide Bayesian NNs are concerned.  A different question is whether that regime is actually useful. - Cons of this work Although this work introduces a new GP covariance function inspired by deep wide NNs,  I am unconvinced of the usefulness of this regime for the cases in which deep learning is useful.  For instance, looking at the experiments, we can see that on MNIST-50k (the one with most data, and therefore, the one that best informs about the "true" underlying NN structure) the inferred depth is 1 for the GP and 2 for the NN, i.e., not deep.  Similarly for CIFAR, where only up to depth 3 is used.  None of these results beat state-of-the-art deep NNs. Also, the results about the phase structure show how increased depth makes the parameter regime in which these networks work more and more constrained.  In [1], it is argued that kernel machines with fixed kernels do not learn a hierarchical representation.  And such representation is generally regarded as essential for the success of deep learning.  My impression is that the present line of work will not be relevant for deep learning and will not beat state-of-the-art results  because of the lack of a structured prior.  In that sense, to me this work is more of a negative result informing that to be successful, deep Bayesian NNs should not be wide and should have more structure to avoid reaching the GP regime. - Other comments:In Fig. 5, use a consistent naming for the axes (bias and variances). In Fig. 1, I didn't find the meaning of the acronym NN with no specified width. Does the unit norm normalization used to construct the covariance disallow ARD input selection? [1] Yoshua Bengio, Olivier Delalleau, and Nicolas Le Roux. The Curse of Dimensionality for Local Kernel Machines. 2005.
This paper presents a generic unbiased low-rank stochastic approximation to full rank matrices that makes it possible to do online RNN training without the O(n^3) overhead of real-time recurrent learning (RTRL). This is an important and long-sought-after goal of connectionist learning and this paper presents a clear and concise description of why their method is a natural way of achieving that goal, along with experiments on classic toy RNN tasks with medium-range time dependencies for which other low-memory-overhead RNN training heuristics fail. My only major complaint with the paper is that it does not extend the method to large-scale problems on real data, for instance work from the last decade on sequence generation, speech recognition or any of the other RNN success stories that have led to their wide adoption (eg Graves 2013, Sutskever, Martens and Hinton 2011 or Graves, Mohamed and Hinton 2013). However, if the paper does achieve what it claims to achieve, I am sure that many people will soon try out UORO to see if the results are in any way comparable.
The paper presents an off-policy actor-critic method for learning a stochastic policy with entropy regularization.  It is a direct extension of maximum entropy reinforcement learning for Q-learning (recently called soft-Q learning), and named soft actor-critic (SAC).  Empirically SAC is shown to outperform DDPG significantly in terms of stability and sample efficiency, and can solve relatively difficult tasks that previously only on-policy (or hybrid on-policy/off-policy) method such as TRPO/PPO can solve stably.  Besides entropy regularization, it also introduces multi-modal policy parameterization through mixture of Gaussians that enables diverse, on-policy exploration.      The main appeal of the paper is the strong empirical performance of this new off-policy method in continuous action benchmarks.  Several design choices could be the key,  so it is encouraged to provide more ablation studies on these, which would be highly valuable for the community.  In particular, - Amortization of Q and \pi through fitting state value function - On-policy exploration vs OU process based off-policy exploration - Mixture vs non-mixture-based stochastic policy - SAC vs soft Q-learning Another valuable discussion to be had is the stability of off-policy algorithm comparing Q-learning versus actor-critic method. Pros: - Simple off-policy algorithm that achieves significantly better performance than existing off-policy baseline algorithms - It allows on-policy exploration in off-policy learning, partially thanks to entropy regularization that prevents variance from shrinking to 0.  It could be considered a major success of off-policy algorithm that removes heuristic exploration noise. Cons: - Method is relatively simple extension from existing work in maximum entropy reinforcement learning.  It is unclear what aspects lead to significant improvements in performance due to insufficient ablation studies.  Other question: - Above Eq. 7 it discusses that fitting a state value function wrt Q and \pi is shown to improve the stability significantly.  Is this comparison with directly estimating state value using finite samples?  If so, is the primary instability due to variance of the estimate, which can be avoided by drawing a lot of samples or do full integration (still reasonably tractable for finite mixture model)?  Or, is the instability from elsewhere?  By having SGD-based fitting of state value function, it appears to simulate slowly changing target values (similar role as target networks).  If so, could a similar technique be used with DDPG and get more stable performance?
This paper describes an attempt of improving information flow in deep networks (but is used and tested here with seq2seq models although it is reality unrelated to seq2seq models per se).  Slightly different from Resnet the information flow is improved by not just adding the outputs from previous layers but instead concatenating the outputs from previous layers with the current outputs.  The authors claim better convergence speed and better results for a similar number of parameters although the differences seems to be in the noise.   Overall this is an OK technique but in my opinion not really novel enough to justify a whole paper about it  as it seems more like a relatively minor architecture tweak.  The results seem to indicate that there were some problems with getting deeper networks to work for the baseline (why is in Table 3 baseline-6L worse than baseline-4L?) for which the reason could be a multitude of issues probably related to hyper-parameter tuning.  What is also missing is a an analysis of the negative consequences of this technique -- for example, doesn't the number of parameters increase with the depth of the network because of the concatenation?  Also, it would have been good to see more experiments with smaller baseline networks as well to match the smaller DenseNet networks in Table 1 and 2.  Finally, the writing of the paper could be improved a lot:  The basic idea is not well described (however, many times repeated)  and the grammar is often wrong  and also there are some typos.