This paper focuses on the problem of "machine teaching", i.e., how to select a good strategy to select training data points to pass to a machine learning algorithm, for faster learning.

The proposed approach leverages reinforcement learning by defining the reward as how fast the learner learns, and use policy gradient to update the teacher parameters.

I find the definition of the "state" in this case very interesting.

The experimental results seem to show that such a learned teacher strategy makes machine learning algorithms learn faster.

Overall I think that this paper is decent.

The angle the authors took is interesting (essentially replacing one level of the bi-level optimization problem in machine teaching works with a reinforcement learning setup).

The problem formulation is mostly reasonable,

and the evaluation seems quite convincing.

The paper is well-written:

I enjoyed the mathematical formulation (Section 3).

The authors did a good job of using different experiments (filtration number analysis, and teaching both the same architecture and a different architecture) to intuitively explain what their method actually does.

At the same time, though, I see several important issues that need to be addressed if this paper is to be accepted.

Details below.

1. As much as I enjoyed reading Section 3, it is very redundant.

In some cases it is good to outline a powerful and generic framework (like the authors did here with defining "teaching" in a very broad sense, including selecting good loss functions and hypothesis spaces) and then explain that the current work focuses on one aspect (selecting training data points).

However, I do not see it being the case here.

In my opinion, selecting good loss functions and hypothesis spaces are much harder problems than data teaching - except maybe when one use a pre-defined set of possible loss functions and select from it.

But that is not very interesting

(if you can propose new loss functions, that would be way cooler).

I also do not see how to define an intuitive set of "states" in that case.

Therefore, I think this section should be shortened.

I also think that the authors should not discuss the general framework and rather focus on "data teaching",

which is the only focus of the current paper.

The abstract and introduction should also be modified accordingly to more honestly reflect the current contributions.

2. The authors should do a better job at explaining the details of the state definition, especially the student model features and the combination of data and current learner model.

3. There is only one definition of the reward - related to batch number when the accuracy first exceeds a threshold.

Is accuracy stable, can it drop back down below the threshold in the next epoch?

The accuracy on a held-out test set is not guaranteed to be monotonically increasing, right?

Is this a problem in practice (it seems to happen on your curves)?

What about other potential reward definitions?

And what would they potentially lead to?

4. Experimental results are averaged over 5 repeated runs

- a bit too small in my opinion.

5. Can the authors show convergence of the teacher parameter \theta?

I think it is important to see how fast the teacher model converges, too.

6. In some of your experiments, every training method converges to the same accuracy after enough training (Fig.2b), while in others, not quite (Fig. 2a and 2c).

Why is this the case?

Does it mean that you have not run enough iterations for the baseline methods?

My intuition is that if the learner algorithm is convex, then ultimately they will all get to the same accuracy level, so the task is just to get there quicker.

I understand that since the learner algorithm is an NN,

this is not the case

- but more explanation is necessary here

- does your method also reduces the empirical possibility to get stuck in local minima?

7. More explanation is needed towards Fig.4c.

In this case, using a teacher model trained on a harder task (CIFAR10) leads to much improved student training on a simpler task (MNIST).

Why?

8. Although in terms of "effective training data points" the proposed method outperforms the other methods,

in terms of time (Fig.5) the difference between it and say, NoTeach, is not that significant (especially at very high desired accuracy).

More explanation needed here.

fact fact evaluation fact evaluation evaluation evaluation evaluation evaluation evaluation evaluation evaluation non-arg evaluation evaluation fact evaluation evaluation evaluation evaluation request request fact request request fact request request request request request fact evaluation request evaluation fact request request evaluation fact fact request request request fact request fact evaluation request


This paper presents an empirical study of whether data augmentation can be a substitute for explicit regularization of weight decay and dropout.

It is a well written and well organized paper.

However, overall I do not find the authors’ premises and conclusions to be well supported by the results and

would suggest further investigations.

In particular: a) Data augmentation is a very domain specific process and limits of augmentation are often not clear.

For example, in financial data or medical imaging data it is often not clear how data augmentation should be carried out and how much is too much.

On the other hand model regularization is domain agnostic

(has to be tuned for each task, but the methodology is consistent and well known).

Thus advocating that data augmentation can universally replace explicit regularization does not seem correct.

b) I find the results to be somewhat inconsistent.

For example, on CIFAR-10, for 100% data regularization+augmentation is better than augmentation alone for both models,

whereas for 80% data augmentation alone seems to be better.

Similarly on CIFAR-100 the WRN model shows mixed trends,

and this model is significantly better than the All-CNN model in performance.

These results also seem inconsistent with authors statement

“…and conclude that data augmentation alone - without any other explicit regularization techniques - can achieve the same performance to higher as regularized models…”

fact evaluation evaluation request evaluation evaluation fact fact fact evaluation evaluation evaluation fact fact evaluation quote


The paper consider a method for "weight normalization" of layers of a neural network.

The weight matrix is maintained normalized, which helps accuracy.

However, the simplest way to normalize on a fully connected layer is quadratic (adding squares of weights and taking square root).

The paper proposes "FastNorm", which is a way to implicitly maintain the normalized weight matrix using much less computation.

Essentially, a normalization vector is maintained an updated separately.

Pros:   Natural method to do weight normalization efficeintly

Cons:   A very natural and simple solution that is fairly obvious.

Limited experiments

fact fact fact fact fact evaluation evaluation evaluation


Paper presents an interesting attention mechanism for fine-grained image classification.

Introduction states that the method is simple and easy to understand.

However, the presentation of the method is bit harder to follow.

It is not clear to me if the attention modules are applied over all  pooling layers.

How they are combined?

Why use cross -correlation as the regulariser?

Why not much stronger constraint such as orthogonality over elements of M in equation 1?

What is the impact of this regularisation?

Why use soft-max in equation 1?

One may use a Sigmoid as well?

Is it better to use soft-max?

Equation 9 is not entirely clear to me.

Undefined notations.

In Table 2, why stop from AD= 2 and AW=2?

What is the performance of AD=1, AW=1 with G?

Why not perform this experiment over all 5 datasets?

Is this performances, dataset specific?

The method is compared against 5 datasets.

Obtained results are quite good.

evaluation fact evaluation evaluation request request request request request non-arg non-arg evaluation fact request request request request fact evaluation


This paper considers the problem of improving sequence generation by learning better metrics.

Specifically, it focuses on addressing the exposure bias problem, where traditional methods such as SeqGAN uses GAN framework and reinforcement learning.

Different from these work, this paper does not use GAN framework.

Instead, it proposed an expert-based reward function training, which trains the reward function (the discriminator) from data that are generated by randomly modifying parts of the expert trajectories.

Furthermore, it also introduces partial reward function that measures the quality of the subsequences of different lengths in the generated data.

This is similar to the idea of hierarchical RL, which divide the problem into potential subtasks, which could alleviate the difficulty of reinforcement learning from sparse rewards.

The idea of the paper is novel.

However, there are a few points to be clarified.

In Section 3.2 and in (4) and (5), the authors explains how the action value Q_{D_i} is modeled and estimated for the partial reward function D_i of length L_{D_i}.

But the authors do not explain how the rewards (or action value functions) of different lengths are aggregated together to update the model using policy gradient.

Is it a simple sum of all of them?

It is not clear why the future subsequences that do not contain y_{t+1} are ignored for estimating the action value function Q in (4) and (5).

The authors stated that it is for reducing the computation complexity.

But it is not clear why specifically dropping the sequences that do not contain y_{t+1}.

Please clarify more on this point.

fact fact fact fact fact evaluation evaluation evaluation fact fact non-arg evaluation fact evaluation request


The authors propose an approach to dynamically generating filters in a CNN based on the input image.

The filters are generated as linear combinations of a basis set of filters, based on features extracted by an auto-encoder.

The authors test the approach on recognition tasks on three datasets: MNIST, MTFL (facial landmarks) and CIFAR10, and show a small improvement over baselines without dynamic filters.

Pros: 1) I have not seen this exact approach proposed before.

2) There method is evaluated on three datasets and two tasks: classification and facial landmark detection.

Cons: 1) The authors are not the first to propose dynamically generating filters,

and they clearly mention that the work of De Brabandere et al. is closely related.

Yet, there is no comparison to other methods for dynamic weight generation.

2) Related to that, there is no ablation study,

so it is unclear if the authors’ contributions are useful.

I appreciate the analysis in Tables 1 and 2,

but this is not sufficient.

Why the need for the autoencoder - why can’t the whole network be trained end-to-end on the goal task?

Why generate filters as linear combination - is this just for computational reasons, or also accuracy?

This should be analyzed empirically.

3) The experiments are somewhat substandard:

- On MNIST the authors use a tiny poorly-performance network,

and it is no surprise that one can beat it with a bigger dynamic filter network.

- The MTFL experiments look most convincing

(although this might be because I am not familiar with SoTA on the dataset),

but still there is no control for the number of parameters,

and the performance improvements are not huge

- On CIFAR10 - there is a marginal improvement in performance,

which, as the authors admit, can also be reached by using a deeper model.

The baseline models are far from SoTA

- the authors should look at more modern architecture such as AllCNN (not particularly new or good, but very simple), ResNet, wide ResNet, DenseNet, etc.

As a comment, I don’t think classification is a good task for showcasing such an architecture

- classification is already working extremely well.

Many other tasks - for instance, detection, tracking, few-shot learning - seem much more promising.

To conclude, the authors propose a new approach to learning convolutional networks with dynamic input-conditioned filters.

Unfortunately, the authors fail to demonstrate the value of the proposed method.

I therefore recommend rejection.

fact fact fact evaluation fact fact fact fact fact evaluation evaluation evaluation request request request evaluation evaluation evaluation evaluation evaluation fact evaluation evaluation fact evaluation request evaluation evaluation evaluation fact evaluation evaluation


This work identifies a mistake in the existing proof of convergence of Adam,

which is among the most popular optimization methods in deep learning.

Moreover, it gives a simple 1-dimensional counterexample with linear losses on which Adam does not converge.

The same issue also affects RMSprop,

which may be viewed as a special case of Adam without momentum.

The problem with Adam is that the "learning rate" matrices V_t^{1/2}/alpha_t are not monotonically decreasing.

A new method, called AMSGrad is therefore proposed, which modifies Adam by forcing these matrices to be decreasing.

It is then shown that AMSGrad does satisfy essentially the same convergence bound as the one previously claimed for Adam.

Experiments and simulations are provided that support the theoretical analysis.

Apart from some issues with the technical presentation (see below),

the paper is well-written.

Given the popularity of Adam, I consider this paper to make a very interesting observation.

I further believe all issues with the technical presentation can be readily addressed.

Issues with Technical Presentation:- All theorems should explicitly state the conditions they require instead of referring to "all the conditions in (Kingma & Ba, 2015)".

- Theorem 2 is a repetition of Theorem 1 (except for additional conditions).

- The proof of Theorem 3 assumes there are no projections,

so this should be stated as part of its conditions.

(The claim in footnote 2 that they can be handled seems highly plausible,

but you should be up front about the limitations of your results.)

- The regret bound Theorem 4 establishes convergence of the optimization method,

so it plays the role of a sanity check.

However, it is strictly worse than the regret bound O(sqrt{T}) for online gradient descent [Zinkevich,2003],

so it cannot explain why the proposed AMSgrad method might be adaptive.

(The method may indeed be adaptive in some sense;

I am just saying the *bound* does not express that.

This is also not a criticism of the current paper;

the same remark also applies to the previously claimed regret bound for Adam.)

- The discussion following Corollary 1 suggests that sum_i hat{v}_{T,i}^{1/2} might be much smaller than d G_infty.

This is true,

but we should always expect it to be at least a constant,

because hat{v}_{t,i} is monotonically increasing by definition of the algorithm,

so the bound does not get better than O(sqrt(T)).

It is also suggested that sum_i ||g_{1:T,i}|| = sqrt{sum_{t=1}^T g_{t,i}^2} might be much smaller than dG_infty,

but this is very unlikely,

because this term will typically grow like O(sqrt{T}), unless the data are extremely sparse,

so we should at least expect some dependence on T.

- In the proof of Theorem 1, the initial point is taken to be x_1 = 1,

which is perfectly fine,

but it is not "without loss of generality", as claimed.

This should be stated in the statement of the Theorem.

- The proof of Theorem 6 in appendix B only covers epsilon=1.

If it is "easy to show" that the same construction also works for other epsilon, as claimed, then please provide the proof for general epsilon.

Other remarks:- Theoretically, nonconvergence of Adam seems a severe problem.

Can you speculate on why this issue has not prevented its widespread adoption?

Which factors might mitigate the issue in practice?

- Please define g_t \circ g_t and g_{1:T,i}

- I would recommend sticking with standard linear algebra notation for the sqrt and the inverse of a matrix and simply using A^{-1} and A^{1/2} instead of 1/A and sqrt{A}.

- In theorems 1,2,3, I would recommend stating the dimension (d=1) of your counterexamples,

which makes them very nice!

Minor issues:- Check accent on Nicol\`o Cesa-Bianchi in bibliography.

- Near the end of the proof of Theorem 6: I believe you mean Adam suffers a "regret" instead of a "loss" of at least 2C-4.

Also 2C-4=2C-4 is trivial in the second but last display.

fact evaluation fact fact evaluation fact fact fact fact evaluation evaluation evaluation evaluation request fact fact request evaluation request fact evaluation fact fact evaluation evaluation non-arg non-arg evaluation fact fact fact fact evaluation evaluation evaluation evaluation fact evaluation fact request fact request evaluation request request request request request evaluation request request evaluation


This work proposes an approach for transcription factor binding site prediction using a multi-label classification formulation.

It is a very interesting problem

and application and the approach is interesting.

Novelty: The method is quite similar to matching networks (Vinyals, 2016) with a few changes in the matching approach.

As such, in order to establish its broader applicability there should be additional evaluation on other benchmark datasets.

The MNIST performance comparison is inadequate

and there are other papers that do better on it.

They should clearly list what the contributions are w.r.t to the work by Vinyals et al 2016.

They should also cite works that learn embeddings in a multi-label setting such as StarSpace.

Impact: In its current form the paper seems to be most relevant to the computational biology / TFBS community.

However, there is no comparison to the exact networks used in the prior works DeepBind/DeepSea/DanQ/Basset/DeepLift or bidirectional LSTMs.

Further there is no comparison to existing one-shot learning techniques either.

This greatly limits the impact of the work.

For biological impact, a comparison to any of the motif learning approaches that are popular in the biology/comp-bio community will help (for instance, HOMER, FIMO).

Cons: The authors claim they can learn TF-TF interactions and it is one of the main biological contributions,

but there is no evidence of why

(beyond very preliminary evaluation using the Trrust database).

Their examples are 200-bp long which does not mean that all TFs binding in that window are involved in cooperative binding.

The prototype loss is too simplistic to capture co-binding tendencies

and the combinationLSTM is not well motivated.

One interesting source of information they could tap into for TF-TF interactions is CAP-SELEX (Jolma et al, Nature 2015).

One of the main drawbacks is the lack of interpretability of their model where approaches like DanQ/DeepLift etc benefit.

The PWM-like filters in some of the prior works help understand what type of sequence properties contribute to binding events.

Can their model lead to an understanding of this sort?

Evaluation: The empirical evaluation itself is not very strong

as there are only modest improvements over simple baselines.

Further there are no error-bars etc to indicate the variance in their performance numbers.

It will be useful to have a TF-level performance split-up to get an idea of which TFs benefit most.

Clarity: The paper can benefit from more clarity in the technical aspects.

It is hard to follow for anyone not already familiar with matching networks.

The objective function, parameters need to be clearly introduced in one place.

For instance, what is y_i in their multi-label framework?

Various choices are not well motivated; for instance cosine similarity, the value of hyperparameter epsilon.

The prototype vectors are not motif-like at all --

can the authors motivate this aspect better?

fact evaluation evaluation evaluation request evaluation fact request request evaluation fact fact evaluation request fact fact evaluation fact evaluation evaluation request evaluation fact non-arg evaluation evaluation fact request request evaluation request request evaluation evaluation request


[Overview]In this paper, the authors proposed a novel model called MemoryGAN, which integrates memory network with GAN.

As claimed by the authors, MemoryGAN is aimed at addressing two problems of GAN training:

1) difficult to model the structural discontinuity between disparate classes in the latent space;

2) catastrophic forgetting problem during the training of discriminator about the past synthesized samples by the generator.

It exploits the life-long memory network and adapts it to GAN.

It consists of two parts, discriminative memory network (DMN) and Memory Conditional Generative Network (MCGN).

DMN is used for discriminating input samples by integrating the memory learnt in the memory network, and MCGN is used for generating images based on random vector and the sampled memory from the memory network.

In the experiments, the authors evaluated memoryGAN on three datasets, CIFAR-10, affine-MNIST and Fashion-MNIST, and demonstrated the superiority to previous models.

Through ablation study, the authors further showed the effects of separate components in memoryGAN.

[Strengths] 1. This paper is well-written.

All modules in the proposed model and the experiments were explained clearly.

I enjoyed much to read the paper.

2. The paper presents a novel method called MemoryGAN for GAN training.

To address the two infamous problems mentioned in the paper, the authors proposed to integrate a memory network into GAN.

Through memory network, MemoryGAN can explicitly learn the data distribution of real images and fake images.

I think this is a very promising and meaningful extension to the original GAN.

3. With MemoryGAN, the authors achieved best Inception Score on CIFAR-10.

By ablation study, the authors demonstrated each part of the model helps to improve the final performance.

[Comments] My comments are mainly about the experiment part:

1. In Table 2, the authors show the Inception Score of images generated by DCGAN at the last row.

On CIFAR-10, it is ~5.35.

As the authors mentioned, removing EM, MCGCN and Memory will result in a conventional DCGAN.

However, as far as I know, DCGAN could achieve > 6.5 Inception Score in general.

I am wondering what makes such a big difference between the reported numbers in this paper and other papers?

2. In the experiments, the authors set N = 16,384, and M = 512, and z is with dimension 16.

I did not understand why the memory size is such large.

Take CIFAR-10 as the example, its training set contains 50k images.

Using such a large memory size, each memory slot will merely count for several samples.

Is a large memory size necessary to make MemoryGAN work?

If not, the authors should also show ablated study on the effect of different memory size;

If it is true, please explain why is that.

Also, the authors should mention the training time compared with DCGAN.

Updating memory with such a large size seems very time-consuming.

3. Still on the memory size in this model.

I am curious about the results if the size is decreased to the same or comparable number of image categories in the training set.

As the author claimed, if the memory network could learn to cluster training data into different category, we should be able to see some interesting results by sampling the keys and generate categoric images.

4. The paper should be compared with InfoGAN (Chen et al. 2016),

and the authors should explain the differences between two models in the related work.

Similar to MemoryGAN, InfoGAN also did not need any data annotations, but could learn the latent code flexibly.

[Summary]

This paper proposed a new model called MemoryGAN for image generation.

It combined memory network with GAN, and achieved state-of-art performance on CIFAR-10.

The arguments that MemoryGAN could solve the two infamous problem make sense.

As I mentioned above, I did not understand why the authors used such large memory size.

More explanations and experiments  should be conducted to justify this setting.

Overall, I think MemoryGAN opened a new direction of GAN and worth to further explore.

fact fact fact fact fact fact fact fact fact evaluation evaluation evaluation fact fact fact evaluation fact fact non-arg fact fact fact fact non-arg fact evaluation fact fact non-arg request request request fact non-arg non-arg fact request request fact non-arg fact fact evaluation non-arg request evaluation


The authors are motivated by two problems: Inputting non-Euclidean data (such as graphs) into deep CNNs, and analyzing optimization properties of deep networks.

In particular, they look at the problem of maze testing, where, given a grid of black and white pixels, the goal is to answer whether there is a path from a designated starting point to an ending point.

They choose to analyze mazes because they have many nice statistical properties from percolation theory.

For one, the problem is solvable with breadth first search in O(L^2) time, for an L x L maze.

They show that a CNN can essentially encode a BFS,

so theoretically a CNN should be able to solve the problem.

Their architecture is a deep feedforward network where each layer takes as input two images: one corresponding to the original maze (a skip connection), and the output of the previous layer.

Layers alternate between convolutional and sigmoidal.

The authors discuss how this architecture can solve the problem exactly.

The pictorial explanation for how the CNN can mimic BFS is interesting

but I got a little lost in the 3 cases on page 4.

For example, what is r?

And what is the relation of the black/white and orange squares?

I thought this could use a little more clarity.

Though experiments, they show that there are two kinds of minima, depending on whether we allow negative initializations in the convolution kernels.

When positive initializations are enforced, the network can more or less mimic the BFS behavior, but never when initializations can be negative.

They offer a rigorous analysis into the behavior of optimization in each of these cases, concluding that there is an essential singularity in the cost function around the exact solution,

yet learning succumbs to poor optima due to poor initial predictions in training.

I thought this was an impressive paper that looked at theoretical properties of CNNs.

The problem was very well-motivated,

and the analysis was sharp and offered interesting insights into the problem of maze solving.

What I thought was especially interesting is how their analysis can be extended to other graph problems;

while their analysis was specific to the problem of maze solving, they offer an approach -- e.g. that of finding "bugs" when dealing with graph objects -- that can extend to other problems.

I would be excited to see similar analysis of other toy problems involving graphs.

One complaint I had was inconsistent clarity:

while a lot was well-motivated and straightforward to understand,

I got lost in some of the details (as an example, the figure on page 4 did not initially make much sense to me).

Also, in the experiments, the authors mention multiple attempt with the same settings --

are these experiments differentiated only by their initialization?

Finally, there were various typos throughout

(one example is "neglect minimua" on page 2 should be "neglect minima").

Pros: Rigorous analysis,

well motivated problem,

generalizable results to deep learning theory

Cons: Clarity

fact fact fact fact fact fact fact fact fact evaluation evaluation request request request fact fact fact fact evaluation evaluation evaluation evaluation fact request evaluation evaluation evaluation fact request fact request evaluation evaluation fact evaluation


This paper describes an extension to the recently introduced Transformer networks which shows better convergence properties and also improves results on standard machine translation benchmarks.

This is a great paper

-- it introduces a relatively simple extension of Transformer networks which only adds very few parameters and speeds up convergence and achieves better results.

It would have been good to also add a motivation for doing this

(for example, this idea can be interpreted as having a variable number of attention heads which can be blended in and out with a single learned parameter, hence making it easier to use the parameters where they are needed).

Also, it would be interesting to see how important the concatenation weight and the addition weight are relative to each other --

do you possibly get the same results even without the concatenation weight?

A suggested improvement: Please check the references in the introduction and see if you can find earlier ones --

for example, language modeling with RNNs has been done for a very long time, not just since 2017 which are the ones you list;

similar for speech recognition etc. (which probably has been done since 1993!).

fact evaluation fact request fact request request request fact fact


The paper proposes an analysis on different adaptive regularization techniques for deep transfer learning.

Specifically it focuses on the use of an L2-SP condition that constraints the new parameters to be close to the ones previously learned when solving a source task.

+ The paper is easy to read and well organized

+ The advantage of the proposed regularization against the more standard L2 regularization is clearly visible from the experiments

- The idea per se is not new:

there is a list of shallow learning methods for transfer learning based on the same L2 regularization choice

[Cross-Domain Video Concept Detection using Adaptive SVMs, ACM Multimedia 2007]

[Learning categories from few examples with multi model knowledge transfer, PAMI 2014]

[From n to n+ 1: Multiclass transfer incremental learning, CVPR 2013]

I believe this literature should be discussed in the related work section

- It is true that the L2-SP-Fisher regularization was designed for life-long learning cases with a fixed task,

however, this solution seems to work quite well in the proposed experimental settings.

From my understanding L2-SP-Fisher can be considered the best competitor of L2-SP

so I think the paper should dedicate more space to the analysis of their difference and similarities both from the theoretical and experimental point of view.

For instance: --  adding the L2-SP-Fisher results in table 2

--  repeating the experiments of figure 2 and figure 3 with L2-SP-Fisher

fact fact evaluation evaluation evaluation fact reference reference reference request fact evaluation evaluation request request request


This paper essentially uses CycleGANs for Domain Adaptation.

My biggest concern is that it doesn't adequately compare to similar papers that perform adaptation at the pixel level

(eg. Shrivastava et al-'Learning from Simulated and Unsupervised Images through Adversarial Training'

and Bousmalis et al - 'Unsupervised Pixel-level Domain Adaptation with GANs',

two similar papers published in CVPR 2017 -the first one was even a best paper- and available on arXiv since December 2016-before CycleGANs).

I believe the authors should have at least done an ablation study to see if they cycle-consistency loss truly makes a difference on top of these works-that would be the biggest selling point of this paper.

The experimental section had many experiments, which is great.

However I think for semantic segmentation it would be very interesting to see whether using the adapted synthetic GTA5 samples would improve the SOTA on Cityscapes.

It wouldn't be unsupervised domain adaptation,

but it would be very impactful.

Finally I'm not sure the oracle (train on target) mIoU on Table 2 is SOTA,

and I believe the proposed model's performance is really far from SOTA.

Pros: * CycleGANs for domain adaptation!

Great idea!

* I really like the work on semantic segmentation,

I think this is a very important direction

Cons: * I don't think Domain separation networks is a pixel-level transformation-

that's a feature-level transformation,

you probably mean to use Bousmalis et al. 2017.

Also Shrivastava et al is missing from the image-level papers.

* the authors claim that Bousmalis et al, Liu & Tuzel and Shrivastava et al ahve only been shown to work for small image sizes.

There's a recent work by Bousmalis et al. (Using Simulation and Domain Adaptation to Improve Efficiency of Deep Robotic Grasping) that shows these methods working well (w/o cycle-consistency) for settings similar to semantic segmentation at a relatively high resolution.

Also it was mentioned that these methods do not necessarily preserve content, when pixel-da explicitly accounts for that with a task loss (identical to the semantic loss used in this submission)

* The authors talk about the content similarity loss on the foreground in Bousmalis et al. 2017,

but they could compare to this method w/o using the content similarity or using a different content similarity tailored to the semantic segmentation tasks, which would be trivial.

* Math seems wrong in (4) and (6).

(4) should be probably have a minus instead of a plus.

(6) has an argmin of a min,

not sure what is being optimized here.

In fact, I'm not sure if eg you use the gradients of f_T for training the generators?

* The authors mention that the pixel-da approach cross validates with some labeled data.

Although I agree that is not an ideal validation,

I'm not sure if it's equivalent or not the authors' validation setting,

as they don't describe what that is.

* The authors present the semantic loss as novel,

however this is the task loss proposed by the pixel-da paper.

* I didn't understand what pixel-only and feat-only meant in tables 2, 3, 4.

I couldn't find an explanation in captions or in text

fact evaluation reference reference fact request evaluation request fact evaluation evaluation evaluation evaluation evaluation evaluation evaluation fact fact evaluation fact fact fact fact fact evaluation fact evaluation fact evaluation evaluation fact evaluation evaluation fact fact fact fact evaluation


There may be some interesting ideas here,

but I think in many places the mathematical\ndescription is very confusing and/or flawed.

To give some examples:\n\n* Just before section 2.1.1, P(T) = \\prod_{p \\in Path(T)} ... : it's not clear \nat all clear that this defines a valid distribution over trees.

There is an\nimplicit order over the paths in Path(T)

that is simply not defined

(otherwise\nhow for x^p could we decide which symbols x^1 ... x^{p-1} to condition\nupon?)\n\n

\"We can write S -> O | v | \\epsilon...\"

with S, O and v defined as sets.\n

This is certainly non-standard notation,

more explanation is needed.\n\n

\"The observation is generated by the sequence of left most \nproduction rules\".

This appears to be related to the idea of left-most\nderivations in context-free grammars.

But no discussion is given,

and\nthe writing is again vague/imprecise.\n\n

\"Although the above grammar is not, in general, context free\"

- I'm not\nsure what is being referred to here.

Are the authors referring to the underlying grammar,\nor the lack of independence assumptions in the model?

The grammar\nis clearly context-free;

the lack of independence assumptions is a separate\nissue.\n\n

\"In a probabilistic context-free grammar (PCFG), all production rules are\nindependent\":

this is not an accurate statement,

it's not clear what is meant\nby production rules being independent.

More accurate would be to say that\nthe choice of rule is conditionally independent of all other information \nearlier in the derivation, once the non-terminal being expanded is\nconditioned upon.

evaluation evaluation evaluation fact fact evaluation quote fact evaluation request quote evaluation fact evaluation quote non-arg non-arg evaluation fact quote evaluation evaluation request


This paper proposes a model for generating pop music melodies with a recurrent neural network conditioned on chord and part (song section) information.

They train their model on a small dataset and compare it to a few existing models in a human evaluation.

I think this paper has many issues, which I describe below.

As a broad overview, the use of what the authors call "word" representations of notes is not novel (appearing first in BachBot and the PerformanceRNN);

I suspect the model may be outputting sequences from the training set;

and the dataset is heavily constrained in a way that will make producing pleasing melodies easily but heavily limits the possible outputs of the model.

The paper is also missing important references and is confusingly laid out (e.g. introducing a GAN model in a few paragraphs in the experiments).

Specific criticism: - "often producing works that are indistinguishable from human works (Goodfellow et al. (2014); Radford et al. (2016); Potash et al. (2015))." I would definitely not say that any of the cited papers produce anything that could be confused as "real";

e.g. the early GAN papers you cite were not even close (maybe somewhat close for images of bedrooms, which is a limited domain and certainly cannot be considered a "work").

- There are many unsubstantiated claims in the second paragraph.

E.g. "there is yet a certain aspect about it that makes it sound like (or not sound like) human-written music."

What is it?

What evidence do we have that this is true?

" notes in the chorus part generally tend to be more high-pitched"

Really?

Where was this measured?

"music is not merely a series of notes, but entails an overall structure of its own"

Sure, but natural images are not merely a series of pixels either, and they certainly have structure, but we are making lots of good progress modeling them.  Etc.

- Your related work section is lacking.

For example, Eck & Schmidhuber in 2002 proposed using LSTMs for music composition, which is not much later than works from "the early days" despite having not "employed rule or template based approach".

Your note/time offset/duration representation is very similar to that of BachBot (by Liang) and Magenta's PerformanceRNN.

GANs were also previously applied to piano roll generation,

see MuseGAN (Dong et al), MidiNet (Yang et al), etc.

Your critique of Jaques et al. is misleading;

"they defined a number of music-theory based rules to set up the reward function"

is the whole point - this is an optional step which improves results, and there is no reason a priori to think that hand-designing regularizers is better than hand-designing RL objectives.

- The analogy to image captioning models is interesting,

but this type of image captioning model is not only model which is effectively a conditional language model - any sequence-to-sequence model can be looked at this way.

I don't think that these image captioning models are even the most commonly known example,

so I'm not sure why the proposed approach is being proposed in analogy to image captioning.

- I don't think you need to reproduce the LSTM equations in your text, they are well-known.

- You should define early on what you mean by "part",

I think you mean the song's section (verse, chorus, etc)

but I have also heard this used to refer to the different instruments in a song.

I don't think you should expect this term to be known outside of musical communities (e.g. the ICLR community).

- It seems simpler (and more in keeping with the current zeigeist, e.g. the image captioning models you refer to) to replace your HMM with a model that

- The regularization is interesting,

but a simpler way to enforce this constraint would be to just only allow the model to produce notes within that predefined range.

Since you effectively constrain it to an octave,

it would be simple to wrap all notes in your training data into this octave.

This baseline is probably worth comparing to

since it is substantially simpler than your regularizer.

- You write that the softmax cost should have \frac{\partial E}{\partial p_i} \mu added to it for the regularizer.

First, you don't define E anywhere, you only introduce it in its derivative

(and of course you can't "define" the derivative of an expression, it's an analytically computed quantity).

Second, are you sure you mean that the partial derivative should be added, and not the cost C itself?

- Your results showing that human raters preferred your models are impressive,

but you have made the task easier for yourself in various ways:

1) Constraining the training data to pop music

2) Making all of the training data in a single (major) key

3) Effectively limiting the melody range to within a single octave.

- It sounds very much like your model is repeating bars, e.g. it generates a melody of length N bars, then repeats this melody.

Is this something you hard-coded into the model?

It would be very surprising if it learned to exhibit this behavior on its own.

If you hard-coded it into the model, I would expect it to sound better to human raters,

but this is a strong heuristic.

- I'd suggest you provide example melodies from your model in isolation (more like the "varying number of bars" examples) rather than as part of a full music mix

- this makes it easier to judge the quality of the model's output.

- The GAN experiments are interesting but come as a big surprise and are largely orthogonal to the other model;

why not include this in your model description section?

The model and training details are not adequately described

and I don't think it adds much to the paper to include it.

Furthermore it's quite similar to the MidiNet and MuseGAN, so maybe it should be introduced as a baseline instead.

- How did you order the notes for chords?

If three notes occur simultaneously (in a chord), there's no a priory correct way to list them sequentially (two with an interval of length zero between notes).

- "Generated instruments sound fairly in tune individually, confirming that our proposed model is applicable to other instruments as well" Assuming you are still using C-major-only melodies, it's not surprising that the generations sound in tune!

- It is not surprising that your model ends up overfitting

because your dataset is very small,

your model is very powerful,

and your regularizer does not really limit the model's capacity much.

I suspect that your model is overfitting even earlier than you think.

You should check that none of the sequences output by your model appear in the training set.

You could easily compute n-gram overlap of the generated sequences vs. the training set.

At what point did you stop training before running the human evaluation?

If you let your model overfit, then of course it will generate very human-sounding melodies,

but this is not a terribly interesting generative model.

fact fact evaluation fact fact fact evaluation evaluation evaluation fact quote evaluation evaluation quote fact fact quote fact evaluation fact evaluation fact reference evaluation quote fact evaluation fact evaluation evaluation evaluation request evaluation non-arg evaluation request evaluation evaluation fact evaluation request evaluation fact fact fact evaluation evaluation fact fact fact fact evaluation non-arg evaluation evaluation evaluation request evaluation evaluation request fact evaluation request non-arg fact evaluation evaluation evaluation evaluation evaluation fact request request non-arg fact evaluation


This paper generates adversarial examples using the fast gradient sign (FGS) and iterated fast gradient sign (IFGS) methods, but replacing the gradient computation with finite differences or another gradient approximation method.

Since finite differences is expensive in high dimensions,

the authors propose using directional derivatives based on random feature groupings or PCA.

This paper would be much stronger if it surveyed a wider variety of gradient-free optimization methods.

Notably, there's two important black-box optimization baselines that were not included:

simultaneous perturbation stochastic approximation ( https://en.wikipedia.org/wiki/Simultaneous_perturbation_stochastic_approximation), which avoids computing the gradient explicitly, and evolutionary strategies ( https://blog.openai.com/evolution-strategies/ ), a similar method that uses several random directions to estimate a better descent direction.

The gradient approximation methods proposed in this paper may or may not be better than SPSA or ES.

Without a direct comparison, it's hard to know.

Thus, the main contribution of this paper is in demonstrating that gradient approximation methods are sufficient for generating good adversarial attacks and applying those attacks to Clarifai models.

That's interesting and useful to know, but is still a relatively small contribution, making this paper borderline.

I lean towards rejection,

since the paper proposes new methods without comparing to or even mentioning well-known alternatives.

fact fact fact evaluation fact fact evaluation evaluation evaluation evaluation evaluation fact


Many black-box optimization problems are "multi-fidelity", in which it is possible to acquire data with different levels of cost and associated uncertainty.

The training of machine learning models is a common example, in which more data and/or more training may lead to more precise measurements of the quality of a hyperparameter configuration.

This has previously been referred to as a special case of "multi-task" Bayesian optimization, in which the tasks can be constructed to reflect different fidelities.

The present paper examines this construction with three twists: using the knowledge gradient acquisition function, using batched function evaluations, and incorporating derivative observations.

Broadly speaking, the idea is to allow fidelity to be represented as a point in a hypercube and then include this hypercube as a covariate in the Gaussian process.

The knowledge gradient acquisition function then becomes "knowledge gradient per unit cost" the KG equivalent to the "expected improvement per unit cost" discussed in Snoek et al (2012),

although that paper did not consider treating fidelity separately.

I don't understand the claim that this is "the first multi-fidelity algorithm that can leverage gradients".

Can't any Gaussian process model use gradient observations trivially, as discussed in the Rasmussen and Williams book?

Why can't any EI or entropy search method also use gradient observations?

This doesn't usually come up in hyperparameter optimization,

but it seems like a grandiose claim.

Similarly, although I don't know of a paper that explicitly does "A + B" for multi-fidelity BO and parallel BO,

it is an incremental contribution to combine them, not least because no other parallel BO methods get evaluated as baselines.

Figure 1 does not make sense to me.

How can the batched algorithm outperform the sequential algorithm on total cost?

The sequential cfKG algorithm should always be able to make better decisions with its remaining budget than 8-cfKG.

Is the answer that "cost" here means "wall-clock time when parallelism is available"?

If that's the case, then it is necessary to include plots of parallelized EI, entropy search, and KG.

The same is true for Figure 2; other parallel BO algorithms need to appear.

fact fact fact fact fact fact fact evaluation fact non-arg fact evaluation evaluation evaluation evaluation evaluation fact non-arg request request


In this paper, the authors trains a large number of MNIST classifier networks with differing attributes (batch-size, activation function, no. layers etc.)

and then utilises the inputs and outputs of these networks to predict said attributes successfully.

They then show that they are able to use the methods developed to predict the family of Imagenet-trained networks and use this information to improve adversarial attack.

I enjoyed reading this paper.

It is a very interesting set up, and a novel idea.

A few comments:The paper is easy to read, and largely written well.

The article is missing from the nouns quite often though

so this is something that should be amended.

There are a few spelling slip ups

("to a certain extend" --> "to a certain extent",

"as will see" --> "as we will see")

It appears that the output for kennen-o is a discrete probability vector for each attribute, where each entry corresponds to a possibility

(for example, for "batch-size" it is a length 3 vector where the first entry corresponds to 64, the second 128, and the third 256).

What happens if you instead treat it as a regression task, would it then be able to hint at intermediates (a batch size of 96) or extremes (say, 512).

A flaw of this paper is that kennen-i and io appear to require gradients from the network being probed (you do mention this in passing), which realistically you would never have access to.

(Please do correct me if I have misunderstood this)

It would be helpful if Section 4 had a paragraph as to your thoughts regarding why certain attributes are easier/harder to predict.

Also, the caption for Table 2 could contain more information regarding the network outputs.

You have jumped from predicting 12 attributes on MNIST to 1 attribute on Imagenet.

It could be beneficial to do an intermediate experiment (a handful of attributes on a middling task).

I think this paper should be accepted

as it is interesting and novel.

Pros ------ - Interesting idea

- Reads well

- Fairly good experimental results

Cons ------ - kennen-i seems like it couldn't be realistically deployed

- lack of an intermediate difficulty task

fact fact fact evaluation evaluation evaluation evaluation request fact request request fact fact request fact non-arg request request fact request evaluation evaluation evaluation evaluation evaluation evaluation fact


This paper proposes a ranking-based similarity metric for distributional semantic models.

The main idea is to learn "baseline" word embeddings, retrofitting those and applying localized centering, to then calculate similarity using a measure called "Ranking-based Exponential Similarity Measure" (RESM), which is based on the recently proposed APSyn measure.

I think the work has several important issues:

1. The work is very light on references.

There is a lot of previous work on evaluating similarity in word embeddings (e.g. Hill et al, a lot of the papers in RepEval workshops, etc.); specialization for similarity of word embeddings (e.g. Kiela et al., Mrksic et al., and many others); multi-sense embeddings (e.g. from Navigli's group); and the hubness problem (e.g. Dinu et al.).

For the localized centering approach, Hara et al.'s introduced that method.

None of this work is cited, which I find inexcusable.

2. The evaluation is limited, in that the standard evaluations (e.g. SimLex would be a good one to add, as well as many others, please refer to the literature) are not used and there is no comparison to previous work.

The results are also presented in a confusing way,

with the current state of the art results separate from the main results of the paper.

It is unclear what exactly helps, in which case, and why.

3. There are technical issues with what is presented, with some seemingly factual errors.

For example, "In this case we could apply the inversion, however it is much more convinient [sic] to take the negative of distance. Number 1 in the equation stands for the normalizing, hence the similarity is defined as follows"

- the 1 does not stand for normalizing, that is the way to invert the cosine distance

(put differently, cosine distance is 1-cosine similarity, which is a metric in Euclidean space due to the properties of the dot product).

Another example, "are obtained using the GloVe vector, not using PPMI"

- there are close relationships between what GloVe learns and PPMI,

which the authors seem unaware of (see e.g. the GloVe paper and Omer Levy's work).

4. Then there is the additional question, why should we care?

The paper does not really motivate why it is important to score well on these tests:

these kinds of tests are often used as ways to measure the quality of word embeddings,

but in this case the main contribution is the similarity metric used *on top* of the word embeddings.

In other words, what is supposed to be the take-away, and why should we care?

As such, I do not recommend it for acceptance -

it needs significant work before it can be accepted at a conference.

Minor points:- Typo in Eq 10

- Typo on page 6 (/cite instead of \cite)

fact fact evaluation evaluation fact fact evaluation evaluation evaluation fact evaluation evaluation quote fact fact quote evaluation evaluation evaluation evaluation fact evaluation request evaluation evaluation fact fact


# Summary This paper presents a new external-memory-based neural network (Neural Map) for handling partial observability in reinforcement learning.

The proposed memory architecture is spatially-structured so that the agent can read/write from/to specific positions in the memory.

The results on several memory-related tasks in 2D and 3D environments show that the proposed method outperforms existing baselines such as LSTM and MQN/FRMQN.

[Pros] - The overall direction toward more flexible/scalable memory is an important research direction in RL.

- The proposed memory architecture is new.

- The paper is well-written.

[Cons] - The proposed memory architecture is new but a bit limited to 2D/3D navigation tasks.

- Lack of analysis of the learned memory behavior.

# Novelty and Significance The proposed idea is novel in general.

Though [Gupta et al.] proposed an ego-centric neural memory in the RL context,

the proposed memory architecture is still new in that read/write operations are flexible enough for the agent to write any information to the memory,

whereas [Gupta et al.] designed the memory specifically for predicting free space.

On the other hand, the proposed method is also specific to navigation tasks in 2D or 3D environment,

which is hard to apply to more general memory-related tasks in non-spatial environments.

But, it is still interesting to see that the ego-centric neural memory works well on challenging tasks in a 3D environment.

# Quality The experiment does not show any analysis of the learned memory read/write behavior especially for ego-centric neural map and the 3D environment.

It is hard to understand how the agent utilizes the external memory without such an analysis.

# Clarity The paper is overall clear and easy-to-follow except for the following.

In the introduction section, the paper claims that "the expert must set M to a value that is larger than the time horizon of the currently considered task" when mentioning the limitation of the previous work.

In some sense, however, Neural Map also requires an expert to specify the proper size of the memory based on prior knowledge about the task.

fact fact fact evaluation evaluation evaluation evaluation fact evaluation fact fact fact fact evaluation evaluation fact evaluation evaluation fact evaluation


The authors propose a model for learning physical interaction skills through trial and error.

They use end-to-end deep reinforcement learning - the DQN model - including the task goal as an input in order to to improve generalization over several tasks, and shaping the reward depending on the visual differences between the goal state and the current state.

They show that the task performance of their model is better than the DQN on two simulated tasks.

The paper is well-written, clarity is good,

it could be slightly improved by updating the title "Toy example with Goal integration" to make it consistent with the naming "navigation task" used elsewhere.

If the proposed model is new given the reviewer's knowledge, the contribution is small.

The biggest change compared to the DQN model is the addition of information in the input.

The authors initially claim that "In this paper, [they] study how an artificial agent can autonomously acquire this intuition through interaction with the environment",

however the proposed tasks present little to no realistic physical interaction:

the navigation task is a toy problem where no physics is simulated.

In the stacking task, only part of the simulation actually use the physical simulation result.

Given that machine learning methods are in general good at finding optimal policies that exploit simulation limitations,

this problem seems a threat to the significance of this work.

The proposed GDQN model shows better performance than the DQN model.

However, as the authors do not provide in-depth analysis of what the network learns (e.g. by testing policies in the absence of an explicit goal),

it is difficult to judge if the network learnt a meaningful representation of the world's physics.

This limitation along with potential other are not discussed in the paper.

Finally, more than a third (10/26) references point to Arxiv papers.

Despite Arxiv definitely being an important tool for paper availability, it is not peer-reviewed and there are also work that are non-finished or erroneous.

It is thus a necessary condition that all Arxiv references are replaced by the peer-reviewed material when it exist (e.g. Lerer 2016 in ICML or Denil 2016 in ICLR 2017), once again to strengthen the author's claim.

fact fact fact evaluation request evaluation evaluation quote fact fact fact fact evaluation fact fact evaluation evaluation fact fact request


* sec.2.2 is about label-preserving translation

and many notations are introduced.

However, it is not clear what label here refers to,

and it does not shown in the notation so far at all.

Only until the end of sec.2.2, the function F(.) is introduced and its revelation - Google Search as label function is discussed only at Fig.4 and sec.2.3.

* pp.5 first paragraph: when assuming D_X and D_Y being perfect, why L_GAN_forward = L_GAN_backward = 0?

To trace back, in fact it is helpful to have at least a simple intro/def. to the functions D(.) and G(.) of Eq.(1).

* Somehow there is a feeling that the notations in sec.2.1 and sec.2.2 are not well aligned.

It is helpful to start providing the math notations as early as sec.2.1,

so labels, pseudo labels, the algorithm illustrated in Fig.2 etc. can be consistently integrated with the rest notations.

* F() is firstly shown in Fig.2 the beginning of pp.3, and is mentioned in the main text as late as of pp.5.

* Table 2: The CNN baseline gives an error rate of 7.80

while the proposed variants are 7.73 and 7.60 respectively.

The difference of 0.07/0.20 are not so significant.

Any explanation for that?

Minor issues: * The uppercase X in the sentence before Eq.(2) should be calligraphic X

fact evaluation evaluation fact fact request evaluation evaluation evaluation evaluation fact fact fact evaluation request request


This paper combines the tensor contraction method and the tensor regression method and applies them to CNN.

This paper is well written and easy to read.

However, I cannot find a strong or unique contribution from this paper.

Both of the methods (tensor contraction and tensor decomposition) are well developed in the existing studies,

and combining these ideas does not seem non-trivial.

--Main question Why authors focus on the combination of the methods?

Both of the two methods can perform independently.

Is there a special synergy effect?

--Minor question The performance of the tensor contraction method depends on a size of tensors.

Is there any effective way to determine the size of tensors?

fact evaluation evaluation evaluation evaluation evaluation fact non-arg fact non-arg


The manuscript proposes a new framework for inference in RNN based upon the Bayes by Backprop (BBB) algorithm.

In particular, the authors propose a new framework to "sharpen" the posterior.

In particular, the hierarchical prior in (6) and (7) frame an interesting modification to directly learning a multivariate normal variational approximation.

In the experimental results, it seems clear that this approach is beneficial, but it's not clear as to why.

In particular, how does the variational posterior change as a result of the hierarchical prior?

It seems that (7) would push the center of the variational structure back towards the MAP point and reduces the variance of the output of the hierarchical prior;

however, with the two layers in the prior it's unclear what actually is happening.

Carefully explaining *what* the authors believe is happening and exploring how it changes the variational approximation in a classic modeling framework would be beneficial to understanding the proposed change and evaluating it.

As a final point, the authors state, "as long as the improvement along the gradient is great than the KL loss incurred...this method is guaranteed to make progress towards optimizing L."

Do the authors mean that the negative log-likelihood will be improved in this case?

Or the actual optimization?

Improving the negative log-likelihood seems straightforward,

but I am confused by what the authors mean by optimization.

The new evaluation metric proposed in Section 6.1.1 is confusing,

and I do not understand what the metric is trying to capture.

This needs significantly more detail and explanation.

Also, it is unclear to me what would happen when you input data examples that are opposite to the original input sequence;

in particular, for many neural networks the predictions are unstable outside of the input domain and inputting infeasible data leads to unusable outputs.

It's completely feasible that these outputs would just be highly uncertain,

and I'm not sure how you can ascribe meaning to them.

The authors should not compare to the uniform prior as a baseline for entropy.

It's much more revealing to compare it to the empirical likelihoods of the words.

fact evaluation evaluation evaluation evaluation evaluation evaluation request quote request request evaluation evaluation evaluation evaluation request evaluation evaluation evaluation evaluation request request


The authors perform a set of experiments in which they inspect the Hessian matrix of the loss of a neural network, and observe that most of the eigenvalues are very close to zero.

This is a potentially important observation,

and the experiments were well worth performing,

but I don't find them fully convincing

(partly because I was confused by the presentation).

They perform four sets of experiments:

1) In section 3.1, they show on simulated data that for data drawn from k clusters, there are roughly k significant eigenvalues in the Hessian of the solution.

2) In section 3.2, they show on MNIST that the solution contains few large eigenvalues, and also that there are negative eigenvalues.

3) In section 3.3, they show (again on MNIST) that at their respective solutions, large batch and small batch methods find solutions with similar numbers of large eigenvalues, but that for the large batch method the magnitudes are larger.

4) In section 4.1, they train (on CIFAR10) using a large batch method, and then transition to a small batch method, and argue that the second solution appears to be better than the first, but that they are a part of the same basin

(since linearly while interpolating between them they don't run into any barriers).

I'm not fully convinced by the second and third experiments,

partly because I didn't fully understand the plots (more on this below),

but also because it isn't clear to me what we should expect from the spectrum of a Hessian,

so I don't know whether the observed specra have fewer large eigenvalues, or more large eigenvalues, then would be "natural".

In other words, there isn't a *baseline*.

For the fourth experiment, it's unsurprising that the small batch method winds up in a different location in the same basin as the large batch method,

since it was initialized to the large batch method's solution

(and it doesn't appear to me, in figure 9, that the small batch solution is significantly different).

Section 2.1 is said to contain an argument that the second term of equation 5 can be ignored, but only says that if \ell' and \nabla^2 of f are uncorrelated, then it can be ignored.

I don't see any reason that these two quantities should be correlated,

but this is not an argument that they are uncorrelated.

Also, it isn't clear to me where this approximation was used--everywhere?

In section 3.2, it sounds as if the exact Hessian is used,

and at the end of this section the authors say that figure 6 demonstrates that the effect of this second term is small,

but I don't see why this is,

and it isn't explained.

My main complaint is that I had a great deal of difficulty interpreting the plots:

it often wasn't clear to me what exactly was being plotted,

and most of the language describing them was frustratingly vague.

For example, figure 6 is captioned "left edge of the spectrum, eigenvalues are scaled by their ratio".

The text explains that "left edge of the spectrum" means "small but negative eigenvalues"

(this would be better in the caption),

but what are the ratios?

Ratio of what to what?

I think it would greatly enhance clarity if every plot caption described exactly, and unambiguously, what quantities were plotted on the horizontal and vertical axes.

Some minor notes:There are a number of places where "it's" is used, where it should be "its".

In the introduction, the definition of \mathcal{L}' is slightly confusing,

since it's an expectation,

but the use of "'" makes one expect a derivative.

Perhaps use \hat{\mathcal{L}} for the empirical loss, and \mathcal{L} for the expected one?

On the bottom of page 4, "if \ell' and \nabla f are not correlated": I think the \nabla should be \nabla^2.

It's "principal components", not "principle components".

fact evaluation evaluation evaluation evaluation fact fact fact fact fact fact evaluation evaluation evaluation evaluation fact evaluation fact evaluation fact fact non-arg evaluation evaluation fact evaluation fact evaluation evaluation evaluation fact fact evaluation evaluation evaluation request request evaluation fact evaluation request request request


The paper "IMPROVING SEARCH THROUGH A3C REINFORCEMENT LEARNING BASED CONVERSATIONAL AGENT" proposes to define an agent to guide users in information retrieval tasks.

By proposing refinements of the query, categorizations of the results or some other bookmarking actions, the agent is supposed to help the user in achieving his search.

The proposed agent is learned via reinforcement learning.

My concern with this paper is about the experiments that are only based on simulated agents, as it is the case for learning.

While it can be questionable for learning

(but we understand why it is difficult to overcome),

it is very problematic for the experiments to not have anything that demonstrates the usability of the approach in a real-world scenario.

I have serious doubts about the performances of such an artificially learned approach for achieving real-world search tasks.

Also, for me the experimental section is not sufficiently detailed, which lead to not reproducible results.

Moreover, authors should have considered baselines

(only the two proposed agents are compared which is clearly not sufficient).

Also, both models have some issues from my point of view.

First, the Q-learning methods looks very complex:

how could we expect to get an accurate model with 10^7 states ?

No generalization about the situations is done here,

examples of trajectories have to be collected for each individual considered state,

which looks very huge (especially if we think about the number of possible trajectories in such an MDP).

The second model is able to generalize from similar situations thanks to the neural architecture that is proposed.

However, I have some concerns about it:

why keeping the history of actions in the inputs since it is captured by the LSTM cell ?

It is a redondant information that might disturb the process.

Secondly, the proposed loss looks very heuristic for me,

it is difficult to understand what is really optimized here.

Particularly, the loss entropy function looks strange to me.

Is it classical ?

Are there some references of such a method to maintain some exploration ability.

I understand the need of exploration,

but including it in the loss function reduces the interpretability of the objective

(wouldn't it be preferable to use a more classical loss but with an epsilon greedy policy?).

Other remarks:    - In the begining of "varying memory capacity" section, what is "100, 150 and 250" ?

Time steps ?

What is the unit ?

Seconds ?

- I did not understand the "Capturing seach context at local and global level" at all

- In the loss entropy formula, the two negation signs could be removed

- Wouldn't it be possible to use REINFORCE or other policy gradient method rather than roll-outs used in the paper (which lead to biased gradient updates) ?

fact fact fact evaluation evaluation evaluation evaluation evaluation evaluation request evaluation evaluation evaluation evaluation fact fact evaluation fact evaluation evaluation evaluation evaluation evaluation evaluation request request evaluation evaluation evaluation non-arg non-arg non-arg non-arg evaluation request request


This paper proposed to combine three kinds of data sources: real, simulated and unlabeled, to help solve "small" data issue occurring in packet stream.

A directed information flow graph was constructed,

a multi-headed network was trained by using Keras and GAN library.

Its use on the packet sequence classification can archive comparable accuracy while relieve operation engineers from heavy background learning.

The presentation of this paper can be improved.

* With the missing citations as "(?)" and not clearly defined concepts, including property of function H (any function? convex?) in (3),

full name of TCP/abbr of GAN when first appear, etc.

reader might need to make guesses to follow.

* P2: You can draw your audience by expanding the "related work" like a story:

more background of GAN etc. and one or two highlight formula to help clear the idea

* P3: What's the purpose of inserting "dummy packets to denote the timeout between two time stamps"?

* P3: Help sell to "non-engineer" by maybe having image example or even plainer language to describe the meaning (deep difference/purpose) of "3 levels of feature engineering"; and when addressing features, mentioned as 1,2,3, while in Table 1, shown as Feature=0,1,2;

* P6: section 4.2 mentioned "only metrics cared by operators", is this what you mean by "relieve operation engineers ...",

and which is or how to decide the cutoff accuracy the engineers should make a Go or No Go decision?

fact fact fact evaluation request request request evaluation request request non-arg request non-arg non-arg


This paper focuses on the problem of \"machine teaching\",

i.e., how to select a good strategy to select training data points to pass to a machine learning algorithm, for faster learning.

The proposed approach leverages reinforcement learning by defining the reward as how fast the learner learns,

and use policy gradient to update the teacher parameters.

I find the definition of the \"state\" in this case very interesting.

The experimental results seem to show that such a learned teacher strategy makes machine learning algorithms learn faster.

\n\nOverall I think that this paper is decent.

The angle the authors took is interesting (essentially replacing one level of the bi-level optimization problem in machine teaching works with a reinforcement learning setup).

The problem formulation is mostly reasonable,

and the evaluation seems quite convincing.

The paper is well-written:

I enjoyed the mathematical formulation (Section 3).

The authors did a good job of using different experiments (filtration number analysis, and teaching both the same architecture and a different architecture) to intuitively explain what their method actually does.

\n\nAt the same time, though, I see several important issues that need to be addressed if this paper is to be accepted.

Details below. \n\n1. As much as I enjoyed reading Section 3, it is very redundant.

In some cases it is good to outline a powerful and generic framework

(like the authors did here with defining \"teaching\" in a very broad sense, including selecting good loss functions and hypothesis spaces)

and then explain that the current work focuses on one aspect (selecting training data points).

However, I do not see it being the case here.

In my opinion, selecting good loss functions and hypothesis spaces are much harder problems than data teaching - except maybe when one use a pre-defined set of possible loss functions and select from it.

But that is not very interesting

(if you can propose new loss functions, that would be way cooler).

I also do not see how to define an intuitive set of \"states\" in that case.

Therefore, I think this section should be shortened.

I also think that the authors should not discuss the general framework and rather focus on \"data teaching\",

which is the only focus of the current paper.

The abstract and introduction should also be modified accordingly to more honestly reflect the current contributions.

\n2. The authors should do a better job at explaining the details of the state definition,

especially the student model features and the combination of data and current learner model.

\n3. There is only one definition of the reward – related to batch number when the accuracy first exceeds a threshold.

Is accuracy stable,

can it drop back down below the threshold in the next epoch?

The accuracy on a held-out test set is not guaranteed to be monotonically increasing, right?

Is this a problem in practice (it seems to happen on your curves)?

What about other potential reward definitions?

And what would they potentially lead to?

\n4. Experimental results are averaged over 5 repeated runs

- a bit too small in my opinion.

\n5. Can the authors show convergence of the teacher parameter \\theta?

I think it is important to see how fast the teacher model converges, too.

\n6. In some of your experiments, every training method converges to the same accuracy after enough training (Fig.2b), while in others, not quite (Fig. 2a and 2c).

Why is this the case?

Does it mean that you have not run enough iterations for the baseline methods?

My intuition is that if the learner algorithm is convex, then ultimately they will all get to the same accuracy level,

so the task is just to get there quicker.

I understand that since the learner algorithm is an NN, this is not the case –

but more explanation is necessary here –

does your method also reduces the empirical possibility to get stuck in local minima?

\n7. More explanation is needed towards Fig.4c.

In this case, using a teacher model trained on a harder task (CIFAR10) leads to much improved student training on a simpler task (MNIST). Why?\n

8. Although in terms of \"effective training data points\" the proposed method outperforms the other methods,

in terms of time (Fig.5) the difference between it and say, NoTeach, is not that significant (especially at very high desired accuracy).

More explanation needed here.

\n\nRead the rebuttal and revision and slightly increased my rating.

fact fact fact fact evaluation evaluation evaluation evaluation evaluation evaluation evaluation evaluation evaluation evaluation evaluation request fact request evaluation evaluation evaluation request evaluation request request evaluation request request request fact non-arg request non-arg non-arg non-arg non-arg fact evaluation request request fact non-arg non-arg evaluation evaluation evaluation request non-arg request non-arg fact evaluation request non-arg


The authors propose an extension to CNN using an autoregressive weighting for asynchronous time series applications.

The method is applied to a proprietary dataset as well as a couple UCI problems and a synthetic dataset, showing improved performance over baselines in the asynchronous setting.

This paper is mostly an applications paper.

The method itself seems like a fairly simple extension for a particular application,

although perhaps the authors have not clearly highlighted details of methodological innovation.

I liked that the method was motivated to solve a real problem, and that it does seem to do so well compared to reasonable baselines.

However, as an an applications paper, the bread of experiments are a little bit lacking

-- with only that one potentially interesting dataset, which happens to proprietary.

Given the fairly empirical nature of the paper in general, it feels like a strong argument should be made, which includes experiments, that this work will be generally significant and impactful.

The writing of the paper is a bit loose with comments like:

“Besides these and claims of secretive hedge funds (it can be marketing surfing on the deep learning hype), no promising results or innovative architectures were publicly published so far, to the best of our knowledge.”

Parts of the also appear rush written, with some sentences half finished:

“"ues of x might be heterogenous, hence On the other hand, significance network provides data-dependent weights for all regressors and sums them up in autoregressive manner.””

As a minor comment, the statement

“however, due to assumed Gaussianity they are inappropriate for financial datasets, which often follow fat-tailed distributions (Cont, 2001).”

Is a bit too broad.

It depends where the Gaussianity appears.

If the likelihood is non-Gaussian, then it often doesn’t matter if there are latent Gaussian variables.

fact fact evaluation evaluation evaluation evaluation evaluation fact request evaluation quote evaluation quote non-arg quote evaluation fact fact


The authors analyze show theoretical shortcomings in previous methods of explaining neural networks and propose an elegant way to remove these shortcomings in their methods PatternNet and PatternAttribution.

The quest of visualizing neural network decision is now a very active field with many contributions.

The contribution made by the authors stands out due to its elegant combination of theoretical insights and improved performance in application.

The work is very detailed and reads very well.

I am missing at least one figure with comparison with more state-of-the-art methods

(e.g. I would love to see results from the method by Zintgraf et al. 2017 which unlike all included prior methods seems to produce much crisper visualizations and also is very related because it learns from the data, too).

Minor questions and comments:* Fig 3: Why is the random method so good at removing correlation from fc6?

And the S_w even better?

Something seems special about fc6.

* Fig 4: Why is the identical estimator better than the weights estimator and that one better than S_a?

* It would be nice to compare the image degradation experiment with using the ranking provided by the work from Zintgraf which should by definition function as a kind of gold standard

* Figure 5, 4th row (mailbox): It looks like the umbrella significantly contributes to the network decision to classify the image as "mailbox" which doesn't make too much sense.

Is is a problem of the visualization  (maybe there is next to no weight on the umbrella), of PatternAttribution or a strange but interesting a artifact of the analyzed network?

* page 8 "... closed form solutions (Eq (4) and Eq. (7))"

The first reference seems to be wrong.

I guess Eq 4. should instead reference the unnumbered equation after Eq. 3.

fact evaluation evaluation evaluation request request request request evaluation request request evaluation evaluation quote fact request


The paper presents a way to regularize a sequence generator by making the hidden states also predict the hidden states of an RNN working backward.

Applied to sequence-to-sequence networks, the approach requires training one encoder, and two separate decoders, that generate the target sequence in forward and reversed orders.

A penalty term is added that forces an agreement between the hidden states of the two decoders.

During model evaluation only the forward decoder is used, with the backward operating decoder discarded.

The method can be interpreted to generalize other recurrent network regularizers, such as putting an L2 loss on the hidden states.

Experiments indicate that the  approach is most successful when the regularized RNNs are conditional generators, which emit sequences of low entropy, such as decoders of a seq2seq speech recognition network.

Negative results were reported when the proposed regularization technique was applied to language models, whose output distribution has more entropy.

The proposed regularization is evaluated with positive results on a speech recognition task and on an  image captioning task, and with negative results (no improvement, but also no deterioration) on a language modeling and sequential MNIST digit generation tasks.

I have one question about baselines: is the proposed approach better than training to forward generators and force an agreement between them (in the spirit of the concurrent ICLR submission https://openreview.net/forum?id=rkr1UDeC-)?

Also, would using the backward RNN, e.g. for rescoring, bring another advantage?

In other words, what is (and is there) a gap between an ensemble of a forward and backward rnn and the forward-rnn only, but trained with the state-matching penalty?

Quality:The proposed approach is well motivated

and the experiments show the limits of applicability range of the technique.

Clarity:The paper is clearly written.

Originality:The presented idea seems novel.

Significance:The method may prove to be useful to regularize recurrent networks,

however I would like to see a comparison with ensemble methods.

Also, as the authors note the method seems to be limited to conditional sequence generators.

Pros and cons:Pros: the method is simple to implement,

the paper lists for what kind of datasets it can be used.

Cons: the method needs to be compared with typical ensembles of models going only forward in time,

it may turn that it using the backward RNN is not necessary

fact fact fact fact fact fact fact fact non-arg non-arg non-arg evaluation fact evaluation evaluation evaluation request fact evaluation fact request evaluation


The paper proposes a CNN-based based approach for speech processing using raw waveforms as input.

An analysis of convolution and pooling layers applied on waveforms is first presented.

An architecture called SimpleNet is then presented and evaluated on two speech tasks: emotion recognition and gender classification.

This paper propose a theoretical analysis of convolution and pooling layers to motivate the SimpleNet architecture.

To my understanding, the analysis is flawed (see comments below).

The SimpleNet approach is interesting but not sufficiently backed with experimental results.

The network analysis is minimal and provides almost no insights.

I therefore recommend to reject the paper.

Detailed comments:Section 1:* “Therefore, it remains unknown what actual features CNNs learn from waveform”.

This is not true,

several works on speech recognition have shown that a convolution layer taking raw speech as input can be seen as a bank of learned filters.

For instance in the context of speech recognition, [9] showed that the filters learn phoneme-specific responses,

[10] showed that the learned filters are close to Mel filter banks

and [7] showed that the learned filters are related to MRASTA features and Gabor filters.

The authors should discuss these previous works in the paper.

Section 2:* Section 2.1 seems unnecessary,

I think it’s safe to assume that the Shannon-Nyquist theorem and the definition of convolution are known by the reader.

* Section 2.2.2 & 2.2.3: I don't follow the justification that stacking convolutions are not needed:

the example provided is correct if two convolutions are directly stacked without non-linearity, but the conclusion does not hold with a non-linearity and/or a pooling layer between the convolutions:

two stacked convolutions with non-linearities are not equivalent to a single convolution.

To my understanding, the same problem is present for the pooling layer:

the presented conclusion that pooling introduces aliasing is only valid for two directly stacked pooling layers and is not correct for stacked blocks of convolution/pooling/non-linearity.

* Section 2.2.5: The ReLU can be seen as a half-wave rectifier if it is applied directly to the waveform.

However, it is usually not the case

as it is applied on the output of the convolution and/or pooling layers. Therefore I don’t see the point of this section.

* Section 2.2.6: In this section, the authors discuss the differences between spectrogram-based and waveforms-based approaches, assuming that spectrogram-based approach have fixed filters.

But spectrogram can also be used as input to CNNs (i.e. using learned filters) for instance in speech recognition [1] or emotion recognition [11].

Thus the comparison could be more interesting if it was between spectrogram-based and raw waveform-based approaches when the filters are learned in both cases.

Section 3:* Figure 4 is very interesting,

and is in my opinion a stronger motivation for SimpleNet that the analysis presented in Section 2.

* Using known filterbanks such as Mel or Gammatone filters as initialization point for the convolution layer is not novel and has been already investigated in [7,8,10] in the context of speech recognition.

Section 4:* On emotion recognition, the results show that the proposed approach is slightly better,

but there is some issues: the average recall metric is usually used for this task due to class imbalance (see [1] for instance).

Could the authors provide results with this metric ?

Also IEMOCAP is a well-used corpus for this task,

could the authors provide some baselines performance for comparison (e.g. [11]) ?

* For gender classification, there is no gain from SimpleNet compared to the baselines.

The authors also mention that some utterances have overlapping speech.

These utterances are easy to find from the annotations provided with the corpus,

so it should be easy to remove them for the train and test set.

Overall, in the current form, the results are not convincing.

* Section 4.3: The analysis is minimal:

it shows that filters changed after training (as already presented in Figure 4).

I don't follow completely the argument that the filters should focus on low frequency.

It is more informative,

but one could expect that the filters will specialized, thus some of them will focus on high frequencies, to model the high frequency events such as consonants or unvoiced event.

It could be very interesting to relate the learned filters to the labels:

are some filters learned to model specific emotions ?

For gender classification, are some filters focusing on the average pitch frequency of male and female speaker ?

* Finally, it would be nice to see if the claims in Section 2 about the fact that only one convolution layer is needed and that stacking pooling layers can hurt the performance are verified experimentally: for instance, experiments with more than one pair of convolution/pooling could be presented.

Minor comments:* More references for raw waveforms-based approach for speech recognition should be added [3,4,6,7,8,9] in the introduction.

* I don’t understand the first sentence of the paper: “In the field of speech and audio processing, due to the lack of tools to directly process high dimensional data …”.

Is this also true for any pattern recognition fields ?

* For the MFCCs reference in 2.2.2, the authors should cite [12].

* Figure 6: Only half of the spectrum should be presented.

References: [1] H. Lee, P. Pham, Y. Largman, and A. Y. Ng. Unsupervised feature learning for audio classification using convolutional deep belief networks. In Advances in Neural Information Processing Systems 22, pages 1096–1104, 2009.

[2] Schuller, Björn, Stefan Steidl, and Anton Batliner. "The interspeech 2009 emotion challenge." Tenth Annual Conference of the International Speech Communication Association. 2009.

[3] N. Jaitly, G. Hinton, Learning a better representation of speech sound waves using restricted Boltzmann machines, in: Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2011, pp. 5884–5887.

[4] D. Palaz, R. Collobert, and M. Magimai.-Doss. Estimating Phoneme Class Conditional Probabilities from Raw Speech Signal using Convolutional Neural Networks, INTERSPEECH 2013, pages 1766–1770.

[5] Van den Oord, Aaron, Sander Dieleman, and Benjamin Schrauwen. "Deep content-based music recommendation." Advances in neural information processing systems. 2013.

[6] Z.Tuske, P.Golik, R.Schluter, H.Ney, Acoustic Modeling with Deep Neural Networks Using Raw Time Signal for LVCSR, in: Proceedings of the Annual Conference of the International Speech Communication Association (INTERSPEECH), Singapore, 2014, pp. 890–894.

[7] P. Golik, Z. Tuske, R. Schlu ̈ter, H. Ney, Convolutional Neural Networks for Acoustic Modeling of Raw Time Signal in LVCSR, in: Proceedings of the Annual Conference of the International Speech Communication Association (INTERSPEECH), 2015, pp. 26–30.

[8] Yedid Hoshen and Ron Weiss and Kevin W Wilson, Speech Acoustic Modeling from Raw Multichannel Waveforms, International Conference on Acoustics, Speech, and Signal Processing, 2015.

[9] D. Palaz, M. Magimai-Doss, and R. Collobert. Analysis of CNN-based Speech Recognition System using Raw Speech as Input, INTERSPEECH 2015, pages 11–15.

[10] T. N. Sainath, R. J. Weiss, A. Senior, K. W. Wilson, and O. Vinyals. Learning the Speech Front-end With Raw Waveform CLDNNs. Proceedings of the Annual Conference of the International Speech Communication Association (INTERSPEECH), 2015.

[11] Satt, Aharon & Rozenberg, Shai & Hoory, Ron. (2017). Efficient Emotion Recognition from Speech Using Deep Learning on Spectrograms. 1089-1093. Interspeech 2017.

[12] S. Davis and P. Mermelstein. Comparison of parametric representations for monosyllabic word recognition in continuously spoken sentences. IEEE Transactions on Acoustics, Speech and Signal Processing, 28(4):357–366, 1980.

fact fact fact fact evaluation evaluation evaluation evaluation quote fact fact fact fact fact request evaluation evaluation evaluation fact fact fact fact fact fact fact fact fact request evaluation evaluation fact evaluation fact request evaluation request fact fact evaluation evaluation evaluation evaluation fact evaluation evaluation evaluation request request request request request evaluation evaluation request request reference reference reference reference reference reference reference reference reference reference reference reference


This paper proposes to automatically recognize domain names as malicious or benign by deep networks (convnets and RNNs) trained to directly classify the character sequence as such.

Pros The paper addresses an important application of deep networks, comparing the performance of a variety of different types of model architectures.

The tested networks seem to perform reasonably well on the task.

Cons There is little novelty in the proposed method/models

-- the paper is primarily focused on comparing existing models on a new task.

The descriptions of the different architectures compared are overly verbose

-- they are all simple standard convnet / RNN architectures.

The code specifying the models is also excessive for the main text

-- it should be moved to an appendix or even left for a code release.

The comparisons between various architectures are not very enlightening

as they aren’t done in a controlled way

-- there are a large number of differences between any pair of models

so it’s hard to tell where the performance differences come from.

It’s also difficult to compare the learning curves among the different models (Fig 1)

as they are in separate plots with differently scaled axes.

The proposed problem is an explicitly adversarial setting

and adversarial examples are a well-known issue with deep networks and other models,

but this issue is not addressed or analyzed in the paper.

(In fact, the intro claims this is an advantage of not using hand-engineered features for malicious domain detection, seemingly ignoring the literature on adversarial examples for deep nets.)

For example, in this case an attacker could start with a legitimate domain name and use black box adversarial attacks (or white box attacks, given access to the model weights) to derive a similar domain name that the models proposed here would classify as benign.

While this paper addresses an important problem,

in its current form the novelty and analysis are limited

and the paper has some presentation issues.

fact evaluation evaluation evaluation fact evaluation evaluation evaluation request evaluation fact evaluation evaluation evaluation fact fact evaluation fact fact fact evaluation evaluation evaluation


This manuscript explores the idea of adding noise to the adversary's play in GAN dynamics over an RKHS.

This is equivalent to adding noise to the gradient update, using the duality of reproducing kernels.

Unfortunately, the evaluation here is wholly unsatisfactory to justify the manuscript's claims.

No concrete practical algorithm specification is given (only a couple of ideas to inject noise listed),

only a qualitative one on a 2-dimensional latent space in MNIST, and an inconclusive one using the much-doubted Parzen window KDE method.

The idea as stated in the abstract and introduction may well be worth pursuing,

but not on the evidence provided by the rest of the manuscript.

fact fact evaluation fact fact evaluation evaluation


The collaborative block that authors propose is a generalized module that can be inserted in deep architectures for better multi-task learning.

The problem is relevant as we are pushing deep networks to learn representation for multiple tasks.

The proposed method while simple is novel.

The few places where the paper needs improvement are: 1. The authors should test their collaborative block on multiple tasks where the tasks are less related.

Ex: Scene and object classification.

The current datasets where the model is evaluated is limited to Faces which is a constrained setting.

It would be great if Authors provide more experiments beyond Faces to test the universality of the proposed approach.

2. The Face datasets are rather small.

I wonder if the accuracy improvements hold on larger datasets and if authors can comment on any large scale experiments they have done using the proposed architecture.

In it's current form I would say the experiment section and large scale experiments are two places where the paper falls short.

fact fact evaluation request request fact request request request evaluation


This paper presents a novel approach to calibrate classifiers for out of distribution samples.

In additional to the original cross entropy loss, the “confidence loss”  was proposed to guarantee the out of distribution points have low confidence in the classifier.

As out of distribution samples are hard to obtain,

authors also propose to use GAN generating “boundary” samples as out of distribution samples.

The problem setting is new and objective (1) is interesting and reasonable.

However, I am not very convinced that objective (3) will generate boundary samples.

Suppose that theta is set appropriately so that p_theta (y|x) gives a uniform distribution over labels for out of distribution samples.

Because of the construction of U(y), which uniformly assign labels to generated out of distribution samples,

the conditional probability p_g (y|x) should always be uniform so p_g (y|x) divided by p_theta (y|x) is almost always 1.

The KL divergence in (a) of (3) should always be approximately 0 no matter what samples are generated.

I also have a few other concerns: 1. There seems to be a related work:

[1] Perello-Nieto et al., Background Check: A general technique to build more reliable and versatile classifiers, ICDM 2016,

Where authors constructed a classifier, which output K+1 labels and the K+1-th label is the “background noise” label for this classification problem.

Is the method in [1] applicable to this paper’s setting?

Moreover, [1] did not seem to generate any out of distribution samples.

2. I am not so sure that how the actual out of distribution detection was done

(did I miss something here?).

Authors repeatedly mentioned “maximum prediction values”,

but it was not defined throughout the paper.

Algorithm 1. is called “minimization for detection and generating out of distribution (samples)”,

but this is only gradient descent, right?

I do not see a detection procedure.

Given the title also contains “detecting”, I feel authors should write explicitly how the detection is done in the main body.

fact fact evaluation fact evaluation evaluation fact fact fact fact fact reference fact request evaluation evaluation non-arg fact fact fact fact fact request


The article "Contextual Explanation Networks" introduces the class of models which learn the intermediate explanations in order to make final predictions.

The contexts can be learned by, in principle, any model including neural networks,

while the final predictions are supposed to be made by some simple models like linear ones.

The probabilistic model allows for the simultaneous training of explanation and prediction parts as opposed to some recent post-hoc methods.

The experimental part of the paper considers variety of experiments, including classification on MNIST, CIFAR-10, IMDB and also some experiments on survival analysis.

I should note, that the quality of the algorithm is in general similar to other methods considered (as expected).

However, while in some cases the CEN algorithm is slightly better, in other cases it appears to sufficiently loose, see for example left part of Figure 3(b) for MNIST data set.

It would be interesting to know the explanation.

Also, it would be interesting to have more examples of qualitative analysis to see, that the learned explanations are really useful.

I am a bit worried, that while we have interpretability with respect to intermediate features, these features theirselves might be very hard to interpret.

To sum up, I think that the general idea looks very natural and the results are quite supportive.

However, I don't feel myself confident enough in this area of research to make strong conclusion on the quality of the paper.

fact fact fact fact fact fact fact request request evaluation evaluation non-arg