The paper proposes a novel way of compressing gradient updates for distributed SGD, in order to speed up overall execution.

While the technique is novel as far as I know (eq. (1) in particular),

many details in the paper are poorly explained (I am unable to understand)

and experimental results do not demonstrate that the problem targeted is actually alleviated.

More detailed remarks: 1: Motivating with ImageNet taking over a week to train seems misplaced when we have papers claiming to train ImageNet in 1 hour, 24 mins, 15 mins...

4.1: Lemma 4.1 seems like you want B > 1, or clarify definition of V_B.

4.2: This section is not fully comprehensible to me.

- It seems you are confusingly overloading the term gradient and words derived (also in other parts or the paper).

What is "maximum value of gradients in a matrix"?

Make sure to use something else, when talking about individual elements of a vector (which is constructed as an average of gradients), etc.

- Rounding: do you use deterministic or random rounding?

Do you then again store the inaccuracy?

- I don't understand definition of d.

It seems you subtract logarithm of a gradient from a scalar.

- In total, I really don't know what is the object that actually gets communicated,

and consequently when you remark that this can be combined with QSGD and the more below it, I don't understand it.

This section has to be thoroughly explained, perhaps with some illustrative examples.

4.3: allgatherv remark: does that mean that this approach would not scale well to higher number of workers?

4.4: Remarks about quantization and mantissa manipulation are not clear to me again, or what is the point in doing so.

Possible because the problems above.

5: I think this section is not too useful unless you can accompany it with actual efficient implementation and contrast the practical performance.

6: Given that I don't understand how you compress the information being communicated, it is hard to believe the utility of the method.

The objective was to speed up training time because communication is bottleneck.

If you provide 12,000x compression, is it any more practically useful than providing 120x compression?

What would be the difference in runtime?

Such questions are never discussed.

Further, if in the implementation you discuss masking mantissa,

I have serious concern about whether the compression protocol is feasible to implement efficiently, without writing some extremely low-level code.

I think the soundness of work addressing this particular problem is damaged if not implemented properly (compared to other kinds of works in current ML related research).

Therefore I highly recommend including proper time comparison with a baseline in the future.

Further, I don't understand 2 things about the Tables.

a) how do you combine the proposed method with Momentum in SGD?

This is not discussed as far as I can see.

b) What is "QSGD, 2bit"

If I remember QSGD protocol correctly, there's no natural mapping of 2bit to its parameters.

fact evaluation evaluation fact evaluation evaluation evaluation evaluation request request request request evaluation fact evaluation evaluation request non-arg evaluation evaluation evaluation evaluation fact request request fact fact evaluation evaluation request evaluation request fact request evaluation


This work addresses the scenario of fine-tuning a pre-trained network for new data/tasks and empirically studies various regularization techniques.

Overall, the evaluation concludes with recommending that all layers of a network whose weights are directly transferred during fine-tuning should be regularized against the initial net with an L2 penalty during further training.

Relationship to prior work: Regularizing a target model against a source model is not a new idea.

The authors miss key connections to A-SVM [1] and PMT-SVM [2] -- two proposed transfer learning models applied to SVM weights, but otherwise very much the same as the proposed solution in this paper.

Though the study here may offer new insights for deep nets,

it is critical to mention prior work which also does analysis of these regularization techniques.

Significance: As the majority of visual recognition problems are currently solved using variants of fine-tuning,

if the findings reported in this paper generalize, then it could present a simple new regularization which improves the training of new models.

The change is both conceptually simple and easy to implement so could be quickly integrated by many people.

Clarity and Questions: The purpose of the paper is clear,

however, some questions remain unanswered.

1) How is the regularization weight of 0.01 chosen?

This is likely a critical parameter.

In an experimental paper, I would expect to see a plot of performance for at least one experiment as this regularization weighting parameter is varied.

2) How does the use of L2 regularization on the last layer effect the regularization choice of other layers?

What happens if you use no regularization on the last layer?

L1 regularization?

3) Figure 1 is difficult to read.

Please at least label the test sets on each sub-graph.

4) There seems to be some issue with the freezing experiment in Figure 2.

Why does performance of L2 regularization improve as you freeze more and more layers, but is outperformed by un-freezing all.

5) Figure 3 and the discussion of linear dependence with the original model in general seems does not add much to the paper.

It is clear that regularizing against the source model weights instead of 0 should result in final weights that are more similar to the initial source weights.

I would rather the authors use this space to provide a deeper analysis of why this property should help performance.

6) Initializing with a source model offers a strong starting point so full from scratch learning isn’t necessary -- meaning fewer examples are needed for the continued learning (fine-tuning) phase.

In a similar line of reasoning, does regularizing against the source further reduce the number of labeled points needed for fine-tuning?

Can you recover L2 fine-tuning performance with fewer examples when you use L2-SP?

[1] J. Yang, R. Yan, and A. Hauptmann. Adapting svm classifiers to data with shifted distributions. In ICDM Workshops, 2007.

[2]  Y. Aytar and A. Zisserman. Tabula rasa: Model transfer for object category detection. In Proc. ICCV, 2011.

fact fact fact fact fact request fact evaluation evaluation evaluation evaluation request evaluation request request request request evaluation request evaluation request evaluation evaluation request fact request request reference reference


The authors present 3 architectures for learning representations of programs from execution traces.

In the variable trace embedding, the input to the model is given by a sequence of variable values.

The state trace embedding combines embeddings for variable traces using a second recurrent encoder.

The dependency enforcement embedding performs element-wise multiplication of embeddings for parent variables to compute the input of the GRU to compute the new hidden state of a variable.

The authors evaluate their architectures on the task of predicting error patterns for programming assignments from Microsoft DEV204.1X (an introduction to C# offered on edx) and problems on the Microsoft CodeHunt platform.

They additionally use their embeddings to decrease the search time for the Sarfgen program repair system.

This is a fairly strong paper.

The proposed models make sense

and the writing is for the most part clear,

though there are a few places where ambiguity arises:

- The variable "Evidence" in equation (4) is never defined.

- The authors refer to "predicting the error patterns",

but again don't define what an error pattern is.

The appendix seems to suggest that the authors are simply performing multilabel classification based on a predefined set of classes of errors,

is this correct?

- It is not immediately clear from Figures 3 and 4 that the architectures employed are in fact recurrent.

- Figure 5 seems to suggest that dependencies are only enforced at points in a program where assignment is performed for a variable,

is this correct?

Assuming that the authors can address these clarity issues, I would in principle be happy for the paper to appear.

fact fact fact fact fact fact evaluation evaluation evaluation evaluation fact fact fact evaluation non-arg evaluation evaluation non-arg evaluation


This paper proposes two regularization terms to encourage learning disentangled representations.

One term is applied to weight parameters of a layer just like weight decay.

The other is applied to the activations of the target layer (e.g., the penultimate layer).

The core part of both regularization terms is a compound hinge loss of which the input is the KL divergence between two softmax-normalized input arguments.

Experiments demonstrate the proposed regularization terms are helpful in learning representations which significantly facilitate clustering performance.

Pros: (1) This paper is clearly written and easy to follow.

(2) Authors proposed multiple variants of the regularization term which cover both supervised and unsupervised settings.

(3) Authors did a variety of classification experiments ranging from time serials, image and text data.

Cons: (1) The design choice of the compound hinge loss is a bit arbitrary.

KL divergence is a natural similarity measure for probability distribution.

However, it seems that authors use softmax to force the weights or the activations of neural networks to be probability distributions just for the purpose of using KL divergence.

Have you compared with other choices of similarity measure, e.g., cosine similarity?

I think the comparison as an additional experiment would help explain the design choice of the proposed function.

(2) In the binary classification experiments, it is very strange to almost randomly group several different classes of images into the same category.

I would suggest authors look into datasets where the class hierarchy is already provided, e.g., ImageNet or a combination of several fine-grained image classification datasets.

Additionally, I have the following questions: (1) I am curious how the proposed method compares to other competitors in terms of the original classification setting, e.g., 10-class classification accuracy on CIFAR10.

(2) What will happen for the multi-layer loss if the network architecture is very large such that you can not use large batch size, e.g., less than 10?

(3) In drawing figure 2 and 3, if the nonlinear activation function is not ReLU, how would you exam the same behavior?

Have you tried multi-class classification for the case “without proposed loss component” and does the similar pattern still happen or not?

Some typos: (1) In introduction, “when the cosine between the vectors 1” should be “when the cosine between the vectors is 1”.

(2) In section 4.3, “we used the DBPedia ontology dataset dataset” should be “we used the DBPedia ontology dataset”.

I would like to hear authors’ feedback on the issues I raised.

fact fact fact fact evaluation evaluation fact fact evaluation evaluation fact request evaluation evaluation request request request request request request request non-arg


This paper presents a new reinforcement learning architecture called Reactor by combining various improvements in deep reinforcement learning algorithms and architectures into a single model.

The main contributions of the paper are to achieve a better bias-variance trade-off in policy gradient updates, multi-step off-policy updates withdistributional RL, and prioritized experience replay for transition sequences.

The different modules are integrated well and the empirical results are very promising.

The experiments (though limited to Atari) are well carried out and the evaluation is performed on both sample efficiency and training time.

Pros: 1. Nice integration of several recent improvements in deep RL, along with a few novel tricks to improve training.

2. The empirical results on 57 Atari games are impressive, in terms of final scores as well as real-time training speed.

Cons: 1. Reactor is still less sample-efficient than Rainbow, with significantly lower scores after 200M frames.

While the reactor trains much faster, it does use more parallel compute,

so the comparison with Rainbow on wall clock time is  not entirely fair.

Would a distributed version of Rainbow perform better in this respect?

2. Empirical comparisons are restricted to the Atari domain.

The conclusions of the paper will be much stronger if results are also shown on other environments like Mujoco/Vizdoom/Deepmind Lab.

3. Since the paper introduces a few new ideas like prioritized sequence replay,

it would help if a more detailed analysis was performed on the impact of these individual schemes, even if in a model simpler than the Reactor.

For instance, one could investigate the impact of prioritized sequence replay in models like multi-step DQN or recurrent DQN.

This will help us understand  the impact of each of these ideas in a more comprehensive fashion.

fact fact evaluation evaluation evaluation evaluation fact fact evaluation request fact request fact request request evaluation


Summary:This paper proposes a simple recipe to preserve proximity to zero mean for activations in deep neural networks.

The proposal is to replace the non-linearity in half of the units in each layer with its "bipolar" version --

one that is obtained by flipping the function on both axes.

The technique is tested on deep stacks of recurrent layers, and on convolutional networks with depth of 28, showing that improved results over the baseline networks are obtained.

Clarity: The paper is easy to read.

The plots in Fig. 2 and the appendix are quite helpful in improving presentation.

The experimental setups are explained in detail.

Quality and significance: The main idea from this paper is simple and intuitive.

However, the experiments to support the idea do not seem to match the motivation of the paper.

As stated in the beginning of the paper, the motivation behind having close to zero mean activations is that this is expected to speed up training using gradient descent.

However, the presented results focus on the performance on held-out data instead of improvements in training speed.

This is especially the case for the RNN experiments.

For the CIFAR-10 experiment, the training loss curves do show faster initial progress in learning.

However, it is unclear that overall training time can be reduced with the help of this technique.

To evaluate this speed up effect, the dependence on the choice of learning rate and other hyperparameters should also be considered.

Nevertheless, it is interesting to note the result that the proposed approach converts a deep network that does not train into one which does in many cases.

The method appears to improve the training for moderately deep convolutional networks without batch normalization

(although this is tested on a single dataset),

but is not practically useful yet

since the regularization benefits of Batch Normalization are also taken away.

fact fact fact fact evaluation evaluation evaluation evaluation fact fact fact fact fact evaluation request fact fact fact evaluation fact


Summary of paper and review: The paper presents the instability issue of training GANs for semi-supervised learning.

Then, they propose to essentially utilize a wgan for semi-supervised learning.

The novelty of the paper is minor,

since similar approaches have been done before.

The analysis is poor,

the text seems to contain mistakes,

and the results don't seem to indicate any advantage or promise of the proposed algorithm.

Detailed comments: - Unless I'm grossly mistaken the loss function (2) is clearly wrong.

There is a cross-entropy term used by Salimans et al. clearly missing.

- As well, if equation (4) is referring to feature matching, the expectation should be inside the norm and not outside

(this amounts to matching random specific random fake examples to specific random real examples, an imbalanced form of MMD).

- Theorem 2.1 is an almost literal rewrite of Theorem 2.4 of [1], without proper attribution.

Furthermore, Theorem 2.1 is not sufficient to demonstrate existence of this issues.

This is why [1] provides an extensive batch of targeted experiments to verify this assumptions.

Analogous experiments are clearly missing.

A detailed analysis of these assumptions and its implications are missing.

- In section 3, the authors propose a minor variation of the Improved GAN approach by using a wgan on the unsupervised part of the loss.

Remarkably similar algorithms (where the two discriminators are two separate heads) to this have been done before (see for example, [2], but other approaches exist after that, see for examples papers citing [2]).

- Theorem 3.1 is a trivial consequence of Theorem 3 from WGAN.

- The experiments leave much to be desired.

It is widely known that MNIST is a bad benchmark at this point,

and that no signal can be established from a minor success in this dataset.

Furthermore, the results in CIFAR don't seem to bring any advantage, considering the .1% difference in accuracy is 1/100 of chance in this dataset.

[1]: Arjovsky & Bottou, Towards Principled Methods for Training Generative Adversarial Networks, ICLR 2017

[2]: Mroueh & Sercu, Goel, McGan: Mean and Covariance Feature Matching GAN, ICML 2017

fact fact evaluation fact evaluation fact fact evaluation fact fact fact evaluation evaluation fact fact fact evaluation evaluation evaluation evaluation evaluation fact fact reference reference


The authors investigate different message passing schedules for GNN learning.

Their proposed approach is to partition the graph into disjoint subregions, pass many messages on the sub regions and pass fewer messages between regions (an approach that is already considered in related literature, e.g., the BP literature), with the goal of minimizing the number of messages that need to be passed to convey information between all pairs of nodes in the network.

Experimentally, the proposed approach seems to perform comparably to existing methods (or slightly worse on average in some settings).

The paper is well-written and easy to read.

My primary concern is with novelty.

Many similar ideas have been floating around in a variety of different message-passing communities.

With no theoretical reason to prefer the proposed approach, it seems like it may be of limited interest to the community if speed is its only benefit (see detailed comments below).

Specific comments:1)  "When information from any one node has reached all other nodes in the graph for the first time, this problem is considered as solved."

Perhaps it is my misunderstanding of the way in which GNNs work, but isn't the objective actually to reach a set of fixed point equations.

If so, then simply propagating information from one side of the graph may not be sufficient.

2)  The experimental results in Section 4.4 are almost impossible to interpret.

Perhaps it is better to plot number of edges updated versus accuracy?

This at least would put them on equal footing.

In addition, the experiments that use randomness should be repeated and plotted on average (just in case you happened to pick a bad schedule).

3)  More generally, why not consider random schedules (i.e., just pick a random edge, update, repeat) or random partitions?

I'm not certain that a fixed set will perform best independent of the types of updates being considered, and random schedules, like the fully synchronous case for an important baseline (especially if update speed is all you care about).

Typos: -pg. 6, "Thm. 2" -> "Table 2"

fact fact fact evaluation evaluation evaluation evaluation quote fact fact evaluation request fact request request evaluation request


This paper's main thesis is that automatic metrics like BLEU, ROUGE, or METEOR is suitable for task-oriented natural language generation (NLG).

In particular, the paper presents a counterargument to "How NOT To Evaluate Your Dialogue System..."

where Wei et al argue that automatic metrics are not correlated or only weakly correlated with human eval on dialogue generation.

The authors here show that the performance of various NN models as measured by automatic metrics like BLEU and METEOR is correlated with human eval.

Overall, this paper presents a useful conclusion: use METEOR for evaluating task oriented NLG.

However, there isn't enough novel contribution in this paper to warrant a publication.

Many of the details unnecessary:

1) various LSTM model descriptions are unhelpful

given the base LSTM model does just as well on the presented tasks

2) Many embedding based eval methods are proposed

but no conclusions are drawn from any of these techniques.

fact fact fact fact evaluation evaluation evaluation evaluation fact evaluation fact


The paper proposes data augmentation as an alternative to commonly used regularisation techniques like weight decay and dropout, and shows for a few reference models / tasks that the same generalization performance can be achieved using only data augmentation.

I think it's a great idea to investigate the effects of data augmentation more thoroughly.

While it is a technique that is often used in literature,

there hasn't really been any work that provides rigorous comparisons with alternative approaches and insights into its inner workings.

Unfortunately I feel that this paper falls short of achieving this.

Experiments are conducted on two fairly similar tasks (image classification on CIFAR-10 and CIFAR-100), with two different network architectures.

This is a bit meager to be able to draw general conclusions about the properties of data augmentation.

Given that this work tries to provide insight into an existing common practice,

I think it is fair to expect a much stronger experimental section.

In section 2.1.1 it is stated that this was a conscious choice because simplicity would lead to clearer conclusions,

but I think the conclusions would be much more valuable if variety was the objective instead of simplicity, and if larger-scale tasks were also considered.

Another concern is that the narrative of the paper pits augmentation against all other regularisation techniques, whereas more typically these will be used in conjunction.

It is however very interesting that some of the results show that augmentation alone can sometimes be enough.

I think extending the analysis to larger datasets such as ImageNet, as is suggested at the end of section 3, and probably also to different problems than image classification, is going to be essential to ensure that the conclusions drawn hold weight.

Comments:- The distinction between "explicit" and "implicit" regularisation is never clearly enunciated.

A bunch of examples are given for both,

but I found it tricky to understand the difference from those.

Initially I thought it reflected the intention behind the use of a given technique;

i.e. weight decay is explicit because clearly regularisation is its primary purpose --

whereas batch normalisation is implicit because its regularisation properties are actually a side effect.

However, the paper then goes on to treat data augmentation as distinct from other explicit regularisation techniques,

so I guess this is not the intended meaning.

Please clarify this, as the terms crop up quite often throughout the paper.

I suspect that the distinction is somewhat arbitrary and not that meaningful.

- In the abstract, it is already implied that data augmentation is superior to certain other regularisation techniques because it doesn't actually reduce the capacity of the model.

But this ignores the fact that some of the model's excess capacity will be used to model out-of-distribution data (w.r.t. the original training distribution) instead.

Data augmentation always modifies the distribution of the training data.

I don't think it makes sense to imply that this is always preferable over reducing model capacity explicitly.

This claim is referred to a few times throughout the work.

- It could be more clearly stated that the reason for the regularising effect of batch normalisation is the noise in the batch estimates for mean and variance.

- Some parts of the introduction could be removed

because they are obvious, at least to an ICLR audience (like "the model would not be regularised if alpha (the regularisation parameter) equals 0").

- The experiments with smaller dataset sizes would be more interesting if smaller percentages were used.

50% / 80% / 100% are all on the same order of magnitude

and this setting is not very realistic.

In practice, when a dataset is "too small" to be able to train a network that solves a problem reliably, it will generally be one or more orders of magnitude too small, not 2x too small.

- The choices of hyperparameters for "light" and "heavy" motivation seem somewhat arbitrary and are not well motivated.

Some parameters which are sampled uniformly at random should be probably be sampled log-uniformly instead,

because they represent scale factors.

It should also be noted that much more extreme augmentation strategies have been used for this particular task in literature, in combination with padding (for example by Graham).

It would be interesting to include this setting in the experiments as well.

- On page 7 it is stated that "when combined with explicit regularization, the results are much worse than without it",

but these results are omitted from the table.

This is unfortunate

because it is a very interesting observation, that runs counter to the common practice of combining all these regularisation techniques together (e.g. L2 + dropout + data augmentation is a common combination).

Delving deeper into this could make the paper a lot stronger.

- It is not entirely true that augmentation parameters depend only on the training data and not the architecture (last paragraph of section 2.4).

Clearly more elaborate architectures benefit more from data augmentation, and might need heavier augmentation to perform optimally

because they are more prone to overfitting

(this is in fact stated earlier on in the paper as well).

It is of course true that these hyperparameters tend to be much more robust to architecture changes than those of other regularisation techniques such as dropout and weight decay.

This increased robustness is definitely useful

and I think this is also adequately demonstrated in the experiments.

- Phrases like "implicit regularization operates more effectively at capturing reality" are too vague to be meaningful.

- Note that weight decay has also been found to have side effects related to optimization

(e.g. in "Imagenet classification with deep convolutional neural networks", Krizhevsky et al.)

fact evaluation evaluation evaluation evaluation fact evaluation fact evaluation fact evaluation evaluation evaluation evaluation evaluation fact evaluation evaluation fact fact fact evaluation request evaluation fact fact fact evaluation fact request request evaluation evaluation fact evaluation fact evaluation request fact fact request fact fact evaluation evaluation evaluation fact evaluation evaluation fact fact evaluation evaluation evaluation fact reference


It is clear that the problem studied in this paper is interesting.

However, after reading through the manuscript, it is not clear to me what are the real contributions made in this paper.

I also failed to find any rigorous results on generalization bounds.

In this case, I cannot recommend the acceptance of this paper.

evaluation evaluation evaluation evaluation


The paper addresses the problem of learners forgetting rare states and revisiting catastrophic danger states.

The authors propose to train a predictive ‘fear model’ that penalizes states that lead to catastrophes.

The proposed technique is validated both empirically and theoretically.

Experiments show a clear advantage during learning when compared with a vanilla DQN.

Nonetheless, there are some criticisms than can be made of both the method and the evaluations:

The fear radius threshold k_r seems to add yet another hyperparameter that needs tuning.

Judging from the description of the experiments this parameter is important to the performance of the method and needs to be set experimentally.

There seems to be no way of a priori determine a good distance

as there is no way to know in advance when a catastrophe becomes unavoidable.

No empirical results on the effect of the parameter are given.

The experimental results support the claim that this technique helps to avoid catastrophic states during initial learning.

The paper however, also claims to address the longer term problem of revisiting these states once the learner forgets about them,

since they are no longer part of the data generated by (close to) optimal policies.

This problem does not seem to be really solved by this method.

Danger and safe state replay memories are kept, but are only used to train the catastrophe classifier.

While the catastrophe classifier can be seen as an additional external memory,

it seems that the learner will still drift away from the optimal policy and then need to be reminded by the classifier through penalties.

As such the method wouldn’t prevent catastrophic forgetting,

it would just prevent the worst consequences by penalizing the agent before it reaches a danger state.

It would therefore  be interesting to see some long running experiments and analyse how often catastrophic states (or those close to them) are visited.

Overall, the current evaluations focus on performance and give little insight into the behaviour of the method.

The paper also does not compare to any other techniques that attempt to deal with catastrophic forgetting and/or the changing state distribution ([1,2]).

In general the explanations in the paper often often use confusing and  imprecise language, even in formal derivations, e.g.  ‘if the fear model reaches arbitrarily high accuracy’ or ‘if the probability is negligible’.

It is wasn’t clear to me that the properties described in Theorem 1 actually hold.

The motivation in the appendix is very informal and no clear derivation is provided.

The authors seem to indicate that a minimal return can be guaranteed because the optimal policy spends a maximum of epsilon amount of time in the catastrophic states and the alternative policy simply avoids these states.

However, as the alternative policy is learnt on a different reward,

it can have a very different state distribution, even for the non-catastrophics states.

It might attach all its weight to a very poor reward state in an effort to avoid the catastrophe penalty.

It is therefore not clear to me that any claims can be made about its performance without additional assumptions.

It seems that one could construct a counterexample using a 3-state chain problem (no_reward,danger, goal) where the only way to get to the single goal state is to incur a small risk of visiting the danger state.

Any optimal policy would therefore need to spend some time e in the danger state, on average.

A policy that learns to avoid the danger state would then also be unable to reach the goal state and receive rewards.

E.g pi* has stationary distribution (0,e,1-e) and return 0*0+e*Rmin + (1-e)*Rmax.

By adding a sufficiently high penalty, policy pi~ can learn to avoid the catastrophic state with distribution (1,0,0) and then gets return 1*0+ 0*Rmin+0*Rmax= 0 < n*_M - e (Rmax - Rmin) = e*Rmin + (1-e)*Rmax - e (Rmax - Rmin).

This seems to contradict the theorem.

It wasn’t clear what assumptions the authors make to exclude situations like this.

[1] T. de Bruin, J. Kober, K. Tuyls and R. Babuška, "Improved deep reinforcement learning for robotics through distribution-based experience retention," 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Daejeon, 2016, pp. 3947-3952.

[2] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A. A., ... & Hassabis, D. (2017). Overcoming catastrophic forgetting in neural networks. Proceedings of the National Academy of Sciences, 201611835.

fact fact fact fact evaluation fact evaluation fact fact fact fact fact fact evaluation fact fact fact fact fact request evaluation fact evaluation evaluation evaluation fact fact evaluation fact evaluation fact fact fact fact fact fact evaluation reference reference


The paper studies different methods for defining hypergraph embeddings, i.e. defining vectorial representations of the set of hyperedges of a given hypergraph.

It should be noted that the framework does not allow to compute a vectorial representation of a set of nodes not already given as an hyperedge.

A set of methods is presented : the first one is based on an auto-encoder technique ;

the second one is based on tensor decomposition ;

the third one derives from sentence embedding methods.

The fourth one extends over node embedding techniques

and the last one use spectral methods.

The two first methods use plainly the set structure of hyperedges.

Experimental results are provided on semi-supervised regression tasks.

They show very similar performance for all methods and variants.

Also run-times are compared

and the results are expected.

In conclusion, the paper gives an overview of methods for computing hypernode embeddings.

This is interesting in its own.

Nevertheless, as the target problem on hypergraphs is left unspecified,

it is difficult to infer conclusions from the study.

Therefore, I am not convinced that the paper should be published in ICLR'18.

* typos * Recent surveys on graph embeddings have been published in 2017 and should be cited as "A comprehensive survey of graph embedding ..." by Cai et al

* Preliminaries. The occurrence number R(g_i) are not modeled in the hypergraphs.

A graph N_a is defined but not used in the paper.

* Section 3.1. the procedure for sampling hyperedges in the lattice shoud be given.

At least, you should explain how it is made efficient when the number of nodes is large.

* Section 3.2. The method seems to be restricted to cases where the cardinality of hyperedges can take a small number of values.

This is discussed in Section 3.6

but the discussion is not convincing enough.

* Section 3.3 The term Sen2vec is not common knowledge

* Section 3.3 The length of the sentences depends on the number of permutations of $k$ elements.

How can you deal with large k ?

* Section 3.4 and Section 3.5. The methods proposed in these two sections should be related with previous works on hypergraph kernels.

I.e. there should be mentions on the clique expansion and star expansion of hypergraphs.

This leads to the question why graph embeddings methods on these expansions have not be considered in the paper.

* Section 4.1. Only hyperedeges of cardinality in [2,6] are considered.

This seems a rather strong limitation

and this hypothesis does not seem pertinent in many applications.

* Section 4. For online multi-player games, hypernode embeddings only allow to evaluate existing teams, i.e. already existing as hyperedges in the input hypergraph.

One of the most important problem for multi-player games is team making where team evaluation should be made for all possible teams.

* Section 5. Seems redundant with the Introduction.

fact fact fact fact fact fact fact fact fact evaluation fact evaluation fact evaluation fact evaluation evaluation request fact fact request request fact fact evaluation evaluation fact request request request evaluation fact evaluation evaluation fact evaluation evaluation


The authors introduce a set of very simple tasks that are meant to illustrate the challenges of learning visual relations.

They then evaluate several existing network architectures on these tasks,

and show that results are not as impressive as others might have assumed they would be.

They show that while recent approaches (e.g. relational networks) can generalize reasonably well on some tasks, these results do not generalize as well to held-out-object scenarios as might have been assumed.

Clarity:  The paper is fairly clearly written.

I think I mostly followed it.

Quality:  I'm intrigued by but a little uncomfortable with the generalization metrics that the authors use.

The authors estimate the performance of algorithms by how well they generalize to new image scenarios when trained on other image conditions.

The authors state that ". . . the effectiveness of an architecture to learn visual-relation problems should be measured in terms of generalization over multiple variants of the same problem, not over multiple splits of the same dataset."

Taken literally, this would rule out a lot of modern machine learning, even obviously very good work.

On the other hand, it's clear that at some point, generalization needs to occur in testing ability to understand relationships.

I'm a little worried that it's "in the eye of the beholder" whether a given generalization should be expected to work or not.

There are essentially three scenarios of generalization discussed in the paper: (a) various generalizations of image parameters in the PSVRT dataset (b) various hold-outs of the image parameters in the sort-of-CLEVR dataset (c) from sort-of-CLEVR "objects" to PSVRT bit patterns

The result that existing architectures didn't do very well at these generalizations (especially b and c) *may* be important -- or it may not.

Perhaps if CNN+RN were trained on a quite rich real-world training set with a variety of real-world three-D objects beyond those shown in sort-of-CLEVR, it would generalize to most other situations that might be encountered.

After all, when we humans generalize to understanding relationships, exactly what variability is present in our "training sets" as compared to our "testing" situations?

How do the authors know that humans are effectively generalizing rather than just "interpolating" within their (very rich) training set?

It's not totally clear to me that if totally naive humans (who had never seen spatial relationships before) were evaluated on exactly the training/testing scenarios described above, that they would generalize particularly well either.

I don't think it can just be assumed a priori that humans would be super good this form of generalization.

So how should authors handle this criticism?

What would be useful would either be some form of positive control.

Either human training data showing very effective generalization (if one could somehow make "novel" relationships unfamiliar to humans), or a different network architecture that was obviously superior in generalization to CNN+RN.

If such were present, I'd rate this paper significantly higher.

Also, I can't tell if I really fully believe the results of this paper.

I don't doubt that the authors saw the results they report.

However, I think there's some chance that if the same tasks were in the hands of people who *wanted* CNNs or CNN+RN to work well, the results might have been different.

I can't point to exactly what would have to be different to make things "work",

because it's really hard to do that ahead of actually trying to do the work.

However, this suspicion on my part is actually a reason I think it might be *good* for this paper to be published at ICLR.

This will give the people working on (e.g.) CNN+RN somewhat more incentive to try out the current paper's benchmarks and either improve their architecture or show that the the existing one would have totally worked if only tried correctly.

I myself am very curious about what would happen and would love to see this exchange catalyzed.

Originality and Significance:  The area of relation extraction seems to me to be very important and probably a bit less intensively worked on that it should be.

However, as the authors here note, there's been some recent work (e.g. Santoro 2017) in the area.

I think that the introduction of baselines  benchmark challenge datasets such as the ones the authors describe here is very useful, and is a somewhat novel contribution.

fact fact evaluation fact evaluation evaluation evaluation fact fact fact fact evaluation fact evaluation evaluation non-arg evaluation evaluation evaluation non-arg request request evaluation evaluation evaluation evaluation evaluation evaluation evaluation evaluation evaluation evaluation fact evaluation


In this paper, the authors propose a novel method for generating adversarial examples when the model is a black-box and we only have access to its decisions (and a positive example).

It iteratively takes steps along the decision boundary while trying to minimize the distance to the original positive example.

Pros:- Novel method that works under much stricter and more realistic assumptions.

- Fairly thorough evaluation.

- The paper is clearly written.

Cons:- Need a fair number of calls to generate a small perturbation.

Would like to see more analysis of this.

- Attack works for making something outside the boundary (not X),

but is less clear how to generate image to meet a specific classification (X).

3.2 attempts this slightly by using an image in the class,

but is less clear for something like FaceID.

- Unclear how often the images generated look reasonable.

Do different random initializations given different quality examples?

fact fact evaluation evaluation evaluation fact request fact evaluation fact evaluation evaluation request


The paper proposes LSD-NET, an active vision method for object classification.

In the proposed method, based on a given view of an object, the algorithm can decide to either classify the object or to take a discrete action step which will move the camera in order to acquire a different view of the object.

Following this procedure the algorithm iteratively moves around the object until reaching a maximum number of allowed moves or until a object view favorable for classification is reached.

The main contribution of the paper is a hierarchical action space that distinguishes between camera-movement actions and classification actions.

At the top-level of the hierarchy, the algorithm decides whether to perform a movement or a classification -type action.

At the lower-level, the algorithm either assign a specific class label (for the case of classification actions) or performs a camera movement (for the case of camera-movement actions).

This hierarchical action space results in reduced bias towards classification actions.

Strong Points - The content is clear and easy to follow.

- The proposed method achieves competitive performance w.r.t. existing work.

Weak Points- Some aspects of the proposed method could have been evaluated better.

- A deeper evaluation/analysis of the proposed method is missing.

Overall the proposed method is sound and the paper has a good flow and is easy to follow.

The proposed method achieves competitive results,

and up to some extent, shows why it is important to have the proposed hierarchical action space.

My main concerns with this manuscript are the following:

In some of the tables a LSTM variant? of the proposed method is mentioned.

However it is never introduced properly in the text.

Can you indicate how this LSTM-based method differs from the proposed method?

At the end of Section 5.2 the manuscript states: "In comparison to other methods, our method is agnostic of the starting point i.e. it can start randomly on any image and it would get similar testing accuracies."

This suggests that the method has been evaluated over different trials considering different random initializations.

However, this is unclear based on the evaluation protocol presented in Section 5.

If this is not the case, perhaps this is an experiment that should be conducted.

In Section 3.2 it is mentioned that different from typical deep reinforcement learning methods, the proposed method uses a deeper AlexNet-like network.

In this context, it would be useful to drop a comment on the computation costs added in training/testing by this deeper model.

Table 3 shows the number of correctly and wrongly classified objects as a function  of the number of steps taken.

Here we can notice that around 50% of the objects are in the step 1 and 12,

which as correctly indicated by the manuscript, suggests that movement does not help for those cases.

Would it be possible to have more class-specific (or classes grouped into intermediate categories) visualization of the results?

This would provide a better insight of what is going on and when exactly actions related to camera movements really help to get better classification performance.

On the presentation side, I would recommend displaying the content of Table 3 in a plot.

This may display the trends more clearly.

Moreover, I would recommend to visualize the classification accuracy as a function of the step taken by the method.

In this regard, a deeper analysis of the effect of the proposed hierarchical action space is a must.

I would encourage the authors to address the concerns raised on my review.

fact fact fact fact fact fact fact evaluation evaluation evaluation evaluation evaluation evaluation evaluation non-arg fact evaluation request quote evaluation evaluation request fact request fact fact evaluation request evaluation request evaluation request request request


Summary: This paper empirically studies adversarial perturbations dx and what the effects are of adversarial training (AT) with respect to shared (dx fools for many x) and singular (only for a single x) perturbations.

Experiments use a (previously published) iterative fast-gradient-sign-method and use a Resnet on CIFAR.

The authors conclude that in this experimental setting: - AT seems to defend models against shared dx's.

- This is visible on universal perturbations,

which become less effective as more AT is applied.

- AT decreases the effectiveness of adversarial perturbations, e.g. AT decreases the number of adversarial perturbations that fool both an input x and x with e.g. a contrast change.

- Singular perturbations are easily detected by a detector model,

as such perturbations don't change much when applying AT.

Pro:- Paper addresses an important problem: qualitative / quantitative understanding of the behavior of adversarial perturbations is still lacking.

- The visualizations of universal perturbations as they change during AT are nice.

- The basic observation wrt the behavior of AT is clearly communicated.

Con:- The experiments performed are interesting directions, although unfocused and rather limited in scope.

For instance, does the same phenomenon happen for different datasets?

Different models?

- What happens when we use adversarial attacks different from FGSM?

Do we get similar results?

- The papers lacks a more in-depth theoretical analysis.

Is there a principled reason AT+FGSM defends against universal perturbations?

Overall:- As is, it seems to me the paper lacks a significant central message (due to limited and unfocused experiments) or significant new theoretical insight into the effect of AT.

A number of questions addressed are interesting starting points towards a deeper understanding of *how* the observations can be explained and more rigorous empirical investigations.

Detailed: -

fact fact fact fact fact fact fact fact fact evaluation evaluation evaluation request request request request evaluation request evaluation evaluation non-arg


SUMMARY: This work is about learning the validity of a sequences in specific application domains like SMILES strings for chemical compounds.

In particular, the main emphasis is on predicting if a prefix sequence could possibly be extended to a complete valid sequence.

In other words, one tries to predict if there exists a valid suffix sequence, and based on these predictions, the goal is to train a generative model that always produces valid sequences.

In the proposed reinforcement learning setting, a neural network models the probability that a certain action (adding a symbol) will result in a valid full sequence.

For training the network, a large set of (validity-)labelled sequences would be needed.

To overcome this problem, the authors introduce an active learning strategy, where the information gain is re-expressed as the conditional mutual information between the the label y and the network weights w, and this mutual information is maximized in a greedy sequential manner.

EVALUATION: CLARITY & NOVELTY: In principle, the paper is easy to read.

Unfortunately, however, for the reader is is not easy to find out what the authors consider their most relevant contribution.

Every single part of the model seems to be quite standard (basically a network that predicts the probability of a valid sequence and an information-gain based active learning strategy)

- so is the specific application to SMILES strings what makes the difference here?

Or is is the specific greedy approximation to the mutual information criterion in the active learning part?

Or is it the way how you augment the dataset?

All these aspects might be interesting,

but somehow I am missing a coherent picture.

SIGNIFICANCE: it is not entirely clear to me if the proposed "pruning" strategy for the completion of prefix sequences can indeed be generally applied to sequence modelling problems,

because in more general domains it might be very difficult to come up with reasonable validity estimates for prefixes that are significantly shorter than the whole sequence.

I am not so familiar with SMILES strings

-- but could it be that the experimental success reported here is mainly a result of the very specific structure of valid SMILES strings?

But then, what can be learned for general sequence validation problems?

fact fact fact fact fact fact evaluation evaluation evaluation non-arg non-arg non-arg evaluation evaluation evaluation evaluation non-arg request request


The authors have addressed the problem of translating natural language queries to SQL queries.

They proposed a deep neural network based solution which combines the attention based neural semantic parser and pointer networks.

They also released a new dataset WikiSQL for the problem.

The proposed method outperforms the existing semantic parsing baselines on WikiSQL dataset.

Pros:1. The idea of using pointer networks for reducing search space of generated queries is interesting.

Also, using extrinsic evaluation of generated queries handles the possibility of paraphrasing SQL queries.

2. A new dataset for the problem.

3. The experiments report a significant boost in the performance compared to the baseline.

The ablation study is helpful for understanding the contribution of different component of the proposed method.

Cons:1. It would have been better to see performance of the proposed method in other datasets (wherever possible).

This is my main concern about the paper.

2. Extrinsic evaluation can slow down the overall training.

Comparison of running times would have been helpful.

3. More details about training procedure (specifically for the RL part) would have been better.

fact fact fact fact evaluation fact fact fact evaluation request evaluation evaluation request request


Summary:This work is about model evaluation for molecule generation and design.

19 benchmarks are proposed, small data sets are expanded to a large, standardized data set

and it is explored how to apply new RL techniques effectively for molecular design.

on the positive side: The paper is well written, quality and clarity of the work are good.

The work provides a good overview about how to apply new reinforcement learning techniques for sequence generation.

It is investigated how several RL strategies perform on a large, standardized data set.

Different RL models like Hillclimb-MLE, PPO, GAN, A2C are investigated and discussed.

An implementation of 19 suggested benchmarks of relevance for de novo design will be provided as open source as an OpenAI Gym.

on the negative side:  There is no new novel contribution on the methods side.

minor comments: Section 2.1. see Fig.2 —> see Fig.1

page 4just before equation 8: the the

fact fact fact evaluation evaluation fact fact fact evaluation request request


The authors propose a new episodic reinforcement learning algorithm based on contextual bandit oracles.

The key specificity of this algorithm is its ability to deal with the credit assignment problem by learning automatically a progressive "reward shaping" (the residual losses) from a feedback that is only provided at the end of the epochs.

The paper is dense but well written.

The theoretical grounding is a bit thin or hard to follow.

The authors provide a few regret theoretical results (that I did not check deeply) obtained by reduction to "value-aware" contextual bandits.

The experimental section is solid.

The method is evaluated on several RL environments against state of the art RL algorithms.

It is also evaluated on bandit structured prediction tasks.

An interesting synthetic experiment (Figure 4) is also proposed to study the ability of the algorithm to work on both decomposable and non-decomposable structured prediction tasks.

Question 1: The credit assignment approach you propose seems way more sophisticated than eligibility traces in TD learning.

But sometimes old and simple methods are not that bad.

Could you develop a bit on the relation between RESLOPE and eligibility traces ?

Question 2: RESLOPE is built upon contextual bandits which require a stationary environment.

Does RESLOPE inherit from this assumption?

Typos: page 1 "scalar loss that output." -> "scalar loss."

", effectively a representation" -> ". By effective we mean effective in term of credit assignment."

page 5 "and MTR" -> "and DR"

page 6 "in simultaneously." -> ???

".In greedy" -> ". In greedy"

fact fact evaluation evaluation fact evaluation fact fact fact evaluation evaluation request fact request request request request request request


This paper proposes a device placement algorithm to place operations of tensorflow on devices.

Pros: 1. It is a novel approach which trains the placement end to end.

2. The experiments are solid to demonstrate this method works very well.

3. The writing is easy to follow.

4. This would be a very useful tool for the community if open sourced.

Cons: 1. It is not very clear in the paper whether the training happens for each model yielding separate agents, or a shared agent is trained and used for all kinds of models.

The latter would be more exciting.

The adjacency matrix varies size for different graphs,

so I guess a separate agent is trained for each graph?

However, if the agent is not shared, why not just use integer to represent each operation in the graph,

since overfitting would be more desirable in this case.

2. Averaging the embedding is hard to understand especially for the output sizes and number of outputs.

3. It is not clear how the adjacency information is used.

fact evaluation evaluation evaluation evaluation evaluation evaluation fact non-arg request fact evaluation evaluation


Paper summary: Existing works on multi-task neural networks typically use hand-tuned weights for weighing losses across different tasks.

This work proposes a dynamic weight update scheme that updates weights for different task losses during training time by making use of the loss ratios of different tasks.

Experiments on two different network indicate that the proposed scheme is better than using hand-tuned weights for multi-task neural networks.

Paper Strengths:- The proposed technique seems simple yet effective for multi-task learning.

- Experiments on two different network architectures showcasing the generality of the proposed method.

Major Weaknesses:- The main weakness of this work is the unclear exposition of the proposed technique.

Entire technique is explained in a short section-3.1 with many important details missing.

There is no clear basis for the main equations 1 and 2.

How does equation-2 follow from equation-1?

Where is the expectation coming from?

What exactly does ‘F’ refer to?

There is dependency of ‘F’ on only one of sides in equations 1 and 2?

More importantly, how does the gradient normalization relate to loss weight update?

It is very difficult to decipher these details from the short descriptions given in the paper.

- Also, several details are missing in toy experiments.

What is the task here?

What are input and output distributions and what is the relation between input and output?

Are they just random noises?

If so, is the network learning to overfit to the data as there is no relationship between input and output?

Minor Weaknesses:- There are no training time comparisons between the proposed technique and the standard fixed loss learning.

- Authors claim that they operate directly on the gradients inside the network.

But, as far as I understood, the authors only update loss weights in this paper.

Did authors also experiment with gradient normalization in the intermediate CNN layers?

- No comparison with state-of-the-art techniques on the experimented tasks and datasets.

Clarifications:- See the above mentioned issues with the exposition of the technique.

- In the experiments, why are the input images downsampled to 320x320?

- What does it mean by ‘unofficial dataset’ (page-4).

Any references here?

- Why is 'task normalized' test-time loss as good measure for comparison between models in the toy example (Section 4)?

The loss ratios depend on initial loss,

which is not important for the final performance of the system.

Suggestions:- I strongly suggest the authors to clearly explain the proposed technique to get this into a publishable state.

- The term ’GradNorm’ seem to be not defined anywhere in the paper.

Review Summary:Despite promising results, the proposed technique is quite unclear from the paper.

With its poor exposition of the technique, it is difficult to recommend this paper for publication.

fact fact fact evaluation fact evaluation evaluation evaluation non-arg non-arg non-arg non-arg non-arg evaluation fact non-arg non-arg non-arg non-arg fact fact fact non-arg fact non-arg non-arg non-arg non-arg non-arg fact evaluation request fact evaluation evaluation


Summary: The authors proposed an unsupervised time series clustering methods built with deep neural networks.

The proposed model is equipped with an encoder-decoder and a clustering model.

First, the encoder employs CNN to shorten the time series and extract local temporal features,

and the CNN is followed by bidirectional LSTMs to get the encoded representations.

A temporal clustering model and a DCNN decoder are applied on the encoded representations and jointly trained.

An additional heatmap generator component can be further included in the clustering model.

The authors compared the proposed method with hierarchical clustering with 4 different temporal similarity methods on several univariate time series datasets.

Detailed comments:The problem of unsupervised time series clustering is important and challenging.

The idea of utilizing deep learning models to learn encoded representations for clustering is interesting and could be a promising solution.

One potential limitation of the proposed method is that it is only designed for univariate time series of the same temporal length,

which limits the usage of this model in practice.

In addition, given that the input has fixed length, clustering baselines for static data can be easily applied

and should be compared to demonstrate the necessity of temporal clustering.

Some important details are missing or lack of explanations.

For example, what is the size of each layer and the dimension of the encoded space?

How much does the model shorten the input time series and how is this be determined?

How does the model combine the heatmap output (which is a sequence of the same length as the time series) and the clustering output (which is a vector of size K) in Figure 1?

The heatmap shown in Figure 3 looks like the negation of the decoded output (i.e., lower value in time series -> higher value in heatmap).

How do we interpret the generated heatmap?

From the experimental results, it is difficult to judge which method/metric is the best.

For example, in Figure 4, all 4 DTC-methods achieved the best performance on one or two datasets.

Though several datasets are evaluated in experiments, they are relatively small.

Even the largest dataset (Phalanges OutlinesCorrect) has only 2 thousand samples,

and the best performance is achieved by one of the baseline, with AUC score only 0.586 for binary classification.

Minor suggestion: In Figure 3, instead of showing the decoded output (reconstruction), it may be more helpful to visualize the encoded time series

since the clustering method is applied directly on those encoded representations.

fact fact fact fact fact fact fact evaluation evaluation fact fact evaluation request evaluation non-arg non-arg non-arg evaluation non-arg evaluation fact evaluation fact fact request fact


Summary: The authors present a simple variation of vanilla recurrent neural networks, which use ReLU hiddens and a fixed identity matrix that is added to the hidden-to-hidden weight matrix.

This identity connection acts as a “surrogate memory” component, preserving hidden activations over time steps.

The experiments demonstrate that this architecture reliably solves the addition task for up to 400 input frames.

It also achieves a very good performance on sequential and permuted MNIST and achieves SOTA performance on bAbI.

The authors observe that the proposed recurrent identity network (RIN) is relatively robust to hyperparameter choices.

After Le et al. (2015), the paper presents another convincing case for the application of ReLUs in RNNs.

Review: I very much like the paper.

The motivation and architecture is presented very clearly

and I am happy to also see explorations of simpler recurrent architectures in parallel to research of gated architectures!

I have a few comments and questions:1) Clarification: In Section 2.2, do you really mean bit-wise multiplication or element-wise?

If bit-wise, can you elaborate why?

I might have missed something.

2) Why does the learning curve of the IRNN stop around epoch 270 in Figure 2c?

Also some curves in the appendix stop abruptly without visible explosions.

Were these experiments run until completion?

If so, would it be possible to plot the complete curves?

3) I think for a fair comparison with LSTMs and IRNNs a limited hyperparameter search should be performed separately on all three architectures at least for the addition task.

Optimal hyperparameters are usually model-specific.

Admittedly, the authors mention that they do not intend to make claims about superior performance to LSTMs,

however the competitive performance of small RINs is mentioned a couple of times in the manuscript.

Le et al. (2015) for instance perform a coarse grid search for each model.

4) I wouldn't say that ResNets are Gated Neural Networks,

as the branches are just summed up.

There is no (multiplicative) gating as in Highway Networks.

5) I think what enables the training of very deep networks or LSTMs on long sequences is the presence of a (close-to-)identity component in forward/backward propagation, not the gating.

The use of ReLU activations in IRNNs (with identity initialization of the hidden-to-hidden weights) and RINs (effectively initialized with identity plus some noise) makes the recurrence more linear than with squashing activation functions.

6) Regarding the absence of gating in RINs: What is your intuition on how the model would perform in tasks for which conditional forgetting is useful.

Consider for example a task with long sequences, outputs at every time step and hidden activations not necessarily being encouraged to estimate last step hidden activations.

Would RINs readily learn to reset parts of the hidden state?

7) Henaff et al. (2016) might be related,

as they are also looking into the addition task with long sequences.

Overall, the presented idea is novel to the best of my knowledge

and the manuscript is well-written.

I would recommend it for acceptance,

but would like to see the above points addressed (especially 1-3 and some comments on 4-6).

After a revision I would consider to increase the score.

References: Henaff, Mikael, Arthur Szlam, and Yann LeCun. "Recurrent orthogonal networks and long-memory tasks." In International Conference on Machine Learning, pp. 2034-2042. 2016.

Le, Quoc V., Navdeep Jaitly, and Geoffrey E. Hinton. "A simple way to initialize recurrent networks of rectified linear units." arXiv preprint arXiv:1504.00941 (2015).

fact fact fact fact fact evaluation evaluation evaluation evaluation non-arg non-arg evaluation request fact non-arg request request evaluation fact fact fact fact fact fact fact fact request fact request evaluation fact evaluation evaluation evaluation request evaluation reference reference


This reviewer has found the proposed approach quite compelling,

but the empirical validation requires significant improvements:

1) you should include in your comparison Query-by- Bagging & Boosting,

which are two of the best out-of-the-box active learning strategies

2) in your empirical validation you have (arbitrarily) split the 14 datasets in 7 training and testing ones,

but many questions are still unanswered:

-  would any 7-7 split work just as well (ie, cross-validate over the 14 domains)

- do you what happens if you train on 1, 2, 3, 8, 10, or 13 domains? are the results significantly different?

OTHER COMMENTS: - p3: both images in Figure 1 are labeled Figure 1.a

- p3: typo "theis" --> "this"

Abe & Mamitsuksa (ICML-1998). Query Learning Strategies Using Boosting and Bagging.

evaluation evaluation request evaluation fact evaluation request request fact request reference


The paper discusses the problem of optimizing neural networks with hard threshold and proposes a novel solution to it.

The problem is of significance because in many applications one requires deep networks which uses reduced computation and limited energy.

The authors frame the problem of optimizing such networks to fit the training data as a convex combinatorial problems.

However since the complexity of such a problem is exponential, the authors propose a collection of heuristics/approximations to solve the problem.

These include, a heuristic for setting the targets at each layer, using a soft hinge loss, mini-batch training and such.

Using these modifications the authors propose an algorithm (Algorithm 2 in appendix) to train such models efficiently.

They compare the performance of a bunch of models trained by their algorithm against the ones trained using straight-through-estimator (SSTE) on a couple of datasets, namely, CIFAR-10 and ImageNet.

They show superiority of their algorithm over SSTE.

I thought the paper is very well written and provides a really nice exposition of the problem of training deep networks with hard thresholds.

The authors formulation of the problem as one of combinatorial optimization

and proposing Algorithm 1 is also quite interesting.

The results are moderately convincing in favor of the proposed approach.

Though a disclaimer here is that I'm not 100% sure that SSTE is the state of the art for this problem.

Overall i like the originality of the paper and feel that it has a potential of reasonable impact within the research community.

There are a few flaws/weaknesses in the paper though, making it somewhat lose.

- The authors start of by posing the problem as a clean combinatorial optimization problem and propose Algorithm 1.

Realizing the limitations of the proposed algorithm, given the assumptions under which it was conceived in,

the authors relax those assumptions in the couple of paragraphs before section 3.1

and pretty much throw away all the nice guarantees, such as checks for feasibility, discussed earlier.

- The result of this is another algorithm (I guess the main result of the paper), which is strangely presented in the appendix as opposed to the main text, which has no such guarantees.

- There is no theoretical proof that the heuristic for setting the target is a good one, other than a rough intuition

- The authors do not discuss at all the impact on generalization ability of the model trained using the proposed approach.

The entire discussion revolves around fitting the training set and somehow magically everything seem to generalize and not overfit.

fact evaluation fact fact fact fact fact fact evaluation fact evaluation evaluation evaluation evaluation evaluation fact evaluation fact fact fact fact fact evaluation


Quick summary: This paper proposes an energy based formulation to the BEGAN model and modifies it to include an image quality assessment based term.

The model is then trained with CelebA under different parameters settings and results are analyzed.

Quality and significance: This is quite a technical paper, written in a very compressed form and is a bit hard to follow.

Mostly it is hard to estimate what is the contribution of the model and how the results differ from baseline models.

Clarity: I would say this is one of the weak points of the paper - the paper is not well motivated and the results are not clearly presented.

Originality: Seems original.

Pros: * Interesting energy formulation and variation over BEGAN

Cons: * Not a clear paper

* results are only partially motivated and analyzed

fact fact evaluation evaluation evaluation evaluation evaluation evaluation evaluation


This paper proposes a method for parameter space noise in exploration.

Rather than the "baseline" epsilon-greedy (that sometimes takes a single action at random)... this paper presents an method for perturbations to the policy.

In some domains this can be a much better approach and this is supported by experimentation.

There are several things to like about the paper:

- Efficient exploration is a big problem for deep reinforcement learning (epsilon-greedy or Boltzmann is the de-facto baseline)

and there are clearly some examples where this approach does much better.

- The noise-scaling approach is (to my knowledge) novel, good and in my view the most valuable part of the paper.

- This is clearly a very practical and extensible idea...

the authors present good results on a whole suite of tasks.

- The paper is clear and well written,

it has a narrative and the plots/experiments tend to back this up.

- I like the algorithm, it's pretty simple/clean and there's something obviously *right* about it (in SOME circumstances).

However, there are also a few things to be cautious of... and some of them serious:

- At many points in the paper the claims are quite overstated.

Parameter noise on the policy won't necessarily get you efficient exploration...

and in some cases it can even be *worse* than epsilon-greedy...

if you just read this paper you might think that this was a truly general "statistically efficient" method for exploration (in the style of UCRL or even E^3/Rmax etc).

- For instance, the example in 4.2 only works because the optimal solution is to go "right" in every timestep...

if you had the network parameterized in a different way (or the actions left/right were relabelled) then this parameter noise approach would *not* work...

By contrast, methods such as UCRL/PSRL and RLSVI https://arxiv.org/abs/1402.0635 *are* able to learn polynomially in this type of environment.

I think the claim/motivation for this example in the bootstrapped DQN paper is more along the lines of "deep exploration"

and you should be clear that your parameter noise does *not* address this issue.

- That said I think that the example in 4.2 is *great* to include...

you just need to be more upfront about how/why it works and  what you are banking on with the parameter-space exploration.

Essentially you perform a local exploration rule in parameter space...

and sometimes this is great

- but you should be careful to distinguish this type of method from other approaches.

This must be mentioned in section 4.2 "does parameter space noise explore efficiently"

because the answer you seem to imply is "yes" ... when the answer is clearly NOT IN GENERAL... but it can still be good sometimes ;D

- The demarcation of "RL" and "evolutionary strategies" suggests a pretty poor understanding of the literature and associated concepts.

I can't really support the conclusion "RL with parameter noise exploration learns more efficiently than both RL and evolutionary strategies individually".

This sort of sentence is clearly wrong and for many separate reasons:

- Parameter noise exploration is not a separate/new thing from RL... it's even been around for ages!

It feels like you are talking about DQN/A3C/(whatever algorithm got good scores in Atari last year) as "RL" and that's just really not a good way to think about it.

- Parameter noise exploration can be *extremely* bad relative to efficient exploration methods

(see section 2.4.3 https://searchworks.stanford.edu/view/11891201)

Overall, I like the paper, I like the algorithm

and I think it is a valuable contribution.

I think the value in this paper comes from a practical/simple way to do policy randomization in deep RL.

In some (maybe even many of the ones you actually care about) settings this can be a really great approach, especially when compared to epsilon-greedy.

However, I hope that you address some of the concerns I have raised in this review.

You shouldn't claim such a universal revolution to exploration / RL / evolution

because I don't think that it's correct.

Further, I don't think that clarifying that this method is *not* universal/general really hurts the paper...

you could just add a section in 4.2 pointing out that the "chain" example wouldn't work if you needed to do different actions at each timestep (this algorithm does *not* perform "deep exploration").

I vote accept.

fact fact evaluation evaluation evaluation evaluation evaluation evaluation evaluation evaluation evaluation evaluation evaluation evaluation evaluation evaluation evaluation evaluation evaluation fact evaluation request evaluation request fact evaluation request request evaluation evaluation evaluation evaluation fact evaluation evaluation reference evaluation evaluation evaluation evaluation request request fact evaluation request evaluation


This paper investigates learning representations for the problem of nearest neighbor (NN) search by exploring various deep learning architectural choices.

The crux of the paper is the connection between NN and the angles between the closest neighbors --

the higher this angle, more data points need to be explored for finding the nearest one, and thus more computational expense.

Thus, the paper proposes to learn a network that tries to reduce the angles between the inputs and the corresponding class vectors in a supervised framework using softmax cross-entropy loss.

Three architectural choices are investigated,

(i) controlling the norm of output layers of the CNN (using batch norm essentially),

(ii) removing relu so that the outputs are well-distributed in both positive and negative orthants,

and (iii) normalizing the class vectors.

Experiments are given on multiMNIST and Sports 1M and show improvements.

Pros: 1) The paper explores different architectural choices for the deep network to some depth and show extensive results.

2) The results do demonstrate clearly the advantage of the various choices and is useful

3) The theoretical connections between data angles and query times are quite interesting,

Cons: 1) Unclear Problem Statement.

I find the problem statement a bit vague.

Standard NN search finds a data point in the database closest to a query under some distance metric.

While, the current paper uses the cosine similarity as the distance, the deep framework is trained on class vectors using cross-entropy loss.

I do not think class labels are usually assumed to be given in the standard definition of NN,

and it is not clear to me how the proposed setup can accommodate NN without class labels.

Thus as such, I see this paper is perhaps proposing a classification problem and not an NN problem per se.

2) Lacks Focus The paper lacks a good organization in my opinion.

Things that are perhaps technically important are moved to the Appendix.

For example, I find the theoretical part of the paper (e.g., Theorem 1) quite elegant and perhaps the main innovation in this paper.

However, that is moved completely to the Appendix.

So it cannot be really considered a contribution.

It is also not clear if those theoretical results are novel.

2) Disconnect/Unclear Assumptions There seems to be some disconnect between LSH and deep learning architectures explored in Sections 2 and 3 respectively.

Are the assumptions used in the theoretical results for LSH also assumed in the deep networks?

For example, as far as I know, the standard LSH works assumes the projection hyperplanes are randomly chosen and the theoretical results are based on such assumptions.

It is not clear how a softmax output of a CNN, which is trained in a supervised way, follow such assumptions.

It would be important if the paper could clarify such assumptions to make sure the sections are congruent.

3) No Related Work

There have been several efforts for adapting deep frameworks into KNN.

The paper ignores all such works.

Thus, it is not clear how significant is the proposed contribution.

There are also not comparisons what-so-ever to competitive prior works.

4) Novelty The main contribution of this paper is basically a set of experiments looking into architectural choices.

However, the results of this study do not provide any surprises.

It appears that batch normalization is essential for good performances,

while using RELU is not so when one wants to use all directions for effective data encoding.

Thus, as such, the novelty or the contributions of this paper are minor.

Overall, while I find there are some interesting theoretical bits in this paper,

it lacks focus,

the experiments do not offer any surprises,

and there are no comparisons with prior literature.

Thus, I do not think this paper is ready to be accepted in its present form.

fact evaluation fact fact fact fact fact fact fact evaluation evaluation evaluation evaluation evaluation fact fact fact evaluation fact evaluation fact evaluation fact evaluation evaluation evaluation request fact evaluation request fact fact fact evaluation fact fact evaluation fact fact evaluation evaluation evaluation evaluation fact evaluation


This paper presents a new approach to determining what to measure and when to measure it, using a novel deep learning architecture.

The problem addressed is important and timely

and advances here may have an impact on many application areas outside medicine.

The approach is evaluated on real-world medical datasets and has increased accuracy over the other methods compared against.

+ A key advantage of the approach is that it continually learns from the collected data, using new measurements to update the model, and that it runs efficiently even on large real-world datasets.

-However, the related work section is significantly underdeveloped, making it difficult to really compare the approach to the state of the art.

The paper is ambitious and claims to address a variety of problems,

but as a result each segment of related work seems to have been shortchanged.

In particular, the section on missing data is missing a large amount of recent and related work.

Normally, methods for handling missing data are categorized based on the missingness model (MAR/MCAR/MNAR).

The paper seems to assume all data are missing at random, which is also a significant limitation of the methods.

-The paper is organized in a nonstandard way, with the methods split across two sections, separated by the related work.

It would be easier to follow with a more common intro/related work/methods structure.

Questions: -One of the key motivations for the approach is sensing in medicine.

However, many tests come as a group (e.g. the chem-7 or other panels).

In this case, even if the only desired measurement is glucose, others will be included as well.

Is it possible to incorporate this?

It may change the threshold for the decision, as a combination of measures can be obtained for the same cost.

fact evaluation evaluation fact fact evaluation evaluation evaluation fact fact evaluation fact request fact fact fact request evaluation


The paper offers a formal proof that gradient descent on the logistic loss converges very slowly to the hard SVM solution in the case where the data are linearly separable.

This result should be viewed in the context of recent attempts at trying to understand the generalization ability of neural networks, which have turned to trying to understand the implicit regularization bias that comes from the choice of optimizer.

Since we do not even understand the regularization bias of optimizers for the simpler case of linear models,

I consider the paper's topic very interesting and timely.

The overall discussion of the paper is well written,

but on a more detailed level the paper gives an unpolished impression, and has many technical issues.

Although I suspect that most (or even all) of these issues can be resolved, they interfere with checking the correctness of the results.

Unfortunately, in its current state I therefore do not consider the paper ready for publication.

Technical Issues: The statement of Lemma 5 has a trivial part and for the other part the proof is incorrect: Let x_u = ||nabla L(w(u))||^2.

- Then the statement sum_{u=0}^t x_u < infinity is trivial,

because it follows directly from ||nabla L(w(u))||^2 < infinity for all u.

I would expect the intended statement to be sum_{u=0}^infinity x_u < infinity,

which actually follows from the proof of the lemma.

- The proof of the claim that t*x_t -> 0 is incorrect:

sum_{u=0}^t x_u < infinity does not in itself imply that t*x_t -> 0, as claimed.

For instance, we might have x_t = 1/i^2 when t=2^i for i = 1,2,... and x_t = 0 for all other t.

Definition of tilde{w} in Theorem 4:  - Why would tilde{w} be unique?

In particular, if the support vectors do not span the space, because all data lie in the same lower-dimensional hyperplane, then this is not the case.

- The KKT conditions do not rule out the case that \hat{w}^top x_n = 1, but alpha_n = 0 (i.e. a support vector that touches the margin, but does not exert force against it).

Such n are then included in cal{S}, but lead to problems in (2.7),

because they would require tilde{w}^top x_n = infinity, which is not possible.

In the proof of Lemma 6, case 2. at the bottom of p.14:  - After the first inequality, C_0^2 t^{-1.5 epsilon_+} should be C_0^2 t^{-epsilon_+}

- After the second inequality the part between brackets is missing an additional term C_0^2 t^{-\epsilon_+}.

- In addition, the label (1) should be on the previous inequality and it should be mentioned that e^{-x} <= 1-x+x^2 is applied for x >= 0 (otherwise it might be false).

In the proof of Lemma 6, case 2 in the middle of p.15:  - In the line of inequality (1) there is a t^{-epsilon_-} missing.

In the next line there is a factor t^{-epsilon_-} too much.

- In addition, the inequality e^x >= 1 + x holds for all x, so no need to mention that x > 0.

In Lemma 1:  - claim (3) should be lim_{t \to \infty} w(t)^\top x_n = infinity

- In the proof: w(t)^top x_n > 0 only holds for large enough t.

Remarks: p.4 The claim that "we can expect the population (or test) misclassification error of w(t) to improve" because "the margin of w(t) keeps improving" is worded a little too strongly,

because it presumes that the maximum margin solution will always have the best generalization error.

In the proof sketch (p.3):  - Why does the fact that the limit is dominated by gradients that are a linear combination of support vectors imply that w_infinity will also be a non-negative linear combination of support vectors?

- "converges to some limit". Mention that you call this limit w_infinity

Minor Issues: In (2.4): add "for all n".

p.10, footnote: Shouldn't "P_1 = X_s X_s^+" be something like "P_1 = (X_s^top X_s)^+"?

A.9: ell should be ell'

The paper needs a round of copy editing.

For instance:  - top of p.4: "where tilde{w} A is the unique"

- p.10: "the solution tilde{w} to TO eq. A.2"

- p.10: "might BOT be unique"

- p.10: "penrose-moorse pseudo inverse" -> "Moore-Penrose pseudoinverse"

In the bibliography, Kingma and Ba is cited twice, with different years.

fact evaluation fact evaluation evaluation evaluation evaluation evaluation fact fact fact evaluation fact fact fact fact fact fact fact fact fact request request request request request request request fact evaluation fact request request request request request request quote quote quote request fact


This paper presents a method for classifying Tumblr posts with associated images according to associated single emotion word hashtags.

The method relies on sentiment pre-processing from GloVe and image pre-processing from Inception.

My strongest criticism for this paper is against the claim that Tumblr post represent self-reported emotions and that this method sheds new insight on emotion representation

and my secondary criticism is a lack of novelty in the method,

which seems to be simply a combination of previously published sentiment analysis module and previously published image analysis module, fused in an output layer.

The authors claim that the hashtags represent self-reported emotions,

but this is not true in the way that psychologists query participants regarding emotion words in psychology studies.

Instead these are emotion words that a person chooses to broadcast along with an associated announcement.

As the authors point out, hashtags and words may be used sarcastically or in different ways from what is understood in emotion theory.

It is quite common for everyday people to use emotion words this way e.g. using #love to express strong approval rather than an actual feeling of love.

In their analysis the authors claim:

“The 15 emotions retained were those with high relative frequencies on Tumblr among the PANAS-X scale (Watson & Clark, 1999)”.

However five of the words the authors retain: bored, annoyed, love, optimistic, and pensive are not in fact found in the PANAS-X scale:

Reference: The PANAS-X Scale: https://wiki.aalto.fi/download/attachments/50102838/PANAS-X-scale_spec.pdf

Also the longer version that the authors cited:

It should also be noted that the PANAS (Positive and Negative Affect Scale) scale and the PANAS-X (the “X” is for eXtended) scale are questionnaires used to elicit from participants feelings of positive and negative affect,

they are not collections of "core" emotion words,

but rather words that are colloquially attached to either positive or negative sentiment.

For example PANAS-X includes words like:“strong” ,“active”, “healthy”, “sleepy” which are not considered emotion words by psychology.

If the authors stated goal is "different than the standard sentiment analysis goal of predicting whether a sentence expresses positive or negative sentiment" they should be aware that this is exactly what PANAS is designed to do -

not to infer the latent emotional state of a person, except to the extent that their affect is positive or negative.

The work of representing emotions had been an field in psychology for over a hundred years

and it is still continuing.

One of the most popular theories of emotion is the theory that there exist “basic” emotions: Anger, Disgust, Fear, Happiness (enjoyment), Sadness and Surprise

(Paul Ekman, cited by the authors).

These are short duration sates lasting only seconds.

They are also fairly specific,

for example “surprise” is sudden reaction to something unexpected,

which is it exactly the same as seeing a flower on your car and expressing “what a nice surprise.”

The surprise would be the initial reaction of “what’s that on my car?  Is it dangerous?”

but after identifying the object as non-threatening, the emotion of “surprise” would likely pass and be replaced with appreciation.

The Circumplex Model of Emotions (Posner et al 2005) the authors refer to actually stands in opposition to the theories of Ekman.

From the cited paper by Posner et al :

"The circumplex model of affect proposes that all affective states arise from cognitive interpretations of core neural sensations that are the product of two independent neurophysiological systems. This model stands in contrast to theories of basic emotions, which posit that a discrete and independent neural system subserves every emotion."

From my reading of this paper, it is clear to me that the authors do not have a clear understanding of the current state of psychology’s view of emotion representation

and this work would not likely contribute to a new understanding of the latent structure of peoples’ emotions.

In the PCA result, it is not "clear" that the first axis represents valence,

as "sad" has a slight positive on this scale

and "sad" is one of the emotions most clearly associated with negative valence.

With respect to the rest of the paper, the level of novelty and impact is "ok, but not good enough."

This analysis does not seem very different from Twitter analysis,

because although Tumblr posts are allowed to be longer than Twitter posts,

the authors truncate the posts to 50 characters.

Additionally, the images do not seem to add very much to the classification.

The authors algorithm also seems to be essentially a combination of two other, previously published algorithms.

For me the novelty of this paper was in its application to the realm of emotion theory,

but I do not feel there is a contribution here.

This paper is more about classifying Tumblr posts according to emotion word hashtags than a paper that generates a new insights into emotion representation or that can infer latent emotional state.

fact fact evaluation evaluation evaluation fact evaluation fact fact evaluation fact quote fact reference fact reference fact fact fact fact fact fact fact fact reference fact reference fact evaluation fact fact evaluation evaluation fact fact quote evaluation evaluation evaluation fact evaluation evaluation evaluation fact fact evaluation fact evaluation evaluation evaluation


This paper proposes an extremely simple methodology to improve the network's performance by adding extra random perturbations (resizing/padding) at evaluation time.

Although the paper is very basic,

it creates a good baseline for defending about various types of attacks and got good results in kaggle competition.

The main merit of the paper is to study this simple but efficient baseline method extensively and shows how adversarial attacks can be mitigated by some extent.

Cons of the paper: there is not much novel insight or really exciting new ideas presented.

Pros: It gives a convincing very simple baseline

and the evaluation of all subsequent results on defending against adversaries will need to incorporate this simple defense method in addition to any future proposed defenses,

since it is very easy to implement and evaluate and seems to improve the defense capabilities of the network to a significant degree.

So I assume that this paper will be influential in the future just by the virtue of its easy applicability and effectiveness.

evaluation evaluation evaluation evaluation evaluation evaluation fact evaluation evaluation


The authors investigate a modified input layer that results in color invariant networks.

The proposed methods are evaluated on two car datasets.

It is shown that certain color invariant "input" layers can improve accuracy for test-images from a different color distribution than the training images.

The proposed assumptions are not well motivated and seem arbitrary.

Why is using a permutation of each pixels' color a good idea?

The paper is very hard to read.

The message is unclear

and the experiments to prove it are of very limited scope, i.e. one small dataset with the only experiment purportedly showing generalization to red cars.

Some examples of specific issues:- the abstract is almost incomprehensible and it is not clear what the contributions are

- Some references to Figures are missing the figure number, eg. 3.2 first paragraph,

- It is not clear how many input channels the color invariant functions use, eg. p1 does it use only one channel and hence has fewer parameters?

- are the training and testing sets all disjoint (sec 4.3)?

- at random points figures are put in the appendix, even though they are described in the paper and seem to show key results (eg "tested on nored-test")

- Sec 4.6: The explanation for why the accuracy drops for all models is not clear.

Is it because the total number of training images drops? If that's the case the whole experimental setup seems flawed.

- Sec 4.6: the authors refer to the "order net" beating the baseline,

however, from Fig 8 (right most) it appears as if all models beat the baseline.

In the conclusion they say that weighted order net beats the baseline on all three test sets w/o red cars in the training set.

Is that Fig 8 @0%?

The baseline seems to be best performing on "all cars" and "non-red cars"

In order to be at an appropriate level for any publication the experiments need to be much more general in scope.

fact fact fact evaluation non-arg evaluation evaluation evaluation evaluation fact evaluation non-arg fact evaluation evaluation fact fact fact non-arg fact request


The paper explores momentum SGD and an adaptive version of momentum SGD which the authors name YF (Yellow Fin).

They compare YF to hand tuned momentumSGD and to Adam in several deep learning applications.

I found the first part which discusses the theoretical motivation behind YF to be very confusing and misleading:

Based on the analysis of 1-dimensional problems, the authors design a framework and an algorithm that  supposedly ensures accelerated convergence.

There are two major problems with this approach:-First: Exploring 1-dim functions is indeed a nice way to get some intuition.

Yet,  algorithms that work in the 1-dim case do not trivially generalize to high dimensions,

and such reasoning might lead to very bad solutions.

-Second: Accelerated GD does not benefit over GD in the 1-dim case.

And therefore, this is not an appropriate setting to explore acceleration.

Concretely, the definition of the generalized condition number $\nu$, and relating it to the standard definition of the condition number $\kappa$, is very misleading.

This is since $\kappa =1$ for 1-dim problems,

and therefore accelerated GD does not have any benefits over non accelerated GD in this case.

However, $\nu$ might be much larger than 1 even in the 1-dim case.

Regarding the algorithm itself: there are too many hyper-parameters (which depend on each other) that are tuned (per-dimension).

And as I have mentioned, the design of the algorithm is inspired by the analysis of 1-dim quadratic functions.

Thus, it is very hard for me to believe that this algorithm works in practice unless very careful fine tuning is employed.

The authors mention that their experiments were done without tuning or with very little tuning, which is very mysterious for me.

In contrast to the theoretical part, the experiments seems very encouraging.

Showing YF to perform very well on several deep learning tasks without (or with very little) tuning.

Again, this seems a bit magical or even too good to be truth.

I suggest the authors to perform a experiment with say a qaudratic high dimensional function, which is not aligned with the axes in order to illustrate how their method behaves and try to give intuition.

fact fact evaluation evaluation evaluation evaluation evaluation fact evaluation evaluation fact evaluation evaluation evaluation evaluation evaluation evaluation evaluation evaluation evaluation request


The paper investigates the iterative estimation view on gated recurrent networks (GNN).

Authors observe that the average estimation error between a given hidden state and the last hidden state  gradually decreases toward zeros.

This suggest that GNN are bias toward an identity mapping and learn to preserve the activation through time.

Given this observation, authors then propose RIN, a new RNN parametrization where the hidden to hidden matrix is decomposed as a learnable weight matrix plus the identity matrix.

Authors evaluate their RIN on the adding, sequential MNIST and the baby tasks and show that their IRNN outperforms the IRNN and LSTM models.

Questions:- Section 2 suggests that use of the gate  in GNNs encourages to learn an identity mapping.

Does the average iteration error behaves differently in case of a tanh-RNN ?

- It seems from Figure 4 (a) that the average estimation error is higher for RIN than IRNN and LSTM and only decrease toward zero at the very end.

What could explain this phenomenon?

- While the LSTM baseline matches the results of Le et al.,

later work such as Recurrent Batch Normalization or Unitary Evolution RNN have demonstrated much better performance with a vanilla LSTM on those tasks (outperforming both IRNN and RIN).

What could explain this difference in the performances?

- Unless I am mistaken, Gated Orthogonal Recurrent Units: On Learning to Forget from Jing et al. also reports better performances for the LSTM (and GRU) baselines that outperform RIN on the baby tasks with mean performances of 58.2 and 56.0 for GRU and LSTM respectively?

- Quality/Clarity:The paper is well written and pleasant to read

- Originality:Looking at RNN from an iterative refinement point of view seems novel.

- Significance:While looking at RNN from an iterative estimation is interesting,

the experimental part does not really show what are the advantages of the propose RIN.

In particular, the LSTM baseline seems to weak compared to other works.

fact fact fact fact fact fact non-arg fact non-arg fact fact non-arg fact evaluation evaluation evaluation fact fact


This paper discusses an application of survival analysis  in social networks.

While the application area seems to be pertinent, the statistics as presented in this paper are suboptimal at best.

There is no useful statistical setup described (what is random? etc etc),

the interplay between censoring and end-of-life is left rather fuzzy,

and mentioned clustering approaches are extensively studied in the statistical literature in so-called frailty analysis.

The setting is also covered in statistics in the extensive literature on repeated measurements  and even time-series analysis.

It's up to the authors discuss similarities and differences of results of the present approach and those areas.

The numerical result is not assessing the different design decisions of the approach (why use a Kuyper loss?) in this empirical paper.

fact evaluation fact evaluation fact fact request evaluation