The paper presents a Depthwise Separable Graph Convolution network that aims at generalizing Depthwise convolutions, that exhibit a nice performance in image related tasks, to the graph domain.

In particular it targets Graph Convolutional Networks.

In the abstract the authors mention that the Depthwise Separable Graph Convolution that they propose is the key to understand the connections between geometric convolution methods and traditional 2D ones.

I am afraid I have to disagree

as the proposed approach is not giving any better understanding of what needs to be done and why.

It is an efficient way to mimic what has worked so far for the planar domain

but I would not consider it as fundamental in "closing the gap".

I feel that the text is often redundant and that it could be simplified a lot.

For example the authors state in various parts that DSC does not work on non-Euclidean data.

Section 2 should be clearer and used to better explain related approaches to motivate the proposed one.

In fact, the entire motivation, at least for me, never went beyond the simple fact that this happens to be a good way to improve performance.

The intuition given is not sufficient to substantiate some of the claims on generality and understanding of graph based DL.

In 3.1, at point (2), the authors mention that DSC filters are learned from the data whereas GC uses a constant matrix.

This is not correct,

as also reported in equation 2.

The matrix U is learned from the data as well.

Equation (4) shows that the proposed approach would weight Q different GC layers.

In practical terms this is a linear combination of these graph convolutional layers.

What is not clear is the \Delta_{ij} definition.

It is first introduced in 2.3 and described as the relative position of pixel i and pixel j on the image, but then used in the context of a graph in (4).

What is the coordinate system used by the authors in this case?

This is a very important point that should be made clearer.

Why is the Related Work section at the end?

I would put it at the front.

The experiments compare with the recent relevant literature.

I think that having less number of parameters is a good thing in this setting

as the data is scarce,

however I would like to see a more in-depth comparison with respect to the number of features produced by the model itself.

For example GCN has a representation space (latent) much smaller than DSCG.

No statistics over multiple runs are reported,

and given the high variance of results on these datasets I would like them to be reported.

I think the separability of the filters in this case brings the right level of simplification to the learning task,

however as it also holds for the planar case it is not clear whether this is necessarily the best way forward.

What are the underlying mathematical insights that lead towards selecting separable convolutions?

Overall I found the paper interesting but not ground-breaking.

A nice application of the separable principle to GCN.

Results are also interesting

but should be further verified by multiple runs.

fact fact fact evaluation fact fact evaluation evaluation fact request evaluation evaluation fact fact fact fact fact fact evaluation fact request request evaluation request fact evaluation evaluation request evaluation fact request evaluation evaluation request evaluation evaluation evaluation request


This paper proposes a new way of sampling data for updates in deep-Q networks.

The basic principle is to update Q values starting from the end of the episode in order to facility quick propagation of rewards back along the episode.

The paper is interesting,

but it lacks the proper comparisons to previously published techniques.

The results presented by this paper shows improvement over the baseline.

But the Atari results is still significantly worse than the current SOTA.

In the non-tabular case, the authors have actually moved away from Q learning and defined an objective that is both on and off-policy.

Some (theoretical) analysis would be nice.

It is hard to judge whether the objective defined in the non-tabular defines a contraction operator at all in the tabular case.

There has been a number of highly relevant papers.

Prioritized replay, for example, could have a very similar effect to proposed approach in the tabular case.

In the non-tabular case, the Retrace algorithm, tree backup, Watkin's Q learning all bear significant resemblance to the proposed method.

Although the proposed algorithm is different from all 3,

the authors should still have compared to at least one of them as a baseline.

The Retrace algorithm specifically has also been shown to help significantly in the Atari case,

and it defines a convergent update rule.

fact fact evaluation evaluation fact fact fact request evaluation evaluation evaluation evaluation evaluation request fact fact


This paper addresses the important problem of understanding mathematically how GANs work.

The approach taken here is to look at GAN through the lense of the scattering transform.

Unfortunately the manuscrit submitted is very poorly written.

Introduction and flow of thoughts is really hard to follow.

In method sections, the text jumps from one concept to the next without proper definitions.

Sorry I stopped reading on page 3.

I suggest to rewrite this work before sending it to review.

Among many things: - For citations use citep and not citet to have () at the right places.

- Why does it seems -> Why does it seem etc.

fact fact evaluation evaluation evaluation evaluation request request request


=== SUMMARY === The paper considers a combination of Reinforcement Learning (RL) and Imitation Learning (IL), in the infinite horizon discounted MDP setting.

The IL part is in the form of an oracle that returns a value function V^e, which is an approximation of the optimal value function.

The paper defines a new cost (or reward) function based on V^e, through shaping (Eq. 1).

It is known that shaping does not change the optimal policy.

A key aspect of this paper is to consider a truncated horizon problem (say horizon k) with the reshaped cost function, instead of an infinite horizon MDP.

For this truncated problem, one can write the (dis)advantage function as a k-step sum of reward plus the value returned by the oracle at the k-th step (cf. Eq. 5).

Theorem 3.3 shows that the value of the optimal policy of the truncated MDP w.r.t. the original MDP is only O(gamma^k eps) worse than the optimal policy of the original problem (gamma is the discount factor and eps is the error between V^e and V*).

This suggests two things: 1) Having an oracle that is accurate (small eps) leads to good performance.

If oracle is the same as the optimal value function, we do not need to plan more than a single step ahead.

2) By planning for k steps ahead, one can decrease the error in the oracle geometrically fast.

In the limit of k —> inf, the error in the oracle does not matter.

Based on this insight, the paper suggests an actor-critic-like algorithm called THOR (Truncated HORizon policy search) that minimizes the total cost over a truncated horizon with a modified cost function.

Through a series of experiments on several benchmark problems (inverted pendulum, swimmer, etc.), the paper shows the effect of planning horizon k.

=== EVALUATION & COMMENTS === I like the main idea of this paper.

The paper is also well-written.

But one of the main ideas of this paper (truncating the planning horizon and replacing it with approximation of the optimal value function) is not new and has been studied before,

but has not been properly cited and discussed.

There are a few papers that discuss truncated planning.

Most closely is the following paper:

Farahmand, Nikovski, Igarashi, and Konaka, “Truncated Approximate Dynamic Programming With Task-Dependent Terminal Value,” AAAI, 2016.

The motivation of AAAI 2016 paper is different from this work.

The goal there is to speedup the computation of finite, but large, horizon problem with a truncated horizon planning.

The setting there is not the combination of RL and IL, but multi-task RL.

An approximation of optimal value function for each task is learned off-line and then used as the terminal cost.

The important point is that the learned function there plays the same role as the value provided by the oracle V^e in this work.

They both are used to shorten the planning horizon.

That paper theoretically shows the effect of various error terms, including terms related to the approximation in the planning process (this paper does not do that).

Nonetheless, the resulting algorithms are quite different.

The result of this work is an actor-critic type of algorithm.

AAAI 2016 paper is an approximate dynamic programming type of algorithm.

There are some other papers that have ideas similar to this work in relation to truncating the horizon.

For example, the multi-step lookahead policies and the use of approximate value function as the terminal cost in the following paper:

Bertsekas, “Dynamic Programming and Suboptimal Control: A Survey from ADP to MPC,” European Journal of Control, 2005.

The use of learned value function to truncate the rollout trajectory in a classification-based approximate policy iteration method has been studied by Gabillon, Lazaric, Ghavamzadeh, and Scherrer, “Classification-based Policy Iteration with a Critic,” ICML, 2011.

Or in the context of Monte Carlo Tree Search planning, the following paper is relevant:

Silver et al., “Mastering the game of Go with deep neural networks and tree search,” Nature, 2016.

Their “value network” has a similar role to V^e.

It provides an estimate of the states at the truncated horizon to shorten the planning depth.

Note that even though these aforementioned papers are not about IL,

this paper’s stringent requirement of having access to V^e essentially make it similar to those papers.

In short, a significant part of this work’s novelty has been explored before.

Even though not being completely novel is totally acceptable,

it is important that the paper better position itself compared to the prior art.

Aside this main issue, there are some other comments: - Theorem 3.1 is not stated clearly and may suggest more than what is actually shown in the proof.

The problem is that it is not clear about the fact the choice of eps is not arbitrary.

The proof works only for eps that is larger than 0.5.

With the construction of the proof, if eps is smaller than 0.5, there would not be any error, i.e., J(\hat{pi}^*) = J(pi^*).

The theorem basically states that if the error is very large (half of the range of value function), the agent does not not perform well.

Is this an interesting case?

- In addition to the papers I mentioned earlier, there are some results suggesting that shorter horizons might be beneficial and/or sufficient under certain conditions.

A related work is a theorem in the PhD dissertation of Ng:

Andrew Ng, Shaping and Policy Search in Reinforcement Learning, PhD Dissertation, 2003.

(Theorem 5 in Appendix 3.B: Learning with a smaller horizon).

It is shown that if the error between Phi (equivalent to V^e here) and V* is small, one may choose a discount factor gamma’ that is smaller than gamma of the original MDP, and still have some guarantees.

As the discount factor has an interpretation of the effective planning horizon,

this result is relevant.

The result, however, is not directly comparable to this work

as the planning horizon appears implicitly in the form of 1/(1-gamma’) instead of k,

but I believe it is worth to mention and possibly compare.

- The IL setting in this work is that an oracle provides V^e, which is the same as (Ross & Bagnell, 2014).

I believe this setting is relatively restrictive

as in many problems we only have access to (state, action) pairs, or sequence thereof, and not the associated value function.

For example, if a human is showing how a robot or a car should move, we do not easily have access to V^e (unless the reward function is known and we estimate the value with rollouts; which requires us having a long trajectory).

This is not a deal breaker,

and I would not consider this as a weakness of the work,

but the paper should be more clear and upfront about this.

- The use of differential operator nabla instead of gradient of a function (a vector field) in Equations (10), (14), (15) is non-standard.

- Figures are difficult to read,

as the colors corresponding to confidence regions of different curves are all mixed up.

Maybe it is better to use standard error instead of standard deviation.

fact fact fact fact fact fact fact fact fact fact fact fact fact evaluation evaluation fact fact fact evaluation reference fact fact fact fact fact fact fact evaluation fact fact fact fact reference fact fact reference evaluation fact fact evaluation evaluation evaluation request evaluation evaluation fact fact fact evaluation fact fact reference quote fact fact evaluation evaluation fact request fact evaluation fact fact evaluation evaluation request fact evaluation fact request


The paper discusses a setting in which an existing dataset/trained model is augmented/refined by adding additional datapoints.

Issues of how to price the new data are discussed in a high level, abstract way, and arguments against retrieving the new data for free or encrypting it are presented.

Overall, the paper is of an expository nature,

discussing high-level ideas rather than actually implementing them,

and does not  experimentally or theoretically substantiate any of its claims.

This makes the technical contribution rather shallow.

Interesting questions do arise, such as how to assess the value of new data and how to price datapoints,

but these questions are never addressed (neither theoretically nor empirically).

Though main points are valid,

the paper is also rife with informal statements  and logical jumps,

perhaps due to the expository/high-level approach taken in discussing these issues.

Detailed comments:The (informal) information theoretic argument has a few holes.

The claim is roughly that every datapoint (~1Mbyte image) contributes ~1M bits of changes in a model,

which can be quite revealing.

As a result, there is no benefit from encrypting the datapoint, as the mapping from inputs to changes is insecure (in an information-theoretic sense) in itself.

This assumes that every step of stochastic gradient descent (one step per image) is done in the clear;

this is not what one would consider secure in cryptography literature.

A secure function evaluation (SFE) would encrypt the data and the computation in an end-to-end fashion;

in particular, it would only reveal the final outcome of SGD over all images in the dataset without revealing any intermediate steps.

Presuming that the new dataset is large (i.e., having N images), the "information theoretic" limit becomes ~N x 1Mbyte inputs for ~1M function outputs (the finally-trained model).

In this sense, this argument that "encryption is hopeless" is somewhat brittle.

Encryption-issues aside, the paper would have been much stronger if it spent more effort in formalizing or evaluating different methods for assessing the value of data.

The authors approach this by treating the ML algorithm as a blackbox, and using influence functions (a la Bastani 2017) to assess the impact of different inputs on the finally trained model

(again, this is proposed but not implemented/explored/evaluated in any way).

This is a design choice, but it is not obvious.

There is extensive literature in statistics and machine learning on the areas of experimental design and active learning.

Both are active, successful research areas, and both can be provide tools to formally reason about the value of data/labels not yet seen;

the paper summarily ignores this literature.

Examples of imprecise/informal statements: "The fairness in the pricing is highly questionable"

"implicit contracts get difficult to verify"

"The fairness in the pricing is dubious"

"As machine learning models become more and more complicated, its (sic) capability can outweigh the privacy guarantees encryption gives us"

"as an image classifier's model architecture changes, all the data would need to be collected and purchased again"

"Interpretability solutions aim to alleviate the notoriety of reasonability of neural networks"

fact fact evaluation fact fact evaluation fact fact evaluation evaluation fact evaluation fact evaluation fact fact evaluation fact fact fact evaluation request fact fact evaluation fact evaluation evaluation quote quote quote quote quote quote


The authors present RDA, the Recurrent Discounted Attention unit, that improves upon RWA, the earlier introduced Recurrent Weighted Average unit, by adding a discount factor.

While the RWA was an interesting idea with bad results (far worse than the standard GRU or LSTM with standard attention except for hand-picked tasks),

the RDA brings it more on-par with the standard methods.

On the positive side, the paper is clearly written and adding discount to RWA, while a small change, is original.

On the negative side, in almost all tasks the RDA is on par or worse than the standard GRU -

except for MultiCopy where it trains faster, but not to better results

and it looks like the difference is between few and very-few training steps anyway.

The most interesting result is language modeling on Hutter Prize Wikipedia,

where RDA very significantly improves upon RWA -

but again, only matches a standard GRU or LSTM.

So the results are not strongly convincing,

and the paper lacks any mention of newer work on attention.

This year strong improvements over state-of-the-art have been achieved using attention for translation ("Attention is All You Need") and image classification (e.g., Non-local Neural Networks, but also others in ImageNet competition).

To make the evaluation convincing enough for acceptance, RDA should be combined with those models and evaluated more competitively on multiple widely-studied tasks.

fact evaluation evaluation evaluation evaluation fact evaluation evaluation fact fact evaluation fact fact request


This manuscript is fairly well-written,

and discusses how the batch normalization step helps to stabilize the scale of the gradients.

Intriguingly, the analysis suggests that using a shallower but wider resnet should provide competitive performance, which is supported by empirical evidence.

This work should help elucidate the structure in the learning, and help to support efforts to improve both learning algorithms and the architecture.

Pros: Clean, simple analysis

Empirical support suggests that theory captures reasonable effects behind learning

Cons: The reasonableness of the assumptions used in the analysis needs a more careful analysis.

In particular, the assumption that all weights are independent is valid only at the first random iteration.

Therefore, the utility of this theory during initialization seems reasonable,

but during learning the theory seems quite tenuous.

I would encourage the authors to discuss their assumptions, and talk about how the math would change as a result of relaxing the assumptions.

The empirical support does provide evidence that the theory is reasonable.

However, it is limited to a single dataset.

It would be nice to see that the effect happens more generally.

Second, it is clear that shallow+wide networks may be better than deep+narrow networks,

but it's not clear about how the width is evaluated and supported.

I would encourage the authors to do more extensive experiments and evaluate the architecture further.

evaluation fact fact evaluation evaluation fact request fact evaluation evaluation request fact fact evaluation evaluation evaluation request


In this paper, an number of very strong (even extraordinary) claims are made:

* The abstract promises "a framework to understand the unprecedented performance and robustness of deep neural networks using field theory."

* Page 8 states that this is "This is a first attempt to describe a neural network with a scalar quantum field theory."

* Page 2 promises the use of the "Goldstone theorem" (no less) to understand phase transition in deep learning

* It also claim that many "seemingly different experimental results can be explained by the presence of these zero eigenvalue weights."

* Three important results are stated as "theorem", with a statement like "Deep feedforward networks learn by breaking symmetries" proven in 5 lines, with no formal mathematics.

These are extraordinary claims,

but  when reaching page 5, one sees that the basis of these claims seems to be the Lagrangian of a simple phi-4 theory,

and Fig. 1 shows the standard behaviour of the so-called mexican hat in physics, the basis of the second-order transition.

Given physicists have been working on neural network for more than three or four decades,

I am surprise that this would enough to solve all these problems!

I tried to understand these many results,

but I am afraid I cannot really understand or see them.

In many case, the explanation seems to be a vague analogy.

These are not without interest,

and maybe there is indeed something deep in this paper, but it is so far hidden by the hype.

Still, I fail to see how the fact that phase transitions and negative direction in the landscape is a new phenomena, and how it explains all the stated phenomenology.

Beside, there are quite a lot of things known about the landscape of these problems

Maybe I am indeed missing something,

but i clearly suspect the authors are simply overselling physics results.

I have been wrong many times,

but I beleive that the authors should probably precise their claim, and clarify the relation between their results and both the physics AND statistics litterature, or better, with the theoretical physics litterature applied to learning, which is ---astonishing-- absent in the paper.

About the content: The main problem for me is that the whole construction using field theory seems to be used to advocate for the appearence of a phase transition in neural nets and in learning.

This rises three comments: (1) So we really need to use quantum field theory for this?

I do not see what should be quantum here

(despite the very vague remarks page 12 "WHY QUANTUM FIELD THEORY?")

(2) This is not new.

Phase transitions in learning in neural nets are being discussed since aboutn 40 years, see for instance all the pionnering work of Sompolinky et al.

one can see for instance the nice review in https://arxiv.org/abs/1710.09553

In non aprticular order, phase transition and symmetry breaking are discussed in * "Statistical mechanics of learning from examples", Phys. Rev. A 45, 6056 – Published 1 April 1992

* "The statistical mechanics of learning a rule", Rev. Mod. Phys. 65, 499 – Published 1 April 1993

* Phase transitions in the generalization behaviour of multilayer neural networks

* Note that some of these results are now rigourous,

as shown in "Phase Transitions, Optimal Errors and Optimality of Message-Passing in Generalized Linear Models", https://arxiv.org/abs/1708.03395

* The landscape of these problems has been studied quite extensivly,

see for instance "Identifying and attacking the saddle point problem in high-dimensional non-convex optimization", https://arxiv.org/abs/1406.2572

(3) There is nothing particular about deep neural net and neural nets about this.

Negative direction in the Hessian in learning problems appears in matrix and tensor factorizaion, where phase transition are well understood (even rigorously, see for instance, https://arxiv.org/abs/1711.05424 ) or in problems such as unsupervised learning, as e.g.:

Here are additional comments: PAGE 1: * "It has been discovered that the training process ceases when it goes through an information bottleneck (ShwartzZiv & Tishby, 2017)".

While this paper indeed make a nice suggestion, I would not call it a discovery yet as this has never been shown on a large network.

Beside, another paper in the conference is claiming exacly the opposite,

see : "On the Information Bottleneck Theory of Deep Learning".

This is still subject of discussion.

* "In statistical terms, a quantum theory describes errors from the mean of random variables. "

Last time I studied quantum theory, it was a theory that aim to explain the physical behaviours at the molecular, atomic and sub-atomic levels, usinge either on the wave function (Schrodinger) or the Matrix operatir formalism (Hesienbger) (or if you want, the path integral formalism of Feynman).

It is certainly NOT a theory that describes errors from the mean of random variables.

This is, i beleive, the field of "statistics" or "probability" for correlated variables.

It is certianly used in physics, and heavily both in statistical physics and in quantum thoery,

but this is not what the theory is about in the first place.

Beside, there is little quantum in this paper,

I think most of what the authors say apply to a statistical field theory

( https://en.wikipedia.org/wiki/Statistical_field_theory )

* "In the limit of a continuous sample space, the quantum theory becomes a quantum field theory."

Again, what is quantum about all this?

This true for a field theory, as well for continous theories of, say, mechanics, fracture, etc...

PAGE 2: * "Using a scalar field theory we show that a phase transition must exist towards the end of training based on empirical results."

So it is a scalar classical field theory after all.

This sounds a little bit less impressive that a quantum field theory.

Note that the fact that phase transition arises in learning, and in a statistical theory applied to any learning process, is an old topic, with a classical litterature.

The authors might be interested by the review "The statistical mechanics of learning a rule", Rev. Mod. Phys. 65, 499 – Published 1 April 1993

PAGE 8: * "In this work we solved one of the most puzzling mysteries of deep learning by showing that deep neural networks undergo spontaneous symmetry breaking."

I am afraid I fail to see what is so mysterious about this nor what the authors showed about it.

In any case, gradient descent break symmetry spontaneously in many systems, including phi-4, the Ising model or (in learning problems) the community detection problem

(see eg https://journals.aps.org/prx/abstract/10.1103/PhysRevX.4.011047).

I am afraid I miss what is new there...

* "This is a first attempt to describe a neural network with a scalar quantum field theory."

Given there seems to be little quantum in the paper,

I fail to see the relevance of the statement.

Secondly, I beleive that field theory has been used, many times and in greater lenght, both for statistical and dynamical problems in neural nets, see eg.

* http://iopscience.iop.org/article/10.1088/0305-4470/27/6/016/meta

* https://arxiv.org/pdf/q-bio/0701042.pdf

* http://www.lps.ens.fr/~derrida/PAPIERS/1987/gardner-zippelius-87.pdf

* http://iopscience.iop.org/article/10.1088/0305-4470/21/1/030/meta

* https://arxiv.org/pdf/cond-mat/9805073.pdf

evaluation fact quote fact quote fact evaluation evaluation fact fact evaluation non-arg evaluation evaluation evaluation evaluation evaluation evaluation non-arg evaluation non-arg evaluation evaluation non-arg evaluation non-arg fact fact reference reference reference reference reference fact reference fact reference evaluation fact reference reference quote evaluation fact reference evaluation quote fact evaluation evaluation fact fact evaluation evaluation reference quote non-arg fact quote fact evaluation fact evaluation quote evaluation fact reference evaluation quote evaluation evaluation evaluation reference reference reference reference reference


Summary:The paper proposes a new dialog model combining both retrieval-based and generation-based modules.

Answers are produced in three phases: a retrieval-based model extracts candidate answers; a generator model, conditioned on retrieved answers, produces an additional candidate; a reranker outputs the best among all candidates.

The approach is interesting:

the proposed ensemble can improve on both the retrieval module and the generation module,

since it does not restrict modeling power (e.g. the generator is not forced to be consistent with the candidates).

I am not aware of similar approaches for this task.

One work that comes to mind regarding the blend of retrieval and generation is Memory Networks

(e.g. https://arxiv.org/pdf/1606.03126.pdf and references):

given a query, a set of relevant memories is extracted from a KB using an inverted index and the memories are fed into the generator.

However, the extracted items in the current work are candidate answers which are used both to feed the generator and to participate in reranking.

The experimental section focuses on the task of building conversational systems.

The performance measures used are 1) a human evaluation score with three volunteers and 2) BLUE scores.

While these methods are not very satisfying,

effective evaluation of such systems is a known difficulty.

The results show that the ensemble outperforms the individual modules, indicating that:

the multi-seq2seq models have learned to use the new inputs as needed and that the ranker is correlated with the evaluation metrics.

However, the results themselves do not look impressive to me:

the subjective evaluation is close to the "borderline" score;

in the examples provided, one is good, the other is borderline/bad, and the baseline always provides something very short.

Does the LSTM work particularly poor on this dataset?

Given that this is a novel dataset, I don't know what the state-of-the-art should be.

Could you provide more insight?

Have you considered adding a benchmark dataset (e.g. a QA dataset)?

Specific questions:1. The paper motivates conditioning on the candidates in two ways.

First, that the candidates bring additional information which the decoder can use (e.g. read from the candidates locations, actions, etc.).

Second, that the probability of universal replies must decrease due to the additional condition.

I think the second argument depends on how the conditioning is performed:

if the candidates are simply appended to the input, the model can learn to ignore them.

2. The copy mechanism is a nice touch, encouraging the decoder to use the provided queries.

Why not copy from the query too, e.g. with some answers reusing part of the query <"Where are you going?", "I'm going to the park">?

3. How often does the model select the generated answer vs. the extracted answers?

In both examples provided the selected answer is the one merging the candidate answers.

Minor issues:- Section 3.2: using and the state

- Section 3.2: more than one replies

- last sentence on page 3: what are the "following principles"?

fact fact evaluation fact fact evaluation evaluation reference fact fact fact fact evaluation evaluation fact fact evaluation evaluation fact request evaluation request request fact fact fact fact fact evaluation request request fact fact fact request


This paper presents an analysis of the communication systems that arose when neural network based agents played simple referential games.

The set up is that a speaker and a listener engage in a game where both can see a set of possible referents (either represented symbolically in terms of features, or represented as simple images) and the speaker produces a message consisting of a sequence of numbers while the listener has to make the choice of which referent the speaker intends.

This is a set up that has been used in a large amount of previous work,

and the authors summarize some of this work.

The main novelty in this paper is the choice of models to be used by speaker and listener,

which are based on LSTMs and convolutional neural networks.

The results show that the agents generate effective communication systems,

and some analysis is given of the extent to which these communications systems develop compositional properties

– a question that is currently being explored in the literature on language creation.

This is an interesting question,

and it is nice to see worker playing modern neural network models to his question and exploring the properties of the solutions of the phone.

However, there are also a number of issues with the work.

1. One of the key question is the extent to which the constructed communication systems demonstrate compositionality.

The authors note that there is not a good quantitative measure of this.

However, this is been the topic of much research of the literature and language evolution.

This work has resulted in some measures that could be applied here,

see for example Carr et al. (2016): http://www.research.ed.ac.uk/portal/files/25091325/Carr_et_al_2016_Cognitive_Science.pdf

2. In general the results occurred be more quantitative.

In section 3.3.2 it would be nice to see statistical tests used to evaluate the claims.

Minimally I think it is necessary to calculate a null distribution for the statistics that are reported.

3. As noted above the main novelty of this work is the use of contemporary network models.

One of the advantages of this is that it makes it possible to work with more complex data stimuli, such as images.

However, unfortunately the image example that is used is still very artificial being based on a small set of synthetically generated images.

Overall, I see this as an interesting piece of work that may be of interest to researchers exploring questions around language creation and language evolution,

but I think the results require more careful analysis and the novelty is relatively limited, at least in the way that the results are presented here.

fact fact evaluation fact evaluation fact fact fact fact evaluation evaluation evaluation evaluation fact evaluation request reference request request request evaluation evaluation evaluation evaluation evaluation


This paper examines the very popular and useful ADAM optimization algorithm, and locates a mistake in its proof of convergence (for convex problems).

Not only that, the authors also show a specific toy convex problem on which ADAM fails to converge.

Once the problem was identified to be the decrease in v_t (and increase in learning rate), they modified the algorithm to solve that problem.

They then show the modified algorithm does indeed converge and show some experimental results comparing it to ADAM.

The paper is well written, interesting  and very important given the popularity of ADAM.

Remarks: - The fact that your algorithm cannot increase the learning rate seems like a possible problem in practice.

A large gradient at the first steps due to bad initialization can slow the rest of training.

The experimental part is limited,

as you state "preliminary",

which is a unfortunate for a work with possibly an important practical implication.

Considering how easy it is to run experiments with standard networks using open-source software,

this can easily improve the paper.

That being said, I understand that the focus of this work is theoretical and well deserves to be accepted based on the theoretical work.

- On page 14 the fourth inequality not is clear to me.

- On page 6 you talk about an alternative algorithm using smoothed gradients which you do not mention anywhere else

and this isn't that clear (more then one way to smooth).

A simple pseudo-code in the appendix would be welcome.

Minor remarks:- After the proof of theorem 1 you jump to the proof of theorem 6

(which isn't in the paper)

and then continue with theorem 2.

It is a bit confusing.

- Page 16 at the bottom v_t= ... sum beta^{t-1-i}g_i should be g_i^2

- Page 19 second line, you switch between j&t and it is confusing.

Better notation would help.

- The cifarnet uses LRN layer that isn't used anymore.

fact fact fact fact evaluation evaluation fact evaluation fact evaluation evaluation request evaluation evaluation fact evaluation request fact fact fact evaluation request evaluation request fact


My problem with this paper that all the theoretical contributions / the new approach refer to 2 arXiv papers,

what's then left is an application of that approach to learning form imperfect demonstrations.

Quality ====== The approach seems sound

but the paper does not provide many details on the underlying approach.

The application to learning from (partially adversarial) demonstrations is a cool idea

but effectively is a very straightforward application based on the insight that the approach can handle truly off-policy samples.

The experiments are OK

but I would have liked a more thorough analysis.

Clarity ===== The paper reads well,

but it is not really clear what the claimed contribution is.

Originality ========= The application seems original.

Significance ========== Having an RL approach that can benefit from truly off-policy samples is highly relevant.

Pros and Cons ============ + good results

+ interesting idea of using the algorithm for RLfD

- weak experiments for an application paper

- not clear what's new

evaluation fact evaluation evaluation evaluation fact evaluation request evaluation evaluation evaluation evaluation evaluation evaluation evaluation evaluation


Summary: This paper presents a thorough examination of the effects of pruning on model performance.

Importantly, they compare the performance of "large-sparse" models (large models that underwent pruning in order to reduce memory footprint of model) and "small-dense" models, showing that "large-sparse" models typically perform better than the "small-dense" models of comparable size (in terms of number of non-zero parameters, and/or memory footprint).

They present results across a number of domains (computer vision, language modelling, and neural machine translation) and model types (CNNs, LSTMs).

They also propose a way of performing pruning with a pre-defined sparsity schedule, simplifying the pruning process in a way which works across domains.

They are able to show convincingly that pruning is an effective way of trading off accuracy for model size (more effective than simply reducing the size of model architecture),

although there does come a point where too much sparsity degrades the model performance considerably;

this suggests that pruning a medium size model to 80%-90% sparsity is likely better than pruning a larger model to >= 95% sparsity.

Review: Quality: The quality of the work is high

--- the experiments are extensive and thorough.

I would have liked to see "small-dense" vs. "large-sparse" comparisons on Inception (only large-sparse results are reported).

Clarity: The paper is clearly written,

though there is room for improvement.

For example, many of the results are presented in a redundant manner (in both tables and figures, where the table and figure are often not next to each other in the document).

Also, it is not clear in several cases exactly which training/heldout/test sets are used, and on which partition of the data the accuracies/BLEU scores/perplexities presented correspond to.

A small section (before "Methods") describing the datasets/features in detail would be helpful.

Also, it would have probably been nice to explain all of the tasks and datasets early on, and then present all the results at once (NIT: include the plots in paper, and move the tables to an appendix).

Originality: Although the experiments are informative,

the work as a whole is not very original.

The method proposed of using a sparsity schedule to perform pruning is simple and effective,

but is a rather incremental contribution.

The primary contribution of this paper is its experiments, which for the most part compare known methods.

Significance: The paper makes a nice contribution,

though it is not particularly significant or surprising.

The primary observations are: (1) large-sparse is typically better than small-dense, for a fixed number of non-zero parameters and/or memory footprint.

(2) There is a point at which increasing the sparsity percentage severely degrades the performance of the model,

which suggests that there is a "sweet-spot" when it comes to choosing the model architecture and sparsity percentage which give the best performance (for a fixed memory footprint).

Result #1 is not very surprising,

given that Han et al (2016) were able to show significant compression without loss in accuracy;

thus, because one would expect a smaller dense model to perform worse than the large dense model,

it would also perform worse than the large sparse model.

Result #2 had already been seen in Han et al (2016) (for example, in Figure 6).

Pros: - Very thorough experiments across a number of domains

Cons: - Methodological contributions are minor.

- Results are not surprising, and are in line with previous papers.

fact fact fact fact evaluation fact evaluation evaluation evaluation request evaluation evaluation evaluation evaluation request request evaluation evaluation evaluation evaluation evaluation evaluation evaluation fact fact evaluation evaluation fact evaluation fact fact evaluation evaluation evaluation


Training GAN in a hierarchical optimization schedule shows promising performance recently (e.g. Zhao et al., 2016).

However, these works utilize the prior knowledge of the data (e.g. image)

and it's hard to generalize it to other data types (e.g. text).

The paper aims to learn these hierarchies directly instead of designing by human.

However, several parts are missing and not well-explained.

Also, many claims in paper are not proved properly by theory results or empirical results.

(1) It is not clear to me how to train the proposed algorithm.

My understanding is train a simple ALI, then using the learned latent as the input and train the new layer.

Do the authors use a separate training ? or a joint training algorithms.

The authors should provide a more clear and rigorous objective function.

It would be even better to have a pseudo code.

(2) In abstract, the authors claim the theoretical results are provided.

I am not sure whether it is sec 3.2

The claims is not clear and limited.

For example, what's the theory statement of [Johnsone 200; Baik 2005].

What is the error measure used in the paper?

For different error, the matrix concentration bound might be different.

Also, the union bound discussed in sec 3.2 is also problematic.

Lats, for using simple standard GAN to learn mixture of Gaussian, the rigorous theory result doesn't seem easy (e.g. [1])

The author should strive for this results if they want to claim any theory guarantee.

(3) The experiments part is not complete.

The experiment settings are not described clearly.

Therefore, it is hard to justify whether the proposed algorithm is really useful based on Fig 3.

Also, the authors claims it is applicable to text data in Section 1, this part is missing in the experiment.

Also, the idea of "local" disentangled LV is not well justified to be useful.

[1] On the limitations of first order approximation in GAN dynamics, ICLR 2018 under review

evaluation fact evaluation fact evaluation evaluation evaluation evaluation non-arg request request fact evaluation evaluation non-arg non-arg evaluation evaluation evaluation request evaluation evaluation evaluation fact evaluation reference


The paper proposes the use of a GAN to learn the distribution of image classes from an existing classifier,

that is a nice and straightforward idea.

From the point of view of forensic analysis of a classifier, it supposes a more principled strategy than a brute force attack based on the classification of a database and some conditional density estimation of some intermediate image features.

Unfortunately, the experiments are inconclusive.

Quality: The key question of the proposed scheme is the role of the auxiliary dataset.

In the EMNIST experiment, the results for the “exact same” and “partly same” situations are good,

but it seems that for the “mutually exclusive” situation the generated samples look like letters, not numbers,

and raises the question on the interpolation ability of the generator.

In the FaceScrub experiment is even more difficult to interpret the results,

basically because we do not even know the full list of person identities.

It seems that generated images contain only parts of the auxiliary images related to the most discriminative features of the given classifier.

Does this imply that the GAN models a biased probability distribution of the image class?

What is the result when the auxiliary dataset comes from a different kind of images?

Due to the difficulty of evaluating GAN results, more experiments are needed to determine the quality and significance of this work.

Clarity: The paper is well structured and written,

but Sections 1-4 could be significantly shorter to leave more space to additional and more conclusive experiments.

Some typos on Appendix A should be corrected.

Originality: the paper is based on a very smart and interesting idea and a straightforward use of GANs.

Significance: If additional simulations confirm the author’s claims, this work can represent a significant contribution to the forensic analysis of discriminative classifiers.

fact evaluation fact evaluation evaluation evaluation evaluation evaluation evaluation fact evaluation request request request evaluation request request evaluation evaluation


The paper describes a neural network-based approach to active localization based upon RGB images.

The framework employs Bayesian filtering to maintain an estimate of the agent's pose using a convolutional network model for the measurement (perception) function.

A convolutional network models the policy that governs the action of the agent.

The architecture is trained in an end-to-end manner via reinforcement learning.

The architecture is evaluated in 2D and 3D simulated environments of varying complexity and compared favorably to traditional (structured) approaches to passive and active localization.

As the paper correctly points out, there is large body of work on map-based localization,

but relatively little attention has been paid to decision theoretic formulations to localization, whereby the agent's actions are chosen in order to improve localization accuracy.

More recent work instead focuses on the higher level objective of navigation, whereby any effort act in an effort to improve localization are secondary to the navigation objective.

The idea of incorporating learned representations with a structured Bayesian filtering approach is interesting,

but it's utility could be better motivated.

What are the practical benefits to learning the measurement and policy model beyond (i) the temptation to apply neural networks to this problem and (ii) the ability to learn these in an end-to-end fashion?

That's not to say that there aren't benefits, but rather that they aren't clearly demonstrated here.

Further, the paper seems to assume (as noted below) that there is no measurement uncertainty and, with the exception of the 3D evaluations, no process noise.

The evaluation demonstrates that the proposed method yields estimates that are more accurate according to the proposed metric than the baseline methods, with a significant reduction in computational cost.

However, the environments considered are rather small by today's standards

and the baseline methods almost 20 years old.

Further, the evaluation makes a number of simplifying assumptions, the largest being that the measurements are not subject to noise

(the only noise that is present is in the motion for the 3D experiments).

This assumption is clearly not valid in practice.

Further, it is not clear from the evaluation whether the resulting distribution that is maintained is consistent (e.g., are the estimates over-/under-confident?).

This has important implications if the system were to actually be used on a physical system.

Further, while the computational requirements at test time are significantly lower than the baselines,

the time required for training is likely very large.

While this is less of an issue in simulation, it is important for physical deployments.

Ideally, the paper would demonstrate performance when transferring a policy trained in simulation to a physical environment (e.g., using diversification, which has proven effective at simulation-to-real transfer).

Comments/Questions:* The nature of the observation space is not clear.

* Recent related work has focused on learning neural policies for navigation, and any localization-specific actions are secondary to the objective of reaching the goal.

It would be interesting to discuss how one would balance the advantages of choosing actions that improve localization with those in the context of a higher-level task (or at least including a cost on actions as with the baseline method of Fox et al.).

* The evaluation that assigns different textures to each wall is unrealistic.

* It is not clear why the space over which the belief is maintained flips as the robot turns and shifts as it moves.

* The 3D evaluation states that a 360 deg view is available.

What happens when the agent can only see in one (forward) direction?

* AML includes a cost term in the objective.

Did the author(s) experiment with setting this cost to zero?

* The 3D environments rely upon a particular belief size (70 x 70) being suitable for all environments.

What would happen if the test environment was larger than those encountered in training?

* The comment that the PoseNet and VidLoc methods "lack a strainghtforward method to utilize past map data to do localization in a new environment" is unclear.

* The environments that are considered are quite small compared to the domains currently considered for

* Minor: It might be better to move Section 3 into Section 4 after introducing notation (to avoid redundancy).

* The paper should be proofread for grammatical errors (e.g., "bayesian" --> "Bayesian", "gaussian" --> "Gaussian")

fact fact fact fact fact fact fact fact evaluation evaluation request evaluation fact fact evaluation fact fact fact fact evaluation evaluation fact evaluation evaluation request evaluation fact request evaluation evaluation fact request fact non-arg fact request evaluation evaluation request request


In the centre loss, the centre is learned.

Now it's calculated as the average of the last layer's features

To enable training with SGD, the authors calculate the centre within a mini batch


The authors train an RNN to perform deduced reckoning (ded reckoning) for spatial navigation,

and then study the responses of the model neurons in the RNN.

They find many properties reminiscent of neurons in the mammalian entorhinal cortex (EC): grid cells, border cells, etc.

When regularization of the network is not used during training, the trained RNNs no longer resemble the EC.

This suggests that those constraints (lower overall connectivity strengths, and lower metabolic costs) might play a role in the EC's navigation function.

The paper is overall quite interesting and the study is pretty thorough:

no major cons come to mind.

Some suggestions / criticisms are given below.

1) The findings seem conceptually similar to the older sparse coding ideas from the visual cortex.

That connection might be worth discussing

because removing the regularizing (i.e., metabolic cost) constraint from your RNNS makes them learn representations that differ from the ones seen in EC.

The sparse coding models see something similar:

without sparsity constraints, the image representations do not resemble those seen in V1,

but with sparsity, the learned representations match V1 quite well.

That the same observation is made in such disparate brain areas (V1, EC) suggests that sparsity / efficiency might be quite universal constraints on the neural code.

2) The finding that regularizing the RNN makes it more closely match the neural code is also foreshadowed somewhat by the 2015 Nature Neuro paper by Susillo et al.

That could be worthy of some (brief) discussion.

Sussillo, D., Churchland, M. M., Kaufman, M. T., & Shenoy, K. V. (2015). A neural network that finds a naturalistic solution for the production of muscle activity. Nature neuroscience, 18(7), 1025-1033.

3) Why the different initializations for the recurrent weights for the hexagonal vs other environments?

I'm guessing it's because the RNNs don't "work" in all environments with the same initialization (i.e., they either don't look like EC, or they don't obtain small errors in the navigation task).

That seems important to explain more thoroughly than is done in the current text.

4) What happens with ongoing training?

Animals presumably continue to learn throughout their lives.

With on-going (continous) training, do the RNN neurons' spatial tuning remain stable, or do they continue to "drift" (so that border cells turn into grid cells turn into irregular cells, or some such)?

That result could make some predictions for experiment,

that would be testable with chronic methods (like Ca2+ imaging) that can record from the same neurons over multiple experimental sessions.

5) It would be nice to more quantitatively map out the relation between speed tuning, direction tuning, and spatial tuning (illustrated in Fig. 3).

Specifically, I would quantify the cells' direction tuning using the circular variance methods that people use for studying retinal direction selective neurons.

And I would quantify speed tuning via something like the slope of the firing rate vs speed curves.

And quantify spatial tuning somehow (a natural method would be to use the sparsity measures sometimes applied to neural data to quantify how selective the spatial profile is to one or a few specific locations).

Then make scatter plots of these quantities against each other.

Basically, I'd love to see the trends for how these types of tuning relate to each other over the whole populations:

those trends could then be tested against experimental data (possibly in a future study).

fact fact fact fact evaluation evaluation non-arg non-arg evaluation request fact evaluation fact evaluation evaluation evaluation request reference non-arg evaluation request non-arg evaluation non-arg evaluation evaluation request request request request request request evaluation


This paper considers the problem of autonomous lane changing for self-driving cars in multi-lane multi-agent slot car setting.

The authors propose a new learning strategy called Q-masking which couples well a defined low level controller with a high level tactical decision making policy.

The authors rightly say that one of the skills an autonomous car must have is the ability to change lanes,

however this task is not one of the most difficult for autonomous vehicles to achieve and this ability has already been implemented in real vehicles.

Real vehicles also decouple wayfinding with local vehicle control, similar to the strategy employed here.

To make a stronger case for this research being relevant to the real autonomous driving problem, the authors would need to compare their algorithm to a real algorithm and prove that it is more “data efficient.”

This is a difficult comparison

since the sensing strategies employed by real vehicles – LIDAR, computer vision, recorded, labeled real maps are vastly different from the slot car model proposed by the authors.

In term of impact, this is a theoretical paper looking at optimizing a sandbox problem where the results may be one day applicable to the real autonomous driving case.

In this paper the authors investigate “the use and place” of deep reinforcement learning in solving the autonomous lane change problem they propose a framework that uses Q-learning to learn “high level tactical decisions” and introduce “Q-masking” a way of limiting the problem that the agent has to learn to force it to learn in a subspace of the Q-values.

The authors claim that “By relying on a controller for low-level decisions we are also able to completely eliminate collisions during training or testing, which makes it a possibility to perform training directly on real systems.”

I am not sure what is meant by this since in this paper the authors never test their algorithm on real systems

and in real systems it is not possible to completely eliminate collisions.

If it were, this would be a much sought breakthrough.

Additionally for their experiment authors use the SUMO top view driving simulator.

This choice makes their algorithm not currently relevant to most autonomous vehicles that use ego-centric sensing.

This paper presents a learning algorithm that can “outperform a greedy baseline in terms of efficiency” and “humans driving the simulator in terms of safety and success” within their top view driving game.

The game can be programmed to have an “n” lane highway, where n could reasonable go up to five to represent larger highways.

The authors limit the problem by specifying that all simulated cars must operate between a preset minimum and maximum and follow a target (random) speed within these limits.

Cars follow a fixed model of behavior, do not collide with each other and cannot switch lanes.

It is unclear if the simulator extends beyond a single straight section of highway, as shown in Figure 1.

The agent is tasked with driving the ego-car down the n-lane highway and stopping at “the exit” in the right hand lane D km from the start position.

The authors use deep Q learning from Mnih et al 2015 to learn their optimal policy.

They use a sparse reward function of +10 for reaching the goal and -10x(lane difference from desired lane) as a penalty for failure.

This simple reward function is possible because the authors do not require the ego car to obey speed limits or avoid collisions.

The authors limit what the car is able to do

– for example it is not allowed to take actions that would get it off the highway.

This makes the high level learning strategy more efficient

because it does not have to explore these possibilities (Q-masking).

The authors claim that this limitation of the simulation is made valid by the ability of the low level controller to incorporate prior knowledge and perfectly limit these actions.

In the real world, however, it is unlikely that any low level controller would be able to do this perfectly.

In terms of evaluation, the authors do not compare their result against any other method.

Instead, using only one set of test parameters, the authors compare their algorithm to a “greedy baseline” policy that is specified a “always try to change lanes to the right until the lane is correct” then it tries to go as fast as possible while obeying the speed limit and not colliding with any car in front.

It seems that baseline is additionally constrained vs the ego car due to the speed limit and the collision avoidance criteria and is not a fair comparison.

So given a fixed policy and these constraints it is not surprising that it underperforms the Q-masked Q-learning algorithm.

With respect to the comparison vs. human operators of the car simulation, the human operators were not experts.

They were only given “a few trials” to learn how to operate the controls before the test.

It was reported that the human participants “did not feel comfortable” with the low level controller on,

possibly indicating that the user experience of controlling the car was less than ideal.

With the low level controller off, collisions became possible.

It is possibly not a fair claim to say that human drivers were “less safe” but rather that it was difficult to play the game or control the car with the safety module on.

This could be seen as a game design issue.

It was not clear from this presentation how the human participants were rewarded for their performance.

In more typical HCI experiments the gender distribution and ages ranges of participants are specified as well as how participants were recruited and how the game was motivated, including compensation (reward) are specified.

Overall, this paper presents an overly simplified game simulation with a weak experimental result.

fact fact evaluation evaluation fact request evaluation evaluation evaluation evaluation quote evaluation fact evaluation fact evaluation fact fact fact fact evaluation fact fact fact evaluation fact fact evaluation fact fact evaluation fact fact evaluation evaluation fact fact fact evaluation evaluation evaluation evaluation evaluation request evaluation


This paper creates a layered representation in order to better learn segmentation from unlabeled images.

It is well motivated,

as Fig. 1 clearly shows the idea that if the segmentation was removed properly, the result would still be a natural image.

However, the method itself as described in the paper leaves many questions about whether they can achieve the proposed goal.

I cannot see from the formulation why would this model work as it is advertised.

The formulation (3-4) looks like a standard GAN, with some twist about measuring the GAN loss in the z space (this has been used in e.g. PPGN and CVAE-GAN).

I don't see any term that would guarantee:1) Each layer is a natural image.

This was advertised in the paper,

but the loss function is only on the final product G_K.

The way it is written in the paper, the result of each layer does not need to go through a discriminator.

Nothing seems to have been done to ensure that each layer outputs a natural image.

2) None of the layers is degenerate.

There does not seem to be any constraint either regularizing the content in each layer, or preventing any layer to be non-degenerate.

3) The mask being contiguous.

I don't see any term ensuring the mask being contiguous,

I imagine normally without such terms doing such kinds of optimization would lead to a lot of fragmented small areas being considered as the mask.

The claim that this paper is for unsupervised semantic segmentation is overblown.

A major problem is that when conducting experiments, all the images seem to be taken from a single category, this implicitly uses the label information of the category.

In that regard, this cannot be viewed as an unsupervised algorithm.

Even with that, the results definitely looked too good to be true.

I have a really difficult time believing why such a standard GAN optimization would not generate any of the aforementioned artifacts and would perform exactly as the authors advertised.

Even if it does work as advertised, the utilization of implicit labels would make it subject to comparisons with a lot of weakly-supervised learning papers with far better results than shown in this paper.

Hence I am pretty sure that this is not up to the standards of ICLR.

fact evaluation fact evaluation evaluation fact fact fact fact fact fact fact fact fact fact fact evaluation fact fact evaluation evaluation evaluation evaluation


This paper presents, and analyzes, a method for learning word relationships based on co-occurrence.

In the method, relationships between pairs of words (A, B) are represented by the terms that tend to occur around co-mentions of A and B in text.

The paper shows the start of some interesting ideas,

but needs revisions and much more extensive experiments.

On the plus side, the method proposed here does perform relatively well (Table 1) and probably merits further investigation.

The experiments in Table 1 can only be considered preliminary, however.

They only evaluate over a small number of relationships (three)

-- looking at 20 or so different relationships would greatly improve confidence in the conclusions.

Beyond Table 1 the paper makes a number of claims that are not supported or weakly supported (the paper uses only a handful of examples as evidence).

An attempt to explain what Word2Vec is doing should be made with careful experiments over many relations and hundreds of examples,

whereas this paper presents only a handful of examples for most of its claims.

Further, whether the behavior of the proposed algorithm actually reflects what word2vec is doing is left as a significant open question.

I appreciate the clarity of Assumption 1 and Proposition 1,

but ultimately this formalism is not used

and because Assumption 1 about which nouns are "semantically related" to which other nouns attempts to trivialize a complex notion (semantics) and is clearly way too strong

-- the paper would be better off without it.

Also Assumption 1 does not actually claim what the text says it claims

(the text says words outside the window are *not* semantically related, but the assumption does not actually say this)

and furthermore is soon discarded and only the frequency of noun occurrences around co-mentions is used.

I think the description of the algorithm could be retained without including Assumption 1.

minor: References to numbered algorithms or assumptions should be capitalized in the text.

what the introduction means about the "dynamics" of the vector equation is a little unclear

A submission shouldn't have acknowledgments, and in particular with names that undermine anonymity

MLE has a particular technical meaning that is not utilized here,

I would just refer to the most frequent words as "most related nouns" or similar

In Table 1, are the "same dataset" results with w2v for the nouns-only corpus, or with all the other words?

The argument made assuming a perfect Zipf distribution (with exponent equal to one) should be made with data.

will likely by observed -> will likely be observed

lions:dolphins probably ends up that way because of "sea lions"

Table 4 caption: frequencies -> currencies

Table 2 -- claim is that improvements from k=10 to k=20 are 'nominal' but they look non-negligible to me

I did not understand how POS lying in the same subspace means that Vec(D) has to be in the span of Vecs A-C.

fact fact evaluation request evaluation evaluation fact request evaluation request evaluation evaluation evaluation fact evaluation request fact fact fact request request evaluation request fact request request request request request request evaluation evaluation


The paper presents a method for navigating in an unknown and partially observed environment is presented.

The proposed approach splits planning into two levels: 1) local planning based on the observed space and 2) a global planner which receives the local plan, observation features, and access to an addressable memory to decide on which action to select and what to write into memory.

The contribution of this work is the use of value iteration networks (VINs) for local planning on a locally observed map that is fed into a learned global controller that references history and a differential neural computer (DNC), local policy, and observation features select an action and update the memory.

The core concept of learned local planner providing additional cues for a global, memory-based planner is a clever idea

and the thorough analysis clearly demonstrates the benefit of the approach.

The proposed method is tested against three problems: a gridworld, a graph search, and a robot environment.

In each case the proposed method is more performant than the baseline methods.

The ablation study of using LSTM instead of the DNC and the direct comparison of CNN + LSTM support the authors’ hypothesis about the benefits of the two components of their method.

While the author’s compare to DRL methods with limited horizon (length 4), there is no comparison to memory-based RL techniques.

Furthermore, a comparison of related memory-based visual navigation techniques on domains for which they are applicable should be considered

as such an analysis would illuminate the relative performance over the overlapping portions problem domains

For example, analysis of the metric map approaches on the grid world or of MACN on their tested environments.

Prior work in visual navigation in partially observed and unknown environments have used addressable memory (e.g., Oh et al.) and used VINs (e.g., Gupta et al.) to plan as noted.

In discussing these methods, the authors state that these works are not comparable as they operate strictly on discretized 2d spaces.

However, it appears to the reviewer that several of these methods can be adapted to higher dimensions and be applicable at least a subclass (for the euclidean/metric map approaches) or the full class of the problems (for Oh et al.),

which appears to be capable to solve non-euclidean tasks like the graph search problem.

If this assessment is correct, the authors should differentiate between these approaches more thoroughly and consider empirical comparisons.

The authors should further consider contrasting their approach with “Neural SLAM” by Zhang et al.

A limitation of the presented method is requirement that the observation “reveals the labeling of nearby states.”

This assumption holds in each of the examples presented: the neighborhood map in the gridworld and graph examples and the lidar sensor in the robot navigation example.

It would be informative for the authors to highlight this limitation and/or identify how to adapt the proposed method under weaker assumptions such as a sensor that doesn’t provide direct metric or connectivity information such as a RGB camera.

Many details of the paper are missing and should be included to clarify the approach and ensure reproducible results.

The reviewer suggests providing both more details in the main section of the paper and providing the precise architecture including hyperparameters in the supplementary materials section.

fact fact fact evaluation evaluation fact fact fact fact request fact fact fact fact evaluation fact request request evaluation fact request request request


This paper proposes a family of first-order stochastic optimization schemes

based on (1)  normalizing (batches of) stochastic gradient descents and (2) choosing from a step size updating scheme.

The authors argue that iterative first-order optimization algorithms can be interpreted as a choice of an update direction and a step size,

so they suggest that one should always normalize the gradient when computing the direction and then choose a step size using the normalized gradient.

\n\nThe presentation in the paper is clear,

and the exposition is easy to follow.

The authors also do a good job of presenting related work and putting their ideas in the proper context.

The authors also test their proposed method on many datasets,

which is appreciated.\n\n

However, I didn't find the main idea of the paper to be particularly compelling.

The proposed technique is reasonable on its own,

but the empirical results do not come with any measure of statistical significance.

The authors also do not analyze the sensitivity of the different optimization algorithms to hyperparameter choice, opting to only use the default.

Moreover, some algorithms were used as benchmarks on some datasets but not others.

For a primarily empirical paper, every state-of-the-art algorithm should be used as a point of comparison on every dataset considered.

These factors altogether render the experiments uninformative in comparing the proposed suite of algorithms to state-of-the-art methods.

The theoretical result in the convex setting is also not data-dependent, despite the fact that it is the normalized gradient version of AdaGrad, which does come with a data-dependent convergence guarantee.\n\n

Given the suite of optimization algorithms in the literature and in use today, any new optimization framework should either demonstrate improved (or at least matching) guarantees in some common (e.g. convex) settings or definitively outperform state-of-the-art methods on problems that are of widespread interest.

Unfortunately, this paper does neither.

\n\nBecause of these points, I do not feel the quality, originality, and significance of the work to be high enough to merit acceptance.

\n\nSome specific comments: \np. 2: \"adaptive feature-dependent step size has attracted lots of attention\".

When you apply feature-dependent step sizes, you are effectively changing the direction of the gradient,

so your meta learning formulation, as posed, doesn't make as much sense.

\np. 2: "we hope the resulting methods can benefit from both techniques\".

What reason do you have to hope for this?

Why should they be complimentary?

Existing optimization techniques are based on careful design and coupling of gradients or surrogate gradients, with specific learning rate schedules.

Arbitrarily mixing the two doesn't seem to be theoretically well-motivated.

\np. 2: \"numerical results shows that normalized gradient always helps to improve the performance of the original methods when the network structure is deep\".

It would be great to provide some intuition for this.

\np. 2: \"we also provide a convergence proof under this framework when the problem is convex and the stepsize is adaptive\".

The result that you prove guarantees a \\theta(\\sqrt{T}) convergence rate.

On the other hand, the AdaGrad algorithm guarantees a data-dependent bound that is O(\\sqrt{T})

but can also be much smaller.

This suggests that there is no theoretical motivation to use NGD with an adaptive step size over AdaGrad.

\np. 2-3: \"NGD can find a \\eps-optimal solution....when the objective function is quasi-convex. ....extended NGD for upper semi-continuous quasiconvex objective functions...\".

This seems like a typo.

How are results that go from quasi-convex to upper semi-continuous quasi-convex an extension?

\np. 3: There should be a reference for RMSProp.

\np. 3: \"where each block of parameters x^i can be viewed as parameters associated to the ith layer in the network\".

Why is layer parametrization (and later on normalization) a good way idea?

There should be either a reference or an explanation.

\np. 4: \"x=(x_1, x_2, \\ldots, x_B)\".

Should these subscripts be superscripts?

\np. 4: \"For all the algorithms, we use their default settings.\"

This seems insufficient for an empirical paper,

since most problems often involve some amount of hyperparameter tuning.

How sensitive is each method to the choice of hyperparameters?

What about the impact of initialization?

\np. 4-8: None of the experimental results have error bars or any measure of statistical significance.

\np. 5: \"NG... is a variant of the NG_{UNIT} method\".

This method is never motivated.

\np. 5-6: Why are SGD and Adam used for MNIST but not on CIFAR?

\np. 5: \"we chose the best heyper-paerameter from the 56 layer residual network.\"

Apart from the typos, are these parameters chosen from the training set or the test set?

\np. 6: Why isn't Adam tested on ImageNet?

fact fact fact fact evaluation evaluation evaluation fact evaluation evaluation evaluation fact fact fact request evaluation fact request fact evaluation quote fact evaluation quote non-arg non-arg fact evaluation quote request quote fact fact fact evaluation quote evaluation evaluation request quote non-arg request quote non-arg quote evaluation fact non-arg non-arg fact quote fact non-arg quote non-arg non-arg


The authors use deep learning to learn a surrogate model for the motion vector in the advection-diffusion equation that they use to forecast sea surface temperature.

In particular, they use a CNN encoder-decoder to learn a motion field, and a warping function from the last component to provide forecasting.

I like the idea of using deep learning for physical equations.

I would like to see a description of the algorithm with the pseudo-code in order to understand the flow of the method.

I got confused at several points

because it was not clear what was exactly being estimated with the CNN.

Having an algorithmic environment would make the description easier.

I know that authors are going to publish the code,

but this is not enough at this point of the revision.

Physical processes in Machine learning have been studied from the perspective of Gaussian processes.

Just to mention a couple of references “Linear latent force models using Gaussian processes” and "Numerical Gaussian Processes for Time-dependent and Non-linear Partial Differential Equations"

In Theorem 2, do you need to care about boundary conditions for your equation?

I didn’t see any mention to those in the definition for I(x,t).

You only mention initial conditions.

How do you estimate the diffusion parameter D?

Are you assuming isotropic diffusion?

Is that realistic?

Can you provide more details about how you run the data assimilation model in the experiments?

Did you use your own code?

fact fact evaluation request evaluation evaluation fact fact evaluation fact reference request fact fact request request evaluation request non-arg


The approach solves an important problem

as getting labelled data is hard.

The focus is on the key aspect, which is generalisation across heteregeneous data.

The novel idea is the dataset embedding

so that their RL policy can be trained to work across diverse datasets.

Pros: 1. The approach performs well against all the baselines, and also achieves good cross-task generalisation in the tasks they evaluated on.

2. In particular, they alsoevaluated on test datasets with fairly different statistics from the training datasets, which isnt very common in most meta-learning papers today,

so it’s encouraging that the method works in that regime.

Cons: 1. The embedding strategy, especially the representative and discriminative histograms, is complicated.

It is unclear if the strategy is general enough to work on harder problems / larger datasets, or with higher dimensional data like images.

More evidence in the paper for why it would work on harder problems would be great.

2. The policy network would have to output a probability for each datapoint in the dataset U,

which could be fairly large,

thus the method is computationally much more expensive than random sampling.

A section devoted to showing what practical problems could be potentially solved by this method would be useful.

3. It is unclear to me if the results in table 3 and 4 are achieved by retraining from scratch with an RBF SVM, or by freezing the policy network trained on a linear SVM and directly evaluating it with a RBF SVM base learner.

Significance/Conclusion: The idea of meta-learning or learning to learn is fairly common now.

While they do show good performance,

it’s unclear if the specific embedding strategy suggested in this paper will generalise to harder tasks.

Comments: There’s lots of typos,

please proof read to improve the paper.

evaluation evaluation fact evaluation fact evaluation evaluation evaluation evaluation evaluation request fact evaluation evaluation request evaluation evaluation evaluation evaluation evaluation request


The authors present a derivation of previous work of [1].

In particular they propose the method of using the error signal of a dynamics model as curiosity for exploration, such as [1], but without any additionaly auxiliary methods.

This the author call Curiosity by Bootstrapping Feature (CBF).\n

\nIn particular they show over  a set of auxiliary learning methods (hindsight ER, inverse dynamics model[1]) there is\nnot a clear cut edge one method has over the other (or over using no auxilirary method all, that is CBF).\n\n

Overall I think the novelty is too limited for acceptance.

The main point of the authors (heterogeneous results\nover different auxilirary learning methods),  is not suprising at all, and to be expected.

The method the authors introduce\nis just a submodule of already published results[1].\n\n

For instance, section 4 discusses challenges related to these class of approaches such as the presence of stochasticity.

Had the authors proposed a solution to these challenges that would have benefited the paper greatly.\n\n

Minor: The light green link color make the paper hard on the eye,

I suggest using [hidelinks] for hyperref.\n

Figure 2 is  very small and hard to read.\n\n\n

[1] Deepak Pathak, Pulkit Agrawal, Alexei A. Efros, and Trevor Darrell. Curiosity-driven exploration by\nself-supervised prediction. In ICML, 2017

fact fact fact fact evaluation evaluation fact fact request evaluation request evaluation reference


The paper is clear and well written.

It is an incremental modification of prior work (ResNeXt) that performs better on several experiments selected by the author;

comparisons are only included relative to ResNeXt.

This paper is not about gating (c.f., gates in LSTMs, mixture of experts, etc) but rather about masking or perhaps a kind of block sparsity,

as the "gates" of the paper do not depend upon the input:

they are just fixed masking matrices (see eq (2)).

The main contribution appears to be the optimisation procedure for the binary masking tensor g.

But this procedure is not justified:

does each step minimise the loss?

This seems unlikely due to the sampling.

Can the authors show that the procedure will always converge?

It would be good to contrast this with other attempts to learn discrete random variables

(for example, The Concrete Distribution: Continuous Relaxation of Continuous Random Variables, Maddison et al, ICLR 2017).

evaluation evaluation fact fact fact fact fact fact request evaluation request request reference


In this work, the objective is to analyze the robustness of a neural network to any sort of attack.

This is measured by naturally linking the robustness of the network to the local Lipschitz properties of the network function.

This approach is quite standard in learning theory,

I am not aware of how original this point of view is within the deep learning community.

This is estimated by obtaining values of the norm of the gradient (also naturally linked to the Lipschitz properties of the function) by backpropagation.

This is again a natural idea.

fact fact evaluation evaluation fact evaluation


The authors proposed to supplement adversarial training with an additional regularization that forces the embeddings of clean and adversarial inputs to be similar.

The authors demonstrate on MNIST and CIFAR that the added regularization leads to more robustness to various kinds of attacks.

The authors further propose to enhance the network with cascaded adversarial training, that is, learning against iteratively generated adversarial inputs, and showed improved performance against harder attacks.

The idea proposed is fairly straight-forward.

Despite being a simple approach, the experimental results are quite promising.

The analysis on the gradient correlation coefficient and label leaking phenomenon provide some interesting insights.

As pointed out in section 4.2, increasing the regularization coefficient leads to degenerated embeddings.

Have the authors consider distance metrics that are less sensitive to the magnitude of the embeddings, for example, normalizing the inputs before sending it to the bidirectional or pivot loss, or use cosine distance etc.?

Table 4 and 5 seem to suggest that cascaded adversarial learning have more negative impact on test set with one-step attacks than clean test set,

which is a bit counter-intuitive.

Do the authors have any insight on this?

Comments: 1. The writing of the paper could be improved.

For example, "Transferability analysis" in section 1 is barely understandable;

2. Arrow in Figure 3 are not quite readable;

3. The paper is over 11 pages.

The authors might want to consider shrink it down the recommended length.

fact fact fact evaluation evaluation evaluation fact request fact evaluation non-arg evaluation evaluation evaluation fact request


The authors propose a mechanism for learning task-specific region embeddings for use in text classification.

Specifically, this comprises a standard word embedding an accompanying local context embedding.

The key idea here is the introduction of a (h x c x v) tensor K, where h is the embedding dim (same as the word embedding size), c is a fixed window size around a target word, and v is the vocabulary size.

Each word in v is then associated with an (h x c) matrix that is meant to encode how it affects nearby words,

in particular this may be viewed as parameterizing a projection to be applied to surrounding word embeddings.

The authors propose two specific variants of this approach, which combine the K matrix and constituent word embeddings (in a given region) in different ways.

Region embeddings are then composed (summed) and fed through a standard model.

Strong points--- + The proposed approach is simple and largely intuitive:

essentially the context matrix allows word-specific contextualization.

Further, the work is clearly presented.

+ At the very least the model does seem comparable in performance to various recent methods (as per Table 2),

however as noted below the gains are marginal

and I have some questions on the setup.

+ The authors perform ablation experiments,

which are always nice to see.

Weak points--- - I have a critical question for clarification in the experiments.

The authors write 'Optimal hyperparameters are tuned with 10% of the training set on Yelp Review Full dataset, and identical hyperparameters are applied to all datasets'

-- is this true for *all* models, or only the proposed approach?

- The gains here appear to be consistent,

but they seem marginal.

The biggest gain achieved over all datasets is apparently .7,

and most of the time the model very narrowly performs better (.2-.4 range).

Moreoever, it is not clear if these results are averaged over multiple runs of SGD or not

(variation due to initialization and stochastic estimation can account for up to 1 point in variance

-- see "A sensitivity analysis of (and practitioners guide to) CNNs..." Zhang and Wallace, 2015.)

- The related work section seems light.

For instance, there is no discussion at all of LSTMs and their application to text classificatio (e.g., Tang et al., EMNLP 2015)

-- although it is noted that the authors do compare against D-LSTM,  or char-level CNNs for the same (see Zhang et al., NIPs 2015).

Other relevant work not discussed includes Iyyer et al. (ACL 2015).

In their respective ways, these papers address some of the same issues the authors consider here.

- The two approaches to inducing the final region embedding (word-context and then context-word in sections 3.2 and 3.3, respectively) feel a bit ad-hoc.

I would have appreciated more intuition behind these approaches.

Small comments---There is a typo in Figure 4 -- "Howerver" should be "However"

fact fact fact fact evaluation fact fact evaluation fact evaluation evaluation fact evaluation fact evaluation evaluation fact request fact fact fact fact evaluation fact reference evaluation fact fact fact fact evaluation evaluation request


General comment ==============  Low-rank decomposing convolutional filters has been used to speedup convolutional networks at the cost of a drop in prediction performance.

The authors a) extended existing decomposition techniques by an iterative method for decomposition and fine-tuning convolutional filter weights,

and b) and algorithm to determine the rank of each convolutional filter.

The authors show that their method enables a higher speedup and lower accuracy drop than existing methods when applied to VGG16.

The proposed method is a useful extension of existing methods but needs to evaluated more rigorously.

The manuscript is hard to read due to unclear descriptions and grammatical errors.

Major comments ============= 1. The authors authors showed that their method enables a higher speedup and lower drop in accuracy than existing methods when applied to VGG16.

The authors should analyze if this also holds true for ResNet and Inception, which are more widely used than VGG16.

2. The authors measured the actual speedup on a single CPU (Intel Core i5).

The authors should measure the actual speedup also on a single GPU.

3. It is unclear how the actual speedup was measured.

Does it correspond to the seconds per update step or the overall training time?

In the latter case, how long were models trained?

4. How and which hyper-parameters were optimized?

The authors should use the same hyper-parameters for all methods (Jaderberg, Zhang, Rank selection).

The authors should also analyze the sensitivity of speedup and accuracy drop depending on the learning rate for ‘Rank selection’.

5. Figure 4: the authors should show the same plot for more convolutional layers at varying depth from both VGG and ResNet.

6. The manuscript is hard to understand and not written clearly enough.

In the abstract, what does ‘two-pass decomposition’, ‘proper ranks’, ‘the instability problem’, or ‘systematic’ mean?

What are ‘edge devices’, ‘vanilla parameters’?

The authors should also avoid uninformative adjectives, clutter, and vague terms throughout the manuscript such as ‘vital importance’ or ‘little room for fine-tuning’.

Minor comments ============= 1. The authors should use ‘significantly’ only if a statistical hypothesis was performed.

2. The manuscript contains several typos and grammatical flaws,

e.g. ‘have been widely applied to have the breakthrough’, ‘The CP decomposition factorizes the tensors into a sum of series rank-one tensors.’, ‘Our two-pass decomposition provides the better result as compared with the original CP decomposition’.

3. For clarity, the authors should express equation 5 in terms of Y_1, Y_2, Y_3, and Y_4.

4. Equation 2, bottom: C_in, W_f, H_f, and C_out are undefined at this point.

fact fact fact fact evaluation evaluation fact request fact request evaluation request request request request request request evaluation non-arg non-arg request request evaluation quote request fact


In this paper, the authors studied the problem of semi-supervised few-shot classification, by extending the prototypical networks into the setting of semi-supervised learning with examples from distractor classes.

The studied problem is interesting,

and the paper is well-written.

Extensive experiments are performed to demonstrate the effectiveness of the proposed methods.

While the proposed method is a natural extension of the existing works (i.e., soft k-means and meta-learning).

On top of that, It seems the authors have over-claimed their model capability at the first place

as the proposed model cannot properly classify the distractor examples but just only consider them as a single class of outliers.

Overall, I would like to vote for a weakly acceptance regarding this paper.

fact evaluation evaluation evaluation evaluation evaluation fact evaluation


The paper is interesting,

but needs more work,

and should provide clear and fair comparisons.

Per se, the model is incrementally new,

but it is not clear what the strengths are,

and the presentations needs to be done more carefully.

In detail: - please fix several typos throughout the manuscript, and have a native speaker (and preferably an ASR expert) proofread the paper

Introduction - please define HMM/GMM model (and other abbreviations that will be introduced later),

it cannot be assumed that the reader is familiar with all of them ("ASG" is used before it is defined, ...)

- The standard units that most ASR systems use can be called "senones",

and they are context dependent sub-phonetic units (see http://ssli.ee.washington.edu/~mhwang/), not phonetic states.

Also the units that generate the alignment and the units that are trained on an alignment can be different

(I can use a system with 10000 states to write alignments for a system with 3000 states)

- this needs to be corrected.

- When introducing CNNs, please also cite Waibel and TDNNs

- they are *the same* as 1-d CNNs, and predate them.

They have been extended to 2-d later on (Spatio-temporal TDNNs)

- The most influential deep learning paper here might be Seide, Li, Yu Interspeech 2011 on CD-DNN-HMMs, rather than overview articles

- Many papers get rid of the HMM pipeline,

I would add https://arxiv.org/abs/1408.2873, which predates Deep Speech

- What is a "sequence-level variant of CTC"?

CTC is a sequence training criterion

- The reason that Deep Speech 2 is better on noisy test sets is not only the fact they trained on more data, but they also trained on "noisy" (matched) data

- how is this an end-to-end approach if you are using an n-gram language model for decoding?

Architecture - MFSC are log Filterbanks ...

- 1D CNNs would be TDNNs

- Figure 2: can you plot the various transition types (normalized, un-normalized, ...) in the plots?

not sure if it would help, but it might

- Maybe provide a reference for HMM/GMM and EM (forward backward training)

- MMI was also widely used in HMM/GMM systems, not just NN systems

- the "blank" states do *not* model "garbage" frames,

if one wants to interpret them, they might be said to model "non-stationary" frames between CTC "peaks",

but these are different from silence, garbage, noise, ...

- what is the relationship of the presented ASG criterion to MMI?

the form of equation (3) looks like an MMI criterion to me?

Experiments - Many of the previous comments still hold,

please proofread

- you say there is no "complexity" incrase when using "logadd"

- how do you measure this?

number of operations?

is there an implementation of "logadd" that is (absolutely) as fast as "add"?

- There is discussion as to what i-vectors model (speaker or environment information)

- I would leave out this discussion entirely here,

it is enough to mention that other systems use adaptation, and maybe re-run an unadapted baselien for comparsion

- There are techniques for incremental adaptation and a constrained MLLR (feature adaptation) approaches that are very eficient, if one wnats to get into this

- it may also be interesting to discuss the role of the language model to see which factors influence system performance

- some of the other papers might use data augmentation, which would increase noise robustness

(did not check, but this might explain some of the results in table 4)

- I am confused by the references in the caption of Table 3

- surely the Waibel reference is meant to be for TDNNs

(and should appear earlier in the paper),

while p-norm came later

(Povey used it first for ASR, I think)

and is related to Maxout

- can you also compare the training times?

Conculsion - can you show how your approach is not so computationally expensive as RNN based approaches?

either in terms of FLOPS or measured times

evaluation request request evaluation evaluation request request request evaluation fact fact fact fact request request fact fact evaluation evaluation request non-arg fact fact fact fact fact request evaluation request evaluation fact fact fact request fact evaluation request fact request request request fact request evaluation fact request evaluation evaluation evaluation fact request fact fact evaluation request request request


This paper proposes to use neural network and gradient descent to automatically design for engineering tasks.

It uses two networks, parameterization network and prediction network to model the mapping from design parameters to fitness.

It uses back propagation (gradient descent) to improve the design.

The method is evaluated on heat sink design and airfoil design.

This paper targets at a potentially very useful application of neural networks that can have real world impacts.

However, I have three main concerns: 1) Presentation. The organization of the paper could be improved.

It mixes the method, the heat sink example and the airfoil example throughout the entire paper.

Sometimes I am very confused about what is being described.

My suggestion would be to completely separate these three parts:

present a general method first,

then use heat sink as the first experiment and airfoil as the second experiment.

This organization would make the writing much clearer.

2) In the paragraph above Section 4.1, the paper made two arguments.

I might be wrong, but I do not agree with either of them in general.

First of all, "neural networks are good at generalizing to examples outside their train set".

This depends entirely on whether the sample distribution of training and testing are similar and whether you have enough training examples that cover important sample space.

This is especially critical if a deep neural network is used since overfitting is a real issue.

Second, "it is easy to imagine a hybrid system where a network is trained on a simulation and fine tuned ...".

Implementing such a hybrid system is nontrivial due to the reality gap.

There is an entire research field about closing the reality gap and transfer learning.

So I am not convinced by these two arguments made by this paper.

They might be true for a narrow field of application.

But in general, I think they are not quite correct.

3) The key of this paper is to approximate the dynamics using neural network (which is a continuous mapping) and take advantage of its gradient computation.

However, many of dynamic systems are inherently discontinuous (collision/contact dynamics) or chaotic (turbulent flow).

In those scenarios, the proposed method might not work well and we may have to resort to the gradient free methods.

It seems that the proposed method works well for heat sink problem and the steady flow around airfoil,

both of which do not fall into the more complex physics regime.

It would be great that the paper could be more explicit about its limitations.

In summary, I like the idea, the application and the result of this paper.

The writing could be improved.

But more importantly, I think that the proposed method has its limitation about what kind of physical systems it can model.

These limitation should be discussed more explicitly and more thoroughly.

fact fact fact fact evaluation request fact evaluation request request request evaluation fact evaluation quote evaluation evaluation quote evaluation fact evaluation evaluation evaluation fact fact evaluation fact fact request evaluation request evaluation request