In their paper "CausalGAN: Learning Causal implicit Generative Models with adv. training" the authors address the following issue: Given a causal structure between "labels" of an image (e.g. gender, mustache, smiling, etc.), one tries to learn a causal model between these variables and the image itself from observational data.

Here, the image is considered to be an effect of all the labels.

Such a causal model allows us to not only sample from conditional observational distributions, but also from intervention distributions.

These tasks are clearly different,

as nicely shown by the authors' example of "do(mustache = 1)" versus "given mustache = 1" (a sample from the latter distribution contains only men).

The paper does not aim at learning causal structure from data

(as clearly stated by the authors).

The example images look convincing to me.

I like the idea of this paper.

IMO, it is a very nice, clean, and useful approach of combining causality and the expressive power of neural networks.

The paper has the potential of conveying the message of causality into the ICLR community and thereby trigger other ideas in that area.

For me, it is not easy to judge the novelty of the approach,

but the authors list related works,

none of which seems to solve the same task.

The presentation of the paper, however, should be improved significantly before publication.

(In fact, because of the presentation of the paper, I was hesitating whether I should suggest acceptance.)

Below, I give some examples (and suggest improvements),

but there are many others.

There is a risk that in its current state the paper will not generate much impact,

and that would be a pity.

I would therefore like to ask the authors to put a lot of effort into improving the presentation of the paper.

- I believe that I understand the authors' intention of the caption of Fig. 1,

but "samples outside the dataset" is a misleading formulation.

Any reasonable model does more than just reproducing the data points.

I find the argumentation the authors give in Figure 6 much sharper.

Even better: add the expression "P(male = 1 | mustache = 1) = 1".

Then, the difference is crystal clear.

- The difference between Figures 1, 4, and 6 could be clarified.

- The list of "prior work on learning causal graphs" seems a bit random.

I would add Spirtes et al 2000, Heckermann et al 1999, Peters et al 2016, and Chickering et al 2002.

- Male -> Bald does not make much sense causally

(it should be Gender -> Baldness)...

Aha, now I understand:

The authors seem to switch between "Gender" and "Male" being random variables.

Make this consistent, please.

- There are many typos and comma mistakes.

- I would introduce the do-notation much earlier.

The paragraph on p. 2 is now written without do-notation

("intervening Mustache = 1 would not change the distribution").

But this way, the statements are at least very confusing

(which one is "the distribution"?).

- I would get rid of the concept of CiGM.

To me, it seems that this is a causal model with a neural network (NN) modeling the functions that appear in the SCM.

This means, it's "just" using NNs as a model class.

Instead, one could just say that one wants to learn a causal model and the proposed procedure is called CausalGAN?

(This would also clarify the paper's contribution.)

- many realizations = one sample (not samples), I think.

- Fig 1: which model is used to generate the conditional sample?

- The notation changes between E and N and Z for the noises.

I believe that N is supposed to be the noise in the SCM,

but then maybe it should not be called E at the beginning.

- I believe Prop 1 (as it is stated) is wrong.

For a reference, see Peters, Janzing, Scholkopf: Elements of Causal Inference: Foundations and Learning Algorithms (available as pdf), Definition 6.32.

One requires the strict positivity of the densities (to properly define conditionals).

Also, I believe the Z should be a vector, not a set.

- Below eq. (1), I am not sure what the V in P_V refers to.

- The concept of data probability density function seems weird to me.

Either it is referring to the fitted model, then it's a bad name,

or it's an empirical distribution, then there is no pdf, but a pmf.

- Many subscripts are used without explanation.

r -> real?

g -> generating?

G -> generating?

Sometimes, no subscripts are used (e.g., Fig 4 or figures in Sec. 8.13)

- I would get rid of Theorem 1 and explain it in words for the following reasons.

(1) What is an "informal" theorem?

(2) It refers to equations appearing much later.

(3) It is stated again later as Theorem 2.

- Also: the name P_g does not appear anywhere else in the theorem, I think.

- Furthermore, I would reformulate the theorem.

The main point is that the intervention distributions are correct

(this fact seems to be there,

but is "hidden" in the CIGN notation in the corollary).

- Re. the formulation in Thm 2: is it clear that there is a unique global optimum

(my intuition would say there could be several),

thus: better write "_a_ global minimum"?

- Fig. 3 was not very clear to me.

I suggest to put more information into its caption.

- In particular, why is the dataset not used for the causal controller?

I thought, that it should model the joint (empirical) distribution over the labels,

and this is part of the dataset.

Am I missing sth?

- IMO, the structure of the paper can be improved.

Currently, Section 3 is called "Background"

which does not say much.

Section 4 contains CIGMs, Section 5 Causal GANs, 5.1. Causal Controller, 5.2. CausalGAN, 5.2.1. Architecture (which the causal controller is part of) etc.

An alternative could be: Sec 1: Introduction Sec 1.1: Related Work Sec 2: Causal Models Sec 2.1: Causal Models using Generative Models (old: CIGM) Sec 3: Causal GANs Sec 3.1: Architecture (including controller) Sec 3.2: loss functions ... Sec 4: Empricial Results (old: Sec. 6: Results)

- "Causal Graph 1" is not a proper reference

(it's Fig 23 I guess).

Also, it is quite important for the paper,

I think it should be in the main part.

- There are different references to the "Appendix", "Suppl. Material", or "Sec. 8"

-- please be consistent

(and try to avoid ambiguity by being more specific

-- the appendix contains ~20 pages).

Have I missed the reference to the proof of Thm 2?

- 8.1. contains copy-paste from the main text.

- "proposition from Goodfellow" -> please be more precise

- What is Fig 8 used for?

Is it not sufficient to have and discuss Fig 23?

- IMO, Section 5.3. should be rewritten (also, maybe include another reference for BEGAN).

- There is a reference to Lemma 15.

However, I have not found that lemma.

- I think it's quite interesting that the framework seems to also allow answering counterfactual questions for realizations that have been sampled from the model, see Fig 16.

This is the case since for the generated realizations, the noise values are known.

The authors may think about including a comment on that issue.

- Since this paper's main proposal is a methodological one,

I would make the publication conditional on the fact that code is released.

fact fact fact evaluation evaluation fact fact evaluation evaluation evaluation evaluation evaluation fact fact request evaluation non-arg evaluation evaluation evaluation request evaluation evaluation evaluation evaluation request evaluation evaluation evaluation request evaluation request non-arg fact request evaluation request fact quote evaluation evaluation request fact fact request evaluation evaluation request fact evaluation evaluation fact fact fact request evaluation evaluation evaluation evaluation evaluation request request request fact request request evaluation fact fact request fact fact fact evaluation evaluation request evaluation request request evaluation fact non-arg evaluation fact evaluation fact request evaluation evaluation evaluation request fact request request fact non-arg fact request request request request fact fact evaluation fact request evaluation request


This paper studies off-policy learning in the bandit setting.

It develops a new learning objective where the empirical risk is regularized by the squared Chi-2 divergence between the new and old policy.

This objective is motivated by a bound on the empirical risk, where this divergence appears.

The authors propose to solve this objective by using generative adversarial networks for variational divergence minimization (f-GAN).

The algorithm is then evaluated on settings derived from supervised learning tasks and compared to other algorithms.

I find the paper well written and clear.

I like that the proposed method is both supported by theory and empirical results.

Minor point: I do not really agree with the discussion on the impact of the stochasticity of the logging policy in section 5.6.

Based on Figure 5 a and b, it seems that the learned policy is performing equally well no matter how stochastic the logging policy is.

So I find it a bit misleading to suggest that the learned policy are not being improved when the logging policy is more deterministic.

Rather, the gap reduces between the two policies

because the logging policy gets better.

In order to better showcase this mechanism, perhaps you could try using a logging policy that does not favor the best action.

quality and clarity: ++ code made available

+ well written and clear

- The proof of theorem 2 is not in the paper nor appendix

(the authors say it is similar to another work).

originality + good extension of the work by Swaminathan & Joachims (2015a): derivation of an alternative objective and use of a deep networks.

This paper leverages a set of diverse results

significance - The proposed method can only be applied if propensity scores were recorded when the data was generated.

- no test on a real setting

++ The proposed method is supported both by theoretical insights and empirical experiments.

+ empirical improvement with respect to previous methods

details/typos: 3.1, p3: R^(h) has an indexed parenthesis

5.2; and we more details

5.3: so that results more comparable

fact fact fact fact fact evaluation evaluation evaluation fact evaluation fact evaluation request fact evaluation fact fact evaluation evaluation fact fact evaluation fact fact fact fact


Clearly presented paper, including a number of reasonable techniques to improve LSTM-LMs.

The proposed techniques are heuristic,

but are reasonable and appear to yield improvements in perplexity.

Some specific comments follow.

re. "ASGD" for Averaged SGD: ASGD usually stands for Asynchronous SGD,

have the authors considered an alternative acronym?

AvSGD?

re. Optimization criterion on page 2, note that SGD is usually taken to minimizing expected loss, not just empirical loss (Bottou thesis 1991).

Is there any theoretical analysis of convergence for Averaged SGD?

re. paragraph starting with "To prevent such inefficient data usage, we randomly select the sequence length for the forward and backward pass in two steps":

the explanation is a bit unclear.

What is the "base sequence length" exactly?

Also, re. the motivation above this paragraph, I'm not sure what "elements" really refers to, though I can guess.

What is the number of training tokens of the datasets used, PTB and WT2?

Can the authors provide more explanation for what "neural cache models" are, and how they relate to "pointer models"?

Why do the sections "Pointer models", "Ablation analysis", and "AWD-QRNN" come after the Experiments section?

evaluation fact evaluation non-arg evaluation request request fact request quote evaluation request evaluation request request request


This paper examines ways of producing word embeddings for rare words on demand.

The key real-world use case is for domain specific terms,

but here the techniques are demonstrated on rarer words in standard data sets.

The strength of this paper is that it both gives a more systematic framework for and builds on existing ideas (character-based models, using dictionary definitions) to implement them as part of a model trained on the end task.

The contribution is clear but not huge.

In general, for the scope of the paper, it seems like what is here could fairly easily have been made into a short paper for other conferences that have that category.

The basic method easily fits within 3 pages,

and while the presentation of the experiments would need to be much briefer,

this seems quite possible.

More things could have been considered.

Some appear in the paper,

and there are some fairly natural other ones such as mining some use contexts of a word (such as just from Google snippets) rather than only using textual definitions from wordnet.

The contributions are showing that existing work using character-level models and definitions can be improved by optimizing representation learning in the context of the final task, and the idea of adding a learned linear transformation matrix inside the mean pooling model (p.3).

However, it is not made very clear why this matrix is needed or what the qualitative effect of its addition is.

The paper is clearly written.

A paper that should be referred to is the (short) paper of

Dhingra et al. (2017): A Comparative Study of Word Embeddings for Reading Comprehension https://arxiv.org/pdf/1703.00993.pdf .

While it in no way covers the same ground as this paper

it is relevant as follows:

This paper assumes a baseline that is also described in that paper of using a fixed vocab and mapping other words to UNK.

However, they point out that at least for matching tasks like QA and NLI that one can do better by assigning random vectors on the fly to unknown words.

That method could also be considered as a possible approach to compare against here.

Other comments:  - The paper suggests a couple of times including at the end of the 2nd Intro paragraph that you can't really expect spelling models to perform well in representing the semantics of arbitrary words (which are not morphological derivations, etc.).

While this argument has intuitive appeal,

it seems to fly in the face of the fact that actually spelling models, including in this paper, seem to do surprisingly well at learning such arbitrary semantics.

- p.2: You use pretrained GloVe vectors that you do not update.

My impression is that people have had mixed results, sometimes better, sometimes worse with updating pretrained vectors or not.

Did you try it both ways?

- fn. 1: Perhaps slightly exaggerates the point being made,

since people usually also get good results with the GloVe or word2vec model trained on "only" 6 billion words – 2 orders of magnitude less data.

- p.4. When no definition is available, is making e_d(w) a zero vector worse than or about the same as using a trained UNK vector?

- Table 1: The baseline seems reasonable (near enough to the quality of the original Salesforce model from 2016 (66 F1) but well below current best single models of around 76-78 F1.

The difference between D1 and D3 does well illustrate that better definition learning is done with backprop from end objective.

This model shows the rather strong performance of spelling models – at least on this task

– which he again benefit from training in the context of the end objective.

- Fig 2: It's weird that only the +dict (left) model learns to connect "In" and "where".

The point made in the text between "Where" and "overseas" is perfectly reasonable,

but it is a mystery why the base model on the right doesn't learn to associate the common words "where" and "in" both commonly expressing a location.

- Table 2: These results are interestingly different.

Dict is much more useful than spelling here.

I guess that is because of the nature of NLI,

but it isn't 100% clear why NLI benefits so much more than QA from definitional knowledge.

- p.7: I was slightly surprised by how small vocabs (3k and 5k words) are said to be optimal for NLI (and similar remarks hold for SQuAD).

My impression is that most papers on NLI use much larger vocabs, no?

- Fig 3: This could really be drawn considerably better:

make the dots bigger and their colors more distinct.

- Table 3: The differences here are quite small and perhaps the least compelling, but the same trends hold.

fact evaluation fact evaluation evaluation evaluation evaluation evaluation evaluation evaluation evaluation evaluation evaluation evaluation evaluation request reference evaluation evaluation fact fact request fact evaluation evaluation fact evaluation non-arg evaluation evaluation request evaluation evaluation evaluation evaluation evaluation evaluation evaluation evaluation evaluation evaluation evaluation evaluation evaluation evaluation request evaluation


The authors present a neural embedding technique using a hyperbolic space.

The idea of embedding data into a space that is not Euclidean is not new.

There have been attempts to project onto (hyper)spheres.

Also, the proposal bears some resemblance with what is done in t-SNE, where an (exponential) distortion of distances is induced.

Discussing this potential similarity would certainly broaden the readership of the paper.

The organisation of the paper might be improved, with a clearer red line and fewer digressions.

The call to the very small appendix via eq. 17 is an example.

The position of Table in the paper is odd as well.

The order of examples in Fig.5 differs from the order in the list.

The experiments are well illustrative but rather small sized.

The qualitative assessment is always interesting

and it is completed with some label prediction task.

Due the geometrical consideretations developed in the paper, other quality criteria like e.g. how well neighbourhoods are preserved in the embeddings would give some more insights.

All in all the idea developed in the paper sounds interesting

but the paper organisation seems a bit loose

and additional aspects should be investigated.

fact fact fact evaluation request request request evaluation fact evaluation evaluation fact evaluation evaluation evaluation request


The authors propose to speed up RL techniques, such as DQN, by utilizing expert demonstrations.

The  expert demonstrations are sequences of consecutive states that do not include actions, which is closer to a real setting of imitation learning.

The goal of this process is to extract a function that maps any given state to a subgoal.

Subgoals are then used to learn different Q-value functions, one per subgoal.

To learn the function that maps states into subgoals, the authors propose a surrogate reward model that corresponds to the angle between: the difference between two consecutive states (which captures velocity or direction) and a given subgoal.

A von Mises- Fisher distribution policy is then assumed to be used by the expert to generate actions that guide the agent toward the subgoal.

Finally, the mapping function state->subgoal is learned by performing a gradient descent on the expected total cost (based on the surrogate reward function, which also has free parameters that need to be learned).

Finally, the authors use the DQN platform to learn a Q-value function using the learned  surrogate reward function that guides the agent to specific subgoals, depending on the situation.

The paper is overall well-written,

and the proposed idea seems interesting.

However, there are rather little explanations provided to argue for the different modeling choices made, and the intuition behind them.

From my understanding, the idea of subgoal learning boils down to a non-parametric (or kernel) regression where each state is mapped to a subgoal based on its closeness to different states in the expert's demonstration.

It is not clear how this method would generalize to new situations.

There is also the issue of keeping tracking of a large number of demonstration states in memory.

This technique reminds me of some common methods in learning from demonstrations, such as those using GPs or GMMs,

but the novelty of this technique is the fact that the subgoal mapping function is learned in an IRL fashion, by tacking into account the sum of surrogate rewards in the expert's demonstration.

The architecture of the action value estimator does not seem novel,

it's basically just an extension of DQN with an extra parameter (subgoal g).

The empirical evaluation seems rather mixed.

Figure 3 shows that the proposed method learns faster than DQN,

but Table I shows that the improvement is not statistically significant, except in two games, DefendCenter and PredictPosition.

Are these the results after all agents had converged?

Overall, this is a good paper,

but focusing on only a single game (Doom) is a weakness that needs to be addressed

because one cannot tell if the choices were tailored to make the method work well for this game.

Since the paper does not provide significant theoretical or algorithmic contribution,

at least more realistic and diverse experiments should be performed.

fact fact fact fact fact fact fact fact evaluation evaluation evaluation fact evaluation fact evaluation evaluation evaluation fact evaluation fact fact non-arg evaluation request fact evaluation request


Summary The paper presents an interesting view on the recently proposed MAML formulation of meta-learning (Finn et al).

The main contribution is a) insight into the connection between the MAML procedure and MAP estimation in an equivalent linear hierarchical Bayes model with explicit priors,

b) insight into the connection between MAML and MAP estimation in non-linear HB models with implicit priors,

c) based on these insights, the paper proposes a variant of MALM using a Laplace approximation (with additional approximations for the covariance matrix.

The paper finally provides an evaluation on the mini ImageNet problem without significantly improving on the MAML results on the same task.

Pro: -            The topic is timely and of relevance to the ICLR community continuing a current trend in building meta-learning system for few-shot learning.

-            Provides valuable insight into the MAML objective and its relation to probabilistic models

Con: -            The paper is generally well-written

but I find (as a non-meta-learner expert) that certain fundamental aspects could have been explained better or in more detail (see below for details).

-            The toy example is quite difficult to interpret the first time around

and does not provide any empirical insight into the converge of the proposed method (compared to e.g. MAML)

-            I do not think the empirical results provide enough evidence that it is a useful/robust method.

Especially it does not provide insight into which types of problems (small/large, linear/ non-linear) the method is applicable to.

Detailed comments/questions: -            The use of Laplace approximation is (in the paper) motivated from a probabilistic/Bayes and uncertainty point-of-view.

It would, however, seem that the truncated iterations do not result in the approximation being very accurate during optimization

as the truncation does not result in the approximation being created at a mode.

Could the authors perhaps comment on: a) whether it is even meaningful to talk about the approximations as probabilistic distribution during the optimization (given the psd approximation to the Hessian), or does it only make sense after convergence?

b) the consequence of the approximation errors on the general convergence of the proposed method (consistency and rate)

-            Sec 4.1, p5: Last equation: Perhaps useful to explain the term $log(\phi_j^* | \theta)$ and why it is not in subroutine 4 .

Should $\phi^*$  be $\hat \phi$ ?

-            Sec 4.2: “A straightforward…”: I think it would improve readability to refer back to the to the previous equation (i.e. H) such that it is clear what is meant by “straightforward”.

-            Sec 4.2: Several ideas are being discussed in Sec 4.2

and it is not entirely clear to me what has actually been adopted here;

perhaps consider formalizing the actual computations in Subroutine 4 – and provide a clearer argument (preferably proof) that this leads to consistent and robust estimator of \theta.

-            It is not clear from the text or experiment how the learning parameters are set.

-            Sec 5.1: It took some effort to understand exactly what was going on in the example and particular figure 5.1;

e.g., in the model definition in the body text there is no mention of the NN mentioned/used in figure 5,

the blue points are not defined in the caption,

the terminology e.g.  “pre-update density” is new at this point.

I think it would benefit the readability to provide the reader with a bit more guidance.

-            Sec 5.1: While the qualitative example is useful (with a bit more text),

I believe it would have been more convincing with a quantitative example to demonstrate e.g. the convergence of the proposal compared to std MAML and possibly compare to a std Bayesian inference method from the HB formulation of the problem (in the linear case)

-            Sec 5.2: The abstract clams increased performance over MAML

but the empirical results do not seem to be significantly better than MAML ?

I find it quite difficult to support the specific claim in the abstract from the results without adding a comment about the significance.

-            Sec 5.2: The authors have left out “Mishral et al” from the comparison due to the model being significantly larger than others.

Could the authors provide insight into why they did not use the ResNet structure from the  tcml paper in their L-MLMA scheme ?

-            Sec 6+7: The paper clearly states that it is not the aim to (generally) formulate the MAML as a HB.

Given the advancement in gradient based inference for HB the last couple of years (e.g. variational, nested laplace , expectation propagation etc) for explicit models,

could the authors perhaps indicate why they believe their approach of looking directly to the MAML objective is more scalable/useful than trying to formulate the same or similar objective in an explicit HB model and using established inference methods from that area ?

Minor: -            Sec 4.1 “…each integral in the sum in (2)…” eq 2 is a product

evaluation evaluation fact fact fact evaluation evaluation evaluation evaluation evaluation fact evaluation evaluation evaluation evaluation fact request request request request request fact evaluation request evaluation evaluation fact fact fact request evaluation request fact evaluation evaluation fact request fact fact request request


Summary of paper: The authors present a novel attack for generating adversarial examples, deemed OptMargin,

in which the authors attack an ensemble of classifiers created by classifying at random L2 small perturbations.

They compare this optimization method with two baselines in MNIST and CIFAR, and provide an analysis of the decision boundaries by their adversarial examples, the baselines and non-altered examples.

Review summary: I think this paper is interesting.

The novelty of the attack is a bit dim,

since it seems it's just the straightforward attack against the region cls defense.

The authors fail to include the most standard baseline attack, namely FSGM.

The authors also miss the most standard defense, training with adversarial examples.

As well, the considered attacks are in L2 norm,

and the distortion is measured in L2,

while the defenses measure distortion in L_\infty (see detailed comments for the significance of this if considering white-box defenses).

The provided analysis is insightful,

though the authors mostly fail to explain how this analysis could provide further work with means to create new defenses or attacks.

If the authors add FSGM to the batch of experiments (especially section 4.1) and address some of the objections I will consider updating my score.

A more detailed review follows.

Detailed comments: - I think the novelty of the attack is not very strong.

The authors essentially develop an attack targeted to the region cls defense.

Designing an attack for a specific defense is very well established in the literature,

and the fact that the attack fools this specific defense is not surprising.

- I think the authors should make a claim on whether their proposed attack works only for defenses that are agnostic to the attack (such as PGD or region based), or for defenses that know this is a likely attack (see the following comment as well).

If the authors want to make the second claim, training the network with adversarial examples coming from OptMargin is missing.

- The attacks are all based in L2,

in the sense that the look for they measure perturbation in an L2 sense (as the paper evaluation does),

while the defenses are all L_\infty based

(since the region classifier method samples from a hypercube, and PGD uses an L_\infty perturbation limit).

This is very problematic if the authors want to make claims about their attack being effective under defenses that know OptMargin is a possible attack.

- The simplest most standard baseline of all (FSGM) is missing.

This is important to compare properly with previous work.

- The fact that the attack OptMargin is based in L2 perturbations makes it very susceptible to a defense that backprops through the attack.

This and / or the defense of training to adversarial examples is an important experiment to assessing the limitations of the attack.

- I think the authors rush to conclude that "a small ball around a given input distance can be misleading".

Wether balls are in L2 or L_\infty, or another norm makes a big difference in defense and attacks,

given that they are only equivalent to a multiplicative factor of sqrt(d) where d is the dimension of the space, and we are dealing with very high dimensional problems.

I find the analysis made by the authors to be very simplistic.

- The analysis of section 4.1 is interesting, it was insightful and to the best of my knowledge novel.

Again I would ask the authors to make these plots for FSGM.

Since FSGM is known to be robust to small random perturbations,

I would be surprised that for a majority of random directions, the adversarial examples are brought back to the original class.

- I think a bit more analysis is needed in section 4.2.

Do the authors think that this distinguishability can lead to a defense that uses these statistics?

If so, how?

- I think the analysis of section 5 is fairly trivial.

Distinguishability in high dimensions is an easy problem (as any GAN experiment confirms, see for example Arjovsky & Bottou, ICLR 2017),

so it's not surprising or particularly insightful that one can train a classifier to easily recognize the boundaries.

- Will the authors release code to reproduce all their experiments and methods?

Minor comments: - The justification of why OptStrong is missing from Table2 (last three sentences of 3.3)

should be summarized in the caption of table 2 (even just pointing to the text),

otherwise a first reader will mistake this for the omission of a baseline.

- I think it's important to state in table 1 what is the amount of distortion noticeable by a human.

evaluation fact fact evaluation evaluation fact evaluation evaluation fact fact fact evaluation evaluation evaluation non-arg evaluation fact evaluation evaluation request evaluation fact fact fact fact evaluation evaluation evaluation evaluation evaluation evaluation evaluation fact evaluation evaluation request fact evaluation request request request evaluation evaluation evaluation non-arg evaluation request evaluation request


The authors propose a loss that is based on a RBF loss for metric learning and incorporates additional per exemplar weights in the index for classification.

Significant improvements over softmax are shown on several datasets.

IMHO, this could be a worthwhile paper,

but the framing of the paper into existing literature is lacking

and thus it appears as if the authors are re-inventing the wheel (NCA loss) under a different name (RBF solver).

The specific problems are: - The authors completely miss the connection to NCA loss (https://papers.nips.cc/paper/2566-neighbourhood-components-analysis.pdf)

and thus appear to be re-inventing the wheel.

- The proposed metric learning scenario is exactly as proposed in the NCA loss works,

while the classification approach adds an interesting twist by learning per exemplar weights.

I haven't encountered this before and it could make an interesting proposal.

Of course the benefit of this should be evaluated in ablation studies

( Tab 3 shows one experiment with marginal improvements).

- The authors' use of 'solver' seems uncommon and confusing.

What is proposed is a loss in addition to building a weighted index in the case of classification.

- In the metric learning comparison with softmax (end of page 9) the authors mentions that a Gaussian standard deviation for softmax is learned.

It appears as if the authors use the softmax logits as embedding

whereas the more common approach is to use the bottleneck layer.

This is also indicated by the discussion at the end of page 10 where the authors mention that softmax is restricted to axis aligned embeddings.

All softmax metric learning experiments should be carried out on appropriately sized bottleneck layers.

- Some of the motivations of what the various methods learn seem flawed,

e.g. triplet loss CAN learn multiple modes per class and there is nothing in the Softmax loss that encourages the classes to fill a large region of the space.

- Why don't the authors compare on ImageNet?

Some positive points: - The authors mention in Sec 3.3 that updating the RBF centres is not required.

This is a crucial point that should be made a centerpiece of this work,

as there are many metric learning works that struggle with this.

Additional experiments that can investigate this point would greatly contribute to a well rounded paper.

- The numbers reported in Tab 1 show very significant improvements

If the paper was re-framed and builds on top of the already existing NCA loss, there could be valuable contributions in this paper.

The experimental comparisons are lacking in some respect,

as the comparison with Softmax as a metric learning method seems uncommon, i.e. using the logits instead of the bottleneck layer.

I encourage the authors to extend the paper and flesh out some of the experiments and then submit it again.

fact fact evaluation evaluation evaluation fact evaluation fact evaluation evaluation request evaluation evaluation fact fact fact fact fact request evaluation fact request fact request evaluation request evaluation request evaluation evaluation request


Although GAN recently has attracted so many attentions,

the theory of GAN is very poor.

This paper tried to make a new insight of GAN from theories

and I think their approach is a good first step to build theories for GAN.

However, I believe this paper is not enough to be accepted.

The main reason is that the main theorem (Theorem 4.1) is too restrictive.

1.	There is no theoretical result for failed conditions.

2.	To obtain the theorem, they assume the optimal discriminator.

However, most of failed scenarios come from the discriminator dynamics as in Figure 2.

3.	The authors could make more interesting results using the current ingredients.

For instance, I would like to check the conditions on eta and T to guarantee d_TV(G_mu*, G_hat{mu})<= delta_1 when |mu*_1 – mu*_2| >= delta_2 and |hat{mu}_1 – hat{mu}_2| >= delta_3.

In Theorem 4.1, the authors use the same delta for delta_1, delta_2, delta_3.

So, it is not clear which initial condition or target performance makes the eta and T.

evaluation evaluation evaluation evaluation evaluation evaluation fact fact fact request request fact evaluation


[Summary] The paper is overall well written

and the literature review fairly up to date.

The main issue is the lack of novelty.

The proposed method is just a straightforward dimensionality reduction based on convolutional and max pooling layers.

Using CNNs to handle variable length time series is hardly novel.

In addition, as always with metric learning, why learning the metric if you can just learn the classifier?

If the metric is not used in some compelling application, I am not convinced.

[Detailed comments and suggestions] * Since "assumptions" is the only subsection in Section 2,

I would use \texbf{Assumptions.} rather than \subsection{Assumptions}.

* Same remark for Section 4.1 "Complexity analysis".

* Some missing relevant citations:

Learning the Metric for Aligning Temporal Sequences. Damien Garreau, Rémi Lajugie, Sylvain Arlot, Francis Bach. In Proc. of NIPS 2014.

Deep Convolutional Neural Networks On Multichannel Time Series For Human Activity Recognition. Jian Bo Yang, Minh Nhut Nguyen, Phyo Phyo San, Xiao Li Li, Shonali Krishnaswamy. In Proc.  of IJCAI 2015.

Time Series Classification Using Multi-Channels Deep Convolutional Neural Networks Yi ZhengQi LiuEnhong ChenYong GeJ. Leon Zhao. In Proc. of International Conference on Web-Age Information Management.

Soft-DTW: a Differentiable Loss Function for Time-Series. Marco Cuturi, Mathieu Blondel. In Proc. of ICML 2017.

evaluation evaluation evaluation evaluation evaluation request evaluation fact request request evaluation reference reference reference reference


This paper proposes the use of an ensemble of regression SVM models to predict the performance curve of deep neural networks.

This can be used to determine which model should be trained (further).

The authors compare their method, named Sequential Regression Models (SRM) in the paper, to previously proposed methods such as BNN, LCE and LastSeenValue and claim that their method has higher accuracy and less time complexity than the others.

They also use SRM in combination with a neural network meta-modeling method and a hyperparameter optimization one and show that it can decrease the running time in these approaches to find the optimized parameters.

Pros: The paper is proposing a simple yet effective method to predict accuracy.

Using SVM for regression in order to do accuracy curve prediction was for me an obvious approach,

I was surprised to see that no one has attempted this before.

Using features sur as time-series (TS), Architecture Parameters (AP) and Hyperparameters (HP) is appropriate,

and the study of the effect of these features on the performance has some value.

Joining SRM with MetaQNN is interesting

as the method is a computation hog that can benefit from such refinement.

The overall structure of the paper is appropriate.

The literature review seems to cover and categorize well the field.

Cons: I found the paper difficult to read.

In particular, the SRM method, which is the core of the paper, is not described properly,

I am not able to make sense of the description provided in Sec. 3.1.

The paper is not talking about the weaknesses of the method at all.

The practicability of the method can be controversial,

the number of attempts require to build the (meta-)training set of runs can be huge and lead to something that would be much more costful that letting the runs going on for more iterations.

Questions: 1. The approach of sequential regression SVM is not explained properly.

Nothing was given about the combination weights of the method.

How is the ensemble of (1-T) training models trained to predict the f(T)?

2.  SRM needs to gather training samples which are 100 accuracy curves for T-1 epochs.

This is the big challenge of SRM

because training different variations of a deep neural networks to T-1 epochs can be a very time consuming process.

Therefore, SRM has huge preparing training dataset time complexity that is not mentioned in the paper.

The other methods use only the first epochs of considered deep neural network to guess about its curve shape for epoch T.

These methods are time consuming in prediction time.

The authors compare only the prediction time of SRM with them

which is really fast.

By the way still, SRM is interesting method if it can be trained once and then be used for different datasets without retraining.

Authors should show these results for SRM.

3. Discussing about the robustness of SRM for different depth is interesting

and I suggest to prepare more results to show the robustness of SRM to violation of different hyperparameters.

4. There is no report of results on huge datasets like big Imagenet

which takes a lot of time for deep training and we need automatic advance stopping algorithms to tune the hyper parameters of our model on it.

5. In Table 2 and Figure 3 the results are reported with percentage of using the learning curve.

To be more informative they should be reported by number of epochs, in addition or not to percentage.

6. In section 4, the authors talk about estimating the model uncertainty in the stopping point and propose a way to estimate it.

But we cannot find any experimental results that is related to the effectiveness of proposed method and considered assumptions.

There are also some  typos.

In section 3.3 part Ablation Study on Features Sets, line 5, the sentence should be “Ap are more important than HP”.

fact fact fact fact evaluation evaluation evaluation evaluation evaluation evaluation evaluation evaluation evaluation evaluation evaluation evaluation evaluation evaluation evaluation evaluation fact request fact evaluation evaluation evaluation fact evaluation fact evaluation evaluation request evaluation request fact evaluation fact request fact fact request request


The authors propose a scheme to generate questions based on some answer sentences, topics and question types.

Topics are extracted from questions using similar words in question-answer pairs.

It is similar to what we find in some Q&A systems (like lexical answer types in Watson).

A sequence classifier is also used to tag the presence of topic words.

Question types correspond mostly to salient questions words.

LSTMs are used to encode the various inputs and generate the questions.

The paper is well written and easy to follow.

I would expect more explanations why sentence classification and labeling results presented in Table 2 are so low.

Experimental results on question generation are convincing and clearly indicate that the approach is effective to generate relevant and well-structured short questions.

The main weakness of the paper is the selected set of question types that seems to be a fuzzy combination of answer types and question types (for ex. yes/no).

Some questions type can be highly ambiguous;

for instance “What” might lead to a definition, a quantity, some named entities...

Hence I suggest you revise your qt set.

I would also suggest, for your next experiments, that you try to generate questions leading to answers with list of values.

fact fact evaluation fact fact fact evaluation request evaluation evaluation evaluation evaluation request request


This paper offers a very promising approach to the processing of the type of sequences we find in dialogues, somewhat in between RNNs which have problem modeling memory, and memory networks whose explicit modeling of the memory is too rigid.

To achieve that, the starting point seems to be a strength GRU that has the ability to dynamically add memory banks to the original dialogue and question sentence representations, thanks to the use of imperative DNN programming.

The use of the reparametrization trick to enable global differentiability is reminiscent of an ICLR'17 paper "Learning graphical state transitions".

Compared to the latter, the current paper seems to offer a more tractable architecture and optimization problem that does not require strong supervision and should be much faster to train.

Unfortunately, this is the best understanding I got from this paper,

as it seems to be in such a preliminary stage that the exact operations of the SGRU are not parsable.

Maybe the authors have been taken off guard by the new review process where one can no longer improve the manuscript during this 2017 review

(something that had enabled a few paper to pass the 2016 review).

After a nice introduction,

everything seems to fall apart in section 4, as if the authors did not have time to finish their write-up.

- N is both the number of sentences and number of word per sentence,

which does not make sense.

- i iterates over both the sentences and the words.

The critical SGRU algorithm is impossible to parse

- The hidden vector sigma, which is usually noted h in the GRU notation, is not even defined

- The critical reset gate operation in Eq.(6) is not even explained, and modified in a way I do not understand compared to standard GRU.

- What is t?

From algorithm 1 in Appendix A, it seems to correspond to looping over both sentences and words.

- The most novel and critical operation of this SGRU, to process the entities of the memory bank, is not even explained.

All we get at the end of section 4.2 is " After these steps are finished, all entities are passed through the strength modified GRU (4.1) to recompute question relevance."

The algorithm in Appendix A does  not help much.

With PyTorch being so readable, I wish some source code had been made available.

Experiments reporting also contains unacceptable omissions and errors:

- The definition of 'failed task', essential for understanding, is not stated (more than 5% error)

- Reported numbers of failed tasks are erroneous:

it should be 1 for DMN+ and 3 for MemN2N.

Page 3: dynanet -> dynet

evaluation fact evaluation evaluation evaluation evaluation evaluation fact evaluation evaluation fact evaluation fact evaluation fact fact request evaluation evaluation fact evaluation request evaluation fact fact request request


Summary The authors argue that ensemble prediction takes too much computation time and resource, especially in the case of deep neural networks.

They then address the problem by proposing an adaptive prediction approach.

The approach is based on the observation that it is most important for ensemble approaches to focus on the "uncertain" examples.

The proposed approach thus conducts early-stopping prediction when the confidence (certainty) of the prediction is high enough, where the confidence is based on the confidence intervals of (multi-class) labels based on the student-t distribution.

Experiments on vision datasets demonstrate that the proposed approach is effective in reducing computation resources while maintaining sufficient accuracy.

Comments * The experiments are limited in the scope of (image) multi-class classification.

It is not clear whether the proposed approach is effective for other classification tasks, or even more sophisticated tasks like multi-label classification or sequence tagging.

* The idea appears elegant but rather straightforward.

One important baseline that is easy but not discussed is to set a static threshold on pairwise comparison (p_max - p_secondmax).

Would this baseline be competitive with the proposed approach?

Such a comparison is able to demonstrate the benefits of using confidence interval.

* The overall improvement in computation time seems to be within a constant scale,

which can be easily achieved by doing ensemble prediction in parallel

(note that the proposed approach would require predicting sequentially).

So are there real applications that can benefit from the improvement?

* typo: p4, line19, neural "netowkrs" -> neural "networks"

fact fact fact fact evaluation evaluation evaluation evaluation evaluation request evaluation fact evaluation fact request request


The paper presents Erdos-Selfridge-Spencer games as environments for investigating deep reinforcement learning algorithms.

The proposed games are interesting and clearly challenging,

but I am not sure what they tell us about the algorithms chosen to test them.

There are some clarity issues with the justification and evaluation which undermine the message the authors are trying to make.

In particular, I have the following concerns: • these games have optimal policies that are expressible as a linear model, meaning that if the architecture or updating of the learning algorithm is such that there is a bias towards exploring these parts of policy space, then they will perform better than more general algorithms.

What does this tell us about the relative merits of each approach?

The authors could do more to formally motivate these games as "difficult" for any deep learning architecture if possible.

• the authors compare linear models with non-linear models at some point for attacker policies,

but it is unclear whether these linear models are able to express the optimal policy.

In fact, there is a level of non-determinism in how the attacker policies are encoded which means that an optimal policy cannot be (even up to soft-max) expressed by the agent

(as I read things the number of pieces chosen in level l is always chosen uniformly randomly).

• As the authors state, this paper is an empirical evaluation,

and the theorems presented are derived from earlier work.

There is possibly too much focus on the proofs of these theorems.

• There are a number of ambiguities and errors which places difficulties on the interpretation (and potential replication) of the experiments.

As this is an empirical study,

this is the yardstick by which the paper should be judged.

In particular, this relates to: ◦ The architecture of each of the tested Deep RL methods.

◦ What is done to select appropriate tuning parameters of the tested Deep RL methods, if anything.

◦ It is unclear whether 'incorrect actions' in the supervised learning evaluations, refer to non-optimal actions, or simply actions that do not preserve the dominance of the defender, e.g. both partitions may have potential >0.5

◦ Fig 4. right looks like a reward signal,

but is labelled Proportion correct.

The text is not clear enough to be sure which it is.

◦ Fig 4. left and right has 4 methods: rl rewards, rl correct actions, sup rewards, and sup correct actions.

The specifics of how these methods are constructed is unclear from the paper.

◦ What parts of the evaluation explores how well these methods are able to represent the states (feature/representation learning)

and what parts are evaluating the propagation of sparse rewards (the reinforcment learning core)?

The authors could be clearer and more targetted with respect to this question.

There is value in this work,

but in its current state I do not think it is ready for publicaiton.

# Detailed notes [p4, end of sec 3] The authors say that the difficulty of the games can be varied with "continuous changes in potential",

but the potential is derived from the discrete initial game state,

so these values are not continuously varying (even though it is possible to adjust them by non-integer amounts).

[p4, sec 4.1] "strategy unevenly partitions the occupied levels...with the proportional difference between the two sets being sampled randomly"

What is meant by this?

The proportional difference between the two sets is discussed as if it is a continuous property,

but must be chosen from the discrete set of all available partitions.

If one partition one is chosen uniformly randomly from all possibly sets A, B (and the potential proportion calculated) then I don't know why it would be written in this way.

That suggests that proportions that are closer to 1:1 are chosen more often than "extreme" partitions,

but how?

This feels a little under-justified.

"very different states A, B (uneven potential, disjoint occupied levels)"

Are these states really "very different", or at least for the reasons indicated.

Later on (Theorem 3) we see how an optimal partition is generated.

This chooses a partition where one part contains all pieces in layer (l+1) and above and one part with all pieces in layer (l-1) and below, with layer l being distributed between the two parts.

The first part will typically have a slightly lower potential than the other

and all layers other than layer l will be disjoint.

[p6, Fig 4] The right plot y-limits vary between -1 and 1

so it cannot represent a proportion of correct actions.

Also, in the text the authors say:

>> The results, shown in Figure 4 are surprising. Reinforcement learning  >> is better at playing the game, but does worse at predicting optimal moves.

I am not sure which plot shows the playing of the game.

Is this the right hand plot?

In which case are we looking at rewards?

In fact, I am a little confused as to what is being shown here.

Is "sup rewards" a supervised learning method trained on rewards, or evaluated on rewards, or both?

And how is this done.

The text is just not clear enough.

[p7 Fig 6 and text] Here the authors are comparing how well agents select the optimal actions as compared to how close they are to the end of the game.

This relates to the "surprising" fact that "Reinforcement learning is better at playing the game, but does worse at predicting optimal moves.".

I think an important point here is how many training/test examples there are in each bin.

If there are more in the range 3-7 moves from the end of the game, than there are outside this range, then the supervised learner will

[p8 proof of theorem 3] "φ(A l+1 ) < 0.5 and φ(A l ) > 0.5."

Is it true that both these inequalities are strict?

"Since A l only contains pieces from levels K to l + 1"

In fact this should read from levels K to l.

"we can move k < m − n pieces from A l+1 to A l"

Do the authors mean that we can define a partition A, B where A = A_{l+1} plus some (but not all) elements in level l (A_{l}\setminus A_{l+1})?

"...such that the potential of the new set equals 0.5"

It will equal exactly 0.5 as suggested,

but the authors could make it more precise as to why (there is a value n+k < l (maybe <=l) such that (n+k)*2^{-(K-l+1)}=0.5 (guaranteed).

They should also indicate why this then justifies their proof (namely that phi(S0)-0.5 >= 0.5).

[p8 paramterising action space] A comment: this doesn't give as much control as the authors suggest.

Perhaps the agent should also chose the proportion of elements in layer l to set A.

For instance, if there are a large number of elements in l, and or phi(A_{l+1}) is very close to 0.5 (or phi(A_l) is very close to 0.5) then this doesn't give the attacker the opportunity to fine tune the policy to select very good partitions.

It is unclear expected level of control that agents have under various conditions (K and starting states).

[p9 Fig 8] As the defender's score is functionally determined by the attackers score,

it doesn't help to include this on the plot.

It just distracts from the signal.

fact evaluation evaluation evaluation evaluation request request fact evaluation fact fact fact fact evaluation evaluation fact evaluation evaluation request evaluation evaluation fact evaluation fact evaluation request request request evaluation evaluation fact fact fact quote request evaluation fact evaluation evaluation request evaluation quote evaluation fact fact evaluation fact fact fact fact quote evaluation non-arg request evaluation request request evaluation fact evaluation evaluation fact quote request quote request quote request quote fact request request evaluation request evaluation evaluation fact request evaluation


The authors propose a technique to compress LSTMs in RNNs by using a group Lasso regularizer which results in structured sparsity, by eliminating individual hidden layer inputs at a particular layer.

The authors present experiments on unidirectional and bidirectional LSTM models which demonstrate the effectiveness of this method.

The proposed techniques are evaluated on two models: a fairly large LSTM with ~66.0M parameters, as well as a more compact LSTM with ~2.7M parameters, which can be sped up significantly through compression.

Overall this is a clearly written paper that is easy to follow, with experiments that are well motivated.

To the best of my knowledge most previous papers in the area of RNN compression focus on pruning or compression of the node outputs/connections, but do not focus as much on reducing the computation/parameters within an RNN cell.

I only have a few minor comments/suggestions which are listed below:

1. It is interesting that the model structure where the number of parameters is reduced to the number of ISSs chosen from the proposed procedure does not attain the same performance as when training with a larger number of nodes, with the group lasso regularizer.

It would be interesting to conduct experiments for a range of \lambda values: i.e., to allow for different degrees of compression, and then examine whether the model trained from scratch with the “optimal” structure achieves performance closer to the ISS-based strategy, for example, for smaller amounts of compression, this might be the case?

2. In the experiment, the authors use a weaker dropout when training with ISS.

Could the authors also report performance for the baseline model if trained with the same dropout (but without the group LASSO regularizer)?

3. The colors in the figures: especially the blue vs. green contrast is really hard to see.

It might be nicer to use lighter colors, which are more distinct.

4. The authors mention that the thresholding operation to zero-out weights based on the hyperparameter \tau is applied “after each iteration”.

What is an iteration in this context?

An epoch, a few mini-batch updates, per mini-batch?

Could the authors please clarify.

5. Clarification about the hyperparameter \tau used for sparsification: Is \tau determined purely based on the converged weight values in the model when trained without the group LASSO constraint?

It would be interesting to plot a histogram of weight values in the baseline model, and perhaps also after the group LASSO regularized training.

6. Is the same value of \lambda used for all groups in the model?

It would be interesting to consider the effect of using stronger sparsification in the earlier layers, for example.

7. Section 4.2: Please explain what the exact match (EM) and F1 metrics used to measure performance of the BIDAF model are, in the text.

Minor Typographical/Grammatical errors: - Sec 1: “... in LSTMs meanwhile maintains the dimension consistency.” → “... in LSTMs while maintaining the dimension consistency.”

- Sec 1: “... is public available” → “is publically available”

- Sec 2: Please rephrase: “After learning those structures, compact LSTM units remain original structural schematic but have the sizes reduced.”

- Sec 4.1: “The exactly same training scheme of the baseline ...” → “The same training scheme as the baseline ...”

fact fact fact evaluation evaluation evaluation evaluation request fact request evaluation request fact request request request request request request request request request request request request