Venue: ICLR 2017 conference

Reviews:

final_score: rating: 6: Marginally above acceptance threshold
review_content: The authors propose a new approach for estimating maximum entropy distributions
subject to expectation constraints. Their approach is based on using
normalizing flow networks to non-linearly transform samples from a tractable
density function using invertible transformations. This allows access to the
density of the resulting distribution. The parameters of the normalizing flow
network are learned by maximizing a stochastic estimate of the entropy
obtained by sampling and evaluating the log-density on the obtained samples.
This stochastic optimization problem includes constraints on expectations with
respect to samples from the normalizing flow network. These constraints are
approximated in practice by sampling and are therefore stochastic. The
optimization problem is solved by using the augmented Lagrangian method. The
proposed method is validated on a toy problem with a Dirichlet distribution and
on a financial problem involving the estimation of price changes from option
price data.

Quality:

The paper seems to be technically sound. My only concern would the the approach
followed to apply the augmented Lagrangian method when the objective and the
constraints are stochastic. The authors propose their own solution to this
problem, based on a hypothesis test, but I think it is likely that this has
already been addressed before in the literature. It would be good if the
authors could comment on this.

The experiments performed show that the proposed approach can outperform Gibbs
sampling from the exact optimal distribution or at least be equivalent, with
the advantage of having a closed form solution for the density.

I am concern about the difficulty of he problems considered.
The Dirichlet distributions are relatively smooth and the distribution in the
financial problem is one-dimensional (in this case you can use numerical
methods to compute the normalization constant and plot the exact density).
They seem to be very easy and do not show how the method would perform in more
challenging settings: high-dimensions, more complicated non-linear constraints,
etc...

Clarity:

The paper is clearly written and easy to follow.

Originality:

The proposed method is not very original since it is based on applying an
existing technique (normalizing flow networks) to a specific problem: that of
finding a maximum entropy distribution. The methodological contributions are
almost non-existing. One could only mention the combination of the normalizing
flow networks with the augmented Lagrangian method. 

Significance:

The results seem to be significant in the sense that the authors are able to
find densities of maximum entropy distributions, something which did not seem
to be possible before. However, it is not clearly how useful this can be in
practice. The problem that they address with real-world data (financial data)
could have been solved as well by using 1-dimensional quadrature. The authors
should consider more challenging problems which have a clear practical
interest.

Minor comments:

More details should be given about how the plot in the bottom right of Figure 2 has been obtained.

"a Dirichlet whose KL to the true p∗ is small": what do you mean by this? Can you give more details on how you choose that Dirichlet?

I changed updated my review score after having a look at the last version of the paper submitted by the authors, which includes new experiments.
review_title: OK method, but lack of strong evaluation on real-world problems and lack of significant methodological contributions

final_score: rating: null
review_content: You refer to the regularity conditions for solving (6), but do not say anything about how those conditions do (or don't) hold in your scenario. Can you say a little about this?
review_title: Berteskas 2014 conditions

final_score: rating: 6: Marginally above acceptance threshold
review_content: This paper applies the idea of normalizing flows (NFs), which allows us to build complex densities with tractable likelihoods, to maximum entropy constrained optimization.

The paper is clearly written and is easy to follow.

Novelty is a weak factor in this paper. The main contributions come from (1) applying previous work on NFs to the problem of MaxEnt estimation and (2) addressing some of the optimization issues resulting from stochastic approximations to E[||T||] in combination with the annealing of Lagrange multipliers.
Applying the NFs to MaxEnt is in itself not very novel as a framework. For instance, one could obtain a loss equivalent to the main loss in eq. (6) by minimizing the KLD between KL[p_{\phi};f], where f is the unormalized likelihood f \propto exp \sum_k( - \lambda_k T - c_k ||T_k||^2  ). This type of derivation is typical in all previous works using NFs for variational inference.
A few experiments on more complex data would strengthen the paper's results. The two experiments provided show good results but both of them are toy problems.


Minor point:

Although intuitive, it would be good to have a short discussion of step 8 of algorithm 1 as well.
review_title: Application of normalizing flows to max-ent

final_score: rating: null
review_content: In the experiment with financial data in section 4.2, how does the Gibbs method and the MEFN differ in the modeling of the tails of the price density? From figure 3 it seems that MEFN produces slightly heavier tails. Perhaps the authors could report QQ plots for bot distributions and the observed quantiles in the test data.
review_title: Performance of MEFN in the modeling of the tails

final_score: rating: 9: Top 15% of accepted papers, strong accept
review_content: Much existing deep learning literature focuses on likelihood based models. However maximum entropy approaches are an equally valid modelling scenario, where information is given in terms of constraints rather than data. That there is limited work in flexible maximum entropy neural models is surprising, but  is due to the fact that optimizing a maximum entropy model requires (a) establishing the effect of the constraints on some distribution, and formulating the entropy of that complex distribution. There is no unbiased estimator of entropy from samples alone, and so an explicit model for the density is needed. This challenge limits approaches. The authors have identified that invertible neural models provide a powerful class of models for solving the maximum entropy network problem, and this paper goes on to establish this approach. The contributions of this paper are (a) recognising that, because normalising flows provide an explicit model for the density, they can be used to provide unbiased estimators for the entropy (b) that the resulting Lagrangian can be implemented as a relaxation of a augmented Lagrangian (c) establishing the practical issues in doing the augmented Lagrangian optimization. As far as the reviewer is aware this work is novel – this approach is natural and sensible, and is demonstrated on an number of models where clear evaluation can be done. Enough experiments have been done to establish this is an appropriate method, though not that it is entirely necessary – it would be good to have an example where the benefits of the flexible flow transformation were much clearer. Further discussion of the computational and scaling aspects would be valuable. I am guessing this approach is probably appropriate for model learning, but less appropriate for inferential settings where a known model is then conditioned on particular instance based constraints? Some discussion of appropriate use cases would be good. The issue of match to the theory via the regularity conditions has been brought up, but it is clear that this can be described well, and exceeds most of the theoretical discussions that occur regarding the numerical methods in other papers in this field.

Quality: Good sound paper providing a novel basis for flexible maximum entropy models.
Clarity: Good.
Originality: Refreshing.
Significance: Significant in model development terms. Whether it will be an oft-used method is not clear at this stage.

Minor issues

Please label all equations. Others might wish to refer to them even if you don’t.
Top of page 4: algorithm 1→ Algorithm 1.
The update for c to overcome stability appears slightly opaque and is mildly worrying.  I assume there are still residual stability issues? Can you comment on why this solves all the problems?
The issue of the support of p is glossed over a little. Is the support in 5 an additional condition on the support of p? If so, that seems hard to encode, and indeed does not turn up in (6). I guess for a Gaussian p0 and invertible unbounded transformations, if the support happens to be R^d, then this is trivial, but for more general settings this seems to be an issue that you have not dealt? Indeed in your Dirichlet example, you explicitly map to the required support, but for more complex constraints this may be non trivial to do with invertible models with known Jacobian? It would be nice to include this in the more general treatment rather than just relegating it to the specific example.

Overall I am very pleased to see someone tackling this question with a very natural approach.
review_title: Flexible Maximum Entropy Models

Meta-review: This paper is an interesting application to maximum entropy problems, which are not widely considered in deep learning. The reviewers have concerns over the novelty of this method and the ultimate applicability and scope of these methods. But this method should be of interest, especially in connecting deep learning to the wider community on information theory and should make an interesting contribution to this year's proceedings.
Decision: Accept (Poster)