Venue: UAI 2022 Conference

Reviews:

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area of AI or moderate to high impact on more than one area of AI, with good to excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations. 
review_content: Q1 Summary and contributions: This paper proposes a new integral-based monotone triangular flow which is called AUTM. The paper shows this is a flow with an explicit inverse and fewer restrictions on the function representations and parameters. The paper benchmarks their results against previous flows on the UCI data sets and for both CIFAR10 and ImageNet32.
Q3 Main strengths: * The authors make a clear comparison between AUTM and previous normalising flow architectures. They do a good job of highlighting the characteristics of the proposed normalising flow.
* As far as I can tell, the theory is presented well and was interesting to read.
* I think others in the community will be interested in this work and it is therefore a contribution. 
* Code is clearly provided and the supplementary materials are useful.
Q4 Main weakness: * While it is challenging to get flows to work on high dimensional problems (especially new ones), Figure 2 seems to mostly show blurs of nondescript objects. How do these samples compare to the baselines shown in Table 3? Why are these images so blurry?
* The conclusion from the paper is no clear. While there is a lot of theory proposed in the paper, the take-away at the moment seems to be that the paper is proposing a new flow that is competitive with previous flows and has some favourable properties. Is there a way to truly show these favourable properties a bit better?
* The paper keeps mentioning this idea of unrestricted parameters, yet AUTM seems to only use simple quadratic polynomials. Why is this the case? Is there a computational complexity associated with using more parameters? If AUTM is restricted in practice to polynomials then it would be helpful to know.
Q5 Detailed comments to the authors: In addition to the comments above:
* The paper might need a bit more clarity of what unrestricted model parameters means. Would the authors be able to provide further details on this?
* I think the readers really need to be told in more detail, when and why AUTM should be used over previous approaches, or if it is more of a theoretical contribution, then the reader should be made more excited towards the end of the paper as to what the introduction of this new NF could mean either now or in the future.
Q2(4) Quality of experiments (Optional): 3: Good: The experimental evaluation is adequate, and the results convincingly support the main claims.
Q7 Justification for your score: I think the paper is well presented and that the theoretical contributions are interesting. The experiments section is mostly complete and the code is provided.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate to high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations. 
review_content: Q1 Summary and contributions: This paper proposes a novel integral-based monotone triangular flow. The contributions of this work fall within unrestricted model classes and parameters, and the proposed method provides an explicit inverse and has some interesting theoretical properties.

Q3 Main strengths: It is interesting that the proposed method has an explicit inverse, which is not always the case of integral-based normalizing flows.

Another interesting property is that the proposed method can be easily incorporated into triangular flow architectures, such as coupling flows and autoregressive flows.

The paper is well written. Experimental results are convincing.

Q4 Main weakness: 
The proposed method satisfy what is defined by “universality”, in the sense that any monotonic normalizing flow is a limit of the proposed flows. It is not clear why this would be a universality property.

It is not clear why the proposed method is called “time machine”. The authors argue that it is because “the model is automatically invertible and the computation of inverse is essentially a reverse of integral limits.” However, this is the case of most, if not all, methods based on integrals for normalizing flows, including the ones cited in the paper.

Q5 Detailed comments to the authors: It is not clear why the proposed method is called “time machine”. The authors argue that it is because “the model is automatically invertible and the computation of inverse is essentially a reverse of integral limits.” However, this is the case of most, if not all, methods based on integrals for normalizing flows, including the ones cited in the paper.

The proposed method satisfy what is defined by “universality”, in the sense that any monotonic normalizing flow is a limit of the proposed flows. It is not clear why this would be a universality property.

There are some spelling and grammatical errors, such as “one of our model performs”, “On GAS datase, our”, “indicate the model use”, …

Q2(4) Quality of experiments (Optional): 3: Good: The experimental evaluation is adequate, and the results convincingly support the main claims.
Q7 Justification for your score: Our overall assessment is based on all the aforementioned comments.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area of AI or moderate to high impact on more than one area of AI, with good to excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations. 
review_content: Q1 Summary and contributions: The paper proposes AUTM Flow, a new class of monotonic flows, based on the idea of integral transforms; the novelty comes from defining the integral wrt to a latent variable; the model can be inverted more straightforwardly than other integral based flows, where one would need a numerical solver and multiple calls to the integrator. The authors prove that the resulting flow is monotone, invertible, universal for monotone mappings and can approximate any coupling or autoregressive flow. 
Q3 Main strengths: The paper presents a novel class of normalizing flows, with competitive performance, and with much higher efficiency than competing flows, e.g. FFJORD. The main computational gains come from the structure of AUTM which is based on a readily invertible transformation. 
Q4 Main weakness: I am not entirely satisfied by some parts of the proof of Theorem 3. See detailed comments below. 
Q5 Detailed comments to the authors: *In the proof of THeorem 3 in many places $t$ is the limit of integration and the variable of integration; please correct
* I am missing a few details in the proof of Theorem 3. The function $g_s(v,t)$ is not uniformly bounded as $s\to 0$ since the density $e^{-t^2/s}C_s$ evaluated at $t=0$ tends to $+\infty$ as $s\to 0$. I would like to see some evidence for the claim just before equation (14) that for any $\epsilon>0$ there exists $\rho\in(0,1)$ such that $|g_\star(v_s(t))-g_\star(v_s(0))|\leq \epsilon$ for all $t\leq \rho$. In particular it seems that the way to establish this would be to use the Lipschitzness of $g_\star$ which then requires bounding $|v_s(t)-v_s(0)|$ uniformly in $s$, which is where I would like to see some more detail. I believe the statement to be true, but I think the proof is incomplete as is, unless there is an obvious argument, in which case you should also definitely include it. I suspect a similar argument may be needed to support the claim that $g_\star(v_s(t))$ is uniformly bounded, appearing just before equation (15). 
*Concerning universality of FFJORD is it not possible to establish using for example arXiv:2012.20414? It would probably require some additional assumptions, but in that case it would be more appropriate to say that FFJORD may be universal under some conditions. 
* In Table 2 it would be useful if you could highlight in bold the best performing method in each column. 
* It seems that choosing $g(v,t)$ to be non-constant in $t$ is crucial for the universality proof provided. In the numerical experiments, $g$ is always taken to be constant wrt $t$. How does this affect universality? Would harder experiments potentially require $g$ to depend non-trivially on $t$? would this impact performance? 
Q2(4) Quality of experiments (Optional): 3: Good: The experimental evaluation is adequate, and the results convincingly support the main claims.
Q7 Justification for your score: The experimental section shows that the method is indeed competitive in performance while being much more computationally efficient. I would consider increasing my score if the issues mentioned, including the proof of Theorem 3, are addressed.
review_title: null

Meta-review: Meta Review: The paper introduces AUTM, an integral-based normalizing flow that has an unrestricted functional form, and can be viewed as a continuous limit of certain monotone normalizing flows. The reviewers and myself agree that this a conceptually interesting class of models, with some interesting theoretical analysis around universality. 
Decision: Accept (Poster)