Venue: ICML 2021 Workshop INNF

Reviews:

final_score: rating: Accept
review_content: summary: The authors propose using copula-based distributions as the base distribution of NFs, with the theoretical and empirical justification that lipschitz NFs with light tailed base distributions (e.g. Gaussian) struggle to model heavy-tailed distributions, and in doing so their lipschitz constants blow up. Using copula, it becomes possible to design a base distritbution with suitable tails by looking at the tails of the marginal of the data distribution.
Justification for rating: - I found it interesting that NFs with light-tailed base distributions struggle to fit heavy-tailed distributions, and it seems sensible to want to use a heavy-tailed base distribution in this case. However it's not clear when you would need anything more complicated than a factorised base distribution whose marginals are heavy tailed (i.e. the Independence Copula used in the paper). If this is sufficient, then drawing on copula theory to design base distributions seems like overkill. I would be curious to see any experiments where using more complex copula makes a noticeable difference.
- Another experiment that I think would be helpful is to ablate against the baseline of using a Gaussian base distribution with larger variance to alleviate the instabilities shown in Figure 1. What would happen for example if you make the variance of the base distribution learnable?
- you say "estimating an appropriate marginal distribution and copula still requires further investigation". The first thing that comes to mind is just plot of histogram of each dim of the data and fit a suitably tailed univariate distribution to this.
- since the experiments in the paper are still on synthetic datasets, I'm curious to know what commonly used datasets for NFs have heavy tails. It would be interesting to identify these datasets, then experiment using heavy-tailed base distributions as drop-in replacements for Gaussian base distributions for SOTA NFs.

Minor comments:
- The copula definition 2.1 is a bit misleading for someone in the NF field who's not familiar with copula. RVs and CDFs seem to be confounded. It would be clearer to write out the mathematical definition in full (e.g. given RV X=(X1,...,Xn) with cts marginal CDFs Fi, U=(F1(X1),...,Fn(Xn)) is the copula of X which has U[0,1] marginals)
- You often refer to the appendix but I think it's been cut off.
review_title: null

final_score: rating: Borderline Accept
review_content: summary: The paper proposes to utilize copulae for modeling a base distribution in normalizing flows. This main idea is to make the base distribution more powerful and allow modeling heavy-tailed distributions. The authors use copulae for that and they show in a toy experiment that such an approach could indeed improve the behavior of training.
Justification for rating: Strengths:
S1: I really like the idea. It is rather straightforward but indeed could be useful.
S2: The Lipschitz perspective is very interesting.  I would love to see more analysis of that in the future.

Weaknesses:
I am fully aware this is a workshop paper. However, I suggest taking a closer look at the following points before preparing the final version of the paper:
W1: The paper is pretty hard to follow. It would be better to formulate the problem more explicitly. At the moment, the problem statement is spread across Sections 1, 2 and 3. There is a feeling that the authors state the problem by showing the solution first (copulae). It would be beneficial to have it well-defined in one section. 
W2: More attention should be paid to learning copulae within normalizing flows. It must be very clear how they are trained, what are training extra costs, potential pitfalls etc.
W3: Obviously, a more thorough empirical analysis is required.
W4: The authors refer to an appendix but I couldn't find one.
review_title: null

Decision: Accept (poster)