Inference and Sampling for Archimax Copulas
Authors: Yuting Ng
Ali Hasan
Vahid Tarokh
Venue: NeurIPS 2022 Accept (2022)
Keywords: copula
Archimax
Archimedean
Pareto
extreme
max-stable
Kendall
Pickands

Abstract: Understanding multivariate dependencies in both the bulk and the tails of a distribution is an important problem for many applications, such as ensuring algorithms are robust to observations that are infrequent but have devastating effects. Archimax copulas are a family of distributions endowed with a precise representation that allows simultaneous modeling of the bulk and the tails of a distribution. Rather than separating the two as is typically done in practice, incorporating additional information from the bulk may improve inference of the tails, where observations are limited. Building on the stochastic representation of Archimax copulas, we develop a non-parametric inference method and sampling algorithm. Our proposed methods, to the best of our knowledge, are the first that allow for highly flexible and scalable inference and sampling algorithms, enabling the increased use of Archimax copulas in practical settings. We experimentally compare to state-of-the-art density modeling techniques, and the results suggest that the proposed method effectively extrapolates to the tails while scaling to higher dimensional data. Our findings suggest that the proposed algorithms can be used in a variety of applications where understanding the interplay between the bulk and the tails of a distribution is necessary, such as healthcare and safety.

Inference and Sampling for Archimax Copulas
Yuting Ng" Duke University yuting ng@duke. edu
Ali Hasan"
Duke University ali www.aaauke....... edu
Vahid Tarokh Duke University vahid. tarokhhduke..dd
Abstract
Understanding multivariate dependencies in both the bulk and the tails of distribu- tion is an important problem for many applications, such as ensuring algorithms are robust to observations that are infrequent but have devastating effects. Archimax copulas are family of distributions endowed with precise representation that allows simultaneous modeling of the bulk and the tails distribution. Rather than separating the two is typically done in practice, incorporating additional information from the bulk may improve inference of the tails, where observations are limited. Building develop a non-parametric inference method and sampling algorithm. Our proposed methods, to the best of our knowledge, are the first that allow for highly flexible and scalable inference and sampling algorithms, enabling the increased use Archimax copulas in practical settings. We experimentally compare state-of- the-art density modeling techniques, and the results suggest that the proposed method effectively extrapolates the tails while scaling to higher dimensional data. Our findings suggest that the proposed algorithms can be used in a variety of applications where understanding the interplay between the bulk and the tails of distribution is necessary, such as healthcare and safety.
the stochastic representation of Archimax copulas, we
1 Introduction
Modeling dependencies between random variables a central task statistics, and understanding the dependence between covariates throughout the distribution is important in characterizing distribution outside of areas of highest density. For example, machine learning contexts, major topic of interest lies in enforcing dependencies between covariates, such spatial dependence in convolutional neural networks or temporal dependence in recurrent neural networks. Copulas are functions that model the joint dependence of random variables, and they have been successfully employed in a variety practical modeling settings due to their case of use and intuitive properties.
Moreover, since copulas are used to represent cumulative distribution functions (CDFs), they have been partieularly useful in situations where tail events are important - events that have high impact but low probability. For example, in computer vision applications, local dependence modeled by convolutions may be sufficient for bulk the data, but for data in the tails of distribution non-local dependencies may be present and need to be modeled. Current successful applications of copulas have largely been limited to a few basic parametric families and low dimensional settings,
"Contributed equally. This work was supported in part by Air Force Office of Scientific Research under award number FA9550-20-1
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

preventing their widespread adoption in settings where the dependencies are complicated or the dimension is large.
Archimax copulas are a class of copulas that merges a tractable form with sufficient expressiveness. Archimax copulas effectively balance the representation of data within the bulk of the distribution while extrapolating to the tails by combining the tractability of Archimedean copulas with the tail properties of extreme-value copulas.
Notably, they remove the simplified symmetry assumption among covariates that is present in Archimedean copulas and the max-stable property of extreme-value copulas. The use of Archimax copulas has resulted in better to data in applications such as healthcare [71] and hydrology B
However, existing computational methods do not allow feasible inference and sampling for Archimax copulas, preventing their widespread use. Therein lies the motivation behind this work. We construct efficient and flexible inference and sampling methods using deep learning techniques and discuss how they compare traditional means of density estimation that use existing copula methods and deep generative models. In addition, we provide numerical studies where the proposed Archimax techniques extrapolate to the tails better than existing methods.
1.1 Related work
Modeling distributions is a major task in machine learning, with techniques such as generative adversarial networks (GANs) [1), normalizing flows (NFs) (78), and variational autoencoders (VAEs) 54 being the major developments for representing complex distributions.
However, these techniques are largely used for modeling the bulk of distribution and may not extrapolate well to out-of-distribution samples or samples within the tails, as discussed in [60]. Some methods have been proposed to represent only the tails, for example in [ However, they disregard the information in the bulk and only focus on the tails. Recently, Bhatia al. [6] considered combining GANs with extreme-valus theory (EVT), However, this and the above methods are used
only for sampling, and do not provide a way of quantifying the dependence the observation.
Copulas are an important technique for representing distribution functions since they allow easy separation of the marginals and the joint dependence structure of distribution. Copulas have been applied in machine learning wherein techniques from machine learning have been used in conjunction with traditional copula theory to model more general classes of densities with examples of such work found in [63 75
R8 please see Appendix AA.6 and A.7 for more background on copulas and application of copulas in machine learning- However, these have generally focused on simplified assumptions such as a symmetric dependence between covariates, or hierarchy of bivariate dependencies. Moreover, these do not extrapolate to tail distributions and do not readily appear to generalize to high dimensions.
With regards to Archimax copulas, several theoretical works have been proposed analyzing the distributions [1] 14 73 In Chatelain et al. [16), the authors proposed a method for inferring stable tail dependence function (stdf) when given an Archimedean generator. However the method assumes knowledge of the Archimedean generator or infers a one-parameter Archimedean generator from pairwise Kendall's taus. Past applications of Archimax copulas were of low dimensions, such as dimensions 2, 3 and 3 in the studies of river flow rates [ rainfall [16] and nutrient intake [71].
Our contributions
We propose methods for filling the gaps in the existing literature by:
1. Developing methods for inferring both the Archimedean generator and stdf;
2. Developing methods for sampling from Archimax copulas;
3. Providing flexible representations for both the radial and spectral components.
Specifically, deep generative models are used to represent the distributions of the radial and spectral components. By taking an expectation, these characterize the Archimedean generator and stdf. The code for this paper is available at :tcttp::/giitin wwt.tauw..ooomuttttataaa www.///wttggg/////...... heee//een-aa AX

---

Figure I: Examples of different radial envelopes and asymptotic dependencies.
2 Background
Copulas are given by separating the marginal distributions from the joint dependence of a random variable. Specifically, consider
F(x)
(1)
where F is a d-variate cumulative distribution function (CDF) of the random variable X (X,..... Xa)) F. Fj is the jth univariate margin, and C the copula describing the dependence between the uniform random variables U (U Ud) (Xi,, Fa(Xa)) [0, 1j
Moreover, if the marginals Fj are continuous, then the copula C is unique [6
As discussed in the introduction, Archimax copulas describe generalization of Archimedean and extreme-value copulas. They are defined as:
Definition (Archimax copula) An Archimax copula is given by
C(u) ( ,4p (wa))):
where yp : (0) + [0, 1] is an Archimedean generator and & [0, oo)a + [0, oc) is a stable tail dependence function (stdf) 170 14 73%.
From the definition, the Archimax copula is completely characterized by these two functions and the objective during inference lies in estimating both the stdf and the Archimedean generator. Both functions have specific properties that must be fulfilled in order to obtain a valid Archimax copula.
The sidf is defined as:
Definition (stable tail dependence function (stdf)) d-variate stdf l (0, + [0, 00) is given by:
((x) d
max (2 dFw(w)) dEw Da- JE(L... .d)
max [z, Jell.-. d)
(3)
with a spectral random variable W E A satisfying moment constraints Ew [W,] = 1/d
j E {1,-..,d 122 821.
Intuitively, the stdf dictates the asymptotic dependence between covariates, with examples given in Figure Notably, the definition completely relies on the distribution of W, and the sidf is homogeneous of order one, i.e. f(Cz1.... ..ra) c((21, ... ,zd) for c > 0.
On the other hand, the Archimedean generator is defined by:
Definition 3 (Archimedean generator) An Archimedean generator (0) + [0, 1] is the Williamson d-transform of a random variable R
x(x) Wn(x) = 1 (1 x m d-1
dFF(() ER (1 x d-17
(4)
R
where (y)4 = ma y) for y E R 722 97V.
The Archimedean generator has one-to-one correspondence with the distribution of R 722 and dictates the shape of the radial envelope applied across all covariates, with examples given in Figure
notation: random variables in uppercase, observations in lowercase, scalars not bold, vectors bold.

---

With these definitions in mind, we leverage Charpentier et al. 14 Theorem 3.3] for inference and sampling, which established that any random vector
4 R x (Si. Sa), X 10,00)
(5)
has dependence that follows an Archimax copula, where R and S are independent and known the radial and simplex components, with supports R > 0. S E [0, 11 ((S) s 1. The marginals of X is given by jo such that the random vector
(p(X)), .((a)). U [0.1]%
(6)
follows Archimax copula. Moreover, every Archimax copula given in has decomposition given by (5) and (6) The radial component R is the same in Definition 3 and the simplex compo- nent S has one-to-one correspondence with the spectral component W from Definition Therefore, representing the R and W in Definitions 3 and respectively provides complete understanding of the distribution.
3 Method
As established in Section the sidf and Archimedean generator are expectations of functions of the spectral component W and radial component respectively. We first model these expectations where W and R are discrete random variables with finite support [31 (3]. We then let them be outputs of generative networks in the limit of infinite support. We begin by describing the inference algorithm for the stdf followed by the inference algorithm for the Archimedean generator. Through sampling the stdf we define the relationship between and W which we make use of in the inference for the Archimedean generator. We finally show how the representations we use can be easily adapted for sampling from Archimax copulas. A flow chart describing the relationship between stdf and Archimedean generator parameter estimation is in Figure
Update te: Asymptotic Dependence Compute {(u, x) min.c(1......) Po (u,))/z // Maximize log (()(E(())) + log (x
Update yot Radial Envelope
Sample (o(S), compute R. T Minimize Eie( ( (so((t)) - w
Figure 2: Flow chart describing relationship between stdf and Archimedean generator estimation.
3.1 Stable tail dependence function inference and sampling
For this section, we suppose that the Archimedean generator yp-) is known and the goal is to infer or sample from the stdf.
3.1.1 Inference for stable tail dependence function
We consider a stdf following Definition
2 where is specified by the spectral decomposition with To summarize the inference for the stdf: we first establish representation of the stdf through the empirical expectation of samples w from generative network Gw. We then define the likelihood of transformed data observations and optimize the parameters of Gw to maximize this likelihood.
spectral component W 122. 47 82
Following Chatelain et al. [16], when given the Archimedean generator +p we can define the transfor- mation u, x) for observation and pseudo-observation x (Z1.-.. xa) E A
(u, x) = min(y² (u1)/21, ... .6.
(uu))......._)/ (ua)
a transformation used for estimating extreme-value copulas 10
In the case of Archimax copulas, and by using the homogeneity property of the stdf (3) the CDF of the random variable {(U,x) x) is expressed as:
P(E(U,x) s x) P(E(U,x) =)= = -[(p(xx) = 1- 1-x(xx(x)).
(8)

---

The full derivation is given in Appendix A.2.1
Following Hasan et al.
38 differentiate the CDF of g(U,x) with respect x and obtain the log-likelihood of the transformed observation as:
log [(;; yp f) = log (-i'(z ((x))) + log({(x)).
(9)
Recalling Definition 2 let Ey be parameterized by generative network Gw such that:
Σ max 3i TyWks (x(x) = je(),ngd .d)
(10)
where Wa = (WKI. Whd) for k = {1 ................. ,1} are samples from Gw with dimension d and output activation given by the Softmax(.) function respect the support W Ad-1- The moment
constraints Ew(Wg [W,] = 1/d for j = {1, dd are scale conveniences and not necessities (3T. To
approximate the moment constraints, we penalize the residual Σø= (E Wxj/t 1/d)2.
We may then train the generative network Gw such that Eg approximates e by minimizing the negative log-likelln loeelikeddood of transformed observations Guu, x), with x uniformly sampled on the unit simplex Ad-1. The full technique is presented in Algorithm| D Appendix A.1
The inverse =1 in the computation of E(u. x) in can be computed numerically by Newton- Raphson. The derivative xp' in the computation of log ((x; (2, in (9) can be calculated explicitly from Definition 3 aa:
('(x) = ER (d-1)
(11)
R
Putting these computations together results in the likelihood of the stdf given the generator.
3.1.2 Sampling the simplex component
We now consider sampling the simplex component S. We noted in the introduction that there is one-to-one relationship between S and W. Using this relationship and the result of Charpentier et al. 14 the stdf may also be written in terms of the simplex component through the following definition:
Definition 4 (survival distribution of simplex component) The survival distribution function (SDF) of the simplex component S is:
sa))a11 P(S > s) = max(0, 1 -.))
(12)
which relates to the so-called generalized Pareto copulas defined as:
Definition 5 (generalized Pareto copula) A generalized Pareto copula is defined as the copula of generalized Pareto distribution with support on (-o0, 0j2 and CDF specified by l as:
P(X, < <I,,".. .Xa= xa) max(0, -(( e(-311''.. -Ira)
(13)
Moreover, it has stochastic representation
U
U Wa
X
(14)
W
where U is uniformly distributed on J0. 1], independent of spectral component W (W1, Wa)
from the spectral decomposition of in Definition 9 29y.
Therefore, we can frame the sampling of the simplex component through the sampling from generalized Pareto copula, such that, given samples (.11,-.. .Idd)... (I(d- ... ()(d-1)a) of size d - 1 from generalized Pareto distribution with CDF in (13) we may obtain a sample of where coordinates 8, for j (1,-.. ... dd} are computed as 14:
8j max( ... ((d-))))
(15)
The coordinate-wei wise maxima is taken over d - 1 samples to get the d - exponent in the SDF of S, corresponding to the exponent the expression of the Archimedean generator çp.
Due to our convenient representation of l as an expectation of a function of W from generative network Gw, we are able sample generalized Pareto distribution from its stochastic representation. The full technique is presented in Algorithm 2 in Appendix A.]
5

---

3.2 Archimedean generator inference and sampling
We now assume that the stdf is known and the goal is to infer the Archimedean generator. We consider a general representation of the Archimedean generator following Definition 3 where is d-monotone function specified by the Williamson dr-rrasfoomm of the radial component R [4 72
To provide an outline for the overall approach: we first consider the so-called Kendall distribution, which describes an integral transform of the copula. We then use the fact that the empirical Kendall distribution converges to the true Kendall distribution as the empirical copula converges to the true copula, providing means of estimation. The Kendall distribution is formally defined as:
Definition 6 (Kendall distribution) Let C be copula, and let the CDF of the random variable U E [0, 1] be the copula C. Define the random variable W := C(U). The Kendall distribution of C' is the multivariate probability integral transform given by the CDF of W:
K ((w) P(C(U) S w), w E [0,1].
(16)
In the case of Archimax copulas, using the stochastic representation and the homogeneity property of the stdf (3), the Kendall distribution is expressed as:
((w)= = P(pp(R((()))))
Sa)) S w).
(17)
The full derivation is provided in Appendix A.3 The empirical Kendall distribution Kn, for given observations (U11.-... Uid), (Uml.-. ... und) is defined as:
1 Σ n
Kn(w)=
1 (ws S w)
(18)
where
W4:
1 (una Wa. Ukd < uid)-
(19)
1
In the case of symmetric dependence in the non paraaaarettic inference of Archimedean copulas [34]. is uniformly distributed on the simplex A and ((S).... .Sa) S₁ +... Sy is discrete with the cardinality of the support the same as W and Kn is the Kendall distribution of
Then, R
unique Archimedean copula 34
In the non-parametric inference of Archimax copulas, we define the random variables:
= ((S1..... ..................... and T
(20)
where is a discrete random variable with support the same size as and Kn, is the Kendall distribution of an Archimax copula. Note that R and Z are independent as R and S are independent.
Using Kn,, we reconstruct the support of R, given é and the support of S, which in turn provides the support of Z. With the final objective of letting R and S be independent and identically distributed (iid) outputs of generative networks, we first (linearly) interpolate K₂ to be equispaced, such that each w; has the same probability k; = 1/(n,,z,), where n,- and n. are the chosen sizes of supports for and Z. We describe the reconstruction procedure for the general case with non-iid random variables including cases where , Zy in Appendix A.3] The sizes of supports n, and n, are chosen empirically with examples given in Appendix B.T
Suppose the supports of the distributions of R, 2, W, T are finite and respectively denoted by: W {wi, W2, .... ]. R (ri, ,r,,, 2 {21, ,2,-.. "nn...)) T R. 21 E } = {t1, t2, ... tn.n, where the n, elements of W are sorted in decreasing (non- increasing) order, and the n, elements of are sorted in increasing non-decreasing) order. This reverse ordering is due to & being decreasing function.
[ryzz rj
We minimize the mean sum of square residuals motivated by the uniform convergence of the empirical process V Kn K) as n + ao established by Barbe et al. 4
Th
(w: - 2)())))
(21)
nl 12

---

where, following Definition
3
1 Mr j=1
d-1
(0(4)) = WR(1.)
= 1
(22)
The finite support assumption of R and Z is not necessary. and we consider a modification where the supports R, are specified by samples from generative networks. The main objective is to learn parameters of generative network G' given the empirical Kendall distribution K,, and samples of Z. Since scaling the support by constant c > 0 does not change the copula, we add regularization term for E&(H] = 1. The algorithm can be understood where the map between R and Z to W via T, and the support R are updated in an alternating fashion. The full technique is presented in Algorithm in Appendix A.I] A.I]
alternating minimization algorithm,
The learned GR also provides a source of samples for the radial component R, which we use to generate full samples from the Archimax copula.
3.3 Inference and sampling for Archimax copulas
We finally summarize the combination of inference and sampling for both the Archimedean and simplex component to obtain the full algorithm for inference and sampling for Archimax copulas. This culminates into an iterative technique that successively updates each component.
3.3.1 Inference for Archimax copulas
We initialize with p(x) exp(-x) and infer l to learn Gw, following Section 3.1 This special combination of &o and & corresponds to extreme-value dependence with the max-stable property. To aid inference for this initialization step, we pre-process We do so by computing the block maxima, a technique from extreme-value copulas where we group observations and take the coordinate-wise maximas within each group. Specifically, given obser- vations [#11, ... Ii4), .... (In1.'.. Ind), the block-maximas (M11,'' ... mid),"" "", (Ma1,". """a)"""""""" mia) for k blocks of size n/k are computed as:
data to have extreme-valuc dependence.
my
(()/k)())))
(23)
for i (1........ k) and j = {1. d}. To determine the block size, with larger block sizes n/k better approximating extreme- value dependence and more blocks & for more observations, we test the block maximas for extreme-value dependence via the max-stable property using test by Kojadinovic et al. (55). and select the first & where the null hypothesis of extreme-value dependence not rejected, starting from n, with details given in Appendix A.4.
Given an estimate of l and learned Gw. we are able to generate many samples of S, thereby providing way to compute Z f(S). We then infer ip, learning GR, following Section 3.2 At this point, we repeat the estimation for Gw with the updated &p to improve our estimate for F. The algorithms may be iterated as needed with suggested convergence criteria such as Cramérvon Mises (CvM) distance between successively estimated copulas.
3.3.2 Sampling for Archimax copulas
Given samples of the radial and simplex components, the stochastic representation of Archimax copulas (5) gives straightforward method for sampling Archimax copulas [1]. Specifically, sample and R, then multiply and normalize by yp; see Algorithm in Appendix A. for details.
4 Experiments
To understand the modeling and sampling capabilities of the proposed algorithms, we conduct a number of empirical studies. We compare the proposed method to number of existing copula based and deep learning based methods for density estimation. Our experiments relate to the main focus
: An alternative initialization scheme based on testing different one me--parmeterr families of Archimedean generators with the log likelihood of transformed observations is also given in Appendix(A.4.
7

---

of the proposed method where we are interested in understanding the dependencies between the variables in both the bulk and the tail. In that sense. we conduct a number of experiments where we wish to extrapolate to the tail. Further experimental details may be found in Appendix [
The metric we use to compare the methods is based the Cramér-von Mises (CvM) statistic, which computes the L distance between the empirical copula and the estimated copula. This statistic is commonly used determine differences between distributions in goodness-of-fn tests [8]]. We use version where we compare the empirical copula of true samples to the empirical copula of generated
samples. Specifically, it can be written as: CvM [[CC., (u) Co.n (u) ddu where C..n the empirical copula of true samples and Ce.n is the empirical copula of generated samples [8].
Inference for Archimedean generator Our first set of experiments involve inference for the Archimedcan generator following the proposed method in Section 3.2 We consider data with dimension d = 10 and sample size of = 1000 given the true stdf f. To the best of our knowledge, our proposed method is the first method for non non-parametric inference of flexible Archimedean generators in Archimax copulas. As such, there are no baselines for comparison. Instead, we compute the results in terms of the map A(w) (w)((( (w)),, w E (0, due to its scale invariant property and known asymptotic variance, which was described as useful metric for how well yp is fit in 33 34
We evaluated our proposed method on the Clayton (C), Frank (F), Joe (J) and Gumbel (G) generators, representing different radial envelopes. for Kendall's tau of r = {0.2, 0.5). representing different associations. The stdf comes from the family of negative scaled extremal Dirichlet (NSD), a flexible class of stdf [5). All estimates were within the asymptotic variance of A(w), w E (0, 1). The next best method is comparing to Clayton generator estimated by pairwise Kendall's s tau I6. We give the results in terms of MSE to À in Table | and plot estimates of X(w) in Figure 5 in Appendix B.I Additional experiment results, including small sample performance n 200, and choices of support sizes n, and n2 are in Appendix B.I
Inference for stable tail dependence function and sampling for simplex component We now consider the reverse scenario where wish to estimate the stdf given the true Archimedean generator şo. We compute the integrated relative absolute error (IRAF) between the estimated to and the true L. The IRAE is given by (RAE Ea) [ JA |((x) Eg (x)))/((x dx (16). For the
same experiment settings as above, results are in Table Additional experiment results, including time taken, are in Appendix B.2
Table 1: Inference of yp given true and inference of l given true yp
C.2
C 0.5
F 0.2
O.5
J.2
J 0.5
G
G 0.5
MSE x1a
[6
0.01
0.04
0.9
9
1
9
0.7
8
MSE x10 (OURS)
0.2
0.2
0.1
0.1
0.3
0.1
0.2
0.1
IRAE #0.01 16
0.05
0.11
0.04
0.04
0.05
1.00
0.06
0.15
IRAE 10.01 (OUrS)
0.06
0.12
0.04
0.05
0.06
0.07
0.08
0.15
Modeling nutrient intake The USDA studied the nutrient intake of women 192
One particular
task is understanding the dependencies between the intake of different nutrients. We can model this using an Archimax copula understand the dependencies between the nutrients in the bulk and the tail. To assess this, we fit and compare models from the literature on representing distributions and compute the CvM goodness-of-fit statistic for each model. Specifically, we break our comparison into two different types of models: copula based models and deep network based models. For the copula models, for methods marked with #, we use the oo described in Section 3.2 and for methods marked with i we use the f described in Section 3.2 For the deep generative models, we use standard methods based on the Wasserstein GAN 2 masked autoregressive flow (MAF) [77), and variational autoencoders (VAE) [54]. The results are presented in Table 2 The proposed method (Gen-AX) with the Archimax has the lowest CvM distance among all the competing methods, suggesting that the proposed method is recovering the true dependency structure. The state of the art Clayton Archimax (C-AX) did not perform well possibly due to the difficulty scaling the single parameter Clayton generator to higher dimension. The Archimedean copula (AC *) possibly benefited from the use of our proposed yo. We additionally provide examples of samples versus the ground truth in Appendix B.3 for the different methods well as an explanation of the different abbreviations.
8

---

Table 2: Goodness-offff nutrient intake data and 100d NSD copula
COPULAS
DEEP NETS
OURS
GC
RV
CV
DV
AC*
HAC
EV
C-AX
GAN
MAF
VAE
Gen-AX
Nutrient CYM x10-*
0,081
0.3
0.2
0.6
0,030
0.2
0.4
0.6
0.033
0,036
0,053
0.026
C-NSD CvM x10-=
16
3
Extrapolating to extreme rainfall Archimax copulas were initially developed as a tool to study the behaviour methods that estimate the joint distribution of extreme events [II]. The extreme-value copula that arises in the limit can be understood from the stdf & and the index of regular variation of the Archimedean generator ş. Unlike extreme-value copulas which emerge from the limiting distribution of extreme events, the motivation for the use of Archimax copulas is to model extreme data, where observations are rare, from a mix of moderately less extreme data, where observations are relatively more abundant.
In this experiment, we consider another realistic dataset which models the monthly rainfall in French Britanny as studied by Chatelain et al. [6]] We are interested in testing how well the proposed method can extrapolate to the extremes from non-extreme data. Specifically, we analyze the monthly rainfall data, which did not pass the test of extreme-value dependence [55 [6].
We first train the models on the full dataset. We then generate many samples from the trained model, compute the block maxima, then estimate the extremal dependence from the block maxima using the CFG estimator [10]. The results presented in Table where compare the proposed method to deep generative models. Additional experiment details and results, including plots samples from the bulk and the extremes,
in Appendix B.4 We did not compare to copula based models since classical copulas are generally Gumbel or independent in the extremes and thus not suitable for this application. However, for the purpose of modeling monthly rainfall without extrapolating to extremes, we compared to a variety of copula based models, including skew-t copulas which The full details and results are given in 99
class of flexible asymmetric copulas 23 Appendix B.4.1
56
87
Out-of-distributte detection Using the same realistic dataset as above, added outliers gener- ated uniformly at random on the unit cube. The AUC and F1 scores for outlier detection based on likelihoods are given in Table Additional experiment details and figures are in Appendix B.5
Table 3: Goodness-of fit to dependence in the extremes
Table 4: But-of-distributii detection
IRAE 0.01
GAN
MAF
VAE
C-AX
OURS
MAF
VAE
OURS
C-NSD
0.52
0.42
0.16
0.12
0.03
AUC
0.82
0.37
0.92
F-NSD
0.10
0.15
0.04
0.04
0.04
F1
0.48
0.04
0.72
J-NSD
0.03
0.48
0.03
0.08
0.03
G-NSD
0.07
0.38
0.08
0.16
0.04
High dimensional modeling We finally consider an experiment where we infer and sample data from a 100-dimensional Clayton-NSD Archimax copula. Scaling high dimensions is an important property of the proposed method since many existing copula methods fail to scale beyond lower dimensions. As such, we only compare with deep generative models since the existing copula models resulted in numerical errors during optimization. We report the CvM statistic in the second line of We additionally provide examples of the samples in Appendix B.6
Table 2
5
Conclusion
We developed highly flexible and scalable inference and sampling algorithms, facilitating the use of Archimax copulas practical settings. We experimentally compare to state-of-the-ar density modeling techniques, and the results suggest that the proposed method effectively extrapolates to tails while scaling to higher dimensional data. The methods are especially useful in scenarios requiring extrapolations to the tails while also incorporating data from the bulk.

---

Limitations and future work A single Archimedean generator +p to describe the radial envelope across all coordinates may not be sufficiently expressive for certain datasets. For these cases, hierarchical Archimax copulas may be more appropriate and direction of future work [42]. Other directions for future work include modifying the generator architectures to allow modeling of temporal dependence and also application of Archimax copulas to describe dependencies of non-tabular data, such as via graph neural networks (68)
Potential negative societal impacts Model misspecification may lead to misspecification of risks, leading to potentially catastrophic outcomes areas such as healthcare, safety and finance. Risks may be mitigated by confirming a reasonable fit between the observations generated by the model and the data.
10

---

References
[1] Michaël Allouche, Stéphane Girard, and Emmanuel Gobet. EV-GAN: Simulation of extreme events with relu neural networks. Journal of Machine Learning Research, 23(150):1-39, 2022. URL http::/ttp:/mmm wttppppppss.////1-1a1.o
html
[2] Martin Arjovsky, Soumith Chintala, and Léon Bottou. Wasserstein generative adversarial networks. In International conference machine learning, pages 214-223. PMLR, 2017.
[3] Tomá Bacigál, Vladimír Jágr, and Radko Mesiar. Non-exchangeable random variables, Archi- max copulas and their fitting to real data. Kybernetika, 47(4):519-533 2011.
[4] Philippe Barbe, Christian Genest, Kilani Ghoudi, and Bruno Rémillard. On Kendall's process. Journal of Multivariate Analysis, 58(2):197-229, 1996.
[5] Léo R. Belzile and Johanna G. Nelehová. Extremal attractors of Liouville copulas. Journal of Multivariate Analysis, 160:68-92. 2017. ISSN 0047-259X. doi: http:///doi.or//10.000666 1016 jjmma.2017.055000 URL
50047259K173004558
[6] Siddharth Bhatia, Arjit Jain, and Bryan Hooi. Exgan: Adversarial generation of extreme samples. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 5750-6758, 2021.
[7] Younes Boulaguiem, Jakob Zscheischler, Edoardo Vignotto, Karin van der Wiel, and Sebastian Engelke. Modeling and simulating spatial extremes by combining extreme value theory with generative adversarial networks. Environmental Data Science, 1:e5, 2022. doi: 10.1017/sds. 2022.4.
[8] Mary Ann Branch, Thomas F. Coleman, and Yuying Li. A subspace, interior, and conjugate gradient method for large-scale cound-constrained minimization problems. SIAM Journal Scientific Computing, 21(1):1-23, 1999. URL pesttps://doct scipy ort.////cccccppp//////////////// reference/ Sewpreencegggeeertte//aiiippppep opt.iipze..iiiiiiee..e.e least 1eastesuuuttssssesessseeee
[9] T. A. Buishand, L. de Haan, and C. Zhou. On spatial extremes: With application to a rainfall problem. The Annals of Applied Statistics, 2(2):624-642, 2008. ISSN 19326157. URL http: http://wwwt http://ww.jj.. jstor org/ stbble/02444220
[10] P. Capéraà, A.- Fougères, and C. Genest. A nonparametric estimation procedure for bivariate extreme value copulas. Biometrika, 84(3)=567-577, 09 1997.
[11] Philippe Capéraà, Anne-Laure Fougères, and Christian Genest. Bivariate distributions with given extreme value attractor. Journal of Multivariate Analysis, 72(1):30 49, 2000.
[12] Bo Chang, Shenyi Pan, and Harry Joe. Vine copula structure learning via monte carlo tree search. In Kamalika Chaudhuri and Masashi Sugiyama, editors, Proceedings of the Twenty-Second International Conference on Artificial Intelligence and Statistics, volume 89 of Pruceedings of Machine Learning Research, pages 353-361. PMLR, 16-18 Apr 2019. URL https://proceedingg mlr wwwpsprsspeee..... / /89 chang/h html
[13] Yale Chang, Yi Li, Adam Ding, and Jennifer Dy. A robust-equitable copula dependence measure for feature selection. In Arthur Gretton and Christian C. Robert, editors, Proceedings of the 19th International Conference Artificial Intelligence and Statistics, volume 51 of Proceedings of Machine Learning Research, pages 84 92, Cadiz, Spain, 09-11 May 2016. PMLR. URL https:t/::/// htt.:/tts//prrcceeingg// mlr. www.s/eeaaaaae. httpa/a//v/cchhaggg html
[14] A. Charpentier, A.-L. Fougères, C. Genest, and J.G. Nelehová. Multivariate Archimax copulas. Journal of Multivariate Analysis, 126:1 18-136, 2014.
[15] Arthur Charpentier, Jevn-David Fermanian, and Olivier Scaillet. The estimation of copulas: Theory and practice. Copulas: From theory to application in finance, pages 35 35-64, 2007.
[16] Simon Chatelain, Anne-Laure Fougères, and Johanna G. Nelehová. Inference for Archimax copulas. The Annals of Statistics, 48(2):1025 1051, 2020.

---

[17] Pawel Chilinski and Ricardo Silva. Neural likelihoods via cumulative distribution functions. In Jonas Peters and David Sontag, editors, Proceedings of the 36th Conference on Uncertainty in A Artificial Intelligence (UAI), volume 124 of Proceedings of Machine Learning Research, pages 420 429. PMLR, 03-06 Aug 2020. URL URl.ptttp://proceeding mlr press/v124/ chilinskinn 20a html]
[18] Stuart G. Coles and Jonathan A. Tawn. Modelling extreme multivariate events. Journal of the Royal Statistical Society. Series B (Methadological), 53(2):377-392, 1991. ISSN 00359246.
URL http::/www.
wwo/ strg/staable2345749
[19] Antonia Creswell, Tom White, Vincent Dumoulin, Kai Arulkumaran, Biswa Sengupta, and Anil A Bharath. Generative adversarial networks: An overview. IEEE Signal Processing Magazine, 35(1):53-65, 2018.
[20] Ruifei Cui, Perry Groot, Moritz Schauer, and Tom Heskes. Learning the causal structure of copula models with latent variables. In UAI. Corvallis: AUAI Press, 2018.
[21] Data to AI Lab at MIT. Copulas. URL
com/ www.://o//dddddeeeeeeeeeeeeeeeee Copulas
[22] L. De Haan, A Spectral Representation for Max-stable Processes. The Annals of Probability, 12(4): 1204, 1984.
[23] Stefano Demarta and Alexander J. McNeil. The Statistical Review / Revue Internationale de Statistique, 1111) 2005. ISSN 03067734, 17515823, URL htttp://www.jstorr /rg/s/tbee//////72233333333
t copula and related copulas. International
[24] Alexandre Drouin, Étienne Marcotte, and Nicolas Chapados. TACTiS: Transformer-aateenioon copulas for time series. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepes- vari, Gang Niu, and Sivan Sabato, editors, Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research, pages 5447-5493. PMLR, 17-23 Jul 2022. URL https: ptttp:://proceedings mlr. press/v162/
drouin22a. html
[25] Elad Eban, Gideon Rothschild, Adi Mizrahi, Israel Nelken, and Gal Elidan. Dynamic copula networks for modeling real-valued time series. In Artificial Intelligence and Statistics, pages 247-255, PMLR, 2013.
[26] Gal Elidan. Copula bayesian networks. In J. Lafferty, C. Williams, J. Shawe-Taylor, R. Zemel, and A. Culotta, editors, Advances in Neural Information Processing Systems, volume 23. pages 559-567. Curran Associates, Inc., 2010. URL https: //tppee//roocceiinggg neurips. cc/ aptp:/0000ff11121107777771111110 Paper pdf
[27] Gal Elidan. Inference-less density estimation using copula bayesian networks. In UAI, 2010.
[28] Gal Elidan. Copula network classifiers (cnes). In Artificial intelligence and statistics, pages 346-354. PMLR, 2012.
[29] Michael Falk, Simone A. Padoan, and Florian Wisheckel. Generalized pareto copulas: A key to multivariate extremes. Journal of Multivariate Analysis, 174:104538, 2019. ISSN 0047-259X.
doi: http:://do..rr//000016/// iww....9.......0............. 104538. URL https://www, http:/wwww.cinceiirrrrttt
[30] Amélie Fils- Villetadd, Armelle Guillou, and Johan Segers. Projection estimators of pickands de- pendence functions. The Canadian Journal of Statistics / La Revue Canadienne de Statistique, 36(3):369-333 382, 2008 ISSN 03195724. URL/tttp://www jstor..ss................r....r org/ (ateaaabe///111/8/
[31] Anne-Laure Fougères, Cécile Mercadier, and John Nolan. Dense classes of multivari- ate extreme value distributions. Journal of Multivariate Analysis, 116:109-129, 2013. ISSN 0047-259X. doi: http:://doo.orgg10.1011///// 101.0000..0.............110 URL httph:::///// /tttp:///ww/ www sciencedirect. htteee/ddreett..cooccoeeeeeeeeee httpa/ccc....cooo......coc.aaaat
[32] Christian Genest and Cath-Catereine Favre. Everything you always wanted to know about copula modeling but were afraid to ask. Journal of Hydrologic Engineering, 12(4):347-368 2007. 10. http:///SS..Saaaooo////..///....
you
10. 1061 http:////...//aaaaaac././/o/oaoa
12

---

[33] Christian Genest and Louis-Paul Rivest. Statistical inference procedures for bivariate archimedean copulas. Journal of the American Statistical Association, 88(423):1034-1046 1993.
[34] Christian Genest, Johanna Nešlehová, and Johanna Ziegel. Inference in multivariate archimedean copula models. TEST: An Official Journal of the Spanish Society of Statis- tics and Operations Research, 20(2):223, 2011.
[35] Jan Górecki and Malte S. Kurz. The HACopula toolbox. URL https: com/
gorecki/HHCopulal
[36] Jan Górecki, Marius Hofert, and Martin Holea. On structure, family and parameter estimation of hierarchical archimedean copulas. Journal of Statistical Computation and Simulation, 87 (17):3261-3324, 2017. doi: 10. 00:18000004495552011116666666
JRtthtpp:://ddt /
1080/00949655. 2017. 1365148
[37] Shaobo Han, Xuejun Liao, David Dunson, and Lawrence Carin. Variational Gaussian copula inference. In Arthur Gretton and Christian C. Robert, editors, Proceedings of the 19th International Conference Artificial Intelligence and Statistics, volume 51 of Proceedings of Machine Learning Research, pages 829-838, Cadiz, Spain, 09-11 May 2016. PMLR. URL
https: https://proceedingge mll. www.ppreeeaepppe http//v11/aaaaln htmll
[38] Ali Hasan, Khalil Elkhalil, Yuting Ng. João M Pereira, Sina Farsiu, Jose Blanchet, and Vahid Tarokh. Modeling extremes with d-max-decreasing neural networks. In The 38th Conference on Uncertainty Artificial Intelligence, 2022.
[39] José Miguel Hernández- James Lloyd, and Daniel Hernández-Lobate Gaussian process conditional copulas with applications to financial time series. In CJJ. Burges, L. Bottou, M. Welling, Z. Ghahramani, and K.Q. Weinberger, editors, Advances in Neural Information Processing Systems, volume 26. Curran Associates, Inc., 2013.
[40] Marcel Hirt, Petros Dellaportas, and Alain Durmus. Copula-like variational inference. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Gar- nett, editors, Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019. URL https::/ http://e......a.a/a///////////// pdf
[41] Marius Hofert, Martin Mächler, and Alexander McNeil. Archimedean Copulas in High Dimensions: Estimators and Numerical Challenges Motivated by Financial Applications. Journal de la société française de statistique, 154(1):25-63 2013. http:/tp///ww/ numdam
org/ htte:ttem/ttt/ JSFF JSFS 2013 154 1144112220 _____00_________
[42] Marius Hofert, Raphaël Huser, and Avinash Prasad. Hicrarchical Archimax copulas. Journal
of Multivariate Analysis, 167:195-211 2018.
[43] Jim Huang and Brendan J Frey. Structured ranking learning using cumulative dis- tribution networks. In D. Koller, D. Schuurmans, Y. Bengio, and L. Bottou, ed- itors, Advances in Neural Information Processing Systems, volume 21. Curran As- sociates, Inc., 2008 URL ltttps:/prrocedd
[44] Jim Huang and Nebojsa Jojic. Maximum-likelihodd learning of cumulative distribution functions on graphs. In Yee Whye Teh and Mike Titterington, editors, Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, volume 9 of Proceedings of Machine Learning Research, pages 342- 349, Chia Laguna Resort, Sardinia. Italy, 13-15 May 2010. PMLR. /RR.hhttps://proceee ngs mlr. www.prrreeee press/ / 99/ h/hhangg00b html
[45] Jim C. Huang and Brendan J. Frey. Cumulative distribution networks and the derivative- sum-product algorithm. In Proceedings of the Twenly-Fourle Conference on Uncertainty in Artificial Intelligence, UAI'08, page 290297, Arlington, Virginia, USA, 2008 AUAI Press. ISBN 0974903949.
13

---

[46] Jim C. Huang and Brendan J. Frey. Cumulative distribution networks and the derivative sum- product algorithm: Models and inference for cumulative distribution functions on graphs. Journal of Machine Learning Research, 12(10)330-3453 2011.
[47] Xu Huang. Statistics of bivariate extreme values. PhD. Tinbergen Institute, 1992.
[48] Tim Janke, Mohamed Ghanmi, and Florian Steinke. Implicit generative copulas. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan, editors, Advances in Neural Information Processing Systems, volume 34, pages Associates, Inc.. 2021. URL https: https://prooceddngss
Curran http://pwoooaoooooooooooooo///// ptt.ppp.././aape///////////// Cicka67bbdc42800113b000 1ad67 ed696f oadeeeo1fff1 Paper pdr
[49] Piotr Jaworski, Fabrizio Durante, Wolfgang Karl Hardle, and Tomasz Rychlik, editors. Copula theory and its applications. Lecture notes in statistics. Springer, Berlin, Germany, 2010 edition, July 2010.
[50] Piotr laworski, Fabrizio Durante, and Wolfgang Karl Hardle, editors. Copulae in mathematical and quantitative finance. Lecture notes statisties. Springer, Berlin, Germany, 2013 edition, June 2013.
[51] Harry Joe. Dependence modeling with copulas. Chapman and Hall/CRC, June 2014. ISBN 9781466583233 doi: 10. 10.1201/b17116, URL
9781466583238
[52] Nebojsa Jojic, Chris Meek, and Jim Huang. Exact inference and learning for cumulative distribution functions loopy graphs. In Lafferty, C. Williams, J. Shawe-Taylor, R. Zemel, and A. Culotta, editors, Advances in Neural Information Processing Systems, volume 23, pages 874-882. Curran Associates, Inc., 2010. URL httpps://proceedinngg neurips. cc/ .dff
[53] Sanket Kamthe, Samuel Assefa, and Mare Deisenroth. Copula flows for synthetic data generation, 2021. URL 0ppp:/2200111//77777244667771111
[54] Diederik P Kingma and Max Welling. Auto-encoding variational Bayes. In International Conference on Learning Representations, 2014.
[55] Ivan Kojadinovic, Johan Segers, and Jun Yan. Large-sample tests of extreme- value dependence for multivariate copulas. LIDAM Reprints ISBA 201 1025, Université catholique de Louvain. Institute of Statistics, Biostatistics and Actuarial Sciences (ISBA), 2011. URL https:// EconPapers rww.cree......m org/RePEc: aiz: louvar: 2011025
[56] Tönu Kollo and Gaida Pettere. Parameter estimation and application of the multivariate skew t-copula. In Piotr Jaworski, Fabrizio Durante, Wolfgang Karl Härdle, and Tomasz Rychlik, editors, Copula Theory and Its Applications, pages 289-298, Berlin, Heidelberg, 2010. Springer Berlin Heidelberg.
[57] Yves-Laurent Kom Samo. Inductive mutual information estimation: A convex maximum- entropy copula approach. In Arindam Banerjee and Kenji Fukumizu, editors, Proceedings of The 24th International Conference Artificial Intelligence and Statistics, volume 130 of Proceedings of Machine Learning Research, pages 2242 2250. PMLR, 13-15 Apr 2021. URL https://proceedingg mlr. press v/t0//13////0///// 11130//om--- sam.21a. html
[58] Eric Landgrebe, Madeleine Udell, et al. Online mixed missing value imputation using Gaussian copula. In Workshop the Art of Learning with Missing Values (Artemiss) hosted by the 37th International Conference on Machine Learning (ICML), 2020.
[59] Mike Laszkiewicz, Johannes Lederer, and Asja Fischer. Copula-based normalizing flows. In ICML Workshop on Invertible Neural Networks, Normalizing Flows, and Explicit Likelihood Models, 2021. URL httlhtttp... htttpp://openreviie
[60] Charline Le Lan and Laurent Dinh. Perfect density models cannot guarantee anomaly detection. Entropy, 23(12):1690 2021.
14

---

[61] Benjamin Letham, Wei Sun, and Anshul Sheopuri. Latent variable copula inference for bundle pricing from retail transaction data. In ICML, pages 217-225, 2014. URL http //proceedings.. mlr. press/ sipess/v3l +32/letham14 html
[62] Zheng Li, Yue Zhao, Nicola Botta, Cezar Ionescu, and Xiyang Hu. Copod: copula-based outlier detection. In 2020 IEEE International Conference on Data Mining (ICDM), pages 1118-1123. IEEE, 2020.
[63] Chun Kai Ling, Fei Fang, and J. Zico Kolter. Deep archimedean copulas. In H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan. and H. Lin, editors, Advances in Neural Information Processing Systems. volume 33, pages 1535-1545. Curran As-
sociates, Inc., 2020. URL lattps://proceeddngg http://pooceddnns..e.eeeeeeeeeee htt.srrrss..cppppp/////////////e
pdf
[64] Weiwei Liu. Copula multi-label learning. In H. Wallach, H. Larochelle, A. Beygelzimer, F. M'Alché-Buc, E. Fox, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019.
neurips Paper. pdf
[65] David Lopez-paz, Jose Hernández-lobato, and Bernhard Schölkopf. Semi-supervised domain adaptation with non-parametric copulas. In F. Pereira, C.J. Burges, L. Bottou, and K.Q. Weinberger, editors, Advances in Neural Information Processing Systems, volume 25. Curran Associates, Inc., 2012. URL
wtw.aiips..ccpppp//////////////e pdf
[66] David Lopez-Paz, Jose Miguel Hemández-Lobato, and Ghahramani Zoubin, Gaussian process vine copulas for multivariate dependence. In International Conference on Machine Learning. pages 10-18. PMLR, 2013.
[67] Jian Ma and Zengqi Sun. Mutual information is copula entropy. Tsinghua Science and Technology, 16(1):51-54, 2011. doi: 10.1016/S1007- 0100007001111111 021000000000100000-666
[68] Jiaqi Ma, Bo Chang, Xuefei Zhang, and Qiaozhu Mei. Copulagnn: Towards integrating representational and correlational roles of graphs in graph neural networks. In International Conference on Learning Representations, 2021.
[69] Jan-Frederik Mai and Matthias Scherer. Simulating Copulas. WORLD SCIENTIFIC, 2nd edition, 2017. doi: 10. 10.1142/10265. URL htop:://wwww.wwt 10. N0.1142/10263
[70] Bijan Mazaheri, Siddharth Jain, and Jehoshua Bruck. Robust correction of sampling bias using cumulative distribution functions. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems, volume 33, pages 3546-3556. Curran Associates, Inc., 2020. URL
neurips cc/ pdf
[71] Alexander McNeil and Johanna Nešlehová. From Archimedean to Liouville copulas. Journal of Multivariate Analysis, 101(8):172--1790, 2010.
[72] Alexander J. McNeil and Johanna Nelehová. Multivariate Archimedean copulas, d d-monotone functions and /l-norm symmetric distributions. The Annals of Statistics. ):059) 3097, 2009. doi: 10. 1/114/477-005566 UtI.httss://do /ttp:pppp///oi..o//11... 22214/077-A05555
[73] Radko Mesiar and Vladimír Jágr. d-dimensional dependence functions and Archimax copulas. Fuzzy Sets and Systems, 228:78-888 87, 2013.
[74] Roger B. Nelsen. An introduction to Copulas. Springer series in statistics. Springer New York, New York, NY, 2. ed. 2006. corr. 2, pr. softcover version of original hardcover edition 2006 edition, 2010. ISBN 9780387286785 978 9781441921093, OCLC: 700190717.
[75] Yuting Ng, Ali Hasan, Khalil Elkhalil, and Vahid Tarokh. Generative Archimedean copulas. In 37th Conference on Uncertainty in Artificial Intelligence (UAI), 2021.
15

---

[76] Georg Ostrovski, Will Dabney, and Rémi Munos. Autoregressive quantile networks for generative modeling. In ICML, pages 3933-3942, 2018. URL http::///prccooddingss mml press//880/ostroskkt 18a. html
[77] George Papamakarios, Theo Pavlakou, and lain Murray. Masked autoregressive flow for density estimation. Advances in neural information processing systems, 30, 2017.
[78] George Papamakarios, Eric Nalisnick, Danilo Jimenez Rezende, Shakir Mohamed, and Balaji Lakshminarayanan, Normalizing flows for probabilistic modeling and inference. Journal of Machine Learning Research, 22(57):1-64, 2021.
[79] James Pickands, Multivariate extreme value distribution. In 43rd Session of the International Statistical Institution, volume 2, pages 199-8788844-999 Buenos Aires, 1981.
[80] Barnabás Póczos, Zoubin Ghahramani, and Jeff Schneider. Copula-based kernel dependency measures. In Proceedings of the 29th International Conference International Conference on Machine Learning, ICML'11, page 16351642, Madison, WI, USA, 2012. Omnipress. ISBN 978 15051
[81] Bruno Remillard and Olivier Scaillet. Testing for equality between two copulas. 8: 229-231, june 2007. doi: com abstract=1014555
URL https:///ssr
[82] Paul Ressel. Homogeneous distributionsand spectral representation of classical mean values and stable tail dependence functions. Journal of Multivariate Analysis, 117:246-256, 2013.
Laurent Callot, Roberto Medico, and Jan [83] David Salinas, Michael multivariate forecasting with low-rank Gaussian copula pro- Gasthaus. High-dimensions
cesses. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d' d'Alché-Buc, E. Fox, and R. Gar- nett, editors, Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019. URL httpps://prooeeedngg: www.ripp..ceppppr//0009000 pdf
[84] David Salinas, Huibin Shen, and Valerio Perrone. quantile-based approach for hy- perparameter transfer learning. In Hal Daumé III and Aarti Singh, editors, Proceed- ings of the 37th International Conference
Machine Learning, volume 119 of Proceed- ings of Machine Learning Research, pages 8438-8448. PMLR, 13-18 Jul 2020. URL https: http:s/////cceeingg..mmm mlr. www.ppeaaaaa.... wttess///11//aaaaa220a.
[85] Ricardo Silva, Charles Blundell, and Yee Whye Teh. Mixed cumulative distribution net- works. In Geoffrey Gordon, David Dunson, and Miroslav Dudík, editors, Proceedings the Fourteenth International Conference on Artificial Intelligence and Statistics, volume 15 of Proceedings Machine Learning Research, pages 670-678, Fort Lauderdale, FL., USA, 11-13 html
Apr 2011. PMLR. URI. J ptttps://proceedingg
mlr. htt..pprees/////illallli...
[86] Abe Sklar. Fonctions de répartition à dimensions leurs marges. 8:229 2211, 1959.
[87] Michael S. Smith, Quan Gan, and Robert J. Kohn. Modelling dependence using skew copulas: Bayesian inference and applications. Journal of Applied Econometrics, 27(3):500 522, 2012. doi: https://doi.orr/1000 wttp:/o/0//r1110//freeeeeoooo 10000/0c..11.1.. 1215. URL httpp: htttps://onlineeirrrre wiley. com/doi/
abs/10.1002// 1215
[88] Natasa Tagasovska, Damien Ackerer, and Thibault Vatter. Copulas as high-dimensional generative models: Vine copula autoencoders. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'A'ccééé Buc. E. Fox, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019. UtRphhttss://proeeeee neurips Paper. pdf
[89] Yaniv Tenzer and Gal Elidan. Speedy model selection (sms) for copula models. In UAI, 2013.
[90] Yaniy Tenzer and Gal Elidan. Helm: Highly efficient learning mixed copula networks. In UAI, pages 790-799, 2014.
16

---

[91] Dustin Tran, David Blei, and Edo M Airoldi. Copula variational inference. In C. Cortes, N. Lawrence, Lee, M. Sugiyama, and R. Gamett, editors. Advances in Neural Information Processing Systems, volume 28. Curran Associates, Inc., 2015. httphhtts:://pttt
seurips.cc/paaee http:///wwee00eeeaaaereee/////er
pdf
[92] USDA. Dataset: CSFII 1985, continuing survey of food intakes by individu- als. women 19-50 years of age and their children 1-5 years of age, 6 waves, 1985. Dataset, U.S. Department of Agriculture, Agriculture Research Ser- vice, 1985. https://wis ars usda.
csfii85 6waves doc.
[93] Hongwei Wang, Lantao Yu, Zhangjie Cao, and Stefano Ermon. Multi-agent imitation learning with copulas. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pages 139-156. Springer, 2021.
[94] Huahua Wang, Farideh Fazayeli, Soumyadeep Chatterjee, and Arindam Banerjee. Gaussian Copula Precision Estimation with Missing Values. Samuel Kaski and Jukka Corander, editors, Proceedings of the Seventeenth International Conference on Artificial Intelligence and Statistics, volume 33 of Proceedings of Machine Learning Research, pages 978-986, Reykjavik, Iceland, 22-25 Apr 2014. PMLR. URL https://pproceedingg mml.inng..........mmmmmm..m press/ v33/ wang1aan html
[95] Ruofeng Wen and Kari Torkkola. Deep generative quantile-copula models for probabilistic forecasting. In ICML., 2019.
[96] Mario Wieser, Aleksander Wieczorek, Damian Murezzan, and Volker Roth. Learning sparse latent representations with the deep copula information bottleneck. In International Con- ference on Learning Representations, 2018. hBRLaatte:///ppaaeeeeeeeet//oooo
HkOwHx-RM RW
[97] R. E. Williamson. Multiply monotone functions and their Laplace transforms. Duke Mathe- matical Journal, 23(2):189- -207, 1956.
[98] Andrew G Wilson and Zoubin Ghahramani. Copula processes. In J. Lafferty, C. Williams, J. Shawe-Taylon Taylor, R. Zemel, and A. Culotta, editors, Advances in Neural Information Processing Systems, volume 23, pages 2460-2468. 2468. Curran As- sociates, Inc., 2010. URL htttps:/prroceed: wttp.//pp...cpppppp///////////// [E0016884444/0000000044445225552
[99] Toshinao Yoshiba. Maximum likelihood estimation of skew-t copulas with its applications stock returns. Journal of Statistical Computation and Simulation, 88(13):2489-2555 2506. 2018. doi: 10. 10.000099946555.0000 22t.2201...400 1469631. 2018. 1469631
[100] Yuxuan Zhao and Madeleine Udell. Matrix completion with quantified uncertainty through low rank Gaussian copula. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin, editors, Advances Neural Information Processing Systems, volume 33, pages 20977-20988 Curran Associates, Inc., 2020. | wwwpeediiinnnnnuuirn.ee..e www.nnuu..... wttppppppccpppppp////2/e// Ei2e:fforrrr22000221111110007110 pdf
17

---

Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's
contributions and scope? [Yes]
(b) Did you descrihe the limitations of your work? [Yes]
(c) Did you discuss any potential negative societal impacts of your work? [Yes]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [Yes]
(b) Did you include complete proofs of all theoretical results? [Yes] See Appendix
3. If you experiments..
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or as URL)? [Yes] See Appendix
B, and the code is available at athtpps: : //t:://gtttt cow sonm/yutinnggggeenaaa AX
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] See Appendix B.
(c) Did you report error bars (e.g., with respect to the random seed after running experi- ments multiple times)? [Yes] See Appendix B.
(d) Did you include the total amount of compute and the type of resources used (c.g., type of GPUs, internal cluster, cloud provider)? [Yes] See Appendix B.
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators? [Yes] Nutrient intake
data [92]. |opplassttts::://iituue //github. wttp.oooooooooee www./oorecii/HaCopulll..eoooeeee PyTorch, SciPy, NumPy, Python.
HAC toolbox https
(b) Did you mention the license of the assets? [No] Most information presented on the USDA Web site is considered public domain information. Public domain information may be freely distributed or copied, but use of appropriate byline/photo/image credits requested. Attribution may cited as follows: "U.S. Department of Agriculture." "
(c) Did you include any new assets either the supplemental material or as URL? [Yes]
The code in the supplemental material
be made available online.
(d) Did you discuss whether and how consent was obtained from people whose data you're using/curating? [No] Data was collected only after consent was given.
(e) Did you discuss whether the data you using/curating contains personally identifiable information or offensive content? [No] Personally identifiable information has been removed.
5. If you used crowdsourcing conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if applicable? [No] The instructions given participants is available at [2].
(b) Did you describe any potential participant risks, with links Institutional Review Board (IRB) approvals, if applicable? [No] The description of potential participant risks available at [92].
(c) Did you include the estimated hourly wage paid to participants and the total amount spent participant compensation? [No] The description of participant compensation available at [92]
18


Reviews:
[{"review_id": "nryJ4J2iNCY", "reviewer": "Reviewer_HCqW", "summary": "This paper proposes novel procedures for both inference and sampling of Archimax copulas. This family of copulas is important due to their ability to represent the bulk and tail of distributions simultaneously which can be suitable for healthcare and hydrology real-world data. The authors propose a 'hybrid' approach mixing copulas and neural networks to allow for more flexible estimation. In experiments, the proposed method is compared to SOTA density modeling techniques and the results suggest that their method can extrapolate to tails and scale to higher dimensions. ", "questions": "1. Complexity of the proposed algorithms is not mentioned. Is there any trade-off between flexibility and scalability?\n2. Are there any ablation studies regarding data efficiency? Seems like such non-parametric methods can be data-hungry\n3. Will this generative network work for non-tabular data? Can there be an example of another NN architecture?\n4. How do the sampled marginals compare to the original data?\n5. To increase the significance/contribution of the paper, could you add an experiment for out-of-distribution detection for example? (since this is mentioned as a limitation of existing methods) maybe one of the experiments you have already can be used to present some uncertainty scores.\n6. related work that is missing:  \nBhatia, Siddharth, Arjit Jain, and Bryan Hooi. \"Exgan: Adversarial generation of extreme samples.\" Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 35. No. 8. 2021.\n\nMinor:\nFigure 1 is not referenced in the text\n\n----------------------------------------------------------------------------------------------------------------------------------------------------------------\n### Update after authors response:\n\nThank you for the thorough response and addressing all of the comments in the updated manuscript.\nI am happy to accordingly raise the score.", "limitations": "The authors have addressed the limitations and potential societal negative impact of their work.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n  - Originality - to the best of my knowledge this is the first work to address flexible density estimation of bulk and tail distribution with Archimax copulas.\n  - Quality - the authors put a lot of effort into including a high level of technical details and experiments in the paper and appendix \n  - Clarity - since copulas are not a straightforward tool in the machine learning community, I appreciate the background overview and related work mentioned throughout the paper and supplementary. \n  - Significance - considering the tails not only bulk of the data is overlooked problem and can be very significant in many real-world applications\n\nWeaknesses:\n  - Originality - although the presented methodology is novel, it does still build on existing work regarding Archimax copulas\n  - Quality - I would expect more challenging/motivating experimental results to support the claims of significance and contribution from the introduction\n  - Clarity - A running example or one real data motivating example could improve/clarify why Archimax copulas are an appropriate/necessary tool in critical scenarios. This can help bring the paper closer to the ML community, make it more relevant for readers.\n  - Significance - since copulas are still not widespread at the ML conferences, I feel like additional motivation for such papers is needed, either to showcase superiority on large-scale real-world datasets or find some new tasks where SOTA models fail. I also appreciate the code submission and effort of implementing everything in Python (rare for statistics methods). \n \n\n\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "dzx55EZlCY3", "reviewer": "Reviewer_5Yjr", "summary": "The authors propose scalable estimation and sampling procedures for Archimax copulas. On simulated and real data, they demonstrate that Archimax copulas fit using their procedure can model complex data with dependencies between extreme values accurately, in comparison to existing deep generative methods.", "questions": "See above for suggestions.", "limitations": "The authors clearly explain some of the method's important limitations.", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The paper bridges two important areas of research: copulas and deep generative modeling. It is highly original (the first method of its kind) and technically excellent. It is also potentially very significant in its impact: modeling rare events is critical to managing risk in real world applications, and relying naively on modern deep generative approaches can potentially be very problematic. The paper is overall quite clear, though there are two areas where I struggled: first, an intuitive explanation as to why Archimax copulas are good for modeling dependencies between extreme events would be helpful to the reader; second, I’d appreciate a concise statement of the complete model up front, explaining how the deep generative model determines the stdf. ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "ciUthYW50V5", "reviewer": "Reviewer_d1gb", "summary": "The authors propose an efficient inference and sampling schemes for archimax copulae based on learning of the generator and stable tail dependence functions through deep learning techniques. ", "questions": "As far as I understand when optimizing the presentations of the copula generator and stable tail dependence function the authors start with an initial copula generator $exp(-x)$. Have you considered other initial generators and how would this affect learning of the copula?\n\nHave you considered estimating several copulae as in a hierarchical archimedean copula?\n\nWhat do you say about modifying the architecture of the generative neural networks?", "limitations": "The authors faithfully address limitations and possible negative impact of the proposed method.", "rating": 7, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "*Originality*. The work is original. To my knowledge, the authors propose a new method to infer and sample from archimax copulas extending the previous work in this area.\n\n*Clarity*. The authors go a long way to make sure the paper is accessible by a larger machine learning community. They provide necessary background info on copulae and their use in machine learning. They also provide extensive derivations in the appendix. That said, the material itself is quite dense. I have not found typos. One minor thing is that Figure 1 does not seem to be referenced anywhere in the text.\n\n*Quality*. The authors build upon a previously developed theory and methods to derive a scheme to infer archimax copulae via the means of deep learning models which also allows for an easy sampling from. The authors provide an analysis of performance of the proposed method comparing it to other state-of-the-art methods. The details of the experiment setups are given in detail in the supplementary material. The authors provide extensive background/related work review in both the main text and the appendix.\n\n*Significance*. Inference of the multivariate distributions from the data is a core statistical/machine learning problem. The authors propose a way to infer multivariate dependencies via archimax copulae representations of which are learned via deep learning models. The method also allows for sampling from the learned distribution. The method serves both the bulk and the tails of a distribution. With that said, a proposed method is of major significance for the field.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "2Fc5CVGD1Xb", "reviewer": "Reviewer_dqaw", "summary": "This paper presents new methods for inference and sampling for Archimax copulas. Archimax copulas are a family of copulas defined through an Archimedian generator and a stable tail dependence function (stdf). In order to discuss the inference and sampling for Archimax copulas, the authors first proposed inferential and sampling methods for Archimedian generator and stdf. Then, combining these methods, the methods for inference and sampling for Archimax copulas are established. In experiments, it is seen that the proposed inferential methods for Archimedian generator and stdf show satisfactory performance. Other experiments are given to illustrate that Archimax copulas outperform, or work as well as, some existing models for a couple of real datasets.", "questions": "(g) For the multivariate Archimedean copula (equation (3.4), [49]), all the pairs of variables have the same strength of dependence, and this is considered a drawback in terms of flexibility in general. How about the Archimax copulas? Is it possible to solve this problem by appropriately defining the spectral random variable $W$?\n\n(h) In lines 107 and 108, it is mentioned that $X$ defined in (5) follows an Archimax copula. However it seems to me that $X$ takes values in $[0, \\infty )^d$ in general and therefore the distribution of $X$ is not a copula which is defined on $[0,1]^d$. I think a correction is necessary to avoid this misunderstanding.\n\n(i) I would appreciate the authors' responses to my comments (e) and (f).\n\n", "limitations": "The authors have addressed the limitations and potential negative societal impacts of their work in Section 5. Depending on the authors' response to my questions (g) and (i), I might claim that the usefulness of Archimax copulas is limited.", "rating": 7, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "*** STRENGTHS ***\n\n(a) [Originality] The presented inferential and sampling methods for Archimax copulas seem new. These methods are derived mainly by combining existing methods for Archimedian generators and stdfs.\n \n(b) [Quality] The paper seems technically sound. Comprehensive experiments are given to assess the performance of the proposed inferential methods and compare submodels of Archimax copulas with some existing models.\n\n(c) [Clarity] The paper is clearly written in general. Section 2 providing the background of the presented theory would be helpful for readers who are not family with copulas.\n\n(d) [Significance] Archimax copulas are flexible models which include Archimedian copulas and extreme-value copulas as special cases. Therefore the proposed inferential and sampling methods for Archimax copulas could be useful in practice when flexible modelling is required.\n \n\n*** WEAKNESSES ***\n\n(e) [Quality] Apart from Archimax copulas, there exist other flexible families of copulas such skew-$t$ copulas (see, e.g., Joe [49], Section 3.17.2). The paper does not sufficiently compare the Archimax copulas with those existing copulas.\n\n(f) [Significance] I am not sure about the popularity of Archimax copulas and the importance of the related theory this paper presents. (Nonetheless I appreciate the results of experiments which suggest the usefulness of the Archimax copulas.)\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept