Original: Gaussian Copula Embeddings
Revised: Gaussian Copula Embeddings

Original: Abstract
Revised: Abstract

Original: Learning latent vector representations via embedding models has been shown promising in machine learning.
Revised: Learning latent vector representations via embedding models has been shown promising in machine learning.

Original: However, most of the embedding models are still limited to a single type of observed data.
Revised: However, most of the embedding models are still limited to a single type of observed data.

Original: We propose a Gaussian copula embedding model to learn latent vectorial representations of items in a heterogeneous-data setting.
Revised: We propose a Gaussian copula embedding model to learn latent vectorial representations of items in a heterogeneous-data setting.

Original: The proposed model can effectively incorporate different types of observed data and, at the same time, yield robust embeddings.
Revised: The proposed model can effectively incorporate different types of observed data and, at the same time, yield robust embeddings.

Original: We demonstrate that the proposed model can effectively learn in many different scenarios, outperforming competing models in modeling quality and task performance
Revised: We demonstrate that the proposed model can effectively learn in many different scenarios, outperforming competing models in modeling quality and task performance

Original:  Introduction
Revised:  Introduction

Original: Representation learning is a prominent machine learning approach for working with originally nonvectorial data.
Revised: Representation learning is a prominent machine learning approach for working with originally nonvectorial data.

Original: Embedding models learn latent vectorial representations for data items that appear together with a context, through modeling the interactions between each center item and its context items.
Revised: Embedding models learn latent vectorial representations for data items that appear together with a context, through modeling the interactions between each center item and its context items.

Original: The approach was first introduced as a language model [14] which learns word representations through modeling the probability of the appearance of a central word given surrounding context words.
Revised: The approach was first introduced as a language model [14] which learns word representations through modeling the probability of the appearance of a central word given surrounding context words.

Original: The word appearance is modeled as an observation from a multinomial word distribution.
Revised: The word appearance is modeled as an observation from a multinomial word distribution.

Original: Building on this notion, exponential family embeddings were proposed [19] which further generalized the original model to a class of models suitable for many observed data types, which have been shown promising in different domains.
Revised: Building on this notion, exponential family embeddings were proposed [19] which further generalized the original model to a class of models suitable for many observed data types, which have been shown promising in different domains.

Original: However, the ability of such models to incorporate heterogeneous data is still limited.
Revised: However, the ability of such models to incorporate heterogeneous data is still limited.

Original: Data in many modern domains is heterogeneous, involving simultaneous observation of different data types such as categorical values, integers and real-valued numbers, and having varied distributions within each data type, hence it is difficult to model them together as observations in vectorial embedding; naive unified solutions that ignore the difference of the data types would not yield good models.
Revised: Data in many modern domains is heterogeneous, involving simultaneous observation of different data types such as categorical values, integers and real-valued numbers, and having varied distributions within each data type, hence it is difficult to model them together as observations in vectorial embedding; naive unified solutions that ignore the difference of the data types would not yield good models.

Original: In particular, the different data types are often distributed over varied scales and with various distributional shapes: naive normalization coupled to modeling with a single distributional assumption would not suffice to yield robust embedding models, and would leave them vulnerable to extreme values and distributional shapes not corresponding to the assumed ones.
Revised: In particular, the different data types are often distributed over varied scales and with various distributional shapes: naive normalization coupled to modeling with a single distributional assumption would not suffice to yield robust embedding models, and would leave them vulnerable to extreme values and distributional shapes not corresponding to the assumed ones.

Original: An equally pressing problem is how to flexibly model relationships (dependencies) between the several observed variables with their differing distributions: naive modeling strategies ignoring the variable dependencies would again yield poor embedding models.
Revised: An equally pressing problem is how to flexibly model relationships (dependencies) between the several observed variables with their differing distributions: naive modeling strategies ignoring the variable dependencies would again yield poor embedding models.

Original: We solve the mentioned challenges by introducing a novel Gaussian copula based latent representation learning model.
Revised: We solve the mentioned challenges by introducing a novel Gaussian copula based latent representation learning model.

Original: The model learns vectorial embedding representations for items leveraging the centercontext item interactions, but unlike previous embedding models the proposed model is able to learn embeddings in a setting with multivariate data having heterogeneous data types and distributions.
Revised: The model learns vectorial embedding representations for items leveraging the centercontext item interactions, but unlike previous embedding models the proposed model is able to learn embeddings in a setting with multivariate data having heterogeneous data types and distributions.

Original: For computational efficiency, we introduce a set of variational auto-encoder based inference algorithms.
Revised: For computational efficiency, we introduce a set of variational auto-encoder based inference algorithms.

Original: In experiments on five different scenarios, the proposed model is shown to be effective, outperforming competitive methods in task-based evaluations and yielding insights in a social media analysis task.
Revised: In experiments on five different scenarios, the proposed model is shown to be effective, outperforming competitive methods in task-based evaluations and yielding insights in a social media analysis task.

Original: Our contributions are:
Revised: Our contributions are:

Original: • We introduce a general-purpose representation learning framework which incorporates multiple, heterogeneous data.
Revised: • We introduce a general-purpose representation learning framework which incorporates multiple, heterogeneous data.

Original: Our work allows embedding models to include multiple data types and distributional assumptions in well-founded probabilistic joint modeling through the Gaussian copula; other approaches such as exponential family embeddings have generalised to different data types but still treat them individually.
Revised: Our work allows embedding models to include multiple data types and distributional assumptions in well-founded probabilistic joint modeling through the Gaussian copula; other approaches such as exponential family embeddings have generalised to different data types but still treat them individually.

Original: • To our knowledge, ours is the first work which brings the advantages of Gaussian copula to learning representation vectors from heterogeneous data.
Revised: • To our knowledge, ours is the first work which brings the advantages of Gaussian copula to learning representation vectors from heterogeneous data.

Original: The Gaussian copula is intuitive and has proven effective in machine learning research, thus it is an attractive solution which was neglected in representation learning.
Revised: The Gaussian copula is intuitive and has proven effective in machine learning research, thus it is an attractive solution which was neglected in representation learning.

Original: We close this gap, and the result shows it handles heterogeneous data well.
Revised: We close this gap, and the result shows it handles heterogeneous data well.

Original: • We develop an efficient inference procedure based on semiparametric estimation and variational autoencoder.
Revised: • We develop an efficient inference procedure based on semiparametric estimation and variational autoencoder.

Original: Previous works have used MCMC for inference and in many such works, lack of scalability has limited their application to larger amounts of data.
Revised: Previous works have used MCMC for inference and in many such works, lack of scalability has limited their application to larger amounts of data.

Original: • We demonstrate the effectiveness of our model on five different scenarios, each with a real-world data set and corresponding quantitative and qualitative evaluation.
Revised: • We demonstrate the effectiveness of our model on five different scenarios, each with a real-world data set and corresponding quantitative and qualitative evaluation.

Original: This paper is organized as follows.
Revised: This paper is organized as follows.

Original: Section 2 introduces the necessary background notions of embedding models and copula models.
Revised: Section 2 introduces the necessary background notions of embedding models and copula models.

Original: Section 3 introduces the Gaussian copula model and Sectiondevelops the inference algorithms.
Revised: Section 3 introduces the Gaussian copula model and Sectiondevelops the inference algorithms.

Original: Five different scenarios of using the proposed model are provided 5, each evaluated with an experiment on a real-world data set.
Revised: Five different scenarios of using the proposed model are provided 5, each evaluated with an experiment on a real-world data set.

Original: Section 6 draws the conclusions.
Revised: Section 6 draws the conclusions.

Original:  Background 2.1 Embedding models
Revised:  Background 2.1 Embedding models

Original: Learning latent representations based on the interactions between the observed item and its contexts has been an imperative topic in machine learning.
Revised: Learning latent representations based on the interactions between the observed item and its contexts has been an imperative topic in machine learning.

Original: It was first introduced as a language model to model relations between words [14].
Revised: It was first introduced as a language model to model relations between words [14].

Original: The framework has been later generalized to model other different co-appearance patterns such as in community embedding [23].
Revised: The framework has been later generalized to model other different co-appearance patterns such as in community embedding [23].

Original: In brief, let the item
Revised: In brief, let the item

Original: i and its context j be two items (such as two words, or two communities).
Revised: i and its context j be two items (such as two words, or two communities).

Original: The probability of them appearing (D = 1) in the same context (such as in the same sliding window) can be modeled as
Revised: The probability of them appearing (D = 1) in the same context (such as in the same sliding window) can be modeled as

Original: [Equation]
Revised: [Equation]

Original: where ρ i and ρ j are the embedding vectors of the items.
Revised: where ρ i and ρ j are the embedding vectors of the items.

Original: Note that instead of using the same kinds of vectors for both roles, the context items can have their own context vectors α , thus the above probability becomes
Revised: Note that instead of using the same kinds of vectors for both roles, the context items can have their own context vectors α , thus the above probability becomes

Original: [Equation]
Revised: [Equation]

Original: Exponential Family Embeddings [19] is an extension which has further generalized the model to data beyond text.
Revised: Exponential Family Embeddings [19] is an extension which has further generalized the model to data beyond text.

Original: Let x ( i ) n be the value of the item i at the location n , which has its context c n .
Revised: Let x ( i ) n be the value of the item i at the location n , which has its context c n .

Original: In exponential family embedding, the value of x n depends on its context c n and is generated from an exponential family distribution
Revised: In exponential family embedding, the value of x n depends on its context c n and is generated from an exponential family distribution

Original: [Equation]
Revised: [Equation]

Original: where η n ( x c n ) is the natural parameter, and t (cid:0) x n,v (cid:1) denotes the sufficient statistics.
Revised: where η n ( x c n ) is the natural parameter, and t (cid:0) x n,v (cid:1) denotes the sufficient statistics.

Original: The natural parameter is modeled as a function of an inner product of the embedding vector ρ and the context vector α
Revised: The natural parameter is modeled as a function of an inner product of the embedding vector ρ and the context vector α

Original: so that
Revised: so that

Original: [Equation]
Revised: [Equation]

Original: As the exponential family can model different observation distributions, the embedding models are no longer limited to modeling co-appearance (binary) observations.
Revised: As the exponential family can model different observation distributions, the embedding models are no longer limited to modeling co-appearance (binary) observations.

Original: It has been applied to different domains such as grouped data [18] and graph data [1].
Revised: It has been applied to different domains such as grouped data [18] and graph data [1].

Original: Negative sampling or sub-sampling is a common practice when training embedding models.
Revised: Negative sampling or sub-sampling is a common practice when training embedding models.

Original: The notion is to consider only a randomly generated subset of the items that do not occur at the location n .
Revised: The notion is to consider only a randomly generated subset of the items that do not occur at the location n .

Original: That is, if an item i does not occur at a location, the probability of that negative occurrence
Revised: That is, if an item i does not occur at a location, the probability of that negative occurrence

Original: P ( D = 

Revised: P ( D =

Original: | i, j ) =
Revised: | i, j ) =

Original: (cid:18) 1 − e − ρ i ⊤ α j (cid:19) is integrated into the objective function.
Revised: (cid:18) 1 − e − ρ i ⊤ α j (cid:19) is integrated into the objective function.

Original: In exponential family embeddings, if the item i is generated as a negative sample, the corresponding pseudo observed value is encoded as x ( i ) n = 0 .
Revised: In exponential family embeddings, if the item i is generated as a negative sample, the corresponding pseudo observed value is encoded as x ( i ) n = 0 .

Original: 2.2 Gaussian copula
Revised: 2.2 Gaussian copula

Original: A J -dimensional copula C is a probability distribution on [0 , 1] J where each of its univariate marginal distributions is a uniform distribution on [0 , 1] .
Revised: A J -dimensional copula C is a probability distribution on [0 , 1] J where each of its univariate marginal distributions is a uniform distribution on [0 , 1] .

Original: That is, given a set of uniform distributed random variables U 1 , . .
Revised: That is, given a set of uniform distributed random variables U 1 , . .

Original: , U J , a copula is the joint cumulative distribution
Revised: , U J , a copula is the joint cumulative distribution

Original: [Equation]
Revised: [Equation]

Original: The key idea of copula modeling is to use the copula to model the dependencies between several variables having arbitary types and marginal distributions.
Revised: The key idea of copula modeling is to use the copula to model the dependencies between several variables having arbitary types and marginal distributions.

Original: Let x be a random vector of length J , and let j ∈ 1 . . .
Revised: Let x be a random vector of length J , and let j ∈ 1 . . .

Original: J index the elements (random variables) in x .
Revised: J index the elements (random variables) in x .

Original: According to Sklars’ theorem [21], the cumulative distributions (CDFs) of the variables in x can be modeled by a copula
Revised: According to Sklars’ theorem [21], the cumulative distributions (CDFs) of the variables in x can be modeled by a copula

Original: [Equation]
Revised: [Equation]

Original: (6) where F is the joint CDF and F j ( x )
Revised: (6) where F is the joint CDF and F j ( x )

Original: = P ( X j ≤ x ) is the j -th marginal CDF.
Revised: = P ( X j ≤ x ) is the j -th marginal CDF.

Original: Since each marginal
Revised: Since each marginal

Original: CDF value is in [0 , 1] , the right-hand side is a copula regardless of what distributions the individual marginal CDFs have.
Revised: CDF value is in [0 , 1] , the right-hand side is a copula regardless of what distributions the individual marginal CDFs have.

Original: If every F j is continuous, then the C is unique.
Revised: If every F j is continuous, then the C is unique.

Original: In this way, the copula encodes the structure of variable dependencies, while allowing each of the variables x 1 . . .
Revised: In this way, the copula encodes the structure of variable dependencies, while allowing each of the variables x 1 . . .

Original: x J to be of a different type and to have differing kinds of marginal CDFs.
Revised: x J to be of a different type and to have differing kinds of marginal CDFs.

Original: In this paper, we consider a Gaussian copula, which is one of the widely used copula models; we introduce the model to the representation learning task.
Revised: In this paper, we consider a Gaussian copula, which is one of the widely used copula models; we introduce the model to the representation learning task.

Original: A Gaussian copula is defined as
Revised: A Gaussian copula is defined as

Original: [Equation]
Revised: [Equation]

Original: where Φ J is a J -dimensional Gaussian CDF with a correlation matrix C , and Φ − 1 is the inverse function of the standard univariate Gaussian CDF.
Revised: where Φ J is a J -dimensional Gaussian CDF with a correlation matrix C , and Φ − 1 is the inverse function of the standard univariate Gaussian CDF.

Original: With the Gaussian copula, the joint CDF of observed data can be modeled as
Revised: With the Gaussian copula, the joint CDF of observed data can be modeled as

Original: [Equation]
Revised: [Equation]

Original: The Gaussian copula can also be expressed in terms of a latent Gaussian variable representation.
Revised: The Gaussian copula can also be expressed in terms of a latent Gaussian variable representation.

Original: First, a latent vector z is generated from a Gaussian distribution
Revised: First, a latent vector z is generated from a Gaussian distribution

Original: [Equation]
Revised: [Equation]

Original: wih covariance matrix Ω which corresponds to the correlation matrix C in equation (7).
Revised: wih covariance matrix Ω which corresponds to the correlation matrix C in equation (7).

Original: Then for each j , the observed data value x j is obtained from the inverse of the univariate marginal F − 1 j according to the generated latent variable z
Revised: Then for each j , the observed data value x j is obtained from the inverse of the univariate marginal F − 1 j according to the generated latent variable z

Original: j so that
Revised: j so that

Original: [Equation]
Revised: [Equation]

Original: where ω jj is the j -th diagonal element of Ω .
Revised: where ω jj is the j -th diagonal element of Ω .

Original: In principle, equations (9) and (10) could be used to derive an equation for the likelihood of the data values x j at each observation, in order to use such likelihoods for parameter fitting.
Revised: In principle, equations (9) and (10) could be used to derive an equation for the likelihood of the data values x j at each observation, in order to use such likelihoods for parameter fitting.

Original: However, instead the practical approach in Gaussian copula research is to derive an extended likelihood , and we will do that for the vectorial embedding task.
Revised: However, instead the practical approach in Gaussian copula research is to derive an extended likelihood , and we will do that for the vectorial embedding task.

Original:  Gaussian Copula Embeddings
Revised:  Gaussian Copula Embeddings

Original: We now introduce Gaussian Copula Embeddings (GCE), which perform representation learning for data with heterogeneous-type observations of items in contexts.
Revised: We now introduce Gaussian Copula Embeddings (GCE), which perform representation learning for data with heterogeneous-type observations of items in contexts.

Original: Let { x 1 . . .
Revised: Let { x 1 . . .

Original: x N } be the observations.
Revised: x N } be the observations.

Original: Each observation x n is of a particular item, occurring in a context; moreover, each occurrence of the item is observed with multiple feature values (variables).
Revised: Each observation x n is of a particular item, occurring in a context; moreover, each occurrence of the item is observed with multiple feature values (variables).

Original: Let x n = x ( i ) n denote that the item i occurs at location n and is observed carrying J variables x ( i ) n =
Revised: Let x n = x ( i ) n denote that the item i occurs at location n and is observed carrying J variables x ( i ) n =

Original: [ x ( i ) n, 1 , . . .
Revised: [ x ( i ) n, 1 , . . .

Original: , x ( i ) n,J ] ⊤ , where the location n comes with a context c n which contains a collection of context item indices.
Revised: , x ( i ) n,J ] ⊤ , where the location n comes with a context c n which contains a collection of context item indices.

Original: The principle of the GCE model is that the multiple heterogeneous observations will be generated based on the relationship of the central item i to its context, characterized by several embedding vectors, and the dependencies between the observations will be characterized by a Gaussian copula.
Revised: The principle of the GCE model is that the multiple heterogeneous observations will be generated based on the relationship of the central item i to its context, characterized by several embedding vectors, and the dependencies between the observations will be characterized by a Gaussian copula.

Original: For each central item i there is an underlying embedding vector ρ i ∈ R K × 1 .
Revised: For each central item i there is an underlying embedding vector ρ i ∈ R K × 1 .

Original: In addition to occurring as a central item, each item may also occur as part of a context.
Revised: In addition to occurring as a central item, each item may also occur as part of a context.

Original: The context c n of the location n will contain several items i ′ .
Revised: The context c n of the location n will contain several items i ′ .

Original: The roles of the items in the context will be characterized by context vectors: unlike a traditional embedding model that generates only one type of observation, here each context item i ′ has a set of variable specific context vectors { α i ′ ,j | j ∈ 1 , . . .
Revised: The roles of the items in the context will be characterized by context vectors: unlike a traditional embedding model that generates only one type of observation, here each context item i ′ has a set of variable specific context vectors { α i ′ ,j | j ∈ 1 , . . .

Original: J } .
Revised: J } .

Original: We develop the GCE model based on the latent representation equations (9) - (10).
Revised: We develop the GCE model based on the latent representation equations (9) - (10).

Original: For each item i the underlying embedding vector ρ i ∈ R K × 1 is generated from a multivariate normal distribution
Revised: For each item i the underlying embedding vector ρ i ∈ R K × 1 is generated from a multivariate normal distribution

Original: ρ i ∼ N (0 , I ) .
Revised: ρ i ∼ N (0 , I ) .

Original: Following the latent variable representation of Gaussian copula, all J observations of an item at a location will be generated based from a latent vector.
Revised: Following the latent variable representation of Gaussian copula, all J observations of an item at a location will be generated based from a latent vector.

Original: The latent variable vector z n is generated as
Revised: The latent variable vector z n is generated as

Original: [Equation]
Revised: [Equation]

Original: (12) where the matrix R n ∈ R K
Revised: (12) where the matrix R n ∈ R K

Original: × J is constructed based on the embedding vectors of the items in the context c n for all observation variables.
Revised: × J is constructed based on the embedding vectors of the items in the context c n for all observation variables.

Original: For each observation variable j , the corresponding column r n,j of the matrix R n is constructed as
Revised: For each observation variable j , the corresponding column r n,j of the matrix R n is constructed as

Original: [Equation]
Revised: [Equation]

Original: where i ′ are the items in the context of the location n , c n .
Revised: where i ′ are the items in the context of the location n , c n .

Original: Here, for simplicity, we set the prior of α to be a multivariate normal distribution
Revised: Here, for simplicity, we set the prior of α to be a multivariate normal distribution

Original: [Equation]
Revised: [Equation]

Original: with a diagonal covariance matrix where the constant λ α is a precision parameter which controls the constraints on α .
Revised: with a diagonal covariance matrix where the constant λ α is a precision parameter which controls the constraints on α .

Original: Using the exchangeability in equation (12), the above generating process can be also written as
Revised: Using the exchangeability in equation (12), the above generating process can be also written as

Original: [Equation]
Revised: [Equation]

Original: The observations are then obtained from the latent variables according to the Gaussian copula equations.
Revised: The observations are then obtained from the latent variables according to the Gaussian copula equations.

Original: The j th observed value x ( i ) n,j is obtained as
Revised: The j th observed value x ( i ) n,j is obtained as

Original: [Equation]
Revised: [Equation]

Original: where z ( i ) n,j is the j th element of the latent vector z ( i ) n and r n,j,k is the k th dimension of the context representation column r n,j , and F − 1 j is the inverse CDF of the marginal distribution of variable j .
Revised: where z ( i ) n,j is the j th element of the latent vector z ( i ) n and r n,j,k is the k th dimension of the context representation column r n,j , and F − 1 j is the inverse CDF of the marginal distribution of variable j .

Original: Inference of the embedding parameters based on the observations will be done with an extended likelihood approach introduced in the next section.
Revised: Inference of the embedding parameters based on the observations will be done with an extended likelihood approach introduced in the next section.

Original: The major difference between our approach and other embedding models is that the GCE can take heterogenous, multivariate observed data into account.
Revised: The major difference between our approach and other embedding models is that the GCE can take heterogenous, multivariate observed data into account.

Original: Another difference is that GCE further specializes the roles of embedding vectors ρ and context vectors α : in traditional exponential family embeddings their roles can be seen as ambiguous, having the same form and a very similar place in the generative equations.
Revised: Another difference is that GCE further specializes the roles of embedding vectors ρ and context vectors α : in traditional exponential family embeddings their roles can be seen as ambiguous, having the same form and a very similar place in the generative equations.

Original: In contrast, here the roles are made distinct: the ρ are used to model a general representation of each item as a central item, and the variable-specific context vectors α j govern the role of each item as a context item for the multiple variables, telling how the different variables interact with the central item as well as controlling the dependencies between the variables.
Revised: In contrast, here the roles are made distinct: the ρ are used to model a general representation of each item as a central item, and the variable-specific context vectors α j govern the role of each item as a context item for the multiple variables, telling how the different variables interact with the central item as well as controlling the dependencies between the variables.

Original:  Inference
Revised:  Inference

Original: The task of the inference is to fit the embedding parameters ρ and α j of all items to the observations.
Revised: The task of the inference is to fit the embedding parameters ρ and α j of all items to the observations.

Original: We will use a variational inference approach; a key aspect of it is to replace direct evaluation of the likelihood by an extended likelihood approach described next.
Revised: We will use a variational inference approach; a key aspect of it is to replace direct evaluation of the likelihood by an extended likelihood approach described next.

Original: 4.1 Extended Likelihood
Revised: 4.1 Extended Likelihood

Original: As pointed out by [7], the naive inverse of equation (16) during the inference is only recommended when the observed data is continuous and follows an easy-to-invert parametric distribution.
Revised: As pointed out by [7], the naive inverse of equation (16) during the inference is only recommended when the observed data is continuous and follows an easy-to-invert parametric distribution.

Original: Moreover, directly inverting the CDF for discrete variables can lead to pushing the latent variables to extremes and affect the validity of the inference.
Revised: Moreover, directly inverting the CDF for discrete variables can lead to pushing the latent variables to extremes and affect the validity of the inference.

Original: Therefore, an extended rank likelihood has been proposed for the inference of Gaussian copula.
Revised: Therefore, an extended rank likelihood has been proposed for the inference of Gaussian copula.

Original: Consider all observations x n,j of variable j at all locations n = 1 , . .
Revised: Consider all observations x n,j of variable j at all locations n = 1 , . .

Original: . , N , and the corresponding latent variables
Revised: . , N , and the corresponding latent variables

Original: z n,j where for brevity we drop the item indices i ; each observation may arise from a different item.
Revised: z n,j where for brevity we drop the item indices i ; each observation may arise from a different item.

Original: Denote the latent variables together by a vector z j and observed variables together by vector x j .
Revised: Denote the latent variables together by a vector z j and observed variables together by vector x j .

Original: Since each z n,j is related to the corresponding observed x n,j by two monotonic functions (a gaussian CDF and an inverse CDF), the rank order of the z n,j is the same as that of the x n,j .
Revised: Since each z n,j is related to the corresponding observed x n,j by two monotonic functions (a gaussian CDF and an inverse CDF), the rank order of the z n,j is the same as that of the x n,j .

Original: The vector
Revised: The vector

Original: z j is one of a set D ( x j ) of vectors having that rank order:
Revised: z j is one of a set D ( x j ) of vectors having that rank order:

Original: [Equation]
Revised: [Equation]

Original: The D ( x j ) is the set of possible z j = ( z 1 ,j , . .
Revised: The D ( x j ) is the set of possible z j = ( z 1 ,j , . .

Original: . , z n,j ) which preserve the ordering of the observed data [20].
Revised: . , z n,j ) which preserve the ordering of the observed data [20].

Original: Let D =
Revised: Let D =

Original: { Z ∈ R J × N : z j ∈ D ( x j ) ∀ 1 ≤ j ≤ J } be the set of possible latent variable combinations, such that the rank order is satisfied for each observed variable j .
Revised: { Z ∈ R J × N : z j ∈ D ( x j ) ∀ 1 ≤ j ≤ J } be the set of possible latent variable combinations, such that the rank order is satisfied for each observed variable j .

Original: Note that ties may happen in the rank orders, for example when discrete variables yield the same value at multiple observations.
Revised: Note that ties may happen in the rank orders, for example when discrete variables yield the same value at multiple observations.

Original: In the Gaussian copula model the observed variables are directly obtained as unique transformations of the latent variables.
Revised: In the Gaussian copula model the observed variables are directly obtained as unique transformations of the latent variables.

Original: Because the possible latent variables must satisfy the rank orders of the observations, the rank perservation can be inserted into the full likelihood which can then be factorized as
Revised: Because the possible latent variables must satisfy the rank orders of the observations, the rank perservation can be inserted into the full likelihood which can then be factorized as

Original: [Equation]
Revised: [Equation]

Original: The P ( Z ∈ D | C ) is then taken as the alternative likelihood.
Revised: The P ( Z ∈ D | C ) is then taken as the alternative likelihood.

Original: It has been proved [8] that it shares the same information bound as using estimator of the full data.
Revised: It has been proved [8] that it shares the same information bound as using estimator of the full data.

Original: Ranks also bring an advantage of robustness as they are unaffected by precise value and thus are less prone to outliers.
Revised: Ranks also bring an advantage of robustness as they are unaffected by precise value and thus are less prone to outliers.

Original: Using an extended likelihood based on the rank of observed data is the current state of the art practice for
Revised: Using an extended likelihood based on the rank of observed data is the current state of the art practice for

Original: Gaussian copula inference [15, 3].
Revised: Gaussian copula inference [15, 3].

Original: 4.2 Amortized variational autoencoder
Revised: 4.2 Amortized variational autoencoder

Original: Most of the previous works taking the extended likelihood use Gibbs sampling for inference [15, 3].
Revised: Most of the previous works taking the extended likelihood use Gibbs sampling for inference [15, 3].

Original: Despite some recent improvement such as [9], the sampling process inevitably must update the latent variables for every location n , which makes the computation inefficient when the volume of data grows large.
Revised: Despite some recent improvement such as [9], the sampling process inevitably must update the latent variables for every location n , which makes the computation inefficient when the volume of data grows large.

Original: To avoid this inefficiency, we develop a stochastic variational inference procedure exploiting the idea of amortized inference [4] and a variational autoencoder [11].
Revised: To avoid this inefficiency, we develop a stochastic variational inference procedure exploiting the idea of amortized inference [4] and a variational autoencoder [11].

Original: Since the essence of the estimator using P ( Z ∈ D | C ) is to keep the ranking of z j corresponding to the ranking of x j , we then employ the Plackett-Luce model with ties [24] as an alternative likelihood.
Revised: Since the essence of the estimator using P ( Z ∈ D | C ) is to keep the ranking of z j corresponding to the ranking of x j , we then employ the Plackett-Luce model with ties [24] as an alternative likelihood.

Original: Let r ( x n,j ) denote the rank of x n,j and r ( x j ) denote the vector of rankings corresponding to x j .
Revised: Let r ( x n,j ) denote the rank of x n,j and r ( x j ) denote the vector of rankings corresponding to x j .

Original: We then have
Revised: We then have

Original: [Equation]
Revised: [Equation]

Original: where q in the product goes over the rank positions, A q = { n :
Revised: where q in the product goes over the rank positions, A q = { n :

Original: [Equation]
Revised: [Equation]

Original: } denotes the items ranked at position q (there may be more than one due to ties), and C q = { n : r ( x n,j ) ≥ q } are items ranked at q or higher.
Revised: } denotes the items ranked at position q (there may be more than one due to ties), and C q = { n : r ( x n,j ) ≥ q } are items ranked at q or higher.

Original: Note that if there are no ties, equation (19) reduces to the standard
Revised: Note that if there are no ties, equation (19) reduces to the standard

Original: Plackett-Luce distribution.
Revised: Plackett-Luce distribution.

Original: The ties are considered in order to handle discrete data.
Revised: The ties are considered in order to handle discrete data.

Original: The likelihood for all variables j is then simply (cid:81)
Revised: The likelihood for all variables j is then simply (cid:81)

Original: j log p ( r ( x j ) | z j ) .
Revised: j log p ( r ( x j ) | z j ) .

Original: Moreover, the likelihood can be optimized using a stochastic procedure with a random subset of each x j .
Revised: Moreover, the likelihood can be optimized using a stochastic procedure with a random subset of each x j .

Original: Let us consider again the J latent variables per location, denoted z ( i ) n .
Revised: Let us consider again the J latent variables per location, denoted z ( i ) n .

Original: To avoid exhaustively updating every z ( i ) n , we follow the framework proposed by [11].
Revised: To avoid exhaustively updating every z ( i ) n , we follow the framework proposed by [11].

Original: According to the “reparameterization trick”, equations (15) can be rewritten as
Revised: According to the “reparameterization trick”, equations (15) can be rewritten as

Original: [Equation]
Revised: [Equation]

Original: Therefore, during the stochastic inference, the latent variables ˜ z can be simulated based on the other parameters and the random noise.
Revised: Therefore, during the stochastic inference, the latent variables ˜ z can be simulated based on the other parameters and the random noise.

Original: This will save computation and memory because the ˜ z only need to be simulated at the subset of positions samples in the ongoing mini-batch of optimization.
Revised: This will save computation and memory because the ˜ z only need to be simulated at the subset of positions samples in the ongoing mini-batch of optimization.

Original: The noise is first sampled from a standard normal distribution and a pseudo variable ˜
Revised: The noise is first sampled from a standard normal distribution and a pseudo variable ˜

Original: z is then generated as
Revised: z is then generated as

Original: [Equation]
Revised: [Equation]

Original: ) where ˆ µ ( i ) n = ˆ ρ ⊤ i 1 | c n | (cid:80) i ′
Revised: 21) where ˆ µ ( i ) n = ˆ ρ ⊤ i 1 | c n | (cid:80) i ′

Original: ∈ c n ˆ α
Revised: ∈ c n ˆ α

Original: i ′ ,j , and ˆ ρ and ˆ α are point estimates of the embedding vectors which we are optimizing.
Revised: i ′ ,j , and ˆ ρ and ˆ α are point estimates of the embedding vectors which we are optimizing.

Original: The log-likelihood function becomes ˜ L ( ρ , α ; x )
Revised: The log-likelihood function becomes ˜ L ( ρ , α ; x )

Original: = (cid:80) j log p ( r ( x j ) | ˜
Revised: = (cid:80) j log p ( r ( x j ) | ˜

Original: z j ) and the objective function is computed as
Revised: z j ) and the objective function is computed as

Original: [Equation]
Revised: [Equation]

Original: In each iteration, the gradients  toupdate ˆ ρ and ˆ αin respect of the log-likelihood can be simply obtained with ˜ z by chain rule via ▽
Revised: In each iteration, the gradients of the log-likelihood with respect to ˆ ρ and ˆ α can be simply obtained with ˜ z by chain rule via ▽

Original: [Equation]
Revised: [Equation]

Revised: =

Revised: [Equation]

Original: z j ∂ ˜ z
Revised: j ∂ ˜ z

Original: j ∂ ρ , and ▽ α j ˜ L = ∂ log p ( x j | ˜
Revised: j ∂ ρ , and ▽ α j ˜ L = ∂ log p ( x j | ˜

Original: z j ) ∂ ˜
Revised: z j ) ∂ ˜

Original: z j ∂ ˜ z
Revised: z j ∂ ˜ z

Original: j ∂ α j .
Revised: j ∂ α j .

Original: The p ( ρ i ) and p ( α i ′ ,j ) are set to N (0 , I ) and N (0 , λ − 1 α I ) according to equations (11) and (14), with corresponding gradients  (cid:80) i ∂ log p ( ρ ) ∂ ρ i and (cid:80) i ′ (cid:80) j ∂ log p ( α ) ∂ α i ′ ,j respectively.
Revised: The p ( ρ i ) and p ( α i ′ ,j ) are set to N (0 , I ) and N (0 , λ − 1 α I ) according to equations (11) and (14). The gradients of the log-priors are (cid:80) i ∂ log p ( ρ ) ∂ ρ i and (cid:80) i ′ (cid:80) j ∂ log p ( α ) ∂ α i ′ ,j respectively.

Revised: The gradient to update ˆ ρ or ˆ α with respect to F is then the sum of the corresponding log-likelihood and log-prior gradients.

Original: The complete stochastic inference procedure is given in Algorithm 1.
Revised: The complete stochastic inference procedure is given in Algorithm 1.

Original: We optimize the embedding vectors iteratively over epochs.
Revised: We optimize the embedding vectors iteratively over epochs.

Original: In each epoch data is partitioned randomly into mini-batches, and negative samples are generated for each batch in addition to its positive samples: a negative sample has the same location n and context as a positive sample but a randomly chosen different item i , and its observed variable values are all set to zero since the item did not occur at that location.
Revised: In each epoch data is partitioned randomly into mini-batches, and negative samples are generated for each batch in addition to its positive samples: a negative sample has the same location n and context as a positive sample but a randomly chosen different item i , and its observed variable values are all set to zero since the item did not occur at that location.

Original: The items for the negative samples are chosen from a distribution proportional to a power of the overall item distribution, as is done in word embedding [14].
Revised: The items for the negative samples are chosen from a distribution proportional to a power of the overall item distribution, as is done in word embedding [14].

Original: In experiments we use M = 1000 mini-batches and 5 negative samples for each positive sample.
Revised: In experiments we use M = 1000 mini-batches and 5 negative samples for each positive sample.

Original: Due to the stochastic partitions, for each epoch the latent variables only need to be simulated at the positions in the mini-batch.
Revised: Due to the stochastic partitions, for each epoch the latent variables only need to be simulated at the positions in the mini-batch.

Original: The optimization then updates the embedding vectors in each epoch by gradient steps with step sizes chosen by the Adam optimizer.
Revised: The optimization then updates the embedding vectors in each epoch by gradient steps with step sizes chosen by the Adam optimizer.

Original: Algorithm 1: Inference Algorithm input : Observations { x 1 . . .

Original: x N } , Context { c 1 . . .

Original: c N } , initial learning rate ξ output : Point estimates of embedding vectors ˆ ρ and context vectors ˆ α foreach epoch do

Original: Divide input data into M random partitions.

Original: Generate negative samples.

Original: for m ← 1 to M do

Original: Use the m -th batch of the data

Original: Simulate ˜ z n with (15) for every n in the mini-batch

Original: Compute gradients ▽ ρ F = (cid:80) j ∂ log p ( x j | ˜ z j ) ∂ ˜

Original: z j ∂ ˜ z

Original: j ∂ ρ + (cid:80) i ∂ log

Original: [Equation]

Original: Update ρ and α with ρ = ρ − ξ ∗ ▽ ρ F , and α = α − ξ ∗ ▽ α F ξ is set with Adam[10] end end

Original: [Table]

Original:  Empirical Case Studies
Revised:  Empirical Case Studies

Original: In this section we describe 5 different scenarios of using GCE to model the observed data.
Revised: In this section we describe 5 different scenarios of using GCE to model the observed data.

Original: The precision parameter λ
Revised: The precision parameter λ

Original: α is set to 0 for a very wide prior for α .
Revised: α is set to 0 corresponding to a very wide prior for α .

Original: We have also tried another setting with a constrained prior, they yield similar results (see supplementary materials).
Revised: We have also tried another setting with a constrained prior, they yield similar results (see supplementary materials).

Original: 5.1 Product rating data
Revised: 5.1 Product rating data

Original: Data.
Revised: Data.

Original: The Anime rating data 1 is a set of user ratings on anime movies and series collected from myanimelist.net.
Revised: The Anime rating data 1 is a set of user ratings on anime movies and series collected from myanimelist.net.

Original: It contains 17562 different anime rated by 325770 different users.
Revised: It contains 17562 different anime rated by 325770 different users.

Original: Unlike typical product rating data sets, the data set provides how many episodes (integer) the user had watched when rating the anime (discrete), thus there are multivariate heterogeneous observed variables.
Revised: Unlike typical product rating data sets, the data set provides how many episodes (integer) the user had watched when rating the anime (discrete), thus there are multivariate heterogeneous observed variables.

Original: Modeling.
Revised: Modeling.

Original: We evaluate GCE by comparing with two other models on their capability to predict held-out ratings.
Revised: We evaluate GCE by comparing with two other models on their capability to predict held-out ratings.

Original: Compared models are exponential family embeddings (Poisson distribution, p-emb) and Poisson matrix factorization model ([2], Pois-MF).
Revised: Compared models are exponential family embeddings (Poisson distribution, p-emb) and Poisson matrix factorization model ([2], Pois-MF).

Original: The p-emb is selecteddue to our model can been seen asan extension of exponential family embeddings; the Pois-MF is a model for the user-rating scenario and it was also a compared method to p-emb in [19].
Revised: The p-emb is selected as a comparison method input : Observations { x 1 . . . x N } , Context { c 1 . . . c N } , initial learning rate ξ output : Point estimates of embedding vectors ˆ ρ and context vectors ˆ α foreach epoch do

Revised: Divide input data into M random partitions.

Revised: Generate negative samples.

Revised: for m ← 1 to M do

Original: For all methods, when training the model, we hold out 10% of the data as the testing data set
Revised: Use the m -th batch of the data Simulate ˜ z n with (15) for every n in the mini-batch Compute gradients [Equation]

Revised: Update ρ and α

Revised: F

Revised: ] end end

Original: [Table]
Revised: [Table]

Original:  and the trained models are used to predict the ratings in the test data.
Revised: since our model can been seen as an extension of exponential family embeddings; the Pois-MF is a model for the user-rating scenario and it was also a compared method to p-emb in [19]. For all methods, when training the model, we hold out 10% of the data as the testing data set, and the trained models are used to predict the ratings in the test data.

Original: For each anime rated by a specific user, other anime rated by the same user are its context.
Revised: For each anime rated by a specific user, other anime rated by the same user are its context.

Original: We train GCE with two variables, anime rating and number watched episodes, whereas p-emb and Pois-MF which can only model one variable are trained to model the anime ratings.
Revised: We train GCE with two variables, anime rating and number watched episodes, whereas p-emb and Pois-MF which can only model one variable are trained to model the anime ratings.

Original: To compute the predictive ratings we use (16) with z ( i ) n,j set to the means µ ( i ) n,j computed from the optimized embedding vectors and with F − 1 j computed as the inverse of the empirical CDF in the training data.
Revised: To compute the predictive ratings we use (16) with z ( i ) n,j set to the means µ ( i ) n,j computed from the optimized embedding vectors and with F − 1 j computed as the inverse of the empirical CDF in the training data.

Original: The held-out mean absolute error (MAE) is used as the performance metric.
Revised: The held-out mean absolute error (MAE) is used as the performance metric.

Original: The results are shown in the Table 1 (Left): our model strongly outperforms p-emb and Pois-MF.
Revised: The results are shown in the Table 1 (Left): our model strongly outperforms p-emb and Pois-MF.

Original: 5.2 Player modeling in online games
Revised: 5.2 Player modeling in online games

Original: Data.
Revised: Data.

Original: The HLTV match record data is collected from HLTV.org and records professional match histories of a multiplayer first-person shooter game Counter-Strike: Global Offensive [22].
Revised: The HLTV match record data is collected from HLTV.org and records professional match histories of a multiplayer first-person shooter game Counter-Strike: Global Offensive [22].

Original: We used a web crawler to gather the histories of 34900 matches of 4751 professional players from the website.
Revised: We used a web crawler to gather the histories of 34900 matches of 4751 professional players from the website.

Original: For each match, we collect the match ID, player ID, and records of each player in the match including number of kills and deaths, again yielding multivariate heterogeneous observations.
Revised: For each match, we collect the match ID, player ID, and records of each player in the match including number of kills and deaths, again yielding multivariate heterogeneous observations.

Original: Modeling.
Revised: Modeling.

Original: We train GCE to learn representation vectors for each player.
Revised: We train GCE to learn representation vectors for each player.

Original: The context for the player in each match is the set of other players in the same match.
Revised: The context for the player in each match is the set of other players in the same match.

Original: Observed data are the numbers of kills and deaths.
Revised: Observed data are the numbers of kills and deaths.

Original: We again compare our model to two exponential family embeddings (normal and Poisson) for they are the methodologically closest approches.
Revised: We again compare our model to two exponential family embeddings (normal and Poisson) because they are the methodologically closest approaches.

Original: We train GCA incorporating the two variables at the same time whereas the exponential family embeddings train the model for each variable separately.
Revised: We train GCA incorporating the two variables at the same time whereas the exponential family embeddings train the model for each variable separately.

Original: The predictions for the variables are done as in Section 5.1.
Revised: The predictions for the variables are done as in Section 5.1.

Original: We measure MAE for both variables separately.
Revised: We measure MAE for both variables separately.

Original: The results are shown in Table 1 (Right): we strongly outperform the comparison methods.
Revised: The results are shown in Table 1 (Right): we strongly outperform the comparison methods.

Original: [Table]
Revised: [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: 5.3 Internet traffic classification
Revised: 5.3 Internet traffic classification

Original: Data.
Revised: Data.

Original: The CIC Dark-net traffic data set [6] contains 141532 records of darknet traffic.
Revised: The CIC Dark-net traffic data set [6] contains 141532 records of darknet traffic.

Original: Each record is categorized into a traffic category (Audio-Stream, Browsing, Chat, Email, P2P, Transfer, VideoStream, and VOIP, 8 categories in total) and contains the source IP, destination IP, and communication observations such as forward and backward bytes, flows, duration, subflow and so on.
Revised: Each record is categorized into a traffic category (Audio-Stream, Browsing, Chat, Email, P2P, Transfer, VideoStream, and VOIP, 8 categories in total) and contains the source IP, destination IP, and communication observations such as forward and backward bytes, flows, duration, subflow and so on.

Original: Model.
Revised: Model.

Original: We use GCE to learn the latent representation for each source IP.
Revised: We use GCE to learn the latent representation for each source IP.

Original: In each traffic record, the destination IP is the context of the source IP.
Revised: In each traffic record, the destination IP is the context of the source IP.

Original: The observed data used to train the GCE model are IP co-appearance (Boolean), TCP Flag counts (SYN, RST, PSH, and ACK; integer), and three subflow related measurements (continuous value).
Revised: The observed data used to train the GCE model are IP co-appearance (Boolean), TCP Flag counts (SYN, RST, PSH, and ACK; integer), and three subflow related measurements (continuous value).

Original: We first train the embedding model with different vector dimensions.
Revised: We first train the embedding model with different vector dimensions.

Original: After training GCE, we incorporate the learned representation vector of each source IP as additional features to the original input variables and train a random forest classifier to predict the traffic category.
Revised: After training GCE, we incorporate the learned representation vector of each source IP as additional features to the original input variables and train a random forest classifier to predict the traffic category.

Original: We compare our model to DeepImage, which is a convolutional neural network based, end-to-end solution proposed in [6].
Revised: We compare our model to DeepImage, which is a convolutional neural network based, end-to-end solution proposed in [6].

Original: The results in Table 2 show that the learned additional features not only improve performance of the random forest classifier, but also outperform
Revised: The results in Table 2 show that the learned additional features not only improve performance of the random forest classifier, but also outperform

Original: DeepImage, the state-of-the-art deep learning based classifier.
Revised: DeepImage, the state-of-the-art deep learning based classifier.

Original: Figure 1 further demonstrates that the learned features play important roles in the classification task.
Revised: Figure 1 further demonstrates that the learned features play important roles in the classification task.

Original: 5.4 Graph embedding with node meta-data
Revised: 5.4 Graph embedding with node meta-data

Original: Data.
Revised: Data.

Original: Spanish Twitch gamers is a subgraph of the Twitch gamers graph data [17]; each node is a
Revised: Spanish Twitch gamers is a subgraph of the Twitch gamers graph data [17]; each node is a

Original: Twitch gamer; an edge denotes mutual friendship.
Revised: Twitch gamer; an edge denotes mutual friendship.

Original: The data has 5538 nodes and 85893 edges.
Revised: The data has 5538 nodes and 85893 edges.

Original: Model.
Revised: Model.

Original: We train the embedding vectors incorporating the node-level observations including number of views and life duration.
Revised: We train the embedding vectors incorporating the node-level observations including number of views and life duration.

Original: Following the customary graph embedding procedure, we first generate random walks on the graph to simulate a node sequence as input data: 80 walks per node with lengthsteps.
Revised: Following the customary graph embedding procedure, we first generate random walks on the graph to simulate a node sequence as input data: 80 walks per node with lengthsteps.

Original: Conventionally, limited by the capability of embedding models, most graph embedding models only take appearance of nodes into account.
Revised: Conventionally, limited by the capability of embedding models, most graph embedding models only take appearance of nodes into account.

Original: With GCE, we incorporate not only the appearance of nodes but also the views and lifetime of the nodes into the model.
Revised: With GCE, we incorporate not only the appearance of nodes but also the views and lifetime of the nodes into the model.

Original: To evaluate our model, we take on link prediction, a classical task for graph embedding models.
Revised: To evaluate our model, we take on link prediction, a classical task for graph embedding models.

Original: We hold out 50% of edges randomly into a test set while keeping the remaining training graph connected.
Revised: We hold out 50% of edges randomly into a test set while keeping the remaining training graph connected.

Original: In both training and test sets, randomly sampled negative edges are added in equal amount to the
Revised: In both training and test sets, randomly sampled negative edges are added in equal amount to the

Original: [Table] [Table]
Revised: [Table] [Table]

Original: positive edges.
Revised: positive edges.

Original: A logistic regression classifier is trained based on the reduced training graph and the training negative edges, using Hadamard product of embeddings of the edge endpoint nodes as input features; the classifier is used to classify the held-out test-set edges.
Revised: A logistic regression classifier is trained based on the reduced training graph and the training negative edges, using Hadamard product of embeddings of the edge endpoint nodes as input features; the classifier is used to classify the held-out test-set edges.

Original: We compare our model to state-of-the-art, random walk based solutions including Deepwalk [16], Node2vec [5] and
Revised: We compare our model to state-of-the-art, random walk based solutions including Deepwalk [16], Node2vec [5] and

Original: Exponential Family Graph Embeddings (EFGE.
Revised: Exponential Family Graph Embeddings (EFGE; [1]).

Original: The results in the Table 3 show our model outperforms the other competitive models.
Revised: The results in the Table 3 show our model outperforms the other competitive models.

Original: 5.5 Social media community interactions
Revised: 5.5 Social media community interactions

Original: Data.
Revised: Data.

Original: The Reddit Hyperlink Network [12] is a data set of 858488 hyperlinks between 55863 subreddits.
Revised: The Reddit Hyperlink Network [12] is a data set of 858488 hyperlinks between 55863 subreddits.

Original: For each hyperlink, the data set records the source and destination subreddit, and the description of the hypertext including, e.g., number of words, sentiments, fractions of 5 different character types (i.e., alphabetical, digits, uppercase characters, special characters, white space) and so on.
Revised: For each hyperlink, the data set records the source and destination subreddit, and the description of the hypertext including, e.g., number of words, sentiments, fractions of 5 different character types (i.e., alphabetical, digits, uppercase characters, special characters, white space) and so on.

Original: Model.
Revised: Model.

Original: We train the model based on the pairs of source and destination subreddits.
Revised: We train the model based on the pairs of source and destination subreddits.

Original: The source subreddit in each hyperlink is the context for the destination subreddit.
Revised: The source subreddit in each hyperlink is the context for the destination subreddit.

Original: The 5 fractions of different character types, the number of words, and the sentiment are taken as the observed variables.
Revised: The 5 fractions of different character types, the number of words, and the sentiment are taken as the observed variables.

Original: We train a GCE model with K = 100 .
Revised: We train a GCE model with K = 100 .

Original: We demonstrate the closest subreddits based on embedding vectors, and three context vectors.
Revised: We demonstrate the closest subreddits based on embedding vectors, and three context vectors.

Original: Each reflects a different aspect: take the r/environment for example, when it comes to fraction of alphabetical characters, closest subreddits are related to resources such as r/climate, r/green, and r/oil, and r/water.
Revised: Each reflects a different aspect: take the r/environment for example, when it comes to fraction of alphabetical characters, closest subreddits are related to resources such as r/climate, r/green, and r/oil, and r/water.

Original: However, when it comes to sentiment, the closest subreddits for r/environment become r/invasivespecies, r/lockcarbon, r/metageopolitics, and r/earthdisaster.
Revised: However, when it comes to sentiment, the closest subreddits for r/environment become r/invasivespecies, r/lockcarbon, r/metageopolitics, and r/earthdisaster.

Original:  Discussions and Conclusions
Revised:  Discussions and Conclusions

Original: We introduced Gaussian copula embeddings (GCE), a representation learning model that can incorporate observed data of different data types.
Revised: We introduced Gaussian copula embeddings (GCE), a representation learning model that can incorporate observed data of different data types.

Original: A stochastic variational inference algorithm based on semi-parametric estimation for efficient computation is introduced.
Revised: A stochastic variational inference algorithm based on semi-parametric estimation for efficient computation is introduced.

Original: The empirical case studies demonstrate that our model is effective in many domains outperforming competitive comparison methods, and can provide analytical insights.
Revised: The empirical case studies demonstrate that our model is effective in many domains outperforming competitive comparison methods, and can provide analytical insights.

Original: Moreover, our model can extend the representation learning task to more complex settings and thus bring more opportunities to the research community.
Revised: Moreover, our model can extend the representation learning task to more complex settings and thus bring more opportunities to the research community.

Original: In this paper we used a streightforward parametric construction of context and its combination with embedding vectors; however, the GCE framework can be flexibly adapted to other parameterizations, such as integrating it as a layer within deep learning architectures, and integrating context selection mechanisms such as [13].
Revised: In this paper we used a streightforward parametric construction of context and its combination with embedding vectors; however, the GCE framework can be flexibly adapted to other parameterizations, such as integrating it as a layer within deep learning architectures, and integrating context selection mechanisms such as [13].

Original: Our method brings a new way of analyzing data through vectorial embedding which has the potential to bring greater understanding of several phenomena; as usual such tools must be used responsibly to avoid negative societal impact.
Revised: Our method brings a new way of analyzing data through vectorial embedding which has the potential to bring greater understanding of several phenomena; as usual such tools must be used responsibly to avoid negative societal impact.

Original: Acknowledgement
Revised: Acknowledgement

Original: This work is supported by the Academy of Finland decision 312395.
Revised: This work is supported by the Academy of Finland decisions 312395 and

Original: Checklist
Revised: Checklist

Original: The checklist follows the references.
Revised: The checklist follows the references.

Original: Please read the checklist guidelines carefully for information on how to answer these questions.
Revised: Please read the checklist guidelines carefully for information on how to answer these questions.

Original: For each question, change the default [TODO] to [Yes] , [No] or [N/A] .
Revised: For each question, change the default [TODO] to [Yes] , [No] or [N/A] .

Original: You are strongly encouraged to include a justification to your answer , either by referencing the appropriate section of your paper or providing a brief inline description.
Revised: You are strongly encouraged to include a justification to your answer , either by referencing the appropriate section of your paper or providing a brief inline description.

Original: For example:
Revised: For example:

Original: • Did you include the license to the code and datasets?
Revised: • Did you include the license to the code and datasets?

Original: [Yes] Provided in supplementary material.
Revised: [Yes] Provided in supplementary material.

Original: • Did you include the license to the code and datasets?
Revised: • Did you include the license to the code and datasets?

Original: [No] The code and the data are proprietary.
Revised: [No] The code and the data are proprietary.

Original: • Did you include the license to the code and datasets?
Revised: • Did you include the license to the code and datasets?

Original: [N/A]
Revised: [N/A]

Original: Please do not modify the questions and only use the provided macros for your answers.
Revised: Please do not modify the questions and only use the provided macros for your answers.

Original: Note that the
Revised: Note that the

Original: Checklist section does not count towards the page limit.
Revised: Checklist section does not count towards the page limit.

Original: In your paper, please delete this instructions block and only keep the Checklist section heading above along with the questions/answers below.
Revised: In your paper, please delete this instructions block and only keep the Checklist section heading above along with the questions/answers below.

Original: For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?
Revised: For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?

Original: [Yes] (b) Did you describe the limitations of your work?
Revised: [Yes] (b) Did you describe the limitations of your work?

Original: [Yes] (c) Did you discuss any potential negative societal impacts of your work?
Revised: [Yes] (c) Did you discuss any potential negative societal impacts of your work?

Original: [Yes] (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
Revised: [Yes] (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?

Original: [Yes] 2.
Revised: [Yes] 2.

Original: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
Revised: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?

Original: [Yes] (b) Did you include complete proofs of all theoretical results?
Revised: [Yes] (b) Did you include complete proofs of all theoretical results?

Original: [N/A] 3.
Revised: [N/A] 3.

Original: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
Revised: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?

Original: [Yes] (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
Revised: [Yes] (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?

Original: [Yes] (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
Revised: [Yes] (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?

Original: [Yes] In supplementary.
Revised: [Yes] In supplementary.

Original: (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?
Revised: (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?

Original: [Yes] In supplementary.
Revised: [Yes] In supplementary.

Original: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
Revised: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?

Original: [Yes] (b) Did you mention the license of the assets?
Revised: [Yes] (b) Did you mention the license of the assets?

Original: [N/A] (c) Did you include any new assets either in the supplemental material or as a URL?
Revised: [N/A] (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [Yes] (d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?
Revised: [Yes] (d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?

Original: [N/A] (e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?
Revised: [N/A] (e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?

Original: [Yes] 5.
Revised: [Yes] 5.

Original: If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?
Revised: If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?

Original: [N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?
Revised: [N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?

Original: [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
Revised: [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?

Original: [N/A]
Revised: [N/A]