Original: H YPEREDGE 2 VEC : D ISTRIBUTED R EPRESENTATIONS FOR H YPEREDGES
Revised: H YPEREDGE 2 VEC : D ISTRIBUTED R EPRESENTATIONS FOR H YPEREDGES

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Data structured in form of overlapping or non-overlapping sets is found in a variety of domains, sometimes explicitly but often subtly.
Revised: Data structured in form of overlapping or non-overlapping sets is found in a variety of domains, sometimes explicitly but often subtly.

Original: For example, teams, which are of prime importance in social science studies are “sets of individuals”; “item sets” in pattern mining are sets; and for various types of analysis in language studies a sentence can be considered as a “set or bag of words”.
Revised: For example, teams, which are of prime importance in social science studies are “sets of individuals”; “item sets” in pattern mining are sets; and for various types of analysis in language studies a sentence can be considered as a “set or bag of words”.

Original: Although building models and inference algorithms for structured data has been an important task in the ﬁelds of machine learning and statistics, research on “set-like” data still remains less explored.
Revised: Although building models and inference algorithms for structured data has been an important task in the ﬁelds of machine learning and statistics, research on “set-like” data still remains less explored.

Original: Relationships between pairs of elements can be modeled as edges in a graph.
Revised: Relationships between pairs of elements can be modeled as edges in a graph.

Original: However, modeling relationships that involve all members of a set, a hyperedge is a more natural representation for the set.
Revised: However, modeling relationships that involve all members of a set, a hyperedge is a more natural representation for the set.

Original: In this work, we focus on the problem of embedding hyperedges in a hypergraph (a network of overlapping sets) to a low dimensional vector space.
Revised: In this work, we focus on the problem of embedding hyperedges in a hypergraph (a network of overlapping sets) to a low dimensional vector space.

Original: We propose a probabilistic deep-learning based method as well as a tensor-based algebraic model, both of which capture the hypergraph structure in a principled manner without loosing set-level information.
Revised: We propose a probabilistic deep-learning based method as well as a tensor-based algebraic model, both of which capture the hypergraph structure in a principled manner without loosing set-level information.

Original: Our central focus is to highlight the connection between hypergraphs (topology), tensors (algebra) and probabilistic models.
Revised: Our central focus is to highlight the connection between hypergraphs (topology), tensors (algebra) and probabilistic models.

Original: We present a number of interesting baselines, some of which adapt existing node-level embedding models to the hyperedge-level, as well as sequence based language techniques which are adapted for set structured hypergraph topology.
Revised: We present a number of interesting baselines, some of which adapt existing node-level embedding models to the hyperedge-level, as well as sequence based language techniques which are adapted for set structured hypergraph topology.

Original: The performance is evaluated with a network of social groups and a network of word phrases.
Revised: The performance is evaluated with a network of social groups and a network of word phrases.

Original: Our experiments show that accuracy wise our methods perform similar to those of baselines which are not designed for hypergraphs.
Revised: Our experiments show that accuracy wise our methods perform similar to those of baselines which are not designed for hypergraphs.

Original: Moreover, our tensor based method is quiet efﬁcient as compared to deep-learning based auto-encoder method.
Revised: Moreover, our tensor based method is quiet efﬁcient as compared to deep-learning based auto-encoder method.

Original: We therefore, argue that we have proposed more general methods which are suited for hypergraphs (and therefore also for graphs) while maintaining accuracy and efﬁciency.
Revised: We therefore, argue that we have proposed more general methods which are suited for hypergraphs (and therefore also for graphs) while maintaining accuracy and efﬁciency.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: In group structured data we have multiple entities related by some form of group relationships.
Revised: In group structured data we have multiple entities related by some form of group relationships.

Original: In fact, such data is more abundantly found in the real world than has been usually studied (Estrada & Rodriguez-Velazquez, 2005).
Revised: In fact, such data is more abundantly found in the real world than has been usually studied (Estrada & Rodriguez-Velazquez, 2005).

Original: There are increasing number of ﬁelds where such data is being found more than ever before.
Revised: There are increasing number of ﬁelds where such data is being found more than ever before.

Original: In social networks domain: team data from massive online multiplayer games (Ahmed et al., 2011) such as World of Warcraft, group communication tools such as
Revised: In social networks domain: team data from massive online multiplayer games (Ahmed et al., 2011) such as World of Warcraft, group communication tools such as

Original: Skype and Google Docs and research collaborations (Subbian et al., 2013; pub).
Revised: Skype and Google Docs and research collaborations (Subbian et al., 2013; pub).

Original: There are other ﬁelds in which structural relationships between entities is important as well, and large datasets capturing them exist.
Revised: There are other ﬁelds in which structural relationships between entities is important as well, and large datasets capturing them exist.

Original: Examples include Natural Language Processing (Bengio & Bengio, 2000),
Revised: Examples include Natural Language Processing (Bengio & Bengio, 2000),

Original: Biology (Hwang et al., 2008; Klamt et al., 2009), e-commerce
Revised: Biology (Hwang et al., 2008; Klamt et al., 2009), e-commerce

Original: Deshpande & Karypis, 2004; Christakopoulou & Karypis, 2014) and Chemistry (Bartholomay, 1960).
Revised: Deshpande & Karypis, 2004; Christakopoulou & Karypis, 2014) and Chemistry (Bartholomay, 1960).

Original: Figure 1 shows such examples for networks of groups, sentences, and item sets.
Revised: Figure 1 shows such examples for networks of groups, sentences, and item sets.

Original: Hypergraph (Berge, 1984), which is a generalization of graphs, is a popular model to naturally capture higher-order relationships between sets of objects (Figure 2) (Estrada & Rodriguez-Velazquez, 2005).
Revised: Hypergraph (Berge, 1984), which is a generalization of graphs, is a popular model to naturally capture higher-order relationships between sets of objects (Figure 2) (Estrada & Rodriguez-Velazquez, 2005).

Original: Within machine learning, algorithms guided by the structure of such higher order networks (Zhou et al., 2006) have found applications in a variety of domains (Tian et al., 2009; Gao et al., 2013; Li & Li, 2013; Sharma et al., 2015; 2017).
Revised: Within machine learning, algorithms guided by the structure of such higher order networks (Zhou et al., 2006) have found applications in a variety of domains (Tian et al., 2009; Gao et al., 2013; Li & Li, 2013; Sharma et al., 2015; 2017).

Original: More recently, the interest of representation learning in NLP (Mikolov et al., 2013a) has stimulated its application in graph embedding (learning low dimensional representation for graph nodes) (Perozzi et al., 2014; Tang et al., 2015b; Grover & Leskovec, 2016;
Revised: More recently, the interest of representation learning in NLP (Mikolov et al., 2013a) has stimulated its application in graph embedding (learning low dimensional representation for graph nodes) (Perozzi et al., 2014; Tang et al., 2015b; Grover & Leskovec, 2016;

Original: Cai et al., 2017).
Revised: Cai et al., 2017).

Original: However, this new line of research is limited to simple graphs and we are unaware of any work that considers hyperedge embeddings (learning representations for hyperedges).
Revised: However, this new line of research is limited to simple graphs and we are unaware of any work that considers hyperedge embeddings (learning representations for hyperedges).

Original: In this paper we aim to develop methods which (1) learn hypergraph embeddings directly, (2) leverage the hypergraph topology and (3) not loose the hyperedge-level joint information.
Revised: In this paper we aim to develop methods which (1) learn hypergraph embeddings directly, (2) leverage the hypergraph topology and (3) not loose the hyperedge-level joint information.

Original: These learned embeddings can then be employed by a supervised or semi-supervised algorithm to perform various predictive tasks pertaining to hyperedges.
Revised: These learned embeddings can then be employed by a supervised or semi-supervised algorithm to perform various predictive tasks pertaining to hyperedges.

Original: For example performance prediction of a team (set of individuals) engaged in a collaborative task or sentiment analysis of a sentence (set of words).
Revised: For example performance prediction of a team (set of individuals) engaged in a collaborative task or sentiment analysis of a sentence (set of words).

Original: Although, there have been a variety of attempts to learn node embeddings for hypergraphs by extending traditional graph embedding methods for hypergraph setting.
Revised: Although, there have been a variety of attempts to learn node embeddings for hypergraphs by extending traditional graph embedding methods for hypergraph setting.

Original: These approaches differ in the extent they address the aims (2) and (3) as pointed previously.
Revised: These approaches differ in the extent they address the aims (2) and (3) as pointed previously.

Original: In ﬁrst category, there are a number of methods that incorporate the hypergraph topology using proxy graphs (Zhou et al., 2006; Hwang et al., 2008), therefore, incur loss of information.
Revised: In ﬁrst category, there are a number of methods that incorporate the hypergraph topology using proxy graphs (Zhou et al., 2006; Hwang et al., 2008), therefore, incur loss of information.

Original: Also, Agarwal et al.
Revised: Also, Agarwal et al.

Original: (2006) squarely criticize that such representations can be learned by constructing graphs, which are proxies for the hypergraph structure.
Revised: (2006) squarely criticize that such representations can be learned by constructing graphs, which are proxies for the hypergraph structure.

Original: Second, are methods which actually, take into account the hyperedge-level information, like the classical Neural Language Model which maintains the higher order sequence information (Bengio & Bengio, 2000).
Revised: Second, are methods which actually, take into account the hyperedge-level information, like the classical Neural Language Model which maintains the higher order sequence information (Bengio & Bengio, 2000).

Original: But their model is designed for sequences and also simply ignores the topology (sequences are linked by common words forming a network of sequences).
Revised: But their model is designed for sequences and also simply ignores the topology (sequences are linked by common words forming a network of sequences).

Original: However recently, there has been interest in modeling “set-like” structures within deep-learning community (Vinyals et al., 2016; Rezatoﬁghi et al., 2016).
Revised: However recently, there has been interest in modeling “set-like” structures within deep-learning community (Vinyals et al., 2016; Rezatoﬁghi et al., 2016).

Original: But they do not consider the hypergraph structure between the sets and therefore, not model hypergraphs in a principled manner.
Revised: But they do not consider the hypergraph structure between the sets and therefore, not model hypergraphs in a principled manner.

Original: Third category are those methods that do not ignore the topology completely and also retain the hyperedge-level information.
Revised: Third category are those methods that do not ignore the topology completely and also retain the hyperedge-level information.

Original: Among this are k -way tensor based methods that explicitly do not work on hypergraph (Shashua et al., 2006; Bul`o & Pelillo, 2009), but the connection exists in the form that k -way tensor represents a k-uniform hypergraph (Qi, 2005).
Revised: Among this are k -way tensor based methods that explicitly do not work on hypergraph (Shashua et al., 2006; Bul`o & Pelillo, 2009), but the connection exists in the form that k -way tensor represents a k-uniform hypergraph (Qi, 2005).

Original: But they are restricted to uniform hypergraphs.
Revised: But they are restricted to uniform hypergraphs.

Original: We therefore, propose two methods, both of which directly learn hyperedge embeddings for general hypergraphs and capture the hypergraph structure in a principled manner.
Revised: We therefore, propose two methods, both of which directly learn hyperedge embeddings for general hypergraphs and capture the hypergraph structure in a principled manner.

Original: First, is an auto-encoder based deep-learning method and second, is a tensor-based algebraic method.
Revised: First, is an auto-encoder based deep-learning method and second, is a tensor-based algebraic method.

Original: We also highlight the connection between hypergraphs (topology), tensors (algebra) and probabilistic models.
Revised: We also highlight the connection between hypergraphs (topology), tensors (algebra) and probabilistic models.

Original: A number of interesting baselines are also developed, some of which adapt existing node-level embedding models to hypergraph setting and also sequence based NLP techniques are adapted for set structured hypergraphs.
Revised: A number of interesting baselines are also developed, some of which adapt existing node-level embedding models to hypergraph setting and also sequence based NLP techniques are adapted for set structured hypergraphs.

Original: We have experimented with social group dataset (network of teams as a hypergraph) from an online game for team (hyperedge) performance prediction as well as language networks (sentence/phrase hypergraphs) for phrase-level sentiment analysis.
Revised: We have experimented with social group dataset (network of teams as a hypergraph) from an online game for team (hyperedge) performance prediction as well as language networks (sentence/phrase hypergraphs) for phrase-level sentiment analysis.

Original: Our experiments show that accuracy wise our methods perform similar to those of baselines which are not designed for hypergraphs.
Revised: Our experiments show that accuracy wise our methods perform similar to those of baselines which are not designed for hypergraphs.

Original: Moreover, our tensor based method is quiet efﬁcient as compared to deep-learning based auto-encoder method.
Revised: Moreover, our tensor based method is quiet efﬁcient as compared to deep-learning based auto-encoder method.

Original: We therefore, argue that we have proposed more general methods which are suited for hypergraphs (and therefore also for graphs) while maintaining accuracy and efﬁciency.
Revised: We therefore, argue that we have proposed more general methods which are suited for hypergraphs (and therefore also for graphs) while maintaining accuracy and efﬁciency.

Original: In summary the main contributions of our paper are:
Revised: In summary the main contributions of our paper are:

Original: • We propose the novel concept of a dual tensor corresponding to the hypergraph dual and allows us to get hyperedge embedding directly.
Revised: • We propose the novel concept of a dual tensor corresponding to the hypergraph dual and allows us to get hyperedge embedding directly.

Original: • We propose a general hypergraph tensor decomposition method designed for general hypergraphs (containing different cardinality hyperedges) unlike simple uniform hypergraph tensor decomposition which is restricted to ﬁxed cardinality hyperedges (i
Revised: • We propose a general hypergraph tensor decomposition method designed for general hypergraphs (containing different cardinality hyperedges) unlike simple uniform hypergraph tensor decomposition which is restricted to ﬁxed cardinality hyperedges (i

Original: e. uniform hypergraph).
Revised: e. uniform hypergraph).

Original: We are unaware of any such works or applications employing this approach.
Revised: We are unaware of any such works or applications employing this approach.

Original: • We propose the novel idea to use of de-noising auto-encoder in a hypergraph setting.
Revised: • We propose the novel idea to use of de-noising auto-encoder in a hypergraph setting.

Original: Moreover, we also develop techniques of creating noise using random-walks over hasse diagram topology, which is original and unique.
Revised: Moreover, we also develop techniques of creating noise using random-walks over hasse diagram topology, which is original and unique.

Original: • We highlight and argue that embeddings from tensor based methods have a natural hypergraph theoretic interpretation unlike the deep learning based black-box approaches.
Revised: • We highlight and argue that embeddings from tensor based methods have a natural hypergraph theoretic interpretation unlike the deep learning based black-box approaches.

Original: • Our proposed methods learn hypergraph embeddings based on the unique approach of leveraging the existing structure present in network data as context, in contrast to context generation based existing graph embedding techniques.
Revised: • Our proposed methods learn hypergraph embeddings based on the unique approach of leveraging the existing structure present in network data as context, in contrast to context generation based existing graph embedding techniques.

Original: Following is the outline for the rest of the paper.
Revised: Following is the outline for the rest of the paper.

Original: In section 2 we describe the problem deﬁnition and statement followed by Section 3 where we describe in detail the various methods proposed in this paper.
Revised: In section 2 we describe the problem deﬁnition and statement followed by Section 3 where we describe in detail the various methods proposed in this paper.

Original: Section 4 describes the datasets, experimental tasks & settings for the models.
Revised: Section 4 describes the datasets, experimental tasks & settings for the models.

Original: Sectionprovides an overview of the related literature followed by conclusion and appendix.
Revised: Sectionprovides an overview of the related literature followed by conclusion and appendix.

Original:  P RELIMINARIES
Revised:  P RELIMINARIES

Original: In this paper we consider the scenario where we have a collection of elements .
Revised: In this paper we consider the scenario where we have a collection of elements .

Original: These elements can represent individual actors in case of social groups or words in sentences or items in itemsets within a transaction database.
Revised: These elements can represent individual actors in case of social groups or words in sentences or items in itemsets within a transaction database.

Original: In other words a social group or a sentence or an item-set are sets which contain these elements .
Revised: In other words a social group or a sentence or an item-set are sets which contain these elements .

Original: Let V = { v 1 , v 2 , ..., v n } represents n elements and we have m sets deﬁned over these elements, denoted by G = { g 1 , g 2 , ..., g m } , where g i ⊆
Revised: Let V = { v 1 , v 2 , ..., v n } represents n elements and we have m sets deﬁned over these elements, denoted by G = { g 1 , g 2 , ..., g m } , where g i ⊆

Original: V represents the i th set.
Revised: V represents the i th set.

Original: The cardinality | g i | represents the number of elements in the set.
Revised: The cardinality | g i | represents the number of elements in the set.

Original: Also each set g i ∈
Revised: Also each set g i ∈

Original: G has an occurrence number R ( g i ) , which denotes the number of times it has occurred.
Revised: G has an occurrence number R ( g i ) , which denotes the number of times it has occurred.

Original: Such overlapping or non-overlapping sets can be modeled as a hypergraph Berge (1984), where the nodes and hyperedges, represent the elements and sets, respectively.
Revised: Such overlapping or non-overlapping sets can be modeled as a hypergraph Berge (1984), where the nodes and hyperedges, represent the elements and sets, respectively.

Original: This hypergraph is represented as
Revised: This hypergraph is represented as

Original: N g =
Revised: N g =

Original: ( V, G ) with G as the collection of hyperedges over the nodes V .
Revised: ( V, G ) with G as the collection of hyperedges over the nodes V .

Original: The incidence matrix H ∈ { 0 , 1 } | G |×| V | for N g represents the presence of nodes in different hyperedges with H ( g i , v ) =
Revised: The incidence matrix H ∈ { 0 , 1 } | G |×| V | for N g represents the presence of nodes in different hyperedges with H ( g i , v ) =

Original: if v ∈ g i else 0 .
Revised: if v ∈ g i else 0 .

Original: A specialization of hypergraph model is the Simplicial complex (Munkres, 1984) (Figure 2), in which additionally each hyperedge has the subset closure property, i.e., each subset of hyperedge is also a valid hyperedge.
Revised: A specialization of hypergraph model is the Simplicial complex (Munkres, 1984) (Figure 2), in which additionally each hyperedge has the subset closure property, i.e., each subset of hyperedge is also a valid hyperedge.

Original: Problem Statement:
Revised: Problem Statement:

Original: Given this setting, our goal is to learn the mapping φ : G → R d from hyperedges to feature representations (i.e., embeddings) that can be used to build predictive models involving sets.
Revised: Given this setting, our goal is to learn the mapping φ : G → R d from hyperedges to feature representations (i.e., embeddings) that can be used to build predictive models involving sets.

Original: Here d is a parameter specifying the number of dimensions of the embedding vector.
Revised: Here d is a parameter specifying the number of dimensions of the embedding vector.

Original: Equivalently, φ can be thought of as a look-up matrix of size | G | × d , where | G | is the total number of sets or hyperedges.
Revised: Equivalently, φ can be thought of as a look-up matrix of size | G | × d , where | G | is the total number of sets or hyperedges.

Original:  M ETHODOLOGY
Revised:  M ETHODOLOGY

Original: As we had mentioned before there are several methods to learn representation of nodes in a graph.
Revised: As we had mentioned before there are several methods to learn representation of nodes in a graph.

Original: Given that a hyperedge is a set of nodes, a natural question arises is that, can we combine the node level embeddings (learned using existing methods) within a given hyperedge to ﬁnd a suitable representation of that hyperedge?
Revised: Given that a hyperedge is a set of nodes, a natural question arises is that, can we combine the node level embeddings (learned using existing methods) within a given hyperedge to ﬁnd a suitable representation of that hyperedge?

Original: However, there are large number of possible ways one can combine the node embeddings.
Revised: However, there are large number of possible ways one can combine the node embeddings.

Original: Therefore, in the following two subsections, we develop two methods that learn the embeddings for hyperedges directly in a more principled manner.
Revised: Therefore, in the following two subsections, we develop two methods that learn the embeddings for hyperedges directly in a more principled manner.

Original: Our tensor based method is focused on retaining the set-level information intact while harnessing the hypergraph network structure.
Revised: Our tensor based method is focused on retaining the set-level information intact while harnessing the hypergraph network structure.

Original: Where the auto-encoder based method harnesses the non-linearity of representation offered by deep learning to generate crisp and powerful embeddings.
Revised: Where the auto-encoder based method harnesses the non-linearity of representation offered by deep learning to generate crisp and powerful embeddings.

Original: Tensor based method has a natural hasse diagram based topological interpretation and can generate both node and hyperedge embeddings.
Revised: Tensor based method has a natural hasse diagram based topological interpretation and can generate both node and hyperedge embeddings.

Original: Whereas deep learning model is more black-box based approach and can only generate hyperedge embeddings.
Revised: Whereas deep learning model is more black-box based approach and can only generate hyperedge embeddings.

Original: 3. H YPEREDGE 2 VEC USING H YPERGRAPH T ENSOR D ECOMPOSITION
Revised: 3. H YPEREDGE 2 VEC USING H YPERGRAPH T ENSOR D ECOMPOSITION

Original: In this section, we develop tensor (higher-order matrices) based linear algebraic methods that learn node as well as hyperedge embedding by taking into account the joint probability over a hyperedge.
Revised: In this section, we develop tensor (higher-order matrices) based linear algebraic methods that learn node as well as hyperedge embedding by taking into account the joint probability over a hyperedge.

Original: The idea behind using tensors is that they retain the set-level information contained in a hypergraph, unlike the proxy graphs (corresponding to hypergraphs) based techniques (used as baselines in our experiments), which approximate hyperedge or set-level information with dyadic edge-level information.
Revised: The idea behind using tensors is that they retain the set-level information contained in a hypergraph, unlike the proxy graphs (corresponding to hypergraphs) based techniques (used as baselines in our experiments), which approximate hyperedge or set-level information with dyadic edge-level information.

Original: Therefore, we design an algorithm which is principally suited for hypergraph structured data.
Revised: Therefore, we design an algorithm which is principally suited for hypergraph structured data.

Original: For a given hypergraph we can extract a sub-hypergraph that only consists of the hyperedges with cardinality k .
Revised: For a given hypergraph we can extract a sub-hypergraph that only consists of the hyperedges with cardinality k .

Original: This sub-hypergraph is a k -uniform hypergraph or k -graph Cooper & Dutle (2012).
Revised: This sub-hypergraph is a k -uniform hypergraph or k -graph Cooper & Dutle (2012).

Original: Corresponding to this k -uniform hypergraph, we can deﬁne a k th order n -dimensional symmetric tensor Qi (2005)
Revised: Corresponding to this k -uniform hypergraph, we can deﬁne a k th order n -dimensional symmetric tensor Qi (2005)

Original: A khyp =
Revised: A khyp =

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: p k ) ∈ R [ k,n ] whose elements are initialized as follows:
Revised: p k ) ∈ R [ k,n ] whose elements are initialized as follows:

Original: [Equation]
Revised: [Equation]

Original: where { v p 1 , v p 2 , ..., v p k } ∈ g i and | g i |
Revised: where { v p 1 , v p 2 , ..., v p k } ∈ g i and | g i |

Original: =
Revised: =

Original: k, ∀ i ∈ { 1 , ..., m } .
Revised: k, ∀ i ∈ { 1 , ..., m } .

Original: Note that symmetry here implies that value of element a p 1 ,p 2 ,..,p k is invariable under any permutation of its indices ( p 1 , p 2 , .., p k ) .
Revised: Note that symmetry here implies that value of element a p 1 ,p 2 ,..,p k is invariable under any permutation of its indices ( p 1 , p 2 , .., p k ) .

Original: Rest all the elements in the tensor are zeros.
Revised: Rest all the elements in the tensor are zeros.

Original: In a similar manner we can also deﬁne a dual tensor , corresponding to hypergraph dual where the roles of nodes and hyperedges are interchanged.
Revised: In a similar manner we can also deﬁne a dual tensor , corresponding to hypergraph dual where the roles of nodes and hyperedges are interchanged.

Original: We consider all the hyperedges in the hypergraph dual that are of cardinality k .
Revised: We consider all the hyperedges in the hypergraph dual that are of cardinality k .

Original: This basically corresponds to all the vertices in the original hypergraph which have a degree of k , i.e., they are part of exactly k hyperedges in the original hypergraph.
Revised: This basically corresponds to all the vertices in the original hypergraph which have a degree of k , i.e., they are part of exactly k hyperedges in the original hypergraph.

Original: Corresponding to this k -uniform hypergraph dual, we can deﬁne a k th order m -dimensional symmetric dual tensor A kdual =
Revised: Corresponding to this k -uniform hypergraph dual, we can deﬁne a k th order m -dimensional symmetric dual tensor A kdual =

Original: ( a q 1 ,q 2 ,..,q k ) ∈ R [ k,m ] whose elements are initialized as follows:
Revised: ( a q 1 ,q 2 ,..,q k ) ∈ R [ k,m ] whose elements are initialized as follows:

Original: [Equation]
Revised: [Equation]

Original: where { g q 1 , g q 2 , ..., g q k } (cid:51) v j and |{ r | H ( r, j ) = 1 }| = k, ∀ j ∈ { 1 , ..., n } .
Revised: where { g q 1 , g q 2 , ..., g q k } (cid:51) v j and |{ r | H ( r, j ) = 1 }| = k, ∀ j ∈ { 1 , ..., n } .

Original: Note that this tensor is also symmetric and rest all the elements in the tensor are zeros.
Revised: Note that this tensor is also symmetric and rest all the elements in the tensor are zeros.

Original: To realize our aim of learning node and hyperedge embeddings we perform joint CP Tensor Decomposition Kolda & Bader (2009) (of the tensors we just described) across different cardinality hyperedges simultaneously.
Revised: To realize our aim of learning node and hyperedge embeddings we perform joint CP Tensor Decomposition Kolda & Bader (2009) (of the tensors we just described) across different cardinality hyperedges simultaneously.

Original: Speciﬁcally, for the node embeddings we solve the following optimization problem:
Revised: Speciﬁcally, for the node embeddings we solve the following optimization problem:

Original: [Equation]
Revised: [Equation]

Original: where,
Revised: where,

Original: [Equation]
Revised: [Equation]

Original: with z r ∈ R n , λ r ∈ R + , Z ∈ R n × d and Z (: , r ) =
Revised: with z r ∈ R n , λ r ∈ R + , Z ∈ R n × d and Z (: , r ) =

Original: z r .
Revised: z r .

Original: Notice, that equation 4 is the standard symmetric CP decomposition but equation 3 is summation of reconstruction errors in different tensor decompositions for different cardinality hyperedges.
Revised: Notice, that equation 4 is the standard symmetric CP decomposition but equation 3 is summation of reconstruction errors in different tensor decompositions for different cardinality hyperedges.

Original: Each error term learns a common latent factor matrix Z using the empirical observed k -uniform sub-hypergraph stored in A khyp tensor.
Revised: Each error term learns a common latent factor matrix Z using the empirical observed k -uniform sub-hypergraph stored in A khyp tensor.

Original: We perform this joint decomposition by augmenting the standard CP Decomposition into the
Revised: We perform this joint decomposition by augmenting the standard CP Decomposition into the

Original: Hypergraph-CP-ALS algorithm 1.
Revised: Hypergraph-CP-ALS algorithm 1.

Original: Lines (4-8) is the standard ( non-symmetric ) CP Decomposition
Revised: Lines (4-8) is the standard ( non-symmetric ) CP Decomposition

Original: (see reference (Kolda & Bader, 2009) for details), which solves the following optimization:
Revised: (see reference (Kolda & Bader, 2009) for details), which solves the following optimization:

Original: [Equation]
Revised: [Equation]

Original: with k u ( j ) r ∈ R n , λ r ∈ R + , k U ( j ) ∈ R n × d , k U ( j ) (: , r ) =
Revised: with k u ( j ) r ∈ R n , λ r ∈ R + , k U ( j ) ∈ R n × d , k U ( j ) (: , r ) =

Original: k u ( j ) r and j ∈ { 1 , .., k } .
Revised: k u ( j ) r and j ∈ { 1 , .., k } .

Original: As we want to learn common representations ( Z ) for all the nodes, we add additional constraint that enforce that k U ( j ) are same for all j ∈ { 1 , .., k } and for all k ∈ [ c min , c max ] .
Revised: As we want to learn common representations ( Z ) for all the nodes, we add additional constraint that enforce that k U ( j ) are same for all j ∈ { 1 , .., k } and for all k ∈ [ c min , c max ] .

Original: Lines (9-15) is where we force all representations for all cardinalities to be average of the k U ( j ) achieved by the last decomposition iteration that occurred in lines (4-8).
Revised: Lines (9-15) is where we force all representations for all cardinalities to be average of the k U ( j ) achieved by the last decomposition iteration that occurred in lines (4-8).

Original: Averaging can be interpreted as equal contribution from the latent factors learned from different cardinality (uniform) sub-hypergraphs.
Revised: Averaging can be interpreted as equal contribution from the latent factors learned from different cardinality (uniform) sub-hypergraphs.

Original: We make repeated pass through the entire hypergraph (by learning via different k -uniform sub-hypergraph (line 3)) until the objective (equation 3) converges.
Revised: We make repeated pass through the entire hypergraph (by learning via different k -uniform sub-hypergraph (line 3)) until the objective (equation 3) converges.

Original: In our implementation we empirically observe that using the above mentioned unbiased averaging heuristic, our algorithm converges successfully.
Revised: In our implementation we empirically observe that using the above mentioned unbiased averaging heuristic, our algorithm converges successfully.

Original: The same algorithm 1 is used to get the hyperedge embeddings, by just passing A dual instead of A hyp .
Revised: The same algorithm 1 is used to get the hyperedge embeddings, by just passing A dual instead of A hyp .

Original: We shall jointly refer to the embeddings achieved for nodes and hyperedges via the above tensor decomposition techniques as t2v .
Revised: We shall jointly refer to the embeddings achieved for nodes and hyperedges via the above tensor decomposition techniques as t2v .

Original: Algorithm
Revised: Algorithm

Original: 1: randomly initialize k U c c
Revised: 1: randomly initialize k U c c

Original: } 2: repeat 3: for k do 4: for do 5:
Revised: } 2: repeat 3: for k do 4: for do 5:

Original: V
Revised: V

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: : end for 9:
Revised: : end for 9:

Original: Z ←
Revised: Z ←

Original: [Table]
Revised: [Table]

Original: 10: for p c min to c do 11: for
Revised: 10: for p c min to c do 11: for

Original: 1 to p do 12: p U j
Revised: 1 to p do 12: p U j

Original: ← Z 13: end for 14: end for 15: end for 16: until ﬁt criteria achieved or maximum number of iterations exceeded 17: return Z
Revised: ← Z 13: end for 14: end for 15: end for 16: until ﬁt criteria achieved or maximum number of iterations exceeded 17: return Z

Original: We would like to highlight a few points regarding the tensor methods.
Revised: We would like to highlight a few points regarding the tensor methods.

Original: The tensors that we have employed are super-symmetric and hence able to capture distribution over sets rather than sequence.
Revised: The tensors that we have employed are super-symmetric and hence able to capture distribution over sets rather than sequence.

Original: But in general we can employ a k -way tensor which is not symmetric to even capture sequence.
Revised: But in general we can employ a k -way tensor which is not symmetric to even capture sequence.

Original: In this sense tensors are more general purpose.
Revised: In this sense tensors are more general purpose.

Original: Another point one can observe that when we initialize the hypergraph tensor A khyp , we have initialized all the permutations of vertices corresponding to a given hyperedge (Eq. 1).
Revised: Another point one can observe that when we initialize the hypergraph tensor A khyp , we have initialized all the permutations of vertices corresponding to a given hyperedge (Eq. 1).

Original: Moreover, we initialize it by the repetition counts.
Revised: Moreover, we initialize it by the repetition counts.

Original: 3. H YPEREDGE 2V EC U SING H ASSE D E - NOISING A UTOENCODER
Revised: 3. H YPEREDGE 2V EC U SING H ASSE D E - NOISING A UTOENCODER

Original: An autoencoder (Bengio et al., 2009), takes an input vector x ∈ [0 , 1] n and maps it to a latent representation y ∈ [0 , 1] d .
Revised: An autoencoder (Bengio et al., 2009), takes an input vector x ∈ [0 , 1] n and maps it to a latent representation y ∈ [0 , 1] d .

Original: This is typically done using an afﬁne mapping followed by a nonlinearity (more so when the input, like in our case, is binary (Vincent et al., 2010)): f en ( x ) =
Revised: This is typically done using an afﬁne mapping followed by a nonlinearity (more so when the input, like in our case, is binary (Vincent et al., 2010)): f en ( x ) =

Original: s ( Wx + b ) , with parameters θ = { W , b } .
Revised: s ( Wx + b ) , with parameters θ = { W , b } .

Original: Here, W is a n × d weight matrix and b is the offset.
Revised: Here, W is a n × d weight matrix and b is the offset.

Original: This latent representation is reconstructed into a vector z = f de ( y )
Revised: This latent representation is reconstructed into a vector z = f de ( y )

Original: =
Revised: =

Original: s ( W (cid:48)
Revised: s ( W (cid:48)

Original: y + b (cid:48) )
Revised: y + b (cid:48) )

Original: , in the input space, z ∈ [0 , 1] n and parameters θ (cid:48) =
Revised: , in the input space, z ∈ [0 , 1] n and parameters θ (cid:48) =

Original: { W (cid:48) , b (cid:48) } .
Revised: { W (cid:48) , b (cid:48) } .

Original: The mappings f en and f de are referred as the encoder and decoder , respectively.
Revised: The mappings f en and f de are referred as the encoder and decoder , respectively.

Original: The representation y is learned by minimizing the following reconstruction error:
Revised: The representation y is learned by minimizing the following reconstruction error:

Original: [Equation]
Revised: [Equation]

Original: where L is a loss function, which in case of binary or bit probabilities is often chosen as the crossentropy loss :
Revised: where L is a loss function, which in case of binary or bit probabilities is often chosen as the crossentropy loss :

Original: [Equation]
Revised: [Equation]

Original: In their paper, Vincent et al.
Revised: In their paper, Vincent et al.

Original: (2010) have shown that minimizing reconstruction amounts to maximizing the lower bound on the mutual information between input ( x ) and the representation y .
Revised: (2010) have shown that minimizing reconstruction amounts to maximizing the lower bound on the mutual information between input ( x ) and the representation y .

Original: However, they have further argued (Vincent et al.
Revised: However, they have further argued (Vincent et al.

Original: that y retaining information about input x is insufﬁcient.
Revised: that y retaining information about input x is insufﬁcient.

Original: They further, propose the idea that the learned representation should be able to recover ( denoising ) the original input even after being trained with corrupted input ( adding noise ).
Revised: They further, propose the idea that the learned representation should be able to recover ( denoising ) the original input even after being trained with corrupted input ( adding noise ).

Original: They generate the corrupted input ( ˜x ), using a stochastic mapping q
Revised: They generate the corrupted input ( ˜x ), using a stochastic mapping q

Original: ( ˜x | x ) .
Revised: ( ˜x | x ) .

Original: Choice of noise is usually either Gaussian for real inputs and Salt-and-pepper noise for discrete inputs.
Revised: Choice of noise is usually either Gaussian for real inputs and Salt-and-pepper noise for discrete inputs.

Original: The denoising autoencoder then learns the representation for each input, x , same as Equation 6, but with the following modiﬁed loss function:
Revised: The denoising autoencoder then learns the representation for each input, x , same as Equation 6, but with the following modiﬁed loss function:

Original: L ( x ( i ) , g θ (cid:48)
Revised: L ( x ( i ) , g θ (cid:48)

Original: ( f θ ( ˜x ( i )))) .
Revised: ( f θ ( ˜x ( i )))) .

Original: We leverage the denoising autoencoder for learning representation for j th hyperedge, by treating each hyperedge as an input, x = H ( j, :) .
Revised: We leverage the denoising autoencoder for learning representation for j th hyperedge, by treating each hyperedge as an input, x = H ( j, :) .

Original: The size of this input vector for each hyperedge is n , which is the number of vertices in the hypergraph.
Revised: The size of this input vector for each hyperedge is n , which is the number of vertices in the hypergraph.

Original: In most natural hypergraphs, specially social networks, n can be quiet high ranging from thousands to millions or even billions (like Facebook for example).
Revised: In most natural hypergraphs, specially social networks, n can be quiet high ranging from thousands to millions or even billions (like Facebook for example).

Original: Therefore, randomly using a discrete noise like salt-and-pepper , might not be reasonable, as there are large number of possible permutations (as size n is large) and not all of them are related.
Revised: Therefore, randomly using a discrete noise like salt-and-pepper , might not be reasonable, as there are large number of possible permutations (as size n is large) and not all of them are related.

Original: Random addition of 1 s or deletion of existing 1 s from x , amounts to randomly adding or deleting vertices to the hyperedge corresponding to x .
Revised: Random addition of 1 s or deletion of existing 1 s from x , amounts to randomly adding or deleting vertices to the hyperedge corresponding to x .

Original: This might end up in new hyperedges that are completely unrelated to the given hyperedge ( x ).
Revised: This might end up in new hyperedges that are completely unrelated to the given hyperedge ( x ).

Original: For example, users (nodes) in a social network from completely different regions of the network suddenly form a group (hyperedge).
Revised: For example, users (nodes) in a social network from completely different regions of the network suddenly form a group (hyperedge).

Original: Such anomalous scenarios rarely happen in practice and social groups evolve in a gradual fashion via simple processes (Sharma et al., 2017; 2015).
Revised: Such anomalous scenarios rarely happen in practice and social groups evolve in a gradual fashion via simple processes (Sharma et al., 2017; 2015).

Original: Rather, we take advantage of the hypergraph structure to systematically guide us in generating this noise.
Revised: Rather, we take advantage of the hypergraph structure to systematically guide us in generating this noise.

Original: A hypergraph can be deﬁned by its corresponding hasse lattice (Sharma et al., 2017).
Revised: A hypergraph can be deﬁned by its corresponding hasse lattice (Sharma et al., 2017).

Original: For a given hyperedge ( x ), we consider the sub-lattice consisting of only those hyperedges that are a distance h from it in the complete lattice.
Revised: For a given hyperedge ( x ), we consider the sub-lattice consisting of only those hyperedges that are a distance h from it in the complete lattice.

Original: On this sub-lattice we sample p hyperedges (nodes in sublattice) by performing random walk starting at the given hyperedge’s node (see Figure 3).
Revised: On this sub-lattice we sample p hyperedges (nodes in sublattice) by performing random walk starting at the given hyperedge’s node (see Figure 3).

Original: We assume that all the nodes in the sub-lattice which correspond to previously occurred hyperedge g i have weight R ( g i ) and rest all nodes have a constant weight τ .
Revised: We assume that all the nodes in the sub-lattice which correspond to previously occurred hyperedge g i have weight R ( g i ) and rest all nodes have a constant weight τ .

Original: During random walk from a given node we choose a neighboring node (as the next node) in proportion to this neighboring node’s weight as compared to the other neighboring nodes.
Revised: During random walk from a given node we choose a neighboring node (as the next node) in proportion to this neighboring node’s weight as compared to the other neighboring nodes.

Original: Our stochastic mapping q
Revised: Our stochastic mapping q

Original: ( ˜x | x ) is therefore, a random walk on the sub-lattice of hyperdge ( x ) containing hyperedges at distance h from it.
Revised: ( ˜x | x ) is therefore, a random walk on the sub-lattice of hyperdge ( x ) containing hyperedges at distance h from it.

Original: Intuitively, the hyperedges coming within a reasonable distance will affect each others representations and will have more similar representations.
Revised: Intuitively, the hyperedges coming within a reasonable distance will affect each others representations and will have more similar representations.

Original: We will refer to the hyperedge representations learned by the above autoencoder technique, as h2v-auto .
Revised: We will refer to the hyperedge representations learned by the above autoencoder technique, as h2v-auto .

Original:  E XPERIMENTS 4. D ATASET D ESCRIPTION
Revised:  E XPERIMENTS 4. D ATASET D ESCRIPTION

Original: As the ﬁrst dataset, we use group interaction log-data of the Sony’s
Revised: As the ﬁrst dataset, we use group interaction log-data of the Sony’s

Original: Online multi-player game EverQuest II ( EQ II ) (www.everquest2.com) for a time period of nine months.
Revised: Online multi-player game EverQuest II ( EQ II ) (www.everquest2.com) for a time period of nine months.

Original: In this game, several players work together as a team to perform various tasks.
Revised: In this game, several players work together as a team to perform various tasks.

Original: Each team earn points after completion of each task, and as the teams progress by earning points, they are escalated to different levels of game play.
Revised: Each team earn points after completion of each task, and as the teams progress by earning points, they are escalated to different levels of game play.

Original: The interestingness of the game increases with each level.
Revised: The interestingness of the game increases with each level.

Original: The points earned by the teams are treated as a measure of group performance.
Revised: The points earned by the teams are treated as a measure of group performance.

Original: Each set of players who played together is treated as a hyperedge.
Revised: Each set of players who played together is treated as a hyperedge.

Original: We treat the number of times same set of players play together again as hyperedge occurrence number ( R ( g i ) ).
Revised: We treat the number of times same set of players play together again as hyperedge occurrence number ( R ( g i ) ).

Original: Players can participate in several teams over time, therefore, resulting in a hypergraph with overlapping hyperedges.
Revised: Players can participate in several teams over time, therefore, resulting in a hypergraph with overlapping hyperedges.

Original: We consider hyperedges of cardinality ∈ [2 , 6] as almost 90% of our hyperedges lie within this range.
Revised: We consider hyperedges of cardinality ∈ [2 , 6] as almost 90% of our hyperedges lie within this range.

Original: The resulting dataset contains a total ofhyperedges (teams) among 6536 nodes (players).
Revised: The resulting dataset contains a total ofhyperedges (teams) among 6536 nodes (players).

Original: Second dataset, is the fully labeled and publicly available sentiment analysis corpus of Stanford
Revised: Second dataset, is the fully labeled and publicly available sentiment analysis corpus of Stanford

Original: Sentiment Treebank ( LangNet ) (Socher et al., 2013).
Revised: Sentiment Treebank ( LangNet ) (Socher et al., 2013).

Original: This dataset is based on the reviews from a movie review website (www.rottentomatoes.com) and contains 215,154 unique phrases.
Revised: This dataset is based on the reviews from a movie review website (www.rottentomatoes.com) and contains 215,154 unique phrases.

Original: Each of the phrases are labeled with a sentiment score (a real number ∈ [0 , 1] , larger value indicates positive sentiment) by human annotators.
Revised: Each of the phrases are labeled with a sentiment score (a real number ∈ [0 , 1] , larger value indicates positive sentiment) by human annotators.

Original: Each phrase is a set or hyperedge of words.
Revised: Each phrase is a set or hyperedge of words.

Original: As there is no occurrence information for a phrase hyperedge we consider R ( g i )
Revised: As there is no occurrence information for a phrase hyperedge we consider R ( g i )

Original: = 1 , ∀ i ∈ { 1 , ..., m } .
Revised: = 1 , ∀ i ∈ { 1 , ..., m } .

Original: Given that words are shared across various phrases, these common words connect the phrase hyperedges, resulting in a phrase hypergraph with overlapping phrase hyperedges.
Revised: Given that words are shared across various phrases, these common words connect the phrase hyperedges, resulting in a phrase hypergraph with overlapping phrase hyperedges.

Original: Again, we only consider hyperedges of cardinality ∈ [2 , 6] .
Revised: Again, we only consider hyperedges of cardinality ∈ [2 , 6] .

Original: After applying this cardinality ﬁlter we are left with 141, hyperedges (phrases) and 21,122 nodes (words).
Revised: After applying this cardinality ﬁlter we are left with 141, hyperedges (phrases) and 21,122 nodes (words).

Original: 4. E VALUATION M ETHODOLOGY AND E XPERIMENTAL S ETUP 4.2. M ETHODS C OMPARED
Revised: 4. E VALUATION M ETHODOLOGY AND E XPERIMENTAL S ETUP 4.2. M ETHODS C OMPARED

Original: As mentioned before we refer to our proposed methods: tensor based hypergraph tensor decomposition and deep auto-encoder based hypergraph auto-encoder , as t2v and h2v-auto , respectively.
Revised: As mentioned before we refer to our proposed methods: tensor based hypergraph tensor decomposition and deep auto-encoder based hypergraph auto-encoder , as t2v and h2v-auto , respectively.

Original: We compare our proposed methods against six baselines which are listed below.
Revised: We compare our proposed methods against six baselines which are listed below.

Original: Methods (1-2) are adapted from sequence based language models for set structured hypergraph data.
Revised: Methods (1-2) are adapted from sequence based language models for set structured hypergraph data.

Original: Methods (3-6) are various kind of dyadic graph based embedding methods adapted for hypergraph setting.
Revised: Methods (3-6) are various kind of dyadic graph based embedding methods adapted for hypergraph setting.

Original: h2v-DM: Hyperedge and node embeddings obtained using sentence embedding using distributed memory (DM) model (refer section A.1)
Revised: h2v-DM: Hyperedge and node embeddings obtained using sentence embedding using distributed memory (DM) model (refer section A.1)

Original: h2v-inv: Hyperedge and node embeddings obtained using node2vec on inverted hypergraph  (refer section A.2) 4.
Revised: h2v-inv: Hyperedge and node embeddings obtained using node2vec on inverted hypergraph and simple graph, respectively (refer section A.2) 4.

Original: h2v-dual: Hyperedge and node embeddings obtained using node2vec on hypergraph dual  (refer section A.2) 5.
Revised: h2v-dual: Hyperedge and node embeddings obtained using node2vec on hypergraph dual and the proxy graph for hypergraph, respectively (refer section A.2) 5.

Original: e2v: Hyperedge and node spectral embeddings obtained using eigen decomposition of inverted hypergraph laplacian and graph laplacian, respectively (refer section A.3) 6.
Revised: e2v: Hyperedge and node spectral embeddings obtained using eigen decomposition of inverted hypergraph laplacian and graph laplacian, respectively (refer section A.3) 6.

Original: e2v-hyp: Hyperedge and node spectral embeddings obtained using eigen decomposition of dual laplacian and hypergraph laplacian, respectively (refer section A.3)
Revised: e2v-hyp: Hyperedge and node spectral embeddings obtained using eigen decomposition of dual laplacian and hypergraph laplacian, respectively (refer section A.3)

Original: The details of these baseline can be found in Appendix A. Except for h2v-auto , each of these methods result in node as well as hyperedge embeddings of dimension d = 128 .
Revised: The details of these baseline can be found in Appendix A. Except for h2v-auto , each of the baselines as well as t2v outputs both node as well as hyperedge embeddings of dimension d = 128 .

Original: We further combine the node and hyperedge embedding using ﬁve different strategies: i) node embedding summation (dimension d = 128 ), ii) node embedding summation and concatenation with hyperedge embedding (dimension 2 × d = 256 ), iii) node embedding averaging (dimension d = 128 ), iv) node embedding averaging and concatenation with hyperedge embedding (dimension 2 × d = 256 ), and v) only hyperedge embedding (dimension d = 128 ).
Revised: We further combine the node and hyperedge embedding using ﬁve different strategies: (i) node embedding summation (dimension d = 128 ), (ii) node embedding summation and concatenation with hyperedge embedding (dimension 2 × d = 256 ), (iii) node embedding averaging (dimension d = 128 ), (iv) node embedding averaging and concatenation with hyperedge embedding (dimension 2 × d = 256 ), and (v) only hyperedge embedding (dimension d = 128 ).

Original: h2v-auto only produces hyperedge embeddings of dimension d = 128 .
Revised: h2v-auto only produces hyperedge embeddings of dimension d = 128 .

Original: But it builds the embeddings using three different scenarios as mentioned in next section.
Revised: But it builds the embeddings using three different scenarios as mentioned in next section.

Original: Therefore, in total we have 38 ( = 35 + 3 ) different scenarios each resulting in a different hyperedge embedding.
Revised: Therefore, in total we have 38 ( = 35+3 ) different scenarios each resulting in a different hyperedge embedding.

Original: 4.2. E VALUATION T ASKS AND S ETUP
Revised: 4.2. E VALUATION T ASKS AND S ETUP

Original: We perform two regression based tasks for the two datasets.
Revised: We perform two regression based tasks for the two datasets.

Original: In EQII dataset each team (hyperedge) has a team performance score associated with it.
Revised: In EQII dataset each team (hyperedge) has a team performance score associated with it.

Original: This team performance score is a real number, equal to the number of points earned by the team while performing one or more tasks within a gaming session.
Revised: This team performance score is a real number, equal to the number of points earned by the team while performing one or more tasks within a gaming session.

Original: We treat the embedding learned for a given team (hyperedge) as its feature vector which is associated with a real number (team performance).
Revised: We treat the embedding learned for a given team (hyperedge) as its feature vector which is associated with a real number (team performance).

Original: We therefore, perform on regression over all the hyperedges (teams) with team performance as the dependent variable.
Revised: We therefore, perform on regression over all the hyperedges (teams) with team performance as the dependent variable.

Original: Similarly, in LangNet dataset each phrase (hyperedge) has a sentiment score associated with it, which again is a real number.
Revised: Similarly, in LangNet dataset each phrase (hyperedge) has a sentiment score associated with it, which again is a real number.

Original: Similar to the team dataset above, we treat the embedding learned for a given phrase (hyperedge) as its feature vector which is associated with a real number (sentiment score).
Revised: Similar to the team dataset above, we treat the embedding learned for a given phrase (hyperedge) as its feature vector which is associated with a real number (sentiment score).

Original: We therefore, treat this as a regression task with sentiment score as the dependent variable and perform regression using the feature matrix containing embeddings of all the phrases.
Revised: We therefore, treat this as a regression task with sentiment score as the dependent variable and perform regression using the feature matrix containing embeddings of all the phrases.

Original: For both the tasks we just described, we perform several evaluation runs.
Revised: For both the tasks we just described, we perform several evaluation runs.

Original: In each run we randomly choose 30 % of hyperedges (teams or phrases) as the test set and learn ridge regression parameters using the remaining 70 % training hyperedges for each of the 38 different embedding scenarios.
Revised: In each run we randomly choose 30 % of hyperedges (teams or phrases) as the test set and learn ridge regression parameters using the remaining 70 % training hyperedges for each of the 38 different embedding scenarios.

Original: Root mean squared error (RMSE) was chosen as the evaluation metric (the lower, the better).
Revised: Root mean squared error (RMSE) was chosen as the evaluation metric (the lower, the better).

Original: RMSE was calculated for each of the 38 scenarios and for each run.
Revised: RMSE was calculated for each of the 38 scenarios and for each run.

Original: Final RMSE score was taken as the average RMSE score across ﬁve runs.
Revised: Final RMSE score was taken as the average RMSE score across ﬁve runs.

Original: Ridge regression’s hyper-parameter was chosen by 5-fold cross-validation.
Revised: Ridge regression’s hyper-parameter was chosen by 5-fold cross-validation.

Original: For the autoencoder method ( h2v-auto ) we consider three scenarios: (1) single hidden layer ( L 1 ) of d = 128 ; (2) two hidden layers ( L 1 & L 2 ) with size of L 1 : d = 96 and of L 2 : d = 32 .
Revised: For the auto-encoder method ( h2v-auto ) we consider three scenarios: (1) single hidden layer ( L 1 ) of d = 128 ; (2) two hidden layers ( L 1 & L 2 ) with size of L 1 : d = 96 and of L 2 : d = 32 .

Original: We concatenate these embedding to get a single d = 128 size embedding; and (3) two hidden layers ( L 1 & L 2 ) with size of L 1 : d = 512 and of L 2 : d = 128 .
Revised: We concatenate these embedding to get a single d = 128 size embedding; and (3) two hidden layers ( L 1 & L 2 ) with size of L 1 : d = 512 and of L 2 : d = 128 .

Original: We use the output of L 2 , which is of dimension d = 128 , as the embedding.
Revised: We use the output of L 2 , which is of dimension d = 128 , as the embedding.

Original: For sampling, we use the distance parameter h
Revised: For sampling, we use the distance parameter h

Original: =for generating the sub-lattice and τ = 0 .
Revised: =for generating the sub-lattice and τ = 0 .

Original: 2 for both datasets.
Revised: 2 for both datasets.

Original: Also, p = 10 & p = 5 number of hyperedges are sampled (corresponding to each hyperedge) from EQII and LangNet , respectively.
Revised: Also, p = 10 & p = 5 number of hyperedges are sampled (corresponding to each hyperedge) from EQII and LangNet , respectively.

Original: 4. R ESULTS AND D ISCUSSION
Revised: 4. R ESULTS AND D ISCUSSION

Original: Tables 1 &2 show the RMSE scores  for the tasks of team performance prediction and sentiment score prediction, respectively.
Revised: Tables 1 & 2 show the RMSE scores of t2v (as compared with baselines) for the tasks of team performance prediction and sentiment score prediction, respectively.

Original: These tables contain scores for all the 35 different scenarios: columns represent 7 ( 6 baselines and the proposed  t2v ) different models while rows represent combination strategies.
Revised: These tables contain scores for all the 35 different scenarios: columns represent 7 ( 6 baselines and the proposed hypergraph tensor decomposition ( t2v )) different models while rows represent combination strategies.

Original: The scores for Hypergraph Auto-encoder are shown in the separate Table 4 as autoencoder only generates hyperedge embeddings.[Table] [Table] [Table]
Revised: The scores for hypergraph auto-encoder ( h2v-auto ) are shown in the separate Table 4 as auto-encoder only generates hyperedge embeddings.

Original: [Table] As we can observe that for our datasets and for both the regression tasks [Table] almost all the embeddings  are performing very similarly in terms of accuracy.
Revised: Accuracy & Run-times As we can observe that for our datasets and for both the regression tasks, almost all the embeddings (baselines and proposed) are performing very similarly in terms of accuracy.

Original: However, we can observer in Table 3 that tensor based method ( t2v ) have signiﬁcant less time than the expensive auto-encoder technique ( h2v-auto ) (Table 4).
Revised: However, we can observer in Table 3 that tensor based method ( t2v ) have signiﬁcant less time than the expensive auto-encoder technique ( h2v-auto ) (Table 4).

Revised: Among the baselines e2v-hyp is the fastest.

Original: This makes tensor basedwell understood and interpretable decomposition techniques more attractive than the black-box type deep learning based auto-encoder method. Over all, we observe that e2v , e2v-hgraph and t2v , all of which are  matrix or tensor based algebraic techniques, perform comparable to state of the art embedding techniques from NLP.
Revised: We also observer that sentence based techniques run faster on text-based LangNet dataset as compared to node2vec based methods which are not designed for text data and vice-versa. Interpretability All the matrix or tensor based algebraic techniques: e2v , e2v-hyp and t2v , have a natural graph theoretic interpretation. e2v and e2v-hyp are both well studied spectral techniques with several eigen-value based interpretations.

Original: Therefore, it provides motivation to further explore algebraic techniques, which are easily interpretable in graph theoretic terms, as a reasonable alternative for learning representations. Another thing we notice, is that in case of LangNet dataset, that Node2Vec based h2v-inv and h2vdual methods are simply unable to run, as the A inv and A dual are quiet large (see N/A in Table 2).
Revised: t2v has a hierarchical hasse diagram based interpretation. In contrast the sentence and node2vec based techniques can be understood only intuitively in terms of the cost function. Apart from the noise which has a random walk based interpretation, h2v-auto is a deep-learning based method which exploits multiple level of non-linearity and is over all a black-box approach.

Original: It seems that in LangNet and possibly in text data in general the hypergraph dual, which contain phrase to phrase edges turns out be containing signiﬁcantly more edges than the number of node to node edges in A hyp .
Revised: Information Loss To reiterate one of the primary aims of this study is to design methods that retain the hyperedge-level joint information. The proposed tensor-based t2v principally capture the joint distribution over various cardinality hyperedges unlike conditional distribution like sentence embedding ( h2v-dm ) (which are more appropriate for sequences). In comparison the other method proposed h2v-auto , although are not directly designed to retain hyperedge-level joint distribution, but we hypothesize that the deep layered neural network should output highly informative nonlinear representations. Several other baselines only retain pair-wise information.

Original: Performing context generation over A inv and A dual graphs turns out be very costly and we were unable to get hyperedge embeddings for h2v-inv and h2v-dual methods.
Revised: For example, the methods developed using Node2Vec in Section A.2 are based on the skip-gram model, which learns embedding of nodes while maximizing the conditional probability of a node given another node in a context (Eq. 14).

Original: Note, we however do get vertex embedding (as mentioned with a * mark below Table 3).
Revised: Similarly, the spectral methods (Section A.3) are inherently two dimensional as they are based on matrix.

Original: But as you can see that even the time time taken for vertex embedding is similar or more than the total time taken by spectral methods for learning both hyperedge as well as vertex embedding, together.
Revised: Same is the case with skip-gram based sentence embedding ( h2v-dbow ). However, sentence embedding based on the DM architecture ( h2v-dm ) maximizes the conditional probability of a word given the previous set of words ( context ) without breaking this context (by concatenating or averaging the embedding of the previous words). Leveraging Hypergraph Topology is another feature that we stress as a desirable property in a method which aims to build hyperedge embedding. Hypergraph topology is an important auxiliary information, if left unused is wastage of resources at hand.

Original: This indicates the robustness of the Spectral methods, specially the e2v-hyp for different kinds of hypergraph data and edge densities (sparsity).
Revised: We observe that except the sentencebased baselines, both proposed methods as well as other baselines, which are adapted for the hypergraph setting, leverage hypergraph topology using various hypergraph representations. Ability to generate hyperedge embeddings directly is another critical aim as was highlighted in the introduction. We would like to highlight that none of the baselines are designed to give hyperedge embeddings in a principled manner. However, both tensor based ( t2v using A kdual ) as well as auto-encoder ( h2v-auto ) methods proposed in this paper are speciﬁcally designed to give hyperedge representations directly.

Original: Scalability Main scalability in the tensor approach arises with the increase in range of cardinality k .
Revised: Scalability Main scalability in the tensor approach arises with the increase in range of cardinality k .

Original: For a hyperedge of cardinality k there are k !
Revised: For a hyperedge of cardinality k there are k !

Original: entries to be ﬁlled in the tensor A khyp .
Revised: entries to be ﬁlled in the tensor A k hyp .

Original: Per iteration complexity of the algorithm is O
Revised: Per iteration complexity of the algorithm is O

Original: ( (cid:80) c
Revised: ( (cid:80) c

Original: max k = c
Revised: max k = c

Original: min nnz ( A khyp )) .
Revised: min nnz ( A khyp )) .

Original: Similarly, for a vertex of degree (i.e. number of hyperedges it is part of) δ there are δ !
Revised: Similarly, for a vertex of degree (i.e. number of hyperedges it is part of) δ there are δ !

Original: entries to be ﬁlled in the tensor A δ dual .
Revised: entries to be ﬁlled in the tensor A δ dual .

Original: Per iteration complexity of the algorithm is O ( (cid:80) δ max δ
Revised: Per iteration complexity of the algorithm is O ( (cid:80) δ max δ

Original: =
Revised: =

Original: δ min nnz ( A δ dual )) .
Revised: δ min nnz ( A δ dual )) .

Original: Now, as the k increases, the number of k cardinality hyperedge decrease, however the A khyp tensor grows exponentially making it increasingly sparse and less informative (in information theoretic sense).
Revised: Now, as the k increases, the number of k cardinality hyperedge decrease, however the A khyp tensor grows exponentially making it increasingly sparse and less informative (in information theoretic sense).

Original: Therefore, depending upon the cardinality distribution of a given dataset, we can choose a threshold value ( α ) for k beyond which we don’t enumerate the permutations.
Revised: Therefore, depending upon the cardinality distribution of a given dataset, we can choose a threshold value ( α ) for k beyond which we don’t enumerate the permutations.

Original: Moreover, for enumeration till k ≤ α in a scalable manner we propose to use hyperedge-centric distributed computation libraries like MESH (Heintz & Chandra, 2015; Heintz et al., 2017) and
Revised: Moreover, for enumeration till k ≤ α in a scalable manner we propose to use hyperedge-centric distributed computation libraries like MESH (Heintz & Chandra, 2015; Heintz et al., 2017) and

Original: HyperX (Huang et al., 2015), for distributed enumeration per hyperedge.
Revised: HyperX (Huang et al., 2015), for distributed enumeration per hyperedge.

Original: Similar argument goes for A δ dual where we ignore vertices with degree beyond a certain cut-off ( β ).
Revised: Similar argument goes for A δ dual where we ignore vertices with degree beyond a certain cut-off ( β ).

Original: These high degree vertices can be interpreted as highly popular person participating in a huge number of teams, but should the embeddings of these teams be related to each other is difﬁcult to infer.
Revised: These high degree vertices can be interpreted as highly popular person participating in a huge number of teams, but should the embeddings of these teams be related to each other is difﬁcult to infer.

Original: In text data these high degree nodes correspond to words occurring in a large number of phrases, but this doesn’t mean the sentences are related, rather its just that this word is quiet common.
Revised: In text data these high degree nodes correspond to words occurring in a large number of phrases, but this doesn’t mean the sentences are related, rather its just that this word is quiet common.

Original: We can again employ MESH or HyperX for scalability till δ ≤ β . 
Revised: We can again employ MESH or HyperX for scalability till δ ≤ β . [Table]

Original:  In case of auto-encoder method the main scalability challenge lies in generating the noisy hyperedges.
Revised: [Table] [Table] [Table] In case of auto-encoder method the main scalability challenge lies in generating the noisy hyperedges.

Original: For a given hyperedge the intermediate sub-lattice from which p samples are drawn is of size O ( n h ) , where h is max distance from the hyperedge considered.
Revised: For a given hyperedge the intermediate sub-lattice from which p samples are drawn is of size O ( n h ) , where h is max distance from the hyperedge considered.

Original: Once this sampling is performed we have total mp hyperedges as input for the auto-encoder, which is linear in the number of hyperedge as p will be a constant.
Revised: Once this sampling is performed we have total mp hyperedges as input for the auto-encoder, which is linear in the number of hyperedge as p will be a constant.

Original: The main challenge therefore, is to generate the noisy samples where we have to tackle exponential size sub-lattice per hyperedge.
Revised: The main challenge therefore, is to generate the noisy samples where we have to tackle exponential size sub-lattice per hyperedge.

Original: Again, as this sampling is done for each hyperedge separately (hyperedge-centric), we can therefore, use distributed computing for hypergraphs using MESH and HyperX, as mentioned before.
Revised: Again, as this sampling is done for each hyperedge separately (hyperedge-centric), we can therefore, use distributed computing for hypergraphs using MESH and HyperX, as mentioned before.

Original: Furthermore, we also observe that even simple element-wise summation or averaging of node embeddings for the nodes (in a given hyperedge) also perform comparably when compared to hyperedge embedding alone.
Revised: Furthermore, we also observe that even simple element-wise summation or averaging of node embeddings for the nodes (in a given hyperedge) also perform comparably when compared to hyperedge embedding alone.

Original: From this we can infer that depending upon the dataset, if we have less hyperedges and more nodes, than we would rather prefer to simply learn the hyperedge embedding directly rather than learning node embeddings and then performing aggregating operation over them.
Revised: From this we can infer that depending upon the dataset, if we have less hyperedges and more nodes, than we would rather prefer to simply learn the hyperedge embedding directly rather than learning node embeddings and then performing aggregating operation over them.

Original: Aggregation might turn out be costly specially if average hyperedge size is large and the choice of aggregation function is an issue.
Revised: Aggregation might turn out be costly specially if average hyperedge size is large and the choice of aggregation function is an issue.

Original: Therefore, learning hyperedge embeddings directly seems to be escape the problem of choosing the aggregation function all together.
Revised: Therefore, learning hyperedge embeddings directly seems to be escape the problem of choosing the aggregation function all together.

Revised: Another thing we notice, is that in case of LangNet dataset, that node2vec based h2v-inv and h2vdual methods are simply unable to run (see N/A in Table 2).

Revised: It seems that in LangNet (and possibly in text data in general) the hypergraph dual, A dual , which contain phrase to phrase edges turns out be containing signiﬁcantly more edges than the number of node to node edges in A hyp .

Revised: Therefore, performing context generation (using node2vec) over A inv and A dual graphs turns out be very costly and we were unable to get hyperedge embeddings for h2v-inv and h2v-dual methods.

Revised: Note, we however do get vertex embedding (as mentioned with a * mark below Table 3).

Revised: But as you can see that even the time time taken for vertex embedding is similar or more than the total time taken by spectral methods for learning both hyperedge as well as vertex embedding, together.

Original:  Choice of Method Both tensor based ( t2v using A kdual ) as well as auto-encoder ( h2v-auto ) methods proposed in this paper are able to generate embeddings for all hyperedges of various cardinalities present in data.
Revised: This indicates the robustness of the spectral methods, specially the e2v-hyp for different kinds of hypergraph data and edge densities (sparsity). Choice of Method Both tensor based ( t2v using A kdual ) as well as auto-encoder ( h2v-auto ) methods proposed in this paper are able to generate embeddings for all hyperedges of various cardinalities present in data.

Original: However, the auto-encoder method cannot generate node embeddings unlike tensor method (using A khyp ).
Revised: However, the auto-encoder method cannot generate node embeddings unlike tensor method (using A khyp ).

Original: On contrary, in auto-encoder technique we don’t need to ﬁgure out α and β parameters, and can work for all cardinality hyperedges.
Revised: On contrary, in auto-encoder technique we don’t need to ﬁgure out α and β parameters and can work for all cardinality hyperedges.

Original: Another, thing to note is that tensors are inherently capable of retaining joint information because of their higher-order structure.
Revised: Another, thing to note is that tensors are inherently capable of retaining joint information because of their higher-order structure.

Original: Auto-encoder method implicitly (possibly lossy manner) leverage the higher order information embedded in network in form of node weights ( R ( g i ) ) while performing the random walk sampling.
Revised: Auto-encoder method implicitly (possibly lossy manner) leverage the higher order information embedded in network in form of node weights ( R ( g i ) ) while performing the random walk sampling.

Original: Main advantage of auto-encoders is the crisp embedding achieved via multiple levels of non-linearity offered by deep neural networks.
Revised: Main advantage of auto-encoders is the crisp embedding achieved via multiple levels of non-linearity offered by deep neural networks.

Original: Although this non-linearity improves the accuracy of auto-encoder methods very slightly (Table 4) as compared to tensor method (bottom right corner of (Table 1 & 2), the computation cost of tensor is far less than the auto-encoder method (Table 3 & 4).
Revised: Although this non-linearity improves the accuracy of auto-encoder methods very slightly (Table 4) as compared to tensor method (bottom right corner of (Table 1 & 2), the computation cost of tensor is far less than the auto-encoder method (Table 3 & 4).

Original: Intuitively, it seems that depending upon the task at hand, in some task the retention of joint hyperedge-level information is more important for modeling and lesser presence of non-linear structure in the data; or vice versa.
Revised: Intuitively, it seems that depending upon the task at hand, in some task the retention of joint hyperedge-level information is more important for modeling and lesser presence of non-linear structure in the data; or vice versa.

Original: But the low computational cost (as reﬂected in our tasks) makes tensor method deﬁnitely more lucrative.
Revised: But the low computational cost (as reﬂected in our tasks) makes tensor method deﬁnitely more lucrative.

Original: Together with the natural graph theoretic interpretation, tensor methods in comparison to black-box approach of deep learning based auto-encoder, makes them even more viable.
Revised: Together with the natural graph theoretic interpretation, tensor methods in comparison to black-box approach of deep learning based auto-encoder, makes them even more viable.

Original: Information Loss Many methods that we have developed so far make use of only pair-wise information.

Original: For example, the methods developed using Node2Vec in Section A.2 are based on the skip-gram model, which learns embedding of nodes while maximizing the conditional probability of a node given another node in a context (Eq. 14).

Original: Similarly, the spectral methods (Section A.3) are inherently two dimensional as they are based on matrix.

Original: Same is the case with skip-gram based sentence embedding ( h2v-dbow ).

Original: However, sentence embedding based on the DM architecture ( h2v-dm ) maximizes the conditional probability of a word given the previous set of words ( context ) without breaking this context (by concatenating or averaging the embedding of the previous words).

Original: However, the tensors principally capture the joint distribution over various cardinality hyperedges unlike conditional distribution like sentence embedding ( h2v-dm ) (which are more appropriate for sequences).

Original: Improvements There are a few possible improvements which we can see in various approaches.
Revised: Improvements There are a few possible improvements which we can see in various approaches.

Original: In tensor approach when we extract a sub-hypergraph of a given k size hyperedges, it is possible that a signiﬁcant part of nodes are left out (as they might not be a part of any k size hyperedge).
Revised: In tensor approach when we extract a sub-hypergraph of a given k size hyperedges, it is possible that a signiﬁcant part of nodes are left out (as they might not be a part of any k size hyperedge).

Original: This can possibly cause a cold start problem in the corresponding tensor decomposition.
Revised: This can possibly cause a cold start problem in the corresponding tensor decomposition.

Original: However, this does not mean that we will not get embedding for those nodes.
Revised: However, this does not mean that we will not get embedding for those nodes.

Original: It simply means that we might not achieve embeddings for these nodes from the decomposition of k -way tensors.
Revised: It simply means that we might not achieve embeddings for these nodes from the decomposition of k -way tensors.

Original: But we can get embedding for these nodes from other tensors for a different k .
Revised: But we can get embedding for these nodes from other tensors for a different k .

Original: However, we can mitigate the cold start problem in a principled manner using a graph regularization term ( k U ( j ) T L k U ( j ) ) in our tensor decomposition objective function 3.
Revised: However, we can mitigate the cold start problem in a principled manner using a graph regularization term ( k U ( j ) T L k U ( j ) ) in our tensor decomposition objective function 3.

Original: Here, L is the graph laplacian of our choice.
Revised: Here, L is the graph laplacian of our choice.

Original: For example we can use L hyp or L graph when we perform tensor decomposition for node embeddings and use L inv or L dual for hyperedge embeddings.
Revised: For example we can use L hyp or L graph when we perform tensor decomposition for node embeddings and use L inv or L dual for hyperedge embeddings.

Original: Lastly, the hypergraph autoencoder ( h2v-auto ) can only produce hyperedge embeddings and not the node embeddings.
Revised: Lastly, the hypergraph autoencoder ( h2v-auto ) can only produce hyperedge embeddings and not the node embeddings.

Original: In this sense it is feeble to other methods.
Revised: In this sense it is feeble to other methods.

Original: Highlight Another distinguishing feature of our work is that we leverage the existing structure (group structure) within the data.
Revised: Highlight Another distinguishing feature of our work is that we leverage the existing structure (group structure) within the data.

Original: Recent attempts along these lines in network literature (Grover &
Revised: Recent attempts along these lines in network literature (Grover &

Original: Leskovec, 2016; Tang et al., 2015b; Perozzi et al., 2014) argue that language models have readymade context in the form of sentences or paragraphs to train the model, which are not available in networks, and therefore, they propose different ways to generate this context.
Revised: Leskovec, 2016; Tang et al., 2015b; Perozzi et al., 2014) argue that language models have readymade context in the form of sentences or paragraphs to train the model, which are not available in networks, and therefore, they propose different ways to generate this context.

Original: In contrast, we focus on networks where the context is already present, e.g. collaboration networks where collaborative teams are hyperedges, or language hyper-networks where sentences are hyperedges.
Revised: In contrast, we focus on networks where the context is already present, e.g. collaboration networks where collaborative teams are hyperedges, or language hyper-networks where sentences are hyperedges.

Original:  R ELATED W ORKS
Revised:  R ELATED W ORKS

Original: Hypergraphs were studied rigorously by Berge (1976; 1984) as a generalization of graphs and directed hypergraphs have been introduced by Bretto (2013).
Revised: Hypergraphs were studied rigorously by Berge (1976; 1984) as a generalization of graphs and directed hypergraphs have been introduced by Bretto (2013).

Original: Hypergraph were argued for the ﬁrst time as a model to naturally capture higher-order relationships between sets of objects across variety of domains by Estrada & Rodriguez-Velazquez (2005).
Revised: Hypergraph were argued for the ﬁrst time as a model to naturally capture higher-order relationships between sets of objects across variety of domains by Estrada & Rodriguez-Velazquez (2005).

Original: Hypergraphs have been used to model complex networks in different ﬁelds including biology (Klamt et al., 2009), databases (Fagin, 1983) and data mining (Han et al., 1997;
Revised: Hypergraphs have been used to model complex networks in different ﬁelds including biology (Klamt et al., 2009), databases (Fagin, 1983) and data mining (Han et al., 1997;

Original: Zhou et al., 2007).
Revised: Zhou et al., 2007).

Original: Within machine learning, algorithms guided by the structure of hypergraph were introduced by Zhou et al.
Revised: Within machine learning, algorithms guided by the structure of hypergraph were introduced by Zhou et al.

Original: and have found applications in a variety of domains (Tian et al., 2009; Gao et al., 2013; Li & Li, 2013; Sharma et al., 2015; 2017).
Revised: and have found applications in a variety of domains (Tian et al., 2009; Gao et al., 2013; Li & Li, 2013; Sharma et al., 2015; 2017).

Original: Simplicial complex (Munkres, 1984) based view of hypergraph using hasse lattice (Skiena, 1990) within machine learning has recently been proposed by Sharma et al.
Revised: Simplicial complex (Munkres, 1984) based view of hypergraph using hasse lattice (Skiena, 1990) within machine learning has recently been proposed by Sharma et al.

Original: Representation learning (RL) (Bengio et al., 2013) focuses on learning features (geometry) of the data (topology).
Revised: Representation learning (RL) (Bengio et al., 2013) focuses on learning features (geometry) of the data (topology).

Original: Machine learning algorithms make use of these features for prediction.
Revised: Machine learning algorithms make use of these features for prediction.

Original: Traditionally these features are readily available within the data-set or are engineered manually.
Revised: Traditionally these features are readily available within the data-set or are engineered manually.

Original: However, this is often a tedious and human labor-intensive process.
Revised: However, this is often a tedious and human labor-intensive process.

Original: RL addresses this issue by learning features automatically in a task-dependent supervised or a task-independent unsupervised manner.
Revised: RL addresses this issue by learning features automatically in a task-dependent supervised or a task-independent unsupervised manner.

Original: Node Representations in Graph: Traditionally unsupervised node embedding learning has been dong using latent models like matrix factorization (Ahmed et al., 2013) or by community detection (Tang & Liu, 2011)
Revised: Node Representations in Graph: Traditionally unsupervised node embedding learning has been dong using latent models like matrix factorization (Ahmed et al., 2013) or by community detection (Tang & Liu, 2011)

Original: based techniques for networks (Roweis & Saul, 2000; Belkin & Niyogi, 2001;
Revised: based techniques for networks (Roweis & Saul, 2000; Belkin & Niyogi, 2001;

Original: Tenenbaum et al., 2000; Cox & Cox, 2000).
Revised: Tenenbaum et al., 2000; Cox & Cox, 2000).

Original: In each case there is a vector of features learned for a node, each of whose entries reﬂects node’s association with some latent dimension or a network community.
Revised: In each case there is a vector of features learned for a node, each of whose entries reﬂects node’s association with some latent dimension or a network community.

Original: More recently, there has been a revived interest in graph embedding in form of context oriented techniques (Perozzi et al., 2014; Tang et al., 2015b; Grover & Leskovec, 2016).
Revised: More recently, there has been a revived interest in graph embedding in form of context oriented techniques (Perozzi et al., 2014; Tang et al., 2015b; Grover & Leskovec, 2016).

Original: These techniques are inspired by recent unsupervised RL methods in NLP (Mikolov et al., 2013b; Le &
Revised: These techniques are inspired by recent unsupervised RL methods in NLP (Mikolov et al., 2013b; Le &

Original: Mikolov, 2014) where word embeddings are learned that are similar to words in a given neighborhood or context .
Revised: Mikolov, 2014) where word embeddings are learned that are similar to words in a given neighborhood or context .

Original: These techniques differ in the manner they generate this context as well as in the objective which they optimize.
Revised: These techniques differ in the manner they generate this context as well as in the objective which they optimize.

Original: Also there are supervised algorithms learn embeddings which are optimal for the speciﬁc task at hand.
Revised: Also there are supervised algorithms learn embeddings which are optimal for the speciﬁc task at hand.

Original: This results in high accuracy but incurs signiﬁcant computational cost for training.
Revised: This results in high accuracy but incurs signiﬁcant computational cost for training.

Original: Recently, several supervised learning algorithms have been proposed for network analysis (Tian et al., 2014; Xiaoyi et al., 2013) and for text networks in a semi-supervised setting (Tang et al., 2015a).
Revised: Recently, several supervised learning algorithms have been proposed for network analysis (Tian et al., 2014; Xiaoyi et al., 2013) and for text networks in a semi-supervised setting (Tang et al., 2015a).

Original: Finally, we refer readers to a very recent and comprehensive survey on graph embedding methods by Cai et al.
Revised: Finally, we refer readers to a very recent and comprehensive survey on graph embedding methods by Cai et al.

Original: Node Representations in Hypergraph: Learning embeddings for nodes within a hypergraph while incorporating the hypergraph topology using proxy graphs is introduced by Zhou et al.
Revised: Node Representations in Hypergraph: Learning embeddings for nodes within a hypergraph while incorporating the hypergraph topology using proxy graphs is introduced by Zhou et al.

Original: Using graph proxy destroys the hypredge-level joint information and thus, incur loss of information.
Revised: Using graph proxy destroys the hypredge-level joint information and thus, incur loss of information.

Original: Also,
Revised: Also,

Original: Agarwal et al.
Revised: Agarwal et al.

Original: (2006) squarely criticize that such representations can be learned by constructing graphs, which are proxies for the hypergraph structure.
Revised: (2006) squarely criticize that such representations can be learned by constructing graphs, which are proxies for the hypergraph structure.

Original: Set Representations: RL for sets using neural networks has been proposed recently (Vinyals et al., 2016), where a memory network is used to compose features sequentially but in an order invariant manner.
Revised: Set Representations: RL for sets using neural networks has been proposed recently (Vinyals et al., 2016), where a memory network is used to compose features sequentially but in an order invariant manner.

Original: In their very recent paper, Rezatoﬁghi et al.
Revised: In their very recent paper, Rezatoﬁghi et al.

Original: (2016) have tried to answer this set ordering issue by the use of random set theory .
Revised: (2016) have tried to answer this set ordering issue by the use of random set theory .

Original: However, they do not consider embedding but focus on learning set-level probabilities.
Revised: However, they do not consider embedding but focus on learning set-level probabilities.

Original: More importantly, both of these works, do not consider the hypergraph structure of overlapping sets which is the main focus of this paper.
Revised: More importantly, both of these works, do not consider the hypergraph structure of overlapping sets which is the main focus of this paper.

Original: Tensors For comprehensive view of tensors, tensor decomposition as well as applications we refer to the survey by Kolda & Bader (2009).
Revised: Tensors For comprehensive view of tensors, tensor decomposition as well as applications we refer to the survey by Kolda & Bader (2009).

Original: The connection between k -way tensor and k-uniform hypergraph eigen values was established by Qi (2005).
Revised: The connection between k -way tensor and k-uniform hypergraph eigen values was established by Qi (2005).

Original: The use of k -way symmetric tensor and their non-negative decomposition for uniform hypergraph partitioning was ﬁrst introduced by Shashua et al.
Revised: The use of k -way symmetric tensor and their non-negative decomposition for uniform hypergraph partitioning was ﬁrst introduced by Shashua et al.

Original: But they are again restricted to uniform hypergraphs.
Revised: But they are again restricted to uniform hypergraphs.

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: In this paper we have proposed two methods to generate higher-order representations for both hyperedges (representing sets of nodes) and hypergraph nodes (that also take into account the hypergraph structure).
Revised: In this paper we have proposed two methods to generate higher-order representations for both hyperedges (representing sets of nodes) and hypergraph nodes (that also take into account the hypergraph structure).

Original: First, is an auto-encoder based deep-learning method and second, is a tensor-based algebraic method.
Revised: First, is an auto-encoder based deep-learning method and second, is a tensor-based algebraic method.

Original: Both learning models are unique in the manner they leverage the existing structure present in network data as context.
Revised: Both learning models are unique in the manner they leverage the existing structure present in network data as context.

Original: While introducing a new idea of a dual tensors corresponding to the hypergraph dual, we develop a novel approach of using factors from joint decomposition of k -way tensors corresponding to k -uniform sub-hypergraphs, as generic node & hyperedge representations.
Revised: While introducing a new idea of a dual tensors corresponding to the hypergraph dual, we develop a novel approach of using factors from joint decomposition of k -way tensors corresponding to k -uniform sub-hypergraphs, as generic node & hyperedge representations.

Original: We show that that both methods perform comparably with several other baselines in terms of accuracy.
Revised: We show that that both methods perform comparably with several other baselines in terms of accuracy.

Original: We also observe that the proposed tensor based methods are more efﬁcient and also have a natural hypergraph theoretic interpretation; unlike deep learning based black-box approach.
Revised: We also observe that the proposed tensor based methods are more efﬁcient and also have a natural hypergraph theoretic interpretation; unlike deep learning based black-box approach.

Original: We therefore, argue that we have proposed more general methods which are principally suited for hypergraphs (and therefore also for graphs) while maintaining accuracy and efﬁciency.
Revised: We therefore, argue that we have proposed more general methods which are principally suited for hypergraphs (and therefore also for graphs) while maintaining accuracy and efﬁciency.

Original: A. H YPEREDGE 2 VEC U SING N ODE 2V EC
Revised: A. H YPEREDGE 2 VEC U SING N ODE 2V EC

Original: Grover & Leskovec (2016) propose a representation method for nodes in a graph called Node2Vec, which uses the skip-gram model Mikolov et al.
Revised: Grover & Leskovec (2016) propose a representation method for nodes in a graph called node2vec , which uses the skip-gram model Mikolov et al.

Original: (2013b) with the intuition that neighborhood nodes within a graph should have similar representations.
Revised: (2013b) with the intuition that neighborhood nodes within a graph should have similar representations.

Original: The objective of the skip-gram model for graphs can be deﬁned as:
Revised: The objective of the skip-gram model for graphs can be deﬁned as:

Original: [Equation]
Revised: [Equation]

Original: where as before, φ and ω denote the input and the output vector representations of the nodes.
Revised: where as before, φ and ω denote the input and the output vector representations of the nodes.

Original: The neighboring nodes N ( v ) form the context for node v .
Revised: The neighboring nodes N ( v ) form the context for node v .

Original: Node2Vec uses a biased random walk which adaptively combines breadth ﬁrst search (BFS) and depth ﬁrst search (DFS) to ﬁnd the neighborhood of a node.
Revised: node2vec uses a biased random walk which adaptively combines breadth ﬁrst search (BFS) and depth ﬁrst search (DFS) to ﬁnd the neighborhood of a node.

Original: The walk attempts to capture two properties of a graph often used for prediction tasks in networks: i) homophily and ii) structural equivalence.
Revised: The walk attempts to capture two properties of a graph often used for prediction tasks in networks: (i) homophily and (ii) structural equivalence.

Original: According to homophily, nodes in the same group or community should have similar representations.
Revised: According to homophily, nodes in the same group or community should have similar representations.

Original: Structural equivalence suggests that nodes with similar structural roles (hub, bridge) should have similar representations.
Revised: Structural equivalence suggests that nodes with similar structural roles (hub, bridge) should have similar representations.

Original: In a real-world network, nodes exhibit mixed properties.
Revised: In a real-world network, nodes exhibit mixed properties.

Original: Our aim is to ﬁnd both node and hyperedge level embeddings by taking into account the hypergraph structure.
Revised: Our aim is to ﬁnd both node and hyperedge level embeddings by taking into account the hypergraph structure.

Original: For the former, we leverage the adjacency matrix associated with a hypergraph Zhou et al.
Revised: For the former, we leverage the adjacency matrix associated with a hypergraph Zhou et al.

Original: (2006), which is deﬁned as:
Revised: (2006), which is deﬁned as:

Original: [Equation]
Revised: [Equation]

Original: (15) where W e is a diagonal matrix containing the weights of each hyperedge and D v is a diagonal matrix containing the degree of each vertex.
Revised: (15) where W e is a diagonal matrix containing the weights of each hyperedge and D v is a diagonal matrix containing the degree of each vertex.

Original: We take the weight of a hyperedge as its occurrence number, i.e. W e ( i, i ) =
Revised: We take the weight of a hyperedge as its occurrence number, i.e. W e ( i, i ) =

Original: R ( g i ) , ∀{ i ∈ { 1 , ..., n }} .
Revised: R ( g i ) , ∀{ i ∈ { 1 , ..., n }} .

Original: The adjacency matrix ( A hyp ∈ R | V |×| V | ) associates a weight between a pair of nodes while taking into account the weights of all the hyperedges that encompass a pair of nodes.
Revised: The adjacency matrix ( A hyp ∈ R | V |×| V | ) associates a weight between a pair of nodes while taking into account the weights of all the hyperedges that encompass a pair of nodes.

Original: The weighted graph associated with A hyp in some sense serves as a proxy to the actual hypergraph structure.
Revised: The weighted graph associated with A hyp in some sense serves as a proxy to the actual hypergraph structure.

Original: We can provide A hyp as input to Node2Vec and hypothesize that the random walk over this proxy hypergraph should allow the skip-gram model to learn more meaningful node level embeddings which can be combined to construct hyperedge level embeddings.
Revised: We can provide A hyp as input to node2vec and hypothesize that the random walk over this proxy hypergraph should allow the skip-gram model to learn more meaningful node level embeddings which can be combined to construct hyperedge level embeddings.

Original: We refer to these node level embeddings as N2V-hyp .
Revised: We refer to these node level embeddings as N2V-hyp .

Original: We still have not met our second objective to learn hyperedge embeddings directly.
Revised: We still have not met our second objective to learn hyperedge embeddings directly.

Original: We again wish to leverage Node2Vec for our purpose.
Revised: We again wish to leverage node2vec for our purpose.

Original: However, Node2Vec only works for graphs by performing random walk over nodes.
Revised: However, node2vec only works for graphs by performing random walk over nodes.

Original: Therefore, we ask ourselves the question, that can we treat hyperedges as nodes?
Revised: Therefore, we ask ourselves the question, that can we treat hyperedges as nodes?

Original: There can be other ways of doing so, but here we suggest two techniques.
Revised: There can be other ways of doing so, but here we suggest two techniques.

Original: In the ﬁrst technique, we simply consider the hypergraph dual Berge (1984), whose incidence matrix is
Revised: In the ﬁrst technique, we simply consider the hypergraph dual Berge (1984), whose incidence matrix is

Original: H dual = H T .
Revised: H dual = H T .

Original: The adjacency matrix associated with the hypergraph dual is:
Revised: The adjacency matrix associated with the hypergraph dual is:

Original: [Equation]
Revised: [Equation]

Original: where W v is a diagonal matrix containing the weights of each node and D e is a diagonal matrix containing the degree of each hyperedge.
Revised: where W v is a diagonal matrix containing the weights of each node and D e is a diagonal matrix containing the degree of each hyperedge.

Original: We assume no weights on the nodes and take W v = I .
Revised: We assume no weights on the nodes and take W v = I .

Original: The matrix A dual ∈ R | G |×| G | represents another hypergraph, but the roles of nodes and hyperedges have now switched.
Revised: The matrix A dual ∈ R | G |×| G | represents another hypergraph, but the roles of nodes and hyperedges have now switched.

Original: For example, in case of words and sentences, the hyperedges were sentences, but in the hypergraph dual the words become hyperedges and the nodes within a word’s hyperedge represent all the sentences in which the word has appeared.
Revised: For example, in case of words and sentences, the hyperedges were sentences, but in the hypergraph dual the words become hyperedges and the nodes within a word’s hyperedge represent all the sentences in which the word has appeared.

Original: We again give A dual (i.e., a graph proxy for the dual hypergraph) as input to Node2Vec, but this time we get output as the embeddings for the hyperedges associated with the nodes in dual.
Revised: We again give A dual (i.e., a graph proxy for the dual hypergraph) as input to node2vec, but this time we get output as the embeddings for the hyperedges associated with the nodes in dual.

Original: We refer to these hyperedge embeddings as h2v-dual .
Revised: We refer to these hyperedge embeddings as h2v-dual .

Original: In the second technique, we consider the following adjacency matrix A inv ∈ R | G |×| G | :
Revised: In the second technique, we consider the following adjacency matrix A inv ∈ R | G |×| G | :

Original: [Equation]
Revised: [Equation]

Original: (17) associated with what we refer to as the inverted hypergraph .
Revised: 17) associated with what we refer to as the inverted hypergraph .

Original: This inverted hypergraph is a graph (unlike the dual which is a hypergraph) and there is an edge between two nodes if the hyperedges corresponding to the nodes in the original hypergraph have nodes in common.
Revised: This inverted hypergraph is a graph (unlike the dual which is a hypergraph) and there is an edge between two nodes if the hyperedges corresponding to the nodes in the original hypergraph have nodes in common.

Original: Weight of this edge is the number of common nodes.
Revised: Weight of this edge is the number of common nodes.

Original: We again give A inv as input to Node2Vec to get embeddings for the hyperedges associated with the nodes in inverted hypergraph.
Revised: We again give A inv as input to node2vec to get embeddings for the hyperedges associated with the nodes in inverted hypergraph.

Original: We refer to these hyperedge embeddings as h2v-inv .
Revised: We refer to these hyperedge embeddings as h2v-inv .

Original: A. H YPEREDGE 2 VEC U SING S PECTRAL E MBEDDINGS
Revised: A. H YPEREDGE 2 VEC U SING S PECTRAL E MBEDDINGS

Original: These set of methods extract embeddings as the eigenvectors associated with Laplacian matrices corresponding to the various adjacency matrices discussed in the previous section.
Revised: These set of methods extract embeddings as the eigenvectors associated with Laplacian matrices corresponding to the various adjacency matrices discussed in the previous section.

Original: We consider the following graph Laplacians:
Revised: We consider the following graph Laplacians:

Original: [Equation]

Original: [Equation]
Revised: [Equation]

Original: where A graph = H T W e H is the weighted graph associated with the graph corresponding to adjacency matrix A ,
Revised: where A graph = H T W e H is the weighted graph associated with the graph corresponding to adjacency matrix A ,

Original: [Equation]
Revised: [Equation]

Original: We get the d eigenvectors associated with the smallest d eigenvalues of the above graph Laplacians as the embeddings.
Revised: We get the d eigenvectors associated with the smallest d eigenvalues of the above graph Laplacians as the embeddings.

Original: We get vertex embeddings using Eq.
Revised: We get vertex embeddings using Eq.

Original: 19 and hyperedge embedding using Eq.
Revised: 19 and hyperedge embedding using Eq.

Original: 21, and we refer to them together as e2v-hyp .
Revised: 21, and we refer to them together as e2v-hyp .

Original: Similarly, we get another pair of vertex embeddings (using Eq. 18) and hyperedge embedding (using Eq. 21).
Revised: Similarly, we get another pair of vertex embeddings (using Eq. 18) and hyperedge embedding (using Eq. 21).

Original: We refer to the later pair as e2v .
Revised: We refer to the later pair as e2v .