# Hypernetwork Representation Learning with the Set Constraint
## 5 Our Method
### Topology-Derived Objective Function  
Continuous Bag-of-Words [22], abbreviated as CBOW, is a popular distributed word representation method whose computational efficiency is higher than that of Skip-Gram. Hence, in order to improve the computational efficiency of node representation vectors, following the idea of CBOW, with such an assumption that the nodes with similar topology context tend to be similar, we aimed to maximize the prediction probabilities of the center node given its contextual nodes, which are defined as the previous nodes and after nodes of the center node with a fixed-size window in the node sequences generated by random walk. We propose a novel negative sampling-based topology-derived objective function in which the node representation vectors are regarded as the parameters. In the optimization procedure, the representation vectors are updated and the finally learned vectors preserve the hypernetwork topology structure.  
A set of node sequences \(C\) is generated by the same random walk on the traditional network abstracted as 2-section graph as Deepwalk. For each node, a node sequence of  
Figure 1: The hypergraph and two-section graph of its transformation. (**a**) Hypergraph and (**b**) two-section graph.  
length \(t\) to start from this node is generated in the random walk iteration, which will repeat \(\tau\) times to generate enough sequences.  
Under the condition of a center node \(n\) and its contextual nodes \(context(n)\), the node \(n\) is regarded as the positive sample, other nodes are the negative samples, and \(NEG(n)\) is the subset of negative samples of the center node \(n\) with a predefined size \(ds\). For \(\forall\ u\in V\), the labels of the node are defined as follows:  
\[L^{n}(u)=\left\{\begin{array}{ll}1,&u\in\{n\}\\ 0,&u\in NEG(n)\end{array}\right. \tag{1}\]  
\(p(u|context(n))\) denotes the prediction probability of the node \(u\) under the condition of the contextual nodes \(context(n)\). On the basis of the node sequences \(C\), we try to maximize the following objective function.  
\[D_{1}=\prod_{n\in C}\prod_{u\in\{\{n\}\cup NEG(n)\}}p(u|context(n)) \tag{2}\]  
As for each node \(n\), this paper designs the embedding vector and the parameter vector, where the embedding vector \(\mathbf{v}_{n}\) is the representation of the node \(n\) when it is treated as the contextual node, while the parameter vector \(\mathbf{\theta}_{n}\) is the representation of \(n\) when it is treated as the center node. \(p(u|context(n))\) in Formula (2) is defined as follows.  
\[p(u|context(n))=\left\{\begin{array}{ll}\sigma(X_{n}^{T}\mathbf{\theta}_{u}),&L ^{n}(u)=1\\ 1-\sigma(X_{n}^{T}\mathbf{\theta}_{u}),&L^{n}(u)=0\end{array}\right. \tag{3}\]  
where \(\sigma(X_{n}^{T}\mathbf{\theta}_{u})=\frac{1}{1+e^{-X_{n}^{T}\mathbf{\theta}_{u}}}\) is a sigmoid function, where \(X_{n}\) is the summing operation of the representation vectors corresponding to \(context(n)\). Formula (3) can also be rewritten as an integral expression.  
\[p(u|context(n))=\left[\sigma(X_{n}^{T}\mathbf{\theta}_{u})\right]^{L^{n}(u)} \cdot\left[1-\sigma(X_{n}^{T}\mathbf{\theta}_{u})\right]^{1-L^{n}(u)} \tag{4}\]  
Accordingly, Formula (2) can be rewritten as follows.  
\[D_{1}=\prod_{n\in C}\prod_{u\in\{\{n\}\cup NEG(n)\}}\left\{\left[\sigma(X_{n }^{T}\mathbf{\theta}_{u})\right]^{L^{n}(u)}\cdot\left[1-\sigma(X_{n}^{T}\mathbf{ \theta}_{u})\right]^{1-L^{n}(u)}\right\} \tag{5}\]  
Formally, maximizing \(D_{1}\) means maximizing the prediction probabilities of positive samples and minimizing the prediction probabilities of negative samples simultaneously, by which the hypernetwork topology structure is encoded into the node representation vectors.