SlowGuess/NeurIPS_2023_OCR huggingface dataset:
It can be rewritten as  $\frac{1}{T} \left( \left( \frac{k p_{n}(u|v)}{p(u|v)} \right)^{2} \frac{1}{p(u|v)} + \frac{k p_{n}(u|v)}{p(u|v)} \frac{1}{p(u|v)} \right)$ . Since  $p(u|v)$  is among the largest proximity scores,  $\frac{1}{p(u|v)}$  is approximately bounded by a finite number  $\frac{1}{n}$ , where  $n$  is the number of nodes in the network. As a result,  $\frac{k p_{n}(u|v)}{p(u|v)} \frac{1}{p(u|v)} \to 0$  and  $(\frac{k p_{n}(u|v)}{p(u|v)})^{2} \frac{1}{p(u|v)} \to 0$ . Therefore, for nodes with large  $p(u|v)$ , the empirical embedding similarity score  $s_{e}(u,v) \approx s^{*}(u,v)$ . With a similar analysis on nodes with small  $p(u|v)$ , we reach the same conclusion that  $s_{e}(u,v) \approx s^{*}(u,v)$ . Based on the above analysis, we can see that the empirical optimal solution can achieve the discrimination property with a negatively correlated  $p_{n}$ : for node pairs with large/small  $p(u|v)$ , their similarity scores approach the ideal optimal solution. However, there might exist some nodes with  $p(u|v) \approx k p_{n}(u|v)$  which are referred to as nodes with intermediate  $p(u|v)$ . For these nodes, the error in Theorem 2 is larger than nodes with small/large  $p(u|v)$  because  $(2 + \frac{k p_{n}(u|v)}{p(u|v)} + \frac{p(u|v)}{k p_{n}(u|v)})^{2}$  achieves its minimum when  $p(u|v) = k p_{n}(u|v)$ . This can also be explained intuitively as follows. Nodes with large  $p(u|v)$  and large  $p_{n}(u|v)$  (small  $p(u|v)$ ) are always sampled as positive samples and negative samples, while nodes with intermediate  $p(u|v)$  and  $p_{n}(u|v)$  might be ignored and are less likely to be sampled as positive samples or negative samples. As such, the embedding similarity scores for these nodes bear more uncertainty and large error between the empirical optimal solution and the ideal optimal solution (Theorem 2).
With these findings, we can now unveil the underlying reason for the competing sampling strategies in existing methods. The first category of methods (e.g., [36, 14, 23]) champion sampling nodes with longer distance from the given central node as positive samples. The essence of this positive sampling strategy is to include more nodes with intermediate  $p(u|v)$  in the positively sampled node pair set. In this way, these methods realize the goal to minimize the error in Theorem 2 for these nodes with intermediate  $p(u|v)$ . However, this design has cost. When choosing more nodes with intermediate  $p(u|v)$  as positive samples, it implicitly decreases the  $p(u|v)$  for nodes with large  $p(u|v)$  (since  $\sum_{u} p(u|v) = 1$ ). As shown on the second line in Figure 1, it hurts the monotonicity between these nodes (round nodes) and nodes with large  $p(u|v)$  (star nodes). On the contrary, for the second category of methods (e.g., [4, 56, 55]), they prefer to select nodes with intermediate  $p(u|v)$  as difficult negative samples in the recommendation task, they successfully fulfill the monotonicity property between nodes with large  $p(u|v)$  (star nodes) and nodes with intermediate  $p(u|v)$  (round nodes). Unfortunately, as our discussion about Eq. (8) has demonstrated, this positively correlated negative sampling distribution could make the empirical error of the embedding similarity scores of nodes with small  $p(u|v)$  quite large. It does harm the discrimination property, which is shown on the last line in Figure 1 (e.g.,  $s(u_8,v)$ ).
To conclude, due to the limited number  $T$  of positive samples, the empirical loss tends to sample nodes with large  $p(u|v)$  or  $p_n(u|v)$  and there always exist some nodes that can not be sampled as positive/negative samples and thus are ignored. Therefore, the discrimination property and the monotonicity property for all node pairs in the network can not be satisfied simultaneously in the empirical situation.
In many real-world applications, given the query node, we primarily care about the top- $K$  recommendation list. This suggests that we only need to satisfy the discrimination property and the partial monotonicity property within nodes that are likely to appear in the top- $K$  recommendation list. Guided by this and the theoretical results in Section 2, we propose a simple yet novel model named SENSEI.
Key Idea. The key idea of SENSEI is to seamlessly integrate two competing sampling strategies into one model together. Concretely, it means that we can decompose the proposed SENSEI into two steps. The first step is to satisfy the discrimination property and the second step is to achieve the monotonicity property within nodes that are likely to appear in the top-  $K$  recommendation list. As shown in Figure 2, in the first step, SENSEI samples nodes with intermediate  $p(u|v)$ , which is the strategy by the first category of methods mentioned in Section 2. SENSEI constructs a combined positive sample set, including nodes with large  $p(u|v)$  and nodes with intermediate  $p(u|v)$ . During the training process, SENSEI treats these nodes equivalently as positive samples and maximizes their similarity scores with the given central node. At the same time, it minimizes the similarity scores for nodes with small  $p(u|v)$  (large  $p_n(u|v)$ ) to fulfill the discrimination property. In the second step, SENSEI focuses on the difficult negative samples, which is the strategy by the second category of methods in Section 2. SENSEI pays attention to the monotonicity property within the combined positive sample set in the first step. In detail, the previously sampled positive nodes have two roles: they act as the positive samples compared to nodes with smaller  $p(u|v)$  in the first step and meanwhile function as the difficult negative samples compared to nodes with larger  $p(u|v)$  in the second step.
Details. For  $p(u|v)$ , we run personalized pagerank [33], which is used throughout SENSEI. In addition, we normalize node embeddings to have unit L2 norm and adopt the dot product  $(\mathbf{F}(u,:)^{\top}\mathbf{F}(v,:))$  as the similarity function  $s(u,v)$ .
Step 1: Fulfill the Discrimination Property. In this step, to satisfy the discrimination property, we construct a combined positive sample set  $\mathcal{P}(v)$  for  $v$  in  $\mathbf{A}$ , which includes nodes with large and intermediate  $p(u|v)$ . Specifically, we rank the proximity  $p(u|v)$  for all nodes in  $\mathbf{A}$  to obtain a descending list  $[p(u_1|v), p(u_2|v), \ldots, p(u_n|v)]$ . Then, we set a threshold  $\tau$ . Nodes with  $p(u|v) > \tau$  form the combined positive sample set  $\mathcal{P}(v)$  for the central node  $v$  and the remaining nodes are added into the set  $\mathcal{N}(v)$  where negative samples will be randomly selected. So, the loss for node  $v$  is:
Figure 2: The training process of SENSEI. Similar to Figure 1, we still utilize star nodes to denote nodes with large  $p(u|v)$ , round nodes to denote nodes with intermediate  $p(u|v)$  and rectangle nodes to denote nodes with small  $p(u|v)$ . In Step 1, SENSEI includes round nodes (nodes with intermediate  $p(u|v)$ ) as positive samples. In Step 2, the round nodes are used as hard negative samples for the star nodes.
$$
J (v) = - \frac {1}{| \mathcal {P} (v) |} \left(\sum_ {u \in \mathcal {P} (v)} \left(k \mathbf {F} (u,:) ^ {\top} \mathbf {F} (v,:) - \sum_ {m = 1} ^ {m = k} \mathbf {F} \left(w _ {m,:}\right) ^ {\top} \mathbf {F} (v,:))\right) \right. \tag {9}
$$
where  $k$  is the negative sample number for each positive sample,  $u$  is the positively sampled node for  $v$  and  $w_{m} \in \mathcal{N}(v)$  is the negatively sampled node. By adding the loss for all  $v$ s in  $\mathbf{A}$ , we obtain the
overall loss in Step 1 as:
$$
J _ {P} = \sum_ {v \in \mathbf {A}} J (v) \tag {10}
$$
Step 2: Fulfill the Partial Monotonicity Property. In this step, we focus on the partial monotonicity property within the combined positive sample set in Step 1. For any two nodes  $u_{l}$  and  $u_{m}$  in the positive sample set  $\mathcal{P}(v)$ , if  $p(u_{l}|v) > p(u_{m}|v)$ , we want to retain the monotonicity between them with  $\mathbf{F}(u_{l},:)^{\top}\mathbf{F}(v,:) > \mathbf{F}(u_{m},:)^{\top}\mathbf{F}(v,:)$ . Node  $u_{m}$  changes from a positive sample for  $v$  in Step 1 to a negative sample for  $v$  compared with  $u_{l}$ , which is consistent with the idea of sampling difficult negative samples. The margin-based ranking loss is:
$$
L (v) = \sum \mathbb {1} _ {p (u _ {l} | v) > p (u _ {m} | v)} \left(\max  \left\{\mathbf {F} (u _ {m},:) ^ {\top} \mathbf {F} (v,:) - \mathbf {F} (u _ {l},:) ^ {\top} \mathbf {F} (v,:) + \gamma , 0 \right\}\right) \tag {11}
$$
where  $u_{l}, u_{m} \in \mathcal{P}(v)$ ,  $\mathbb{1}_{p(u_l|v) > p(u_m|v)}$  is the indicator function and  $\gamma$  is a positive margin. When  $p(u_l|v) > p(u_m|v)$ ,  $\mathbb{1}_{p(u_l|v) > p(u_m|v)} = 1$ , otherwise,  $\mathbb{1}_{p(u_l|v) > p(u_m|v)} = 0$ . Similar to Eq. (10) in Step 1, we have the loss  $L_{p}$  in Step 2 as:
$$
L _ {P} = \sum_ {v \in \mathbf {A}} L (v). \tag {12}
$$
In fact, Step 2 can be regarded as a fine-tuning process to fulfill the partial monotonicity property. Therefore, this step usually has a small learning rate and a small positive margin  $\gamma$ . The pseudocode of this algorithm can be found in Appendix A.2.
Complexity Analysis. We give a complexity analysis of SENSEI here. For SENSEI, the time complexity can be analyzed as the following: (1) Calculating personalized pagerank has the complexity  $O(\mathrm{iter}_{\mathrm{max}} \cdot \| \mathcal{E} \|)$ , where  $\| \mathcal{E} \|$  is the number of edges in the graph and  $\mathrm{iter}_{\mathrm{max}}$  is the maximum iterations or hops for personalized pagerank; (2) the time complexity for sorting the proximity distribution  $p(u|v)$  and sampling positive/negative nodes for node  $v$  is  $O(n \cdot \log(n))$ ; (3) Calculating  $J(v)$  takes  $O(\| \mathcal{P}(v) \| \cdot k \cdot d)$ , where  $\| \mathcal{P}(v) \|$  is the positive sample set,  $k$  is the number of negative samples and  $d$  is the dimension of node embedding. Obtaining  $J_P$  takes  $O(n)$  time; (4) Computing  $L(v)$  in Eq. (11) takes  $O(\| \mathcal{P}(v) \|^2 \cdot d)$  and calculating  $L_p$  costs  $O(n)$ . Most of these computations can be further parallelized.
In this section, we evaluate the effectiveness of the proposed algorithm (SENSEI) for solving link prediction and node recommendation simultaneously in plain networks.
Datasets. We use 4 public real-world datasets to evaluate the proposed SENSEI model: C.ele [51], Cora [39], CiteSeer [39], NS [31].
Baselines. We compare the proposed SENSEI with the following 5 plain network embedding methods: node2vec [14], VGAE [22], GAT [43], ARGVA [35] and RBGE [17].
Metrics. For link prediction and node recommendation in plain networks, we randomly split edges in every dataset into  $70 / 10 / 20\%$  for training, validation, and test. The same amount of additionally sampled non-existent edges are taken as negative edges for training, validation and test in the link prediction task.
Settings. For the link prediction task in plain networks, we use Area Under the ROC and Precision-Recall Curves (i.e., AUC-ROC and AUC-PR) as the metrics to evaluate the performance of different methods. For the node recommendation task in plain networks, we take Hit@  $K$  as the metric. Only when the model ranks the correct node as top-  $K$  in the recommendation list, it is counted as a hit for this query node in the test set. Therefore, the average hit number for all query nodes is Hit@  $K$ . We set  $K = 10$  for SENSEI.
2The node recommendation task in plain networks is to recommend potential nodes that are most likely to have edges connected to the query node.
Additional Contents. In Appendix, we provide more contents related to experiments, including (1) dataset statistics (Appendix A.3); (2) implementation details (Appendix A.3); (3) the comparison with additional GCN-based methods: GraphSAGE [15], Cluster-gcn [5], APPNP [13], GPRGNN [6] and H2GCN [63] (Appendix A.4).
4.2 Effectiveness of SENSEI
Table 1: The AUC-ROC (±std) and AUC-PR (±std) of link prediction in plain networks (\%).
Models	Cora	CiteSeer	NS	C.ele
AUC-ROC	AUC-PR	AUC-ROC	AUC-PR	AUC-ROC	AUC-PR	AUC-ROC	AUC-PR
node2vec[14]	75.96±1.18	82.73±0.69	69.63±0.97	77.38±0.89	86.48±0.76	91.36±0.75	79.36±1.14	74.27±1.33
VGAE [22]	78.28±0.53	81.07±0.59	72.58±0.74	78.00±0.40	86.45±0.95	89.99±0.58	77.78±1.39	73.50±2.38
GAT [43]	76.48±0.77	78.90±1.03	71.79±1.40	76.04±1.18	85.89±0.84	88.47±1.92	74.16±4.33	71.21±1.69
ARGVA [35]	75.99±0.97	78.11±1.14	71.96±1.10	74.74±0.59	88.55±1.12	90.32±1.36	77.04±1.71	71.29±2.42
RBGE [17]	79.33±0.95	83.11±0.86	73.89±1.11	77.48±1.30	83.98±4.21	88.77±2.77	74.65±4.54	72.07±3.70
SENSEI	81.27±0.65	83.59±1.11	76.28±0.90	82.02±0.79	91.42±1.25	93.10±0.77	77.13±0.81	77.28±1.53
The results of link prediction on all 4 plain networks are presented in Table 1. Our proposed SENSEI generally outperforms all baselines on all 4 datasets. In particular, SENSEI achieves about  $2\%$  improvement in AUC-PR compared to the best competitor (e.g., VGAE on Citeseer and node2vec on C.ele). For the node recommendation task, as shown in Table 2, node2vec obtains the best Hit@10 on Cora and NS among all baselines, which indicates that including more positive samples  $(T)$  by setting a larger walk length (80) can indeed reduce the error in Theorem 2. Notice that the proposed SENSEI still outperforms node2vec on all 4 datasets, which demonstrates the ability of SENSEI to fulfill the partial monotonicity property.
Table 2: Hit@10 (±std) of node recommendation in plain networks (\%).
Models	Cora	Citeseer	NS	C.ele
node2vec[14]	19.78±0.84	18.50±1.30	59.88±2.68	9.43±2.41
VGAE [22]	8.19±1.62	8.93±0.91	37.26±2.36	9.34±1.32
GAT [43]	5.77±0.64	5.99±2.25	23.96±5.39	9.71±3.00
ARGVA [35]	5.65±1.22	4.71±0.56	36.78±5.20	9.71±2.39
RBGE [17]	17.19±1.24	18.99±1.17	52.48±5.18	9.48±4.86
SENSEI(w/o Step 2)	19.82±1.18	18.17±1.31	61.01±4.22	11.07±2.18
SENSEI	20.51±1.13	19.03±1.50	61.31±3.46	11.52±2.13
4.3 Ablation Study and Sensitivity Study
In this subsection, we conduct the ablation study to validate the importance of Step 2 of SENSEI. As illustrated in Table 2, for the node recommendation task (Hit@10), Step 2 consistently improves the performance of SENSEI. The results verify that Step 2 is indeed beneficial by fine-tuning node embeddings to satisfy the partial monotonicity property. In addition, we carry out a sensitivity study on different  $K$ s for Hit@K, which is presented in Figure 3. We observe that when  $K$  increases from 1 to 30, Hit@K gradually increases and Hit@30 is much higher than Hit@1, which is expected.
Network Embeddings. Network embedding maps nodes in the network to low dimensional vectors. It can be traced back to matrix low rank approximation [26] and spectral clustering [40]. DeepWalk [36] and node2vec [14] rely on random walk to encode the topological information of homogeneous networks. For multi-layered networks or heterogeneous networks, metapath2vec [7] and Hin2vec [10] design metapaths to capture the connectivity between different layers or different types of nodes. Since graph convolutional network (GCN) [21] emerges, many GCN-based network embedding methods have been proposed. For example, variational graph autoencoder (VGAE) [22] adopts the same message-passing mechanism as GCN to embed the homogeneous network. More recently, GCN has been integrated with random walk based heterogeneous network embedding methods, which leads to heterogeneous graph convolutional networks like MAGNN [11] and HAN [50]. MANE [25] successfully employs network embedding on multi-layered networks and DMGC [29] accomplishes both network embedding and clustering by utilizing cross-layer links as regularization. In addition, many embedding algorithms for knowledge graph have been proposed such as translational distance model [3] and semantic matching model [32].
Sampling in Network Embedding. Sampling is an important technique, which appears in recommendation [34] and text embedding [30] to speed up the training process. For network embedding algorithms, both the positive sampling strategy and the negative sampling strategy have been applied and improved. For matrix factorization [26], spectral clustering [40] and LINE [41], direct or two-hop neighboring nodes act as positive samples and negative nodes are sampled uniformly or according to the degree distribution. In DeepWalk [36], node2vec [14] and metapath2vec [7], the positive sampling strategy is modified based on truncated random walk starting from the central node. The positive and negative sampling are implemented implicitly with a message-passing mechanism in GCN-based methods, where nodes to be aggregated can be viewed as positive samples and the
Figure 3: Hit@  $K$  for SENSEI.
remaining nodes are implicit negative samples. For the GCN related works, efforts are mainly made to improve the positive sampling strategy. For example, APPNP [13] manipulates positive samples with personalized pagerank [33] to catch long distance information and PGNN [58] randomly fixes an anchor node set for aggregation in each training epoch. Recently, more attention has been paid to improve the negative sampling strategy. Many works suggest to sample closer nodes as difficult negative samples. For instance, MCNS [55] proposes that the negative sampling distribution should be positively correlated with the positive sampling distribution. KBGAN [4] samples most similar entities to replace the groundtruth positive entity in knowledge graph embedding and RecNS [56] tends to sample negative nodes from the intermediate distance region.
In this paper, we study the sampling strategies of network embedding. To uncover the underlying relationships of existing competing sampling strategies, we conduct theoretical analysis on the sampling-based network embedding process. In the analysis, we identify two desirable properties for the similarity scores of node embedding, including the discrimination property and the monotonicity property. Furthermore, we prove that there always exists an error gap between the empirical and ideal optimal embedding similarity scores. Guided by such analysis, we propose a simple yet novel model (SENSEI), which creatively integrates the two competing sampling strategies to fulfill the the discrimination property and the partial monotonicity property. The effectiveness of SENSEI is verified by extensive experiments.
This paper studies sampling strategies of network embedding, which has no negative ethical impacts on society. The limitations of our paper lie in that the theoretical analysis is conducted on proximity scores and the proposed SENSEI model is designed for plain (non-attributed) networks rather than attributed networks. The key to generalize the proposed properties/SENSEI model to attributed networks is a new definition of the node pair similarity, which we leave for future exploration.
This work is supported by NSF (1947135, 2134079, 2316233, 1939725, and 2324770), DARPA (HR001121C0165), NIFA (2020-67021-32799), DHS (17STQAC00001-07-00), ARO (W911NF2110088). The content of the information in this document does not necessarily reflect the position or the policy of the Government, and no official endorsement should be inferred. The U.S. Government is authorized to reproduce and distribute reprints for Government purposes notwithstanding any copyright notation here on.
[1] Yikun Ban, Yuchen Yan, Arindam Banerjee, and Jingrui He. Ee-net: Exploitation-exploration neural networks in contextual bandits. arXiv preprint arXiv:2110.03177, 2021.
[61] Qinghai Zhou, Liangyue Li, Nan Cao, Lei Ying, and Hanghang Tong. Admiring: Adversarial multi-network mining. In 2019 IEEE International Conference on Data Mining (ICDM), pages 1522-1527. IEEE, 2019.
[62] Qinghai Zhou, Liangyue Li, Xintao Wu, Nan Cao, Lei Ying, and Hanghang Tong. Attent: Active attributed network alignment. In Proceedings of the Web Conference 2021, pages 3896-3906, 2021.
[63] Jiong Zhu, Yujun Yan, Lingxiao Zhao, Mark Heimann, Leman Akoglu, and Danai Koutra. Beyond homophily in graph neural networks: Current limitations and effective designs. Advances in Neural Information Processing Systems, 33:7793-7804, 2020.
- Subsection A.1 gives the proof of Theorem 2;
- Subsection A.2 gives the detailed algorithm of SENSEI;
- Subsection A.3 introduces the statistics of datasets and the implementation details of SENSEI;
- Subsection A.4 compares the performance of SENSEI with 5 additional GCN baselines (Cluster-gcn, GraphSAGE, APPNP, GPRGNN and H2GCN) on Cora and CiteSeer.
Theorem 2. Mean Squared Error of  $S$  between the Ideal Loss and the Empirical Loss. For the mean squared error between  $S^{*}$  and  $S_{e}$ , we have
$$
\begin{array}{l} \mathbb {E} [ | | (S _ {e} - S ^ {*}) _ {u} | | ^ {2} ] = \\ \frac {1}{T \left(2 + \frac {k p _ {n} (u | v)}{p (u | v)} + \frac {p (u | v)}{k p _ {n} (u | v)}\right) ^ {2}} \left(\frac {1}{p (u | v)} + \frac {1}{k p _ {n} (u | v)} - 1 - \frac {1}{k}\right) \tag {13} \\ \end{array}
$$
where  $\mathbb{E}$  is the expectation and  $S^{*} = \left[\frac{p(u_{1}|v)}{p(u_{1}|v) + kp_{n}(u_{1}|v)},\dots ,\frac{p(u_{n}|v)}{p(u_{n}|v) + kp_{n}(u_{n}|v)}\right]$ .
Proof. In this proof, since  $S = [s(u_1,v), s(u_2,v), \ldots, s(u_n,v)]$  is the parameter to be optimized, we can write the empirical loss  $J_e$  as a function of  $S$  as  $J_e(S)$ . We prove this theorem with the help of Taylor expansion of  $\nabla_S J_e(S_e)$  around  $S^*$ . Because  $S_e$  is the solution to minimize  $J_e$ ,  $\nabla_S J_e(S_e) = 0$ . So, it can be expressed as:
$$
\nabla_ {S} J _ {e} \left(S _ {e}\right) = \nabla_ {S} J _ {e} \left(S ^ {*}\right) + \nabla_ {S} ^ {2} J _ {e} \left(S ^ {*}\right) \left(S _ {e} - S ^ {*}\right) + O \left(\left\| S _ {e} - S ^ {*} \right\| ^ {2}\right) = 0 \tag {14}
$$
Thus, up to terms of order  $O(||S_e - S^* ||^2)$ , we obtain
$$
S _ {e} - S ^ {*} = - \left(\nabla_ {S} ^ {2} J _ {e} \left(S ^ {*}\right)\right) ^ {- 1} \nabla_ {S} J _ {e} \left(S ^ {*}\right) \tag {15}
$$
Then, we discuss the term  $-(\nabla_S^2 J_e(S^*))^{-1}$  and the term  $\nabla_{S}J_{e}(S^{*})$  respectively.
For the term  $-(\nabla_{S}^{2}J_{e}(S^{*}))^{-1}$ , let  $\mathbf{e}_{(u_i)}$  be the one-hot vector, where only the  $u_{i}$ -th entry is 1.
$$
\begin{array}{l} J _ {e} (S) = - \frac {1}{T} \sum_ {i = 1} ^ {T} \log (s (u _ {i}, v)) - \frac {1}{T} \sum_ {i = 1} ^ {k T} \log (1 - s (w _ {i}, v)) \\ \nabla_ {S} J _ {e} (S) = - \frac {1}{T} \sum_ {i = 1} ^ {T} \frac {1}{s (u _ {i} , v)} \mathbf {e} _ {(u _ {i})} - \frac {1}{T} \sum_ {i = 1} ^ {k T} \frac {1}{s (w _ {i} , v) - 1} \mathbf {e} _ {(w _ {i})} \tag {16} \\ \nabla_ {S} ^ {2} J _ {e} (S) = \frac {1}{T} \sum_ {i = 1} ^ {T} \frac {1}{s (u _ {i} , v) ^ {2}} \mathbf {e} _ {(u _ {i})} \mathbf {e} _ {(u _ {i})} ^ {\top} + \frac {1}{T} \sum_ {i = 1} ^ {k T} \frac {1}{(s (w _ {i} , v) - 1) ^ {2}} \mathbf {e} _ {(w _ {i})} \mathbf {e} _ {(w _ {i})} ^ {\top} \\ - \nabla_ {S} ^ {2} J _ {e} (S) = - \frac {1}{T} \sum_ {i = 1} ^ {T} \frac {1}{s (u _ {i} , v) ^ {2}} \mathbf {e} _ {(u _ {i})} \mathbf {e} _ {(u _ {i})} ^ {\top} - \frac {1}{T} \sum_ {i = 1} ^ {k T} \frac {1}{(s (w _ {i} , v) - 1) ^ {2}} \mathbf {e} _ {(w _ {i})} \mathbf {e} _ {(w _ {i})} ^ {\top} \\ \end{array}
$$
Since  $s^*(u_i, v) = \frac{p(u_i | v)}{p(u_i | v) + kp_n(u_i | v)}$  and  $s^*(w_i, v) = \frac{p(w_i | v)}{p(w_i | v) + kp_n(w_i | v)}$ ,
$$
\begin{array}{l} - \nabla_ {S} ^ {2} J _ {e} \left(S ^ {*}\right) = - \frac {1}{T} \sum_ {i = 1} ^ {T} \frac {\left(p \left(u _ {i} \mid v\right) + k p _ {n} \left(u _ {i} \mid v\right)\right) ^ {2}}{p \left(u _ {i} \mid v\right) ^ {2}} \mathbf {e} _ {\left(u _ {i}\right)} \mathbf {e} _ {\left(u _ {i}\right)} ^ {\top} \tag {17} \\ - \frac {1}{T} \sum_ {i = 1} ^ {k T} \frac {\left(p \left(w _ {i} | v\right) + k p _ {n} \left(w _ {i} | v\right)\right) ^ {2}}{\left(k p _ {n} \left(w _ {i} | v\right)\right) ^ {2}} \mathbf {e} _ {\left(w _ {i}\right)} \mathbf {e} _ {\left(w _ {i}\right)} ^ {\top} \\ \end{array}
$$