SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Junliang Li $^{1,2}$ , Yajun Yang $^{1,2*}$ , Qinghua Hu $^{1}$ , Xin Wang $^{1}$ , Hong Gao $^{3}$
<sup>1</sup>College of Intelligence and Computing, Tianjin University, China
$^{2}$ State Key Laboratory of Communication Content Cognition, Beijing, China
<sup>3</sup>College of Mathematics and Computer Science, Zhejiang Normal University, China
1{lijunliang,yjyang,huqinghua,wangx}@tju.edu.cn,  $^3$  honggao@zjnu.edu.cn
Trending topic diffusion and prediction analysis is an important problem and has been well studied in social networks. Representation learning is an effective way to extract node embeddings, which can help for topic propagation analysis by completing downstream tasks such as link prediction and node classification. In real world, there are often several trending topics or opinion leaders in public opinion space at the same time and they can be regarded as different centers of public opinion. A public opinion field will be formed surrounding every center. These public opinion fields compete for public's attention and it will potentially affect the development of public opinion. However, the existing methods do not consider public opinion field effect for trending topics diffusion. In this paper, we introduce three well-known observations about public opinion field effect in media and communication studies, and propose a novel and effective heterogeneous representation learning framework to incorporate public opinion field effect and social circle influence effect. To the best of our knowledge, our work is the first to consider these effects in representation learning for trending topic diffusion. Extensive experiments on real-world datasets validate the superiority of our model.
With the rapid development of the Internet, hundreds of millions of people access and share information on social medias. Social networks have become the most important platform for producing and spreading public opinions. Various trending topics widely spread on social networks will bring a series of consequences. For example, fake news quickly spreads on social networks, will cause governments or organizations to fall into a huge crisis of public opinion [11]. Studying the diffusion and prediction of trending topics can effectively help governments and enterprises to timely discover potential public opinion crisis and deal with it to avoid their reputation and economic losses [52].
The existing studies on trending topic diffusion are mainly divided into two categories: content-based and network-based. The content-based methods [54, 21, 55, 26] aim to predict the dissemination trends, e.g., number of comments and retweets, mainly for a blog based on content features such as tags of text, images, and videos, but not cosider the network features. For network-based methods, various prediction model based on graph embedding techniques [6, 53, 34, 62] or random walk [33, 51, 2, 22] are proposed to calculate the information propagation probability for every node in social networks. Recently, representation learning on heterogeneous graph becomes an effective way to extract a low-dimensional node embedding by considering the network structure and node attribute features, which can be applied to downstream tasks of propagation analysis such as link prediction and node classification [18, 28, 27, 61, 30].
(a) Public opinion network.
Figure 1: Public opinion field effect.
(b) An example of several pocs and their pofs.
(c) Four cases for nodes in PON.
However, the existing representation learning models only consider the information of social network itself but neglect the influence of "public opinion field effect" for trending topic diffusion. In real life, there are often several trending topics or opinion leaders in the public opinion space at the same time. Every topic or opinion leader with the people actively following it in a social network will form a public opinion field. Research on media and communication has shown that the total amount of public's attention is always limited [29]. These public opinion field compete for public's attention and such competition will potentially affect the development of public opinion. Obviously, a topic with higher popularity may attract more attention from people who are following other topics. The existence of public opinion field effect makes the existing representation learning model cannot work well on the trending topic diffusion. A few works on media and communication [6, 53, 34] only present some statistics of public opinion field, e.g., the number of people who talk about different topics, but not consider to incorporate the public opinion field effect into representation learning for trending topic diffusion. Moreover, in real world, people are always involved in a topic propagation if their social circle is actively talking about the same topic, even though they are not interested in this topic at the beginning. This effect is also not considered in existing representation learning model.
In this paper, we study how to incorporate public opinion field effect in representation learning. To the best of our knowledge, our work is the first one regarding this problem for trending topic diffusion. Our proposed model includes "topic propagation representation" and "social circle influence representation". In topic propagation representation, three kinds of nodes, event nodes, information source nodes and opinion leader nodes, are regarded as the centers of public opinion for their corresponding topics. We define public opinion field based on center node and propose a kind of paradigm to quantify the "energy" of public opinion field, which can reflect the attraction power of public opinion field. We introduce three well-known observations about public opinion effect in media and communication studies, "Attention Competition Effect", "Popularity Dominance Effect" and "Negative Bias Effect", and design the model to obtain the higher quality representation by considering these effects in trending topic diffusion. In social circle influence representation, the social circle influence effect is considered in representation learning. The main contributions are summarized below:
(1) We formalize public opinion field effect and propose a novel and effective heterogeneous representation learning model by incorporating public opinion field effect for the first time.
(2) We consider the social circle influence, which extensively exists in real world, to enhance the quality of node representation.
(3) We conduct extensive experiments on three real-world datasets. The experimental results demonstrate the superiority of our method.
2 Formalization of Public Opinion Field Effect
In order to better investigate the diffusion of public opinion events in social network, we introduce the "public opinion field effect" in representation learning for the first time. In real applications, the public
opinion space includes various public opinion events attracting users' attention in social network. A public opinion field can be regarded as a public opinion event with the users paying attention to it or a high influence user with his/her followers. The interaction of different public opinion fields influences the development of public opinion events. To formalize the public opinion field, we first introduce Public Opinion Network (PON), which is essentially a heterogeneous network consisting of different kinds of entities and links. All of the entities in a PON can be categorized into three types: (i) users in the social network, (ii) public events and (iii) sources that publish the events. All of the links in a PON can be categorized into four types: (i) friendship link (user to user), (ii) concern link (user to event), (iii) publication link (source to event) and (iv) citation link (source to source). The definition of PON is given below:
Definition 1 (Public Opinion Network): A public opinion network (PON) is a heterogeneous directed graph  $G = (V, E)$ , where  $V = U \cup I \cup S$  is the set of nodes and  $E \subseteq (U \times U) \cup (U \times I) \cup (S \times I) \cup (S \times S)$  is the set of links among the nodes in  $V$ . Note that  $U, I$  and  $S$  are disjoint sets of the nodes representing the users in social network, public opinion events and sources publishing the events respectively.
Fig. 1(a) shows an example of a public opinion network. A public opinion network depicts the public opinion space in a social network. In a public opinion network, every event node, source node and opinion leader node is also called a "public opinion center node", or poc node for simplicity. Note that opinion leader nodes identifying in social networks has been well studied and thus we do not discuss it, but instead consider them as input to our model in this paper. We use  $U_h$  and  $C$  to denote the sets of highly influential user nodes and poc nodes respectively, i.e.,  $C = I \cup S \cup U_h$ . Next we give the definition of Public Opinion Field.
Definition 2 (Public Opinion Field): Given a public opinion network  $G$  and a public opinion center node  $v_{i} \in C$ , a public opinion field (or pof for simplicity)  $P_{i}$  of  $v_{i}$  is defined as an induced subgraph of  $G$  on  $N_{i} \cup \{v_{i}\}$ , where  $N_{i}$  is the neighbor set of  $v_{i}$  in  $G$ .
Fig. 1(b) illustrates an example of several poc nodes and their corresponding pofs in a PON. Note that in a social network a user may be in several distinct pofs at the same time. A poc node can be regarded as the center of "whirlpool" of a public topic and a pof can be regarded as a view in  $G$  focusing on a poc node with the users pay attention to it. Existing works neglect "public opinion field effect" in representation learning for public event diffusion. However, in practice, the interaction between different pofs always affects the development of public opinion. There are three well-known observations in media and communication studies as follows:
Observation 1 (Attention Competition Effect): The popularity of news decreases with the number of competing items that are simultaneously available. A user's breadth of attention remains essentially constant and the diversity of memes to which a user can pay attention is bound [29].
Observation 2 (Popularity Dominance Effect): In public opinion space, the topics with the higher popularity are more powerful to attract users' attention [45].
Observation 3 (Negative Bias Effect): The negative events, e.g., fake news or rumors, are easier to spread in social networks [20].
It is noteworthy that we recognize the existence of other effects (e.g., "Empathy Effect" [4, 50] and "Primacy Effect" [16, 56]) in the real world. However these effects are not closely relevant to the influence of coexisting trending topics on information diffusion. In addition, there also are some other effects (e.g., "Snowball Effect" [24, 25] and "Herd Effect" [23, 14]) in media and communication studies, but the meaning behind these effects are very similar to three effects introduced in this paper and thus we do not discuss them.
Observation 1 indicates the amount of attention in public opinion space is always constant, the attention on a current public topic may be seized by another rising topic, e.g., "Feng Wang Effect" in China. Observations 2 and 3 indicate that trending topics and negative events probably grab more attention in public opinion space. For fusion of above three effects in representation learning, we define "energy" of public opinion field to quantify the attraction power of a poc node in PON  $G$ .
Definition 3 (Public Opinion Field Energy): Given a poc node  $v_{i}$  with its pof  $P_{i}$ , the energy of  $P_{i}$ , denoted as  $E(P_{i})$ , is defined as  $E(P_{i}) = \sum_{v_{j}\in N_{i}}w_{i,j}\tau_{i,j}$ , where  $N_{i}$  is the neighbor set of poc node
Feng Wang is a well-known singer in China. Whenever he announces something important, there are always other more trending topics emerging and competing with Feng Wang's event for users' attention.
Figure 2: Overall framework of our model.
$v_{i},w_{i,j}\in \mathbb{R}$  is the popularity contribution of  $v_{j}$  for a poc node  $v_{i}$  and  $\tau_{i,j}\in \mathbb{R}$  is the attention degree of  $v_{j}$  for a topic represented by poc  $v_{i}$ .
Here  $w_{i,j}$  reflects how much popularity or attention can be brought if  $v_{j}$  participates in the discussion or propagation of public topic  $v_{i}$ . It is obvious that Michael Jordan brings more attention than the ordinary person when discussing basketball topics.  $\tau_{i,j}$  reflects how much attention that  $v_{j}$  intends to pay on the topic represented by poc node  $v_{i}$ .
Note that researchers can calculate  $w_{i,j}$  and  $\tau_{i,j}$  in any reasonable ways and we utilize attention mechanism function and covariance function to compute them respectively in this paper. We will discuss it in the next section and then prove that attention defined in our representation learning follows Observation 1, i.e., the total amount of attention in the public opinion fields is constant.
If a poc node  $v_{i}$  is related to a negative topic, an energy amplification factor  $\eta_{i}$  ( $\eta_{i} > 1$ ) is utilized to amplify the energy of  $\mathsf{pof} P_{i}$  by  $\eta_{i} \cdot E(P_{i})$  and thus Observation 3 is satisfied in our representation learning framework.
Notably, the reason we use only 1-hop neighbors to construct pof and pof energy is that they are closest to poc. There are also someone know a trending topic indirectly or post their comments on a retweeted content. In fact, this phenomenon has been taken account into our social circle influence representation in Section 3.2. Moreover, our model can be easily extended to handle  $k$ -hop neighbors by the following equation  $E(P_{i}^{(k)}) = \sum_{t=1}^{k} \sum_{d_{i,j}=t} w_{i,j} \tau_{i,j}$ , where  $d_{i,j}$  denotes the distance between nodes  $v_{i}$  and  $v_{j}$ . We will experimentally illustrate the soundness of the choice of 1-hop neighbors in Section 4.5.
3 Representation Learning
We investigate "topic propagation representation" and "social circle influence representation" in our model. In topic propagation representation, public opinion field effect is introduced to reflect that how users actively participate in trending topics diffusion. However, in practice, some users are passively involved in a trending topic diffusion because most of his/her friends have participated in the discussion about this topic. For example, someone not interested in computer science does not care about the topic about ChatGPT when he/she knows it from news for the first time, but his/her interest will rise if most of his/her friends are talking about ChatGPT and thus he/she will participate in this topic diffusion. We consider such effect in social circle influence representation.
As shown in Fig. 2, our model is divided into three parts: topic propagation representation, social circle influence representation and objective optimization. In topic propagation representation, we first calculate the pof energy, i.e.,  $\theta_{j} = E(P_{j}) = \sum_{v_{i}\in N_{j}}w_{j,i}\tau_{j,i}$ . Then, pof energy will be combined with the heterogeneous graph convolution module to perform feature aggregation for the four types of nodes in the PON. In social circle influence representation, we combine the representation obtained
by topic propagation and the PON network to enhance the quality of representation learning. Finally, we optimize our model by supervised entropy loss and unsupervised proximity loss.
3.1 Topic Propagation Representation
In this section, we introduce how to consider public opinion field effect in topic propagation representation. Because the nodes with different types always have different feature spaces, we first utilize  $\pmb{h}_i = \pmb{A}\pmb{x}_i$  to project the features of all the nodes in  $G$  into the same space, where  $\pmb{x}_i$  is original feature of node  $v_i$  and  $\pmb{A}$  is a projection matrix. Note that  $\pmb{A}$  is different depending on the type of node, i.e.,  $U, I$  and  $S$ . Next the  $w_{i,j}$  and  $\tau_{i,j}$  mentioned in Definition 3 can be calculated as follows:
$$
w _ {i, j} = \lambda_ {j} \cdot \operatorname {a t t} \left(\boldsymbol {h} _ {i}, \boldsymbol {h} _ {j}\right) = \lambda_ {j} \cdot \sigma \left(\boldsymbol {\Phi} ^ {\top} \cdot \left(\boldsymbol {W h} _ {i} \oplus \boldsymbol {W h} _ {j}\right)\right), \tau_ {i, j} = \sigma \left(\operatorname {c o v} \left(\boldsymbol {W h} _ {i}, \boldsymbol {W h} _ {j}\right)\right) \tag {1}
$$
where  $\mathrm{att}(h_i, h_j)$  is graph attention mechanism function, which reflects the importance of  $v_j$  with respect to poc node  $v_i$ ,  $\lambda_j$  is used to measure the influence of  $v_j$  in a social network. Previous studies propose various influence measures such as degree centrality, betweenness centrality and so on, and we use normalized degree centrality in this paper.  $\Phi$  is the weight vector for performing graph attention,  $\mathbf{W}$  is a shared linear transformation weight matrix and  $\oplus$  is concatenate operation. We select covariance function  $\operatorname{cov}(\mathbf{x}, \mathbf{y}) = (\sum_{i=1}^{f} (x_i - \bar{x})(y_i - \bar{y})) / (f - 1)$ , which indicates the correlation between two vectors  $\mathbf{x}, \mathbf{y} \in \mathbb{R}^f$ , to measure the amount of attention that  $v_j$  intends to pay on the topic represented by poc node  $v_i$ . It is reasonable since most people prefer to talk about the topics that are more relevant to them.
From Observation 1, we know that the attention amount of every user is limited. If someone intends to pay more attention to a new trending topic, his/her interests in other topics will be dropped off at the same time. As per the discussion,  $\tau_{i,j}$  essentially reflects the willingness of  $v_{j}$  to participate in a topic represented by a poc node  $v_{i}$ , thus we use a normalization operation on  $\tau_{i,j}$  for every user  $v_{j} \in U$ :
$$
\tau_ {i, j} \leftarrow \frac {w _ {i , j} \tau_ {i , j}}{\sum_ {v _ {i} \in C \cap N _ {j}} w _ {i , j} \tau_ {i , j}} \tag {2}
$$
Obviously, normalization operation guarantees that the "Attention Competition Effect (Observation 1)" is satisfied in our representation learning model. Theorem 1 shows that the total amount of attention in public opinion space that is defined in our representation model is constant.
Theorem 1: Given a PON  $G$ ,  $C$  and  $C'$  are any two distinct poc node sets at different times  $t$  and  $t'$  on  $G$  respectively, the amount of attention for the same user set  $U$  in  $G$  is constant for different times, i.e.,  $\sum_{v_i \in C, v_j \in U} \tau_{i,j} = \sum_{v_i \in C', v_j \in U} \tau_{i,j}$ .
PROOF: Because  $\sum_{v_i\in C}\tau_{i,j} = \sum_{v_i\in C'}\tau_{i,j} = 1$  , then  $\sum_{v_i\in C,v_j\in U}\tau_{i,j} = \sum_{v_i\in C',v_j\in U}\tau_{i,j}$
In our representation learning model, a normalization operation is also necessary on  $E(P_{i})$  to satisfy Observation 3, that is
$$
E \left(P _ {i}\right) \leftarrow \frac {\eta_ {i} \cdot E \left(P _ {i}\right)}{\sum_ {v _ {i} \in C} \eta_ {i} \cdot E \left(P _ {i}\right)} \tag {3}
$$
where  $\eta_{i} > 1$  if poc node  $v_{i}$  is related to negative topic and otherwise  $\eta_{i} = 1$ .
All the nodes in PON  $G$  can be categorized into four cases: (1) poc nodes; (2) user nodes  $v_{i}$  whose all neighbors are normal user nodes, i.e.,  $U_{1} = \{v_{i}|v_{i}\in U\land N_{i}\subseteq U - U_{h}\}$ ; (3) user nodes  $v_{i}$  whose all neighbors are poc nodes, i.e.,  $U_{2} = \{v_{i}|v_{i}\in U\land N_{i}\subseteq C\}$ ; and (4) users nodes  $v_{i}$  whose neighbors include both poc nodes and normal user nodes, i.e.,  $U_{3} = \{v_{i}|v_{i}\in U\land N_{i}\nsubseteq C\land N_{i}\nsubseteq U - U_{h}\}$ . We give an illustration of all cases in Fig. 1(c) and investigate node representation in trending topics propagation for these four cases respectively.
Case (1) and (2): We use the self-attention mechanism to update the features of every poc node  $v_{i} \in C$  and every node  $v_{i} \in U_{1}$ . Specifically, the node representation  $h_{i}^{\prime}$  of  $v_{i}$  is given below:
$$
\boldsymbol {h} _ {i} ^ {\prime} = \sigma \left(\sum_ {v _ {j} \in N _ {i} \cup \{v _ {i} \}} \alpha_ {i, j} \cdot \boldsymbol {W h} _ {j}\right), \alpha_ {i, j} = \operatorname {s o f t m a x} \left(\operatorname {a t t} \left(\boldsymbol {h} _ {i}, \boldsymbol {h} _ {j}\right)\right) \tag {4}
$$
where  $\mathrm{att}(\cdot)$  is similar to Eq. (1),  $\alpha_{i,j}$  is the attention coefficient between  $v_{i}$  and its neighbor  $v_{j}$ . By Eq. (4), we know  $h_i'$  of a poc node  $v_{i}$  also can be regarded as a representation of  $\operatorname{pof} P_i$  because it aggregates all the features of nodes in  $P_i$ .
Case (3): For each user node  $v_{i} \in U_{2}$ , its different poc neighbor nodes will compete for user  $v_{i}$ 's attention (Attention Competition Effect). The poc neighbor node with the larger pof energy will gain more attention from  $v_{i}$  (Popularity Dominance Effect). We consider both self attention mechanism and public opinion field effect in node representation  $h_{i}'$  for  $v_{i} \in U_{2}$ :
$$
\boldsymbol {h} _ {i} ^ {\prime} = \sigma \left(\sum_ {v _ {j} \in N _ {i} \cup \{v _ {i} \}} \alpha_ {i, j} \cdot \tilde {\boldsymbol {h}} _ {j}\right), \tilde {\boldsymbol {h}} _ {j} = \boldsymbol {W} \boldsymbol {h} _ {j} + \theta_ {j} \cdot \boldsymbol {h} _ {j} ^ {\prime} \tag {5}
$$
$$
\alpha_ {i, j} = \operatorname {s o f t m a x} \left(\operatorname {a t t} \left(\boldsymbol {h} _ {i}, \boldsymbol {h} _ {j}, \boldsymbol {\Theta}\right)\right), \operatorname {a t t} \left(\boldsymbol {h} _ {i}, \boldsymbol {h} _ {j}, \boldsymbol {\Theta}\right) = \theta_ {j} \cdot \operatorname {a t t} \left(\boldsymbol {h} _ {i}, \boldsymbol {h} _ {j}\right) \tag {6}
$$
Note that  $\pmb{h}_{j}^{\prime}$  is node representation of poc node  $v_{j}$  calculated in case (1),  $\theta_{j}$  is the normalized poc energy of  $v_{j}$ , i.e.,  $\theta_{j} = E(P_{j})$ , and thus  $\Theta = (\theta_{1},\dots ,\theta_{|C|})$  is the vector of all the normalized poc energy. Specifically, we need to second normalize the poc energy of all poc neighbors of  $v_{i}\in U_{2}$ :  $\theta_{j}\gets \theta_{j} / \sum_{v_{j}\in N_{i}}\theta_{j}$ . In Eq. (5), the term  $\theta_{j}\cdot \pmb{h}_{j}^{\prime}$  is utilized to reflect the influence of poc  $P_{j}$  for representation of  $v_{i}$ , because  $\pmb{h}_{j}^{\prime}$  is the representation of  $P_{i}$  as explained in case (1).
Case (4): For every node  $v_{i} \in U_{3}$ , it has two kinds of neighbors, poc neighbors  $v_{j} \in C$  and normal user neighbors  $v_{j} \in U - U_{h}$ . To consider heterogeneity, RGCN [44] designs different feature projection matrices for different types of links. Inspired by this, we use two different weight matrices  $\Phi_{1}$  and  $\Phi_{2}$  to calculate two different attention coefficient  $\alpha_{i,j}^{1}$  and  $\alpha_{i,j}^{2}$ , that is
$$
\alpha_ {i, j} ^ {k} = \operatorname {s o f t m a x} \left(\operatorname {a t t} \left(\boldsymbol {\Phi} _ {k}, \boldsymbol {h} _ {i}, \boldsymbol {h} _ {j}\right)\right), \operatorname {a t t} \left(\boldsymbol {\Phi} _ {k}, \boldsymbol {h} _ {i}, \boldsymbol {h} _ {j}\right) = \sigma \left(\boldsymbol {\Phi} _ {k} ^ {\top} \cdot \left(\boldsymbol {W h} _ {i} \oplus \boldsymbol {W h} _ {j}\right)\right) \tag {7}
$$
where  $k = 1$  if  $v_{j} \in S_{1} = N_{i} \cap C$  and  $k = 2$  if  $v_{j} \in S_{2} = N_{i} \cap (U - U_{h})$ . Therefore, the node representation  $\pmb{h}_i'$  of  $v_{i} \in U_{3}$  is given below:
$$
\boldsymbol {h} _ {i} ^ {\prime} = \sigma \left(\sum_ {v _ {j} \in S _ {1}} \alpha_ {i, j} ^ {1} \cdot \tilde {\boldsymbol {h}} _ {j} + \sum_ {v _ {j} \in S _ {2}} \alpha_ {i, j} ^ {2} \cdot \boldsymbol {W h} _ {j}\right), \tilde {\boldsymbol {h}} _ {j} = \boldsymbol {W h} _ {j} + \theta_ {j} \cdot \boldsymbol {h} _ {j} ^ {\prime} \tag {8}
$$
Like case (3),  $h_j'$  is node representation for poc node  $v_j \in S_1$  calculated in case (1).
3.2 Social Circle Influence Representation