SlowGuess/ACL_2020_OCR huggingface dataset:
Xingshan Zeng $^{1,2}$ , Jing Li $^{3}$ , Lu Wang $^{4}$ , Zhiming Mao $^{1,2}$ , Kam-Fai Wong $^{1,2}$
<sup>1</sup>The Chinese University of Hong Kong, Hong Kong, China
$^{2}$ MoE Key Laboratory of High Confidence Software Technologies, China
$^{3}$ Department of Computing, The Hong Kong Polytechnic University, Hong Kong, China
<sup>4</sup>Khoury College of Computer Sciences, Northeastern University, Boston, United States
1,2{xszeng,zmmao,kfwong}@se.cuhk.edu.hk
3jing-amelia.li@polyu.edu.hk, 4luwang@ccs.neu.edu
Trending topics in social media content evolve over time, and it is therefore crucial to understand social media users and their interpersonal communications in a dynamic manner. In this research we study dynamic online conversation recommendation, to help users engage in conversations that satisfy their evolving interests. Different from works in conversation recommendation which assume static user interests, our model captures the temporal aspects of user interests. Moreover, our model can cater for cold start problem where conversations are new and unseen in training. We propose a neural architecture to analyze changes of user interactions and interests over time, whose result is used to predict which discussions the users are likely to enter. We conduct experiments on large-scale collections of Reddit conversations. Results on three subreddits show that our model significantly outperforms state-of-the-art models based on static assumption of user interests. We further evaluate performance in cold start, and observe consistently better performance by our model when considering various degrees of sparsity of user's chatting history and conversation contexts. Lastly, our analysis also confirms the change of user interests. This further justifies the advantage and efficacy of our model.
Online social media platforms are popular outlets for individuals to exchange viewpoints and discuss topics they are interested in. However, the huge volume of online conversations produced daily hinders people's capability of finding the information they are interested in. As a result, there is pressing demand for developing a conversation recommendation engine that tracks ongoing conversations and recommends suitable ones to users.
Viewing the deluge of information streaming through social media, it is not hard to envision that
Figure 1: Four chatting snippets posted by the same user  $U$  on Reddit. Arrows linking conversation 1 to 4 follow the chronological order.  $U$ 's interests shifted from Internet security (conversations 1 and 2) to operation system (conversation 3 and 4).
users' tastes, stances, and behaviors evolve over time (Wu et al., 2017). Nonetheless, existing work on recommending conversations (Chen et al., 2011; Zeng et al., 2018, 2019b) assume users' discussion preferences do not change over time. Moreover, the common practice of recommendation is via collaborative filtering (CF), which relies on rich user interaction history for model training (Zeng et al., 2018, 2019b). When a conversation is entirely absent from training data, the model performance is inevitably compromised. This phenomenon is referred to as conversation cold start. As a result, existing methods which ignore the time-evolving user interests is insurmountable to tackle a common problem in practice, i.e., to predict future conversations created after the model is trained.
To overcome this predicament, we explore dynamic conversation recommendation, which can model the change of user interests over time (henceforth user interest dynamics). To illustrate such change, Figure 1 shows multiple conversation turns posted by user  $U$  in four Reddit discussion snippets:  $C_1$  to  $C_4$  in the chronological order. As can
Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 3331-3341
be seen,  $U$  used to like discussing Internet security, indicated by "encryption", "privacy", and "surveillance" in  $C_1$  and  $C_2$ . After a period of time,  $U$ 's interests changed to a different topic, operating system, as "ksplice", "oracle", and "Ubuntu" were later mentioned in  $C_3$  and  $C_4$ .
We design the model to capture user interests from both what they said in the past, and how they interacted with each other in the conversation structure. We first capture time-variant representations from user chatting history, where we assume user interests may change over time and therefore apply a gated recurrent unit (GRU) (Cho et al., 2014) to model time dependency. User interactions in the conversation context are then explored with both bidirectional gated recurrent unit (Bi-GRU) (Cho et al., 2014) for conversation turns' chronological order and graph convolutional networks (GCN) (Marcheggiani and Titov, 2017) for in-reply-to relations. Both representations are learned to encode how participants formed the conversation structure, including what they said and whom they replied to. Next, we propose a user-aware attention to convey the user interest dynamics, which is further put over an interaction-encoded conversation to measure whether its ongoing contexts fit a user's current interests. Finally, we predict how likely a user will engage in a conversation, as a result of recommendation. To the best of our knowledge, we are the first to study dynamic online conversation recommendation and to explore the effects of user interests change over time learned from both chatting content and interaction behavior. For this reason, we are capable of recommending future conversations based on users' interests at the time.
For experiments<sup>1</sup>, we collect Reddit conversations from three subreddits — “technology”, “todaylearned”, and “funny”, each exhibiting different data statistics, discussion topics, and language styles. An absolute date is used to separate training data (before the date) from test and validation data (after the date). In this way, most conversations in the test and validation parts are new conversations that have not been counted before. This presents a more realistic setup than previous studies (Zeng et al., 2018, 2019b), which let training data contain partial context for any conversations to allow the possibility of predicting users' future engagement
for recommendation.
Experimental results in main comparisons show that our model significantly outperforms all previous methods that ignore the change of user interests or interactions within contexts. For example, we achieve 0.375 MAP in discussions of "technology", compared with 0.222 yielded by our previous state-of-the-art model (Zeng et al., 2019b). Further study shows that we consistently perform better both in conversation cold start and with varying degrees of sparsity of user history and conversation contexts. Lastly, to provide more insights into user interest dynamics, we inspect our model outputs and find that users indeed tend to engage in different types of conversations at different times, confirming the usefulness of tracking user preferences in real-time for conversation recommendation.
User Response Prediction. This work is in line with user response prediction, such as message popularity forecast with handcrafted response features (Artzi et al., 2012; Backstrom et al., 2013) and conversation trajectory with user interaction structures (Cheng et al., 2017b; Jiao et al., 2018; Zeng et al., 2019a). These works predict responses from general public, while we work on personalized recommendation and focus on user interest modeling. For recommendation, there are extensive efforts on post-level recommendation (Chen et al., 2012; Yan et al., 2012) and conversation-level (Chen et al., 2011; Zeng et al., 2018, 2019b). In contrast with them which assume static user interests, we capture how user interests change over time and take advantage of the recent advancement of dynamic product recommendation (Wu et al., 2017; Beutel et al., 2018). To recommend conversations, we aim to learn user interest dynamics from chatting content and interaction behavior, which have never been explored in previous research.
Conversation Structure Modeling. Our work is also related to previous work to understand how participants interact with each other in conversation structure. Earlier efforts focus on discovering word statistic patterns via probabilistic graphical models (Ritter et al., 2010; Louis and Cohen, 2015), which are unable to capture deep semantics embedded in complex interactions. Recent research points out the effectiveness to understand conversation structure from temporal dynamics (Cheng et al., 2017a; Jiao et al., 2018) and replying struc
1The datasets and codes are available at: https:// github.com/zxshamson/dy-conv-rec
Figure 2: Overall structure of our model. The left module is to model user interest dynamics, whose results together with conversation representations derived from the right part are used for producing final prediction. Predicted score  $\hat{y}_{u,c}$  indicates how likely  $u$  will engage in  $c$ . "Msg Encoder" mainly contains two layers: word embedding layer and CNN modeling layer.
ture (Miura et al., 2018; Zayats and Ostendorf, 2018; Zeng et al., 2019b). The two factors are coupled in our interaction modeling and their joint effects for dynamic conversation recommendation, ignored by prior work, will be extensively studied here.
3 Our Dynamic Conversation Recommendation Model
This section describes our dynamic conversation recommendation model, whose overall structure is shown in Figure 2. In the following, we will first introduce how we model the user interest dynamics with their chatting history in Section 3.1, followed by the description of conversation modeling in Section 3.2. Afterwards, Section 3.3 will present how we produce final recommendation outputs. Objective function and learning procedures will be finally presented in Section 3.4.
3.1 User Interest Dynamic Modeling
Given a sequence of chronologically ordered historical messages  $\langle m_1, m_2, \dots, m_{|u|} \rangle$  of a user  $u$  ( $|u|$  is the message number of  $u$ ), a message therein corresponds to a word sequence  $\mathbf{w}_m$ . Our goal is to capture the temporal patterns from the sequence of user chatting messages and then produce the user interest representation. We employ two-level modeling — message level and user level.
Message-level Modeling. We model message-level representation from its word sequence. Specifically, given  $u$ 's historical message  $m$ , we first use a pre-trained word embedding layer to map each word into a vector space, and then employ a Convolutional Neural Network (CNN) (Kim, 2014) encoder to model word occurrence with their neighbors. Afterwards, we output representation  $z_{m}$  to reflect  $m$ 's content.
User-level Modeling. As shown in Wu et al. (2017), some user interests may change rapidly and some may last for a long time. For the latter, we adopt a user embedding layer  $I^{UF}(\cdot)$  to capture the time-invariant interest factor and define  $u$ 's factor as  $r_u^{UF}$ .
For the time-variant interests, we are inspired by previous work (Beutel et al., 2018) and employ a GRU (Cho et al., 2014) encoder to capture how user interests change based on sequential chatting messages. For each time state  $t$ , we update user's current interests  $h_{u,t}^{U}$  conditioned on the previous interests  $h_{u,t - 1}^{U}$  and the current behavior  $z_{mt}$  (derived from the aforementioned message-level modeling, reflecting  $m$ 's content):
$$
\boldsymbol {h} _ {u, t} ^ {U} = G R U \left(\boldsymbol {h} _ {u, t - 1} ^ {U}, \boldsymbol {z} _ {m _ {t}}\right) \tag {1}
$$
Further, to leverage time-invariant features in the modeling of user interest dynamics, we initialize GRU's hidden states based on the learned user factor  $\boldsymbol{r}_u^{UF}$  following linear transformation:  $\boldsymbol{h}_{u,0}^{U} = W^{U}\boldsymbol{r}_{u}^{UF} + b^{U}$ . And the last GRU states, i.e.,  $\boldsymbol{r}_u^U = \boldsymbol{h}_{u,t_{|u|}}^U$ , conveying the latest view of user interest dynamics, will be later used in conversation modeling and recommendation prediction.
3.2 User-aware Conversation Modeling
Here we introduce how we encode a conversation in aware of user interests. Each conversation  $c$  is formed with a sequence of chronologically ordered turns  $\langle t_1, t_2, \dots, t_{|c|} \rangle$  ( $|c|$  is the turn number of  $c$ ). A turn  $t$  therein is in form of a word sequence  $\mathbf{w}_t$ , its author's ID  $u_t$ , and the turn it replies to for later exploiting in-reply-to structure.
To learn  $c$ 's representation, we encode both word occurrence in each turn (via turn-level modeling) and interactions between conversation turns (via conversation-level modeling). Afterwards, to identify turns that match target user's interests, we propose a user-aware attention over turns.
Turn-level Modeling. For each turn  $t \in c$ , similar to message-level modeling in Section 3.1, we use a CNN encoder over pre-trained word embeddings to capture content representation,  $z_{t}$ . Further,  $z_{t}$  is concatenated with author  $u_{t}$ 's user embedding  $\boldsymbol{r}_{u_t}^{UF}$  (see Section 3.1) to yield turn-level representation  $\boldsymbol{r}_t^T$ , conveying both what is said and who says that. Based on the turn-level representations, we then learn turn interactions.
Conversation-level Modeling. To explore turn interactions, we exploit turn's chronological order and replying structure, both useful in conversation modeling (Zeng et al., 2019b).
Chronological Order: We employ a Bi-GRU (Cho et al., 2014) to capture how a turn interacts with the turns posted right before and after it, whose hidden states are updated as followings:
$$
\overrightarrow {\boldsymbol {h} _ {c , t} ^ {G R U}} = \overrightarrow {G R U} \left(\boldsymbol {h} _ {c, t - 1} ^ {G R U}, \boldsymbol {r} _ {t} ^ {T}\right) \tag {2}
$$
$$
\overleftarrow {\boldsymbol {h} _ {c , t} ^ {G R U}} = \overleftarrow {G R U} \left(\boldsymbol {h} _ {c, t + 1} ^ {G R U}, \boldsymbol {r} _ {t} ^ {T}\right) \tag {3}
$$
We then concatenate the forward and backward hidden states to produce chronology-encoded turn representations:  $\pmb{h}_{c,t}^{GRU} = [\overleftarrow{\pmb{h}_{c,t}^{GRU}}; \overleftarrow{\pmb{h}_{c,t}^{GRU}}]$ .
Replying Structure. To further encode who-replies-to-whom in conversation structure, we put a Graph Convolutional Network (GCN) (Marcheggiani and Titov, 2017) over the chronology-encoded turn representations (learned by Bi-GRU see above). Graph encoder is empirically better than sequential ones because replying relations usually exhibit tree structure (a post may lead to multiple replies). Concretely, we first build a directed graph for a conversation via adding edges from a turn to its replies. We then define turn interactions therein in three directions: predecessors to successors  $(Pre)$ , successors to predecessors  $(Suc)$ , and self interactions  $(Self)$ . Next, we update a turn's hidden state with the formula below:
$$
\begin{array}{l} \boldsymbol {h} _ {c, t} ^ {G C N} = \sum_ {i \in P r e (t)} g _ {i, t} \left(W ^ {P r e} \boldsymbol {h} _ {c, i} ^ {G R U} + b ^ {P r e}\right) + \\ \sum_ {j \in S u c (t)} g _ {j, t} \left(W ^ {S u c} \mathbf {h} _ {c, j} ^ {G R U} + b ^ {S u c}\right) + \tag {4} \\ g _ {t, t} \left(W ^ {S e l f} h _ {c, t} ^ {G R U} + b ^ {S e l f}\right) \\ \end{array}
$$
$Pre(t)$  and  $Suc(t)$  represent turn  $t$ 's predecessors and successors in replying graph;  $g_{i,j}$  is a scalar gate controlling weights of turn interactions:
$$
g _ {i, j} = \sigma \left(W ^ {D i r (i, j)} \mathbf {h} _ {c, i} ^ {G R U} + b ^ {D i r (i, j)}\right) \tag {5}
$$
where  $Dir(i,j)$  indicates the type of  $i - j$  direction (Pre, Suc, or Self).
The process described above can be viewed as one GCN layer. Multiple layers can be stacked, with a ReLU (Rectified Linear Unit) activated function to connect two succinct layers. It enables the networks to explore deeper interaction effects.
User-aware Attention. To identify conversation turns that better match target user's interests, we design a user-aware attention mechanism over interaction-encoded turns. The attention weights are defined to reflect the similarity between a conversation turn's representation  $h_{c,i}^{GCN}$  and the target user's latest interests  $r_u^U$  (see Section 3.1):
$$
a _ {i} = \operatorname {s o f t m a x} \left(\boldsymbol {r} _ {u} ^ {U} \cdot \boldsymbol {h} _ {c, i} ^ {G C N}\right) \tag {6}
$$
Finally, we compute the attentive sum of all turns and obtain the conversation representations conveying both interactions and user interests:
$$
\boldsymbol {r} _ {c} ^ {C} = \sum_ {i} a _ {i} \boldsymbol {h} _ {c, i} ^ {G C N} \tag {7}
$$
3.3 Recommendation Prediction
To predict whether a user  $u$  will engage in conversation  $c$ , we compute how  $u$ 's interest dynamics (carried by  $r_u^U$  in Section 3.1) are similar to  $c$ 's content and interaction styles (reflected by  $r_c^C$  in Section 3.2). We adopt a two-way interactions via MLP mechanism (He et al., 2017) to measure the similarity:
$$
\boldsymbol {r} _ {u, c} = \alpha \left(W _ {2} ^ {T} \left(\alpha \left(W _ {1} ^ {T} \left[ \boldsymbol {r} _ {u} ^ {U}; \boldsymbol {r} _ {c} ^ {C} \right] + b _ {1}\right)\right) + b _ {2}\right) \tag {8}
$$
where  $\alpha (\cdot)$  is ReLU-activated function.
For recommendation, we predict  $\hat{y}_{u,c} \in [0,1]$ , which signals how likely  $u$  will engage in  $c$ . The equation for the final output layer will be:
$$
\hat {y} _ {u, c} = \sigma \left(\boldsymbol {v} ^ {T} \boldsymbol {r} _ {u, c} + b\right) \tag {9}
$$
where  $\sigma$  represents sigmoid activation function.
3.4 Learning Objective
Following Zeng et al. (2019b), we adopt weighted binary cross-entropy loss as our objective function, which assigns more weights to positive feedbacks (i.e.  $u$  engages in  $c$ ):
$$
\mathcal {L} = - \sum_ {(u, c) \in \mathcal {T}} \left[ \lambda \cdot y _ {u, c} \log \left(\hat {y} _ {u, c}\right) + (1 - y _ {u, c}) \log \left(1 - \hat {y} _ {u, c}\right) \right] \tag {10}
$$
Tech	Learn	Fun
Number of Users	13,927	67,255	112,345
Number of Convs	8,286	42,220	67,908
Number of Turns	43,705	233,213	375,550
Hist Number / User	2.78	3.05	2.94
Turn Number / Conv	5.10	5.34	5.35
User Number / Conv	4.15	4.45	4.79
New User Rate (%)	8.20	8.24	7.81
New Conv Rate (%)	99.64	99.40	99.51
Table 1: Data statistics. "Conv": conversation; "Hist": historical messages. New user rate is the number of users newly appeared in May's data (for test) divided by number of May's users. New conversation rate is similar.
where  $\mathcal{T}$  is the training set,  $y_{u,c}$  denotes the binary ground-truth label, and  $\lambda (\lambda >1)$  is a hyperparameter to trade off the weights of positive and negative instances. We weigh more on positive feedbacks because they are more reliable, while the negative ones sometimes cannot reflect user's interests, owing to many unpredictable issues (e.g., users' busy time). For the same reason, we adopt the negative sampling strategy (He et al., 2017) in training, which also speeds up the training process.
Datasets. For experiments, we collect online conversations from Reddit, a popular online platform. To build our datasets, we first downloaded a large corpus publicly available on Reddit<sup>2</sup>, which consists of posts and comments created since early 2006. Then, we gathered data posted from January to May 2015 on three subreddits reflecting discussion topics on "technology" (Tech), "today-learned" (Learn), and "funny" (Fun). We chose these three subreddits as they were popular subreddits with different discussed topics and language styles. For each subreddit, posts and comments were connected with in-reply-to relations (indicated by comments' "parent_id" field) to form conversations. Finally, we removed conversations with only one turn and produced three conversation datasets of different topics.
In model training and evaluation, we use conversation turns created from January to April for training. For those posted in May, we randomly select half of them for validation and the other half for
(a) User History Dist.
(b) Conversation Turn Dist.
Figure 3: Distribution of users' historical message count (upper) and conversation turn count (lower).
test. This reflects a more realistic scenario where the model is trained with past data and applied to future recommendation, as opposed to prior work which assumes all conversations can be split between training and test (Zeng et al., 2018, 2019b).
Data Analysis. The dataset statistics are displayed on Table 1. Although differ in size, conversations therein exhibit similar average characteristics, likely because they come from the same platform. Moreover, over  $99\%$  of the conversations in test sets are future conversations (i.e. all turns were posted in May), highlighting the challenge of conversation cold start.
We further plot the distributions of message (turn) number in Figure 3 (3(a) for users and 3(b) for conversations). It is seen from Figure 3(a) that a large proportion of users were involved in less than 10 conversation turns, where about  $8\%$  (shown in Table 1) of users are absent in the training data. For conversations (Figure 3(b)), their turn numbers follow a power-law distribution. Therefore, for both users and conversations, the sparse interaction history presents additional challenges for recommendation.
In addition, Figure 4 shows distributions of conversation replying structure with 1, 2, and more root-to-leaf paths to characterize users' interaction structure. We find that more than  $60\%$  of con
2https://www.reddit.com/r/datasets/comments/3bxlg7/i_have EVERY_publicly-available.reddit_COMMENT/
Figure 4: Distributions of conversation structure. "One-path", "Two-path", and "More-path" indicate the conversation has 1, 2, and more root-to-leaf paths.
versations contain two or more paths, illustrating complex who-replies-to-whom interactions in the tree structure (with the original post as the root node and in-reply-to relations as edges). Therefore, graph-structured encoder may be a suitable alternative for capturing rich turn interactions in Reddit conversations.
Preprocessing. For all datasets, we applied open source natural language toolkit (NLTK) (Loper and Bird, 2002) for tokenization. Further, links were replaced by a generic tag “<URL>” and all number tokens were removed. In the experiments, we maintained a vocabulary with all the remaining tokens (including punctuation and emoticons).
Model Settings. In training, we adopt negative sampling with sampling ratio of 5 (see Section 3.4). We also randomly sample 100 negative instances for each positive one during validation and test, to avoid unbalanced labels.
For parameters, we initialize the word embedding layer with 300-dim Common Crawl version of Glove embedding (Pennington et al., 2014), and the dimension of user factor embedding is set to 20. For the CNN turn encoders, we use filter windows of 2, 3, and 4, each with 100 feature maps. As for the GRU models for both user and conversation modeling, the hidden state size is set to 200 (100 for each direction in Bi-GRU). The same hidden state size is applied to the GCN interaction model. We also set the layer number of GCN (see in Section 3.2) to 1, based on validation results. In training, the batch size is set to 256 and Adam optimizer (Kingma and Ba, 2014) is adopted with an initial learning rate of 0.001. As for the trade off weight in loss function, we set  $\lambda = 100$ .
Evaluation. Our evaluation metrics follow the common practice in conversation recommendation (Zeng et al., 2018, 2019b). Mean average
precision (MAP), precision at 1  $(\mathrm{P}@\mathrm{1})$ , and normalized Discounted Cumulative Gain at 5  $(\mathrm{nDCG}@\mathrm{5})$  are adopted to measure the ranking list of conversations to be recommended to a user. These metrics all have a value range of 0.0 to 1.0, and greater value indicates better performance.
Comparisons. We first consider two simple baselines: 1) ranking conversations based on POPULARITY, measured by the number of participants. 2) TOPICRANK (Chen et al., 2011): ranking conversations by topic relevance to the target user's historical messages, where topics are learned from both LDA (Blei et al., 2003) and TF-IDF statistics.