Latent Chained Comments to Retweet
Extraction on Twitter
Ryusei Takagi1 and Yasunobu Sumikawa1
Depat. of Computer Science, Takushoku University, 815-1 Tatemachi, Hachioji-shi,
Tokyo, Japan
r88446@st.takushoku-u.ac.jp, ysumikaw@cs.takushoku-u.ac.jp

Abstract. Twitter, one of the social networking services, has a retweet
function that displays tweets posted by other users on the retweeter’stimeline. As this retweet function spreads information, past studies havebeen conducted to analyze posts that are spread by many people. However, some Twitter users use the retweet function not only to spread
information, but also to discuss their opinions. In these discussions, theytend to do retweet ﬁrst, followed by multiple tweets of their own opinions.
We call the multiple tweets representing the opinions latent comments.
In this study, we propose algorithms that collect tweets which talk about
their previous retweets and are continuously posted. We have created anovel dataset including latent comments and evaluated our algorithm on
the dataset. We have conﬁrmed that the algorithm performs well on thedataset.
Keywords: Topic detection and tracking · Natural language processing· Machine learning · Twitter · commentary tweet

1

Introduction

Twitter provides the retweet function that allows Twitter users to share information posted by other users by showing the content on the users’ timeline. As thisfunction is eﬀective in delivering information to many users, it is also used forcorporate advertising. Indeed, the number of times a tweet has been retweeted
is one of the important indicators of popularity [4, 8].
The number of retweets on Twitter is an important measure of popularity;
however, as text is the basic way of communicating on Twitter, Twitter users
sometimes do retweets to start a discussion on the retweeted contents. in thisstudy, we deﬁne latent comments if the following two conditions are satisﬁed:
1) tweets classiﬁed as one of the three types: rule-, direct-, and indirect-basedreferring tweets related to the retweeted content, and 2) continuously chainedon the user’s timeline. Fig. 1 shows an example of latent comments. The tweet1
shown in Fig. 1(a) is an original tweet whereas the tweet2 in Fig. 1(b) asserts
the user’s opinion to the original tweet. Looking at Fig. 1(b), the user does the12

https://mobile.twitter.com/takeda828/status/1458052542269558786
https://mobile.twitter.com/takeda828/status/1458053054398222341

2

R. Takagi and Y. Sumikawa

Fig. 1. Example of a latent comment.

assertion by posting an original tweet instead of posting the tweet as a quote
tweet. Several Twitter users post tweets representing their opinions as original
tweets as well as the tweet referred to in the example. Therefore, it is necessary
to design algorithms for detecting such opinion tweets.
In this study, we propose three algorithms for automatically collecting latentcomments. The proposed algorithms are designed for the three types of latentcomment: 1) rule-based latent comments contain commonly used notations suchas “>RT” regardless of the topic, 2) direct-based latent comments contain thesame words as the retweeted text, and 3) indirect-based latent comments do notcontain the same words as the retweeted text.
To check the eﬀectiveness of our algorithm, we collected 3,293 tweets posted
by news agencies’ oﬃcial accounts and manually collected 100 test data including
latent comments of Twitter users who retweeted the tweets. Using these tweet
data, we found that the proposed algorithm achieved a high accuracy of 85.9%
as the F1 score.
Deﬁnitions: We deﬁne a tweet set as a latent comment if the content of thetweets is related to the previous retweet and the tweets are continuously posted
without including non-latent comments between them. Fig. 2 shows the concept of the latent comments. The solid and dotted lines indicate the connected
tweets as latent comments and non-latent comments, respectively. There are
two retweeted news tweets about an Olympic game and a trading event. Tweet1
follows the ﬁrst retweet about the Olympic game and contains “Opening ceremony”; thus, we regard this tweet as a latent comment. As Tweet2 includes the
keyword “Olympic” and follows Tweet1 and Retweet1, this tweet is also a latentcomment for Retweet1. The text of the last tweet, Tweet3, is “I will get up early
tomorrow”; it is unclear if the tweet points to the Olympic game. In addition,
there is a retweet between Retweet1 and Tweet3. We do not consider Tweet3 as
a latent comment for Retweet1.

Latent Chained Comments to Retweet Extraction on Twitter
Retweet1

Tweet1

Tweet2Retweet2

Tweet3RT
Olympic

Opening
Ceremony

Olympic
championRT
Trading

I will get up
early tomorrow

3

Fig. 2. Examples of chained latent comments

2

Related Work

2.1

Topic Detection and Tracking

Identifying latent comments is to extract tweet chains; thus, this study is oneof the topic detection and tracking (TDT) studies. Looking at previous TDT
studies, Radinsky and Davidovich proposed an algorithm that extracted news
chains representing causal relationships from the descriptions of past events topredict possible future events [10]. Comparing with our study, the past study
assumptions are based on long descriptions whereas our study assumptions are
based on short descriptions. For TDT studies on Twitter, there are methods todetect and track hot events from online news streams [9], detect global and local
hot events using local community detection mechanisms [11], and reporting reallife events to users in the human-readable form [7]. These past events use not
only tweet texts but also context information such as trending and geography.
By contrast, using this context information is diﬃcult in our study; therefore,
we use tweet text only.
2.2

Using Explicit Popularity on SNS

Twitter is sometimes used as a medium that reﬂects current trends or inﬂuences.
Studies have been conducted over the past decade to predict which tweets will
receive the most likes and retweets. [5] proposed a model that predicts the number of retweets within a given timeframe. Some recent studies proposed methods
to discover inﬂuencers on Twitter in addition to the popularity of each tweet,
and the number of retweets is used as one of the indicators of inﬂuencers [13,12, 2]. In the aforementioned studies, retweets themselves are considered as aninﬂuence. While previous studies consider retweets as an indicator of inﬂuence,
this study uses them as starting points for discussions and detects their tweet
chains.

3

Data Collection

3.1

Tweet collection

We collected all tweets by the Twitter oﬃcial API. Twitter provides three kindsof tweets: tweets, retweets and quote tweets. If a Twitter user posts an original

4

R. Takagi and Y. Sumikawa
Table 1. Statistics of collected tweet dataset
Number of news tweet
Number of news articles
Number of retweet IDs
Period of timestamps
Number of news categories

3,293
3,293
2,267,322
24 Oct. 2021 – 2 Mar. 2022
51

text, it is deﬁned as a tweet. If a tweet is re-posted by a user using Twitter’soﬃcial retweet function, it is called a retweet. If a tweet is re-posted with newcontent, it is called a quote tweet; thus, a quote tweet can be called as a re-tweet
with a comment. In this study, we treat all quote tweets as original tweets asthey include additional information/text.
We collected tweets from oﬃcial accounts of news agencies to create ourdataset. The reason of this selection is twofold. The ﬁrst one is that the accounts
have numerous numbers of tweets. The second one is that the number of peopleviewing those tweets is also large. These reasons are necessary to perform adetailed analysis including what kind of tweets are latent referring tweets as
discussed in Section 5.
We collected news tweets only from Japanese news agencies’ accounts andperformed manual dataset creation so that we can properly analyze whether
Twitter users’ tweets are references to news. The news agencies we collected were
the Yomiuri Shimbun, Asahi Shimbun, NHK, Mainichi Shimbun, and Yahoo
News3 . For these accounts, we used user timeline4 an oﬃcial Twitter API to
retrieve tweets from the news agencies’ accounts.

3.2

Latent comment candidate collection

In this study, we used two Twitter oﬃcial APIs, retweeters5 and user timeline
API, to collect the Twitter users who retweeted each news tweet from the news
agency tweets and their latent comments.
The detailed collection procedure is as follows. First, to collect the tweetsthat retweeted the news agencies’ tweets, we used retweeters to obtain the user
IDs of the users who retweeted them. For these user IDs, we retrieved past tweets
using the user timeline API and collected 10 tweets each before and after the
retweet of the news tweet. Tab. 1 shows the statistics of the collected tweets.
3

The names of the news agency accounts we collected are Yomiuri Online, asahi, nhk news,
mainichi, YahooNewsTopics.
4
https://developer.twitter.com/en/docs/twitter-api/v1/tweets/timelines/api-reference/
get-statuses-user_timeline
5
https://developer.twitter.com/en/docs/twitter-api/v1/tweets/post-and-engage/api-reference/
get-statuses-retweeters-ids

Latent Chained Comments to Retweet Extraction on Twitter

4

5

Algorithm

Our algorithm determines whether the given tweet is a latent comment to thegiven retweet or not. If the algorithm decides that the tweet is a latent comment,
the algorithm returns true; otherwise, it returns false. If there are other tweets
between the two given IDs, the algorithm traces them one by one to determine if
each tweet is a comment on the given retweet. For this determination, we propose
three types of latent comments: rule-, direct-, and indirect-based comments. The
details of identifying chained comment tweets and the 3 types of tweets and thealgorithms for detecting them are in the remainder of this section.
4.1

Chained comment tweet identiﬁcation

This algorithm determines whether a given tweet is a comment on another given
retweet. As a single tweet can only contain 280 characters, Twitter users sometimes post their opinions in multiple tweets. To track such tweets, we retrieve
other tweets between a given tweet ID and retweet ID. As the purpose of thisretrieving is to reveal posts that divide one’s opinion into multiple posts, it isassumed that there are no other retweets between them. We apply the proposed
algorithm recursively to the obtained tweets to analyze whether they are latentcomments or not. If we ﬁnd other retweets posted at this time, then the recursive analysis is terminated and false is returned. We also terminate the recursive
analysis if false is obtained by applying the proposed algorithm described in thefollowing sections. If the recursive search visits the retweet ID and determinesthat it is a latent comment, the algorithm returns true. Once this recursive search
returns false, our algorithm considers the given tweet as a non-latent comment.
4.2

Rules in latent comments

On Twitter, there are widely used keywords such as “> RT” when expressing
own opinion after a retweet. We consider rule-based latent comments as tweets
including these keywords in their texts. The algorithm to ﬁnd this type of tweet
is based on the presence of such keywords in the text.
4.3

Direct-based latent comments

We call tweets direct-based latent comments when the tweets explicitly use words
contained in the retweeted content. To collect these tweets, we use the Jaccard
coeﬃcient that calculates the similarity between two texts based on the numberof words they share. If the score is over a given threshold, we consider thetweets as latent comments. In this study, we set 0.2 as the threshold. The formal
deﬁnition is as follows:| TA ∩ TB |Jaccard (A, B) =(1)
| TA ∪ TB |where: | · | is the size of the set and TA and TB are the tokens included in tweet
A and B, respectively. The higher the score of the measurements, the morecorrelated they are.

6

4.4

R. Takagi and Y. Sumikawa

Indirect-based latent comments

We call tweets indirect-based latent comments if the tweets use indirect-referringwords to refer to the retweets instead of using words contained in the retweeted
content. To collect these tweets, we cannot use direct keywords as a ﬁlter; thus
we perform latent semantic analysis such as LDA [1], LSA [3], and Doc2Vec[6].
We ﬁrst create feature vectors with the latent semantic analysis models equippedwith TF-IDF model. TF-IDF indicates the importance of a word to a documentin the dataset. This score is a multiplication of the term frequency and inverse
document frequency. Term frequency refers to how frequently each term (word)
occurs in each document, whereas the inverse document frequency represents how
rarely each term occurs in all documents. The formal deﬁnition is as follows.
TFIDF (w , d , D) = tf w,d ∗|D|| {d′ ∈ D | w ∈ d′ } |(2)where tf w,d is the number of times a word w occurs in a document d and | • | isthe size of •. The second term of this equation gives the number of all labeleddata divided with labeled data including w.
We then compute the cosine similarity between the feature vectors of retweets
and news agencies’ tweets. We regard the tweet as latent comment if the scoreof cosine similarity is over 0.5 in this study.
4.5

Overview of identifying latent referring tweet algorithm

At the beginning, we input two tweet IDs: an original tweet ID and a retweet
ID. We then collect other tweets between the given two tweet IDs described inSec. 4.1. Thereafter, we apply morphological analysis, lemmatization, stop word
removal and other fundamental natural language processing techniques to create
tokens for each tweet. We then sequentially apply the procedures described inSec. 4.2, 4.3, and 4.4 to analyze if each tweet is a comment on a given retweet.
After applying the three processes, we return the result of applying logical OR
to all the results obtained.

5

Experimental Evaluations

5.1

Experimental setting

Dataset. We have created a dataset including latent comments for this study
because there was no ground truth dataset for latent comment extraction task.
We ﬁrst collected 3,293 news tweets as described in Section 3.1 from Oct. 24,
2021 to Mar. 2, 2022. We then randomly selected 1,000 news tweets and collected
tweets posted after retweeting the news tweets. We manually checked whether
the tweets are latent comments for retweets. If the tweets were determined aslatent comments for the retweets, we combined them as pairs in our dataset.
This was performed by two workers, including Ph.D. holder working on machineLatent Chained Comments to Retweet Extraction on Twitter

7

Table 2. Rules used in latent comments. We add English words to Japanese to makeit easier to understand though we used only Japanese in the experimental evaluation.
> RT
↓

→ RT
承前 (continue)(RT)（RT 参照） (RT reference)

RT >

RT

RT:

Table 3. Micro-average precision, recall, and F-scores of latent comment or not classiﬁcation by each algorithm.

Rule
Direct reference
Indirect reference
All

P
52.0%
86.0%
81.0%
86.0%

R
52.0%
86.0%
81.0%
86.0%

F1
52.0%
85.9%
81.0%
85.9%

learning research. Then, the collected pairs were veriﬁed on agreement of thetwo workers, and the veriﬁed pairs were kept for the experiment. As results, thedataset contained 50 latent comments. We also added 50 non-latent comments
into the dataset. After removing the test data, we trained LDA on the collected
tweet data to use the model as an indirect-based latent comment identiﬁer.
Except for the correct answer data above, the authors extracted rules from
5,000 randomly selected tweets that could be used in a rule-based algorithm.
Tab. 2 lists all the keywords obtained from this manual process. These keywords
are used in this study.
Evaluation Criteria. We evaluated the proposed algorithm as a binary classiﬁcation problem to determine whether the result is a latent comment or not.
Therefore, we evaluated the algorithm with micro-average precision (P ), recall(R), and F1-score (F1 ), which are widely used to evaluate classiﬁcation. To
better understand the accuracy of the proposed algorithms, we evaluated theprediction results of each algorithm as a multi-class classiﬁcation problem. As
the number of test data is diﬀerent for each label, we used the macro-averages
precision (maP ), recall (maR), and F1-score (maF ).
5.2

Results

Q. How accurate is the proposed algorithm in detecting whether a commentis latent or not?
A. The proposed algorithm achieved 86% as P , R, and F1 scores.
Tab. 3 shows P , R, and F1 scores of four cases where each of the threealgorithms proposed in this study is applied and where all results, represented asAll, are combined by OR operator. We can see that using all the three algorithms
and Direct reference are the best among the four cases. As all the P , R, and F1

8

R. Takagi and Y. Sumikawa

Table 4. Macro-average precision, recall, and F-scores of multiclasss classiﬁcation byeach algorithm.

Rule
Direct reference
Indirect reference
All

maP
37.7%
45.0%
43.1%
95.2%

maR
49.5%
46.3%
27.7%
73.6%

maF
41.7%
45.4%
26.0%
75.3%

scores achieved approximately 86%, we can conclude that our algorithm works
well.
Q. How accurate is each algorithm for identifying each latent comment
type?
A. The algorithm of ﬁnding each latent comment type achieved a score of
approximately 26% to 45% as maF scores.
A. The algorithm combining the three algorithms achieved 75% as a maF
score.
Tab. 4 shows maP , maR, and maF scores for identifying each type of the latent comment by each of the three algorithms and All proposed. This result also
indicates that using all the three algorithms is the best among the four cases.
Especially, our algorithm achieved 75% as maF score.
To analyze the features of the proposed algorithm in detail, Fig. 3 shows
the results of our algorithm in correctly predicting whether a tweet is a latentcomment or not. The results show that our algorithm correctly predicted all nonlatent comments as non-latent; however, it mispredicted some latent comments
as non-latent. To analyze what kind of data was mispredicted, the confusion matrix summarizing the prediction results of each algorithm is shown in Fig. 4. We
can see that all rule-based latent comments were correctly predicted. By contrast,
many indirect-based latent comments were mispredicted as not latent comments.
Our dataset includes only 10% indirect-based latent commenting tweets; thus,
this misprediction had a small impact on the overall evaluation results. However,for improving accuracy, more sophisticated algorithms can be used to properly
discover indirect-based latent comments rather than the LSA used in this study.
5.3

Limitations

Although we focused on Japanese news tweets in this study, several Twitterusers post latent comments without limiting to news tweets. To analyze theexact overall trend of Twitter users, we need to collect not only news tweets but
also all tweets and their retweets. If we can solve the problem of limited use of
Twitter API and collecting all tweets or Twitter providing the data, the above
bias can be removed with the proposed algorithm. However, currently we cannot
solve the issue of data collection; thus we intend to study this issue as a futurework.

Latent Chained Comments to Retweet Extraction on Twitter

9

0.9
0.8

Not comment

1.0

Not comment
0.8

0.7
0.6

Direct0.6

Indirect

0.4

0.5
0.4
0.3

Comment0.2
0.1

Not comment

0.2

Rule
0.0

Comment
Not comment Direct Indirect

Fig. 3. Confusion matrix for binaryclassiﬁcation on identifying latent comment or not.

6

Rule

Fig. 4. Confusion matrix for prediction
results of each algorithm.

Conclusions

In this study, we have proposed algorithms to detect latent comments for the
retweets. These latent comments are tweets in which Twitter users discuss theiropinions after retweeting others’ tweets. The algorithms are designed for detecting three types of latent comments: rule-, direct-, and indirect-based latent comments. To evaluate the eﬀectiveness of our algorithm, we created a test dataset
and tested as classiﬁcation problems. The results showed that we conﬁrmed that
our algorithm obtained about 86% micro-averaged and 75% macro-averaged F1scores.
There are two possible directions for future research. The ﬁrst one is toestimate authentic popularity measures on Twitter. There have been studies
using Twitter popularity; however, most of them simply use the number ofretweets. However, as the deﬁnition proposed in this study, Twitter users sometimes retweet to start discussions or express their opinions. We will combine thealgorithm proposed in this study with sentiment analysis to identify whether
the latent comments for each retweet are positive or negative. The second one isthat we will apply our algorithm to other language tweets as this paper focused
only on Japanese tweets.

References
1. Blei, D.M., Ng, A.Y., Jordan, M.I.: Latent dirichlet allocation. J. Mach. Learn.
Res. 3, 993–1022 (Mar 2003)
2. De Salve, A., Mori, P., Guidi, B., Ricci, L., Pietro, R.D.: Predicting inﬂuential
users in online social network groups. ACM Trans. Knowl. Discov. Data 15(3)(Apr 2021)
3. Deerwester, S., T. Dumais, S., W. Furnas, G., Thomas K., L., Harshman, R.:
Indexing by latent semantic analysis. J. Amer. Soc. Inform. Sci. 41(6), 391–407(1990)

10

R. Takagi and Y. Sumikawa

4. Hong, L., Dan, O., Davison, B.D.: Predicting popular messages in twitter. In:
Proceedings of the 20th International Conference Companion on World Wide Web.
pp. 57–58. WWW ’11, Association for Computing Machinery, New York, NY, USA(2011)
5. Kupavskii, A., Ostroumova, L., Umnov, A., Usachev, S., Serdyukov, P., Gusev, G.,
Kustarev, A.: Prediction of retweet cascade size over time. In: Proceedings of the
21st ACM International Conference on Information and Knowledge Management.
pp. 2335–2338. CIKM ’12, Association for Computing Machinery, New York, NY,
USA (2012)
6. Le, Q., Mikolov, T.: Distributed representations of sentences and documents.
vol. 32, pp. 1188–1196. ICML’14, Bejing, China (22–24 Jun 2014)
7. Lei, Z., Wu, L.d., Zhang, Y., Liu, Y.c.: A system for detecting and tracking internet
news event. pp. 754–764. PCM 05, Springer-Verlag, Berlin, Heidelberg (2005)
8. Naveed, N., Gottron, T., Kunegis, J., Alhadi, A.C.: Bad news travel fast: A contentbased analysis of interestingness on twitter. In: Proceedings of the 3rd International
Web Science Conference. WebSci ’11, Association for Computing Machinery, New
York, NY, USA (2011)
9. Qi, Y., Zhou, L., Si, H., Wan, J., Jin, T.: An approach to news event detection andtracking based on stream of online news. vol. 2, pp. 193–196 (2017)
10. Radinsky, K., Davidovich, S.: Learning to predict from textual data. J. Artif. Int.
Res. 45(1), 641–684 (Sep 2012)
11. Tan, Z., Zhang, P., Tan, J., Guo, L.: A multi-layer event detection algorithm fordetecting global and local hot events in social networks. Procedia Computer Science29, 2080–2089 (2014), 2014 International Conference on Computational Science
12. Zhang, Z., Zhao, W., Yang, J., Paris, C., Nepal, S.: Learning inﬂuence probabilities
and modelling inﬂuence diﬀusion in twitter. In: Companion Proceedings of The
2019 World Wide Web Conference. pp. 1087–1094. WWW ’19, Association for
Computing Machinery, New York, NY, USA (2019)13. Zheng, C., Zhang, Q., Young, S., Wang, W.: On-demand inﬂuencer discovery onsocial media. In: Proceedings of the 29th ACM International Conference on Information & Knowledge Management. pp. 2337–2340. CIKM ’20, Association for
Computing Machinery, New York, NY, USA (2020)