Counterfactual Neural Temporal Point Process for Estimating Causal Influence of Misinformation on Social Media
Authors: Yizhou Zhang
Defu Cao
Yan Liu
Venue: NeurIPS 2022 Accept (2022)
Keywords: Causal Inference
Temporal Point Process
Misinformation Influence
Counterfactual Analysis
Fake News
Social Media
Deep Learning

Abstract: Recent years have witnessed the rise of misinformation campaigns that spread specific narratives on social media to manipulate public opinions on different areas, such as politics and healthcare. Consequently, an effective and efficient automatic methodology to estimate the influence of the misinformation on user beliefs and activities is needed. However, existing works on misinformation impact estimation either rely on small-scale psychological experiments or can only discover the correlation between user behaviour and misinformation. To address these issues, in this paper, we build up a causal framework that model the causal effect of misinformation from the perspective of temporal point process. To adapt the large-scale data, we design an efficient yet precise way to estimate the \textbf{Individual Treatment Effect} (ITE) via neural temporal point process and gaussian mixture models. Extensive experiments on synthetic dataset verify the effectiveness and efficiency of our model. We further apply our model on a real-world dataset of social media posts and engagements about COVID-19 vaccines. The experimental results indicate that our model recognized identifiable causal effect of misinformation that hurts people's subjective emotions toward the vaccines.

Counterfactual Neural Temporal Point Process for Estimating Causal Influence of Misinformation on Social Media
Yizhou Zhang,, Defu Cao', Yan Liu
Department of Computer Science
Viterbi School of Engineering University of Southern California
fehangyiz,defucce yanliu. cs) Cu)cusc.eeee edu
Abstract
Recent years have witnessed the rise of misinformation campaigns that spread specific narratives on social media manipulate public opinions on different areas, such as politics and healthcare. Consequently, an effective and efficient automatic methodology estimate the influence of the misinformation on user beliefs and activities is needed. However, existing works misinformation impact estimation either rely on small-scale psychological experiments or can only discover the corre- lation between user behaviour and misinformation. To address these issues, in this paper, we build up a causal framework that model the causal effect misinforma- tion from the perspective of temporal point process. To adapt the large-scale data, we design an efficient yet precise way to estimate the Individual Treatment Effect (ITE) via neural temporal point process and gaussian mixture models. Extensive experiments on synthetic dataset verify the effectiveness and efficiency of our model. We further apply our model real-world dataset of social media posts and engagements about COVID-19 vaccines. The experimental results indicate that our model recognized identifiable causal effect of misinformation that hurts people's 's subjective emotions toward the vaccines.
Introduction
Recent researches reveals that widespread fake news and misleading information have been exploited by misinformation campaigns to manipulate public opinions in different areas, such as healthcare
18 and politics (20 To address this crucial challenge, research efforts from different perspectives have been devoted, such as fake news detection and coordination detection 35 36 38
36
However, an essential associated research question has not been explored sufficiently: how to know a piece of misinformation' 's causal influence on a user's beliefs and activities on a large-scale social media. Precisely estimating such impact is crucial for misinformation mitigation in various areas, e.g. delivering the corresponding clarification contents to the users that are most likely to be affected, allocating resources for more efficient and effective misinformation mitigation, and helping researchers understand misinformation campaigns better. Nevertheless, most of existing researches in social media analysis focus understanding correlation between misinformation and user activities,
rather than causal effect (26] 38 [8 45 As a result, they can not distinguish the effect from personal prior beliefs and engagement with misinformation. Current researches misinformation causal influence on people are mainly from psychology field [5 42]. They are usually based on carefully designed psychological randomised controlled trials recruited subjects. Thus, is impossible
"Equal Contribution
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

extend them onto large-scale social media platforms due to the high cost to recruit enough subjects and ethical risk in conducting such a large-scale psychology experiments.
Since personal beliefs are usually unobservable, researchers usually apply the feature of the tweets generated retweeted by the users as a proxy 38] However, the lack of appropriate algorithmic tools to conduct causal analysis on social media posts prevents researchers from understanding causal effect of misinformation. The processes that social media users generate original posts and engage with existing posts are typical temporal point processes. But existing methodologies for temporal causal effect estimation mostly focus on covariates and outcomes continuously distributed on timeline 4 3 2 & To), rather than discrete event points randomly scattered on timeline. Although the essential theory for counterfactual analysis of point process is already established (30], most works are motivated by healthcare and thus focus on the hazard models, e.g. survival analysis I or the chance to catch cancer [3I], which only consider the single occurrence of the most recent future event. However, on social media, we care more about multiple events happening in time window. [14] and 24 are rare works studying the causal effect on multiple occurrences in temporal point process. mainly focus on simulating the counterfactual events given intervened intensity function rather than learning the treatment effect of specific factors on the process. As for [14] one of its assumption is that the event marks must be categorized to finite number of classes, leaving no space for the rich continuous features of social media posts, such as user sentiment and subjectivity scores.
But 2
In this work, we propose a framework that models the causal effect of given piece of information on user beliefs and activities via counterfactual analysis on temporal point process 77
48
with continuous features. We first define a causal structure model that characterizes the misinformation impact as how the engagement with the misinformation change a user's intensity function of generating original posts. In this model, the engagement with misinformation is considered as the treatment, and the user's future conditional intensity function is considered as the outcome. Then we design functional that converts the change of two functions to a vector with intuitive physical meaning (3.. To estimate the effect, we design a neural temporal point process model. disentangles the distribution of event timestamp and post feature (e.g. the text embedding). Then it models the distribution of post features and event timestamp with Gaussian Mixture Model and temporal point process respectively. Such design enables it to acquire closed-form solution of the feature expect without losing expressive power, leading to a balance between precision and efficiency.
51 25 Fl
A critical challenge in training neural networks to recognize causal effect is the hidden bias in the dataset. In social media data, the most crucial bias is from information cocoons 500 users tend more to engage with the contents that they are interested in, and thus personalized recommendation systems will deliver user more contents that they are interested in to increase user engagement. Such bias leads to data distribution different from randomised controlled trials and thus make neural networks give biased estimation. To decorrelate time-varying treatment from user's covariates and history in point process, we apply adversarial training optimize a min-max game. More specifically, the encoder tries to minimize the likelihood of the observed treatments while a treatment predictor tries to maximize it. Our theoretic analysis proves that any balanced solution of the min-mas game, rather than the global optimal solution in existing works [4), can help us remove the bias from information cocoons. In addition, the extensive experiments on synthetic datasets and real-world datasets indicate that our framework is able to approximate unbiased and identifiable estimation on the causal effect. In conclusion, the main contributions of the proposed model are as follows:
We propose a novel research problem on misinformation impact, which aims to find the causal effect of misinformation on users' belief and activities on social media.
We propose a causal structure model to quantify the causal effect of misinformation and further design a neural temporal process model to conduct unbiased estimation to the effect.
We evaluate model synthetic datasets examine its effectiveness and efficiency and ues it to recognize identifiable causal effect of misinformation from real-world data.
Related Work
2.1 Influence of Misinformation
Recent researches about misinformation mainly focus on detecting fake news 35| mation campaign detection t7 40 and understanding how fake news attract user engagement 8 g
27
misinfor- 32 KE

---

Some researcher attempts to study the relation between misinformation and people's behaviours However, most of them focus on mining the correlation between misinformation and people behaviours rather than causal effects. Only a limited amount of works, such as 42 try to understand the causal effect. However, they are usually from psychology field, and mainly rely on carefully designed randomised controlled trials. Extending such trials on large-scale social media platforms brings not only high cost but also potential ethical risk.
15
22
3R T
2.2 Temporal Point Process
The process that a user retweet posts tweets usually be modeled temporal point process with
event feature 67 52 B3 10 A temporal point process (TPP) with event feature is a stochastic process
whose realization is sequence of discrete events in continuous timeline: S [(f1, f1), (f2, f2), where A is the event feature (a scalar or a vector) and t is the timestamp of the event. A TPP is fully characterized by an intensity function X(f. = defined in the following integral equation:
((N T2) = df L
T2 ^(f. d(
(1)
where F' is an area in the feature space, Sr, is the historical sequence of all events happening before time Ti. (NF, , T1, T2) is the number of events whose feature vectors are in F and timestamps in the range [Ti, T2]. The meaning of X(f. t(.) is the expected instantancous speed that the user generate posts at point f in the feature space on time The process after time t; is fully described by A Sp)) 77. Recent works propose to apply neural networks to model the À function
2.3 Counterfactual Analysis on Temporal Point Process and Continuous Time Series
The works focusing on studying the causal effect on multiple occurrences in temporal point process are rare. In [24), the authors mainly focus on the sampling of counterfactual events rather than the learning the influence of specific factors on the intensity function. Another work [4] proposes counterfactual analysis framework to understand the causal influence of event pairs in temporal point process. It defines the individual treatment effect (ITE) of an event toward future process as:
1 1+T T /.
ITE = (t, (t, T)
(t, +
X1(0) -(()))((tddt
(2)
where My is the expect of the event number of type y per unit time in time range [t, T). 1i indicate the case that a treatment is applied (exposed misinformation) and n1) in contrary. However, this metric is only suitable in the case that the events can be categorized to finite discrete types. This is not applicable for social media post because most meaningful features of the posts, such as geographical information, sentiment score and subjective score, are naturally continuous. Forcibly discretizing them will lose meaningful information. Besides, for counterfactual analysis of time 4 proposes CRN, neural model that can learn unbiased estimation to counterfactual world and causal effect. novel neural controlled differential equation model. [2] introduces new causal prior graph to avoid the undesirable explanations that include confounding or noise and use a multivariate Gaussian distribution to model the real continuous values. However, all of them focus modeling an observable variable existing on a continuous timeline instead of temporal point process. Unless getting heavily revised, such previous work can not be simply transferred to our problem scenario.
series,
3 proposes to analyze counterfactual estimation using synthetic controls via
3 Proposed Causal Structure Model and Treatment Effect
3.1 Causal Structure Model
In this study, we focus on understanding how a user's engagement with the misinformation post causally influence the characters of the posts generated by the user in fixed future time window. We formulate the process that a user interact the post shared by others and generate social media posts as
We use bold font to emphasize that f feature vector rather than function.

---

two temporal point process where each event carries a continuous outcome vector. We denote the process of engaging the diffusion of post as P. and the process generating new posts as pg. Then the realization of the two temporal point process are respectively two sequences S. and Sg of discrete events with continuous outcome vector in continuous time range:
S. = 1( ( (11) ,1,)), Sg = 1(1 (1)).. (1) (0) )....
where f is the feature vector char- acterizing the event and is the time stamp. For Se, each event corre- spond an interaction (e.g- "like" or comment), and the vector f(+) is the feature of the content (like the text representation, sentiment score and metadata) from others. Simi- larly, for S.. the vector f(s) is the feature of the content generated by the user. To examine the causal ef- fect of an interaction event on the posts generated by the user in the future, we construct the following causal structure model, formulated as < X, >, where is the covariate, Y is the outcome and I' is the treatment. In this model, given an interaction event (f((e) {(()) whose causal effect is to be examined, we consider the this event as the treatment Tr, and all the events, including both
Covariates:
History Activities
Post
Post
Post
Treatments:
Post
Engage
Post
Eingagement with Misinfomaaion
Sampling
Information
Engage Engage
Post
NEWS
The proposed causal structured model describing the impact of a piece of information on user.
Figure 1:
engagement events and posting events, that happen before (.()) are considered as the covariates. As
for the outcome, rather than simply considering the most next generation event after {((). we need a representation that can reflect the change of the whole generating process Sg in fixed future time window T. Thus we apply the conditional intensity function of the future process, denoted as X(f, 4 U X), as outcome. As discussed in the related work section, À function completely describe the future process. The overview of the model is presented Figure.
3.2 Treatment Effect Evaluation
In traditional counterfactual analysis works, the outcome is usually a scalar vector with finite dimensions. Thus, the treatment effect can be trivially computed by comparing the difference the outcomes from real world and counterfactual world. However, in framework, it is coiviiivial to compute the difference of two functions. To overcome this challenge, we propose to first apply a functional F to project the À function to vector with finite dimensions:
ptt,t Tr p(t,t+ T,A,+T,ATTUX.
(A(T,TUU
(4)
(1,.+T,,,TTUUN) ES-P(SIATIOII
Σ fil
(5)
M(,,LTT,A,TUUN = BS-PI(SAATUX)
(6)
where Tr is the distribution of the event sequence S sampled from the temporal point process described by -(Tr X). T is the time window that is a hyper- parameter, sup(f) is the UX). support set of (the area where the probability density is larger than 0), and S is a sub-sequence of 5. SEL4T contains every event in S that happens at a time between t and + T. The intuitive meaning of F is the expected mean feature vector of all posts generated by a user. Thus, by comparing the outputs F in real world and counterfactual world, we can see how the engagement with a specific post change the average features, e.g. general sentiment scores or text embedding. With this functional, we can simply compute the individual treatment effect as:
ITE Fy (A,TrUx) - Fr,
where is an empty set, (A X) is the functional from counterfactual world which we assume that the treatment is not applied (e.g. the misinformation post is not recommended or labeled as

---

misinformation). For brief, we write ( as (A, The overall impact of the treatment can be represented with the average treatment effect:
ATE = E (A,
- (A)
(8)
where U is the set of users who engaged with the treatment post.
3.3 Treatment Effect Calculation
In the above sections, we define the causal structure model and the treatment effect. However, the above formulas are hard to compute, Therefore, in this subsection, we will derive a computable formulation of the treatment effect. We will start from the following theorem:
Theorem 1. For user u, if the intensity function X(f, 1 U X) is known, then we have:
L+T
(,, A1, TrUX)=
df
Xf UT)d
(9)
su((s)
J.mA) fdf S. Xf
(10)
The first equation can be trivially proved by replacing the F in Equation 25 with the support set. The second one can be proved with the Camphell's Theorem [5]. A detailed proof provided in the Appendix I The above formulas contain double integral, which is inefficient to compute. To transform the double integral to single integral, based on previous work in spatial-tempon point process we have:
A(f. A.f. UX) A(t]Tr U (f Tr Ir, XX)
(11)
Thus, we can disentangle X(f,t) t) and respectively model A(t)) and p(flt). More importantly, we can simply model j
J. ++T
I+T
u(t,t+T,A,TUU)=-
X((T+UX)dt
A(ETrUX)dt
smplf)
(12)
And with this formula, we have:
@ TTTTUTATUUU
suppl))
XUTTUUX)N J.NC fN(fI,TTUXXj
(13)
The above formulas contain only single integrals. Thus, they can be efficiently approximated with summation: f fx))d== x (12-011/4 f(a + Arr)Ar
4 Neural Estimation of Treatment Effects
The above section construct causal framework that can measure the impact of a given social media post based on the change of A(t|Tr U X)) and p(f, Tr U X). In this section, as shown in Figure we will further discuss how to estimate the impact with a neural temporal point process model.
4.1 Learning Conditional Intensity Function via Maximum Likelihood Estimation
The log-likelln log-liiel of an observed event t)) (no matter an engagement event or an generation event) can be written as:
[ X( UUTrUXXdt bogg
log (/Tr log X(t
(14)
where f, the timestamp the last event in the set Tr X. The above equation provides us with a way learn A(t U X) and p p(flt. Tr XX) by maximizing the likelihood of each event given the historical information (treatment and covariates). To enable the model to make correct prediction for both A( (|Tr X) and X(f, (X)., we construct two kinds of samples train the functions:
5

---

Samples with valid Treatment: If
2(eIH(TT UX)
P( U .), P(1(((2).
for a generating event (f )(a),
((((8(0))
((T)((())
its most recent previous event is an engagement event (f(e), ) (in other words, the user does not have other activity between the en- gagement event and the generat- ing event). then we can construct sample (Y, Tr. X), where Y (f(l), t())), Tr = (f(e), g(e)), and X is a sequence that contains all engagement events and generation events before Tr.
Intensity Function
Feature Distribution
Treatment
Predictor
GRL: adversarial
u(TruX)
H(X)
training
H())
Share Encoder (H)
(X.Trux)
(?r,x)
Samples without Treatment: If for generating event (f(m), t(0), its most recent previous event is still a generating event (in other words, the user generate two original posts without engaging with other posts), then we can construct a sample (Y,X), where Y = (f(1),, (()) and X is sequence that contains all en- gagement events and generation events before In this sample, between the last generation event in X and Y, there is no interruption from treatment. Thus, it helps the model to learn X(t]X) and p(f
Engage
Post
Post
Engage
Post
Figure 2: The proposed neural model estimate the impact of misinformation.
For sample (Y,T)UX) or XY, we first use a shared encoder (-).) to project (Tr X) (or X) representation vector ([r X) (or h = (X) for the case where treatment NULL). Then we model the intensity function and feature distribution as ((t(h) and p(flt,) h). For A(([h), following FullyNN, we use multi-layer perceptron M LL t) model its integral fr, A(t)h)dt. The MLP's partial derivative with respect to is X(([h).
To model p(f it. h). straightforward solution borrowed from generative deep learning is to apply neural network, i.e. the decoder, to transform simple distribution, c.g. Gaussian distribution whose parameters are decided based on and t, to a complicated distribution. The decoder can be trained via diffferent loss function, like reconstruction eπror (variational auto encoder) and likelihood (normalizing flow 44 However, this method has an important drawback: its conditional expect E[flt, ht] does not have closed-form solution. To compute the expect, we only apply sampling or approximation, e.g. forwarding the expect of the simple distribution into the decode HI To address the above challenge, we propose explicitly model p(f(t, h) with a mixture of Gaussian distributions:
p(flthh) Σ (ns(t,hof Je uz(t,
(15)
(t,
(c h) softmax(MI.PP..000 f)), o (xpp))))).. 4hh MLP (h,t) (16)
where wy is the mixture weight, oj is a scalar, uj is a vector with the same dimension as f. and g(-) is a standard multivariate gaussian distribution \/ I) whose covariant matrix an identical matrix. Although the formula of each component is simple, their mixture has a theoretical guarantee on universal approximation to all distributions 44).. Because the expect of a mixture distribution the mixture of the expects, we have closed-form solution for E[f h]:
E STSpUELNN
Σ (1,
(17)
'GAN not suitable hecause for fixed h t we only have one sample, which can casily memorized. Beecasse the decoder D is non-line lonerr function, [x))(x)) is usually different from ((B[a))

---

4.2 Adversarial Balanced Neural Temporal Point Process
As discussed in the related work section, by maximizing the likelihood of the posts generated by the users, we can train a neural network that predict X(t[Tr U X) and p(f it. Tr U X). However, previous works have proved that if we do not balance the bias from the correlation between treatment and covariates, the model will tend to give biased prediction and thus can not give precise estimation of the treatment effect. A crucial bias social media data is information cocoon: personalized recommendation systems will deliver user the contents that they are interested in. For example, it will deliver more anti-vaccine posts to anti-vaccine users because they are more likely to be interested in those contents. As result, the anti-vaccine users will engage more with anti-vaccine posts.
To address the above issue, following previous works in neural counterfactual prediction, we apply domain adversarial training to learn a representation h that is invariant to such a bias More specifically, we hope learn a encoder H such that for any two users with different history X1 and X2, p(Tr) (X,)) = p(Tr| space, the probability that the two users interact with the same post at the same time should be same, which is the same as psychology experiments that divide the experimental and controlled groups randomly. To achieve this object, we apply adversarial training to remove the information about future treatment from the representation of covariates. More specifically, we additionally train treatment predictor ([T H (X)) by modeling (ex(((()) h) and ptr (f-(ff( h) with the encoding h of the historical covariates X. However, between the treatment predictor and the encoder, we insert gradient reversal layer (GRL) [E 28 ! to reverse the sign of the gradient. Thus, when we optimize the treatment predictor to maximize the likelihood of the observed treatment T'r = (f(e), (g(e),,())), the GRL will make the encoder to minimize the likelihood. This process leads to the following min-max game:
for the same Tr. In other words, in the representation
y max, Ba.maaaxx....peeeee.
(X))
(18)
The following theorem (proof provided in Appendix A. provide theoretic guarantee that the above adversarial training reduce the bias introduced by treatment-covariate correlation (e.g. recommenda- tion system and personal interest):
Theorem 2. Given the following min-max game:
Pr,,mmaappP,,,.ttttee log ((Tr)((())))
(19)
the min gamer's Nash balanced solution H", ennureesfor any X1, X2, the following equation holds:
p(Tr|H (X:)) = ((Tr(
(20)
where p denote the ground-truid truth conditional distribution of treatment given encoding.
Experiments
On real- world social media platforms, the ground truth causal effects of user engagement with posts, no matter misinformation post or information post, are unknown. To address the unknown ground truth causal effect, previous works of causality amalysis evaluate their models on synthetic dataset. In this paper, following previous works, we evaluate the performance of our model and compare it with baselines on synthetic dataset. Then we apply our proposed method to evaluate the impact misinformation on a real-world social media data about COVID-19 vaccine collected from Twitter
5.1 Experiments on Synthetic Data
Synthetic Data Generation:
To simulate the real-world social media, we generate 15000 users and 120 post of news. Each user is represented with a hidden vector u, which correspond to the status of social media user. Each piece news has two randomly generated feature vectors: a topic vector Ptopie (n) and an inherent influence vector ven (n). Each user has two kinds of activities: (1) engaging with one of the 120 news post and (2) posting post with original contents. The chance that user engage with a post is decided by V.. (n) and u, simulating information cocoons. Engagement event with news post will change the hidden status u of the user. The scale and direction of the change are decided by the current user status, the topic vector and the inherent influence vector jointly. For
*Code and data will be provided in http:://ggttuub..oo/////hhooseee
7

---

Table 1: Estimation Error the ground truth ITE
Method
Accuracy f
RAE
RRSE
Decoder Inference Time
FullyNN
73.0%
0.865
0.901
7.13ms
CNTPP- -APE (Approximation)
85.9%
0.279
0.503
4.05ms
CNTPP-VAL -AE (Sampling)
87.8%
0.237
0.454
ms
CNTPP((Our)
88.0 %
0.234
0.448
7.12ms
each user, the engagement events and the posting events are modeled through two Hawkes process respectively. Both Hawkes processes are influenced by user status u. Also, the feature each posting event f is drawn from distribution P(f [u, t) characterized by random parameterized multi-layer perceptron (MLP) taking (u, t) and random noises as input and output f. Thus, the engagement with the news post will have causal effects on the two processes. Since we have all parameters of the model, we can calculate the ground-truth ITE defined in Eq. 7 for the synthetic dataset. The details of the data generation algorithm is included in the Appendix B.2
Baselines: To the best our knowledge, the causality effect on temporal user behaviour from misinformation is not explored by previous works. Thus, we lack vell-established baselines for this specific task. To address this issue, we select some baselines from previous works on temporal point process and temporal causal inference and extend them to adapt our setting. FullyNN 25 is non-causal neural temporal point process that predict the user future behaviours without considering causal effect. We select it because has the same neural architecture our model. It can also be regarded as our model's 's variant w/o adversarial balancing. Neural-CIF is an extension of CIP pairs in temporal point process. We further
[1], which aims discovering causal effect of compare our model with an ablation variant: ANTPP-VAE. It replace our GMM-based decoder with Variational Auto-Encoder [77 Since VAE does not have closed form solution of feature expect, we report the results applying sampling and approximation separately.
In this work, we will evaluate the proposed model in two aspects:
ITE Estimation: we will evaluate the model by comparing ITE estimated by the model and the ground-truth ITE. We report three metrics: Accuracy (the model need to correctly predict whether the engage- ment increase or decrease each dimension of the expected average feature), Relative Absolute Error (RAE) and Relative Root Square Error (RRSE). addition, we also report the inference time of our model to reflect our model's 's efficiency.
Table 2: Causal Effect Inference
Method
MatDis
LinCort
Neural-CIP
0.90
0.04
FullyNN
0.93
0,236
CNTPP-VAS (Approximation)
0.84
0.303
CNTPP-VAL (Sampling)
0.76
0.287
CNTPP (Ours)
0.77
0.310
As shown in Table I our model establishes new state-of-the-art on all three quantitative metrics. This means that our model can fully utilize the causal information from the multivariates point process for unbiased treatment effect estimation. In particular, the model with causal analysis outperforms the model with direct neural network prediction (FullyNN), which leads to results that are not causally related. Simultaneously. our model outperforms all baselines in all three metrics of estimation precision. Although CRN-VAE incorporated with sampling method can achieve performance very close to us, it spends substantially longer inference time.
Causal Effect Inference: CIP defines the treatment effect way different from our model. Thus ITE estimation experiment is not fair for it. For a fair comparison, and also to further prove that our model can achieve unbiased estimation, we use all the models to predict the ATE of each news post. Then we evaluate the correlation between the learnt ATES and ground-truth average change of the news post to all users' hidden statuses. The more correlated the learnt ATE is, the better it reflect the inherent causal effect of the engagement on users. To evaluate the correlation, we apply the following two metrics: MatDis and LinCor. MatDis evaluates the similarity between the ATE-Distance matrix and StaidennStatu--Diii
matrix. LinCor evaluates the linear correlation between the learnt ATE
Because the authors did not opensource the code of original CIP and one important precious work that is crucial for CIP, implement version of CIP that apply neural network rather than graphical causal model.

---

Table 3: Comparison of (normalized) Average Sum of Distances with different methods on the real-world dataset. This metric reflect how well the group of data points is clustered.
Methods
ASD.
ASD
ASD mis
Event Feature
0.123
0.123
0.122
FullyNN
0.073
0.069
0.072
CNTPP (Ours)
0.045
0.042
0.044
and the ground-truth average hidden status change. Details of the two metrics can be found in Appendix B.4 From T the ATE of our model best reflect the ground-truth average change of the news post on the simulated data for both two evaluation metries. This suggests that our model has the potential to discover the influence of misinformation on social media users' hidden status, e.g. interest and idea.
5.2 Experiments Real World Data
In this section, we apply our proposed model on the Twitter dataset to estimate misinformation impact on social media scenario. We apply the data set collected in [49 38), including a total of 16,9008 tweets with labels from 24,192 191 users over a 5-month period from 2020/12/09 to 2021/04/24. Notably, focus on understand how the tweets that users retweeted influence their behavior of posting original tweets. For each post, its feature f includes: text representation (extracted with a pre-trained BERT), sentiment score and subjectivity score. We discover the following phenomenons with our model.
0.050
Information
0.2
Information
Misinformation
Misinformation
Subjectivity Score
0.025
PCA Vector2
0.0
0.000
-0.025
-0.2
-0.550
-0.01
0.00
0.01
00.02
00.01
0.00
0.01
Sentiment Score
PCA Vectorl
(a) Impact sentiments and subjectivity.
Impact text representation.
Figure 3: Analysis real world social media data
Identifiability between misinformation and information in influencing people's narratives: For the desired outcome, we analyze the distinguishability between "retweeting fake news" and "retweeting true news" events. More specifically, for each retweeting event, we use its ITE estimated with our model as feature (dimension reduced via PCA (11] and whether the content is information or misinformation as label. As shown on Figure 3(b] we can verify the identifiability of our proposed method the treatment effect of two types of the news are substantially different. This discovery not only supports that misinformation and information influence people's behaviour in different ways, but also provides us with new paradigm to detect fake news. We also calculate Normalized Averaged Sum of Distances (NASD, details in Appendix B.7) for information cluster, misinformation cluster and their joint set. The lower these metrics are, the better that information and misinformation are distinguished. The comparison of our model against FullyNN and event features on this metric is shown in Table B As
the ITE learnt by model can identify information and misinformation better than the baselines.
Misinformation is hurting people's subjective emotion related to COVID vaccine: To understand influence of misinformation in a more intuitive way, we analyze the impact of retweeting events on users' average sentiment score and subjectivity score in the future. The higher subjectivity the content gets, the more personal opinions rather than factual information text contains. Then, we use the proposed model to generate the estimated ITE for each retweeting event and plot the results

---

of fake news and real news, As shown on Figure ((a) (x-axis for sentiment score and y-axis for subjectivity score), we find that both information and misinformation do not substantially influence the users' subjectivity. However, information tends to make people optimistic about vaccines (true news increases the sentiment score), while fake news tends to make people feel negative about vaccines. This discovery strongly supports the hypothesis that misinformation is hurting people's subjective emotion toward COVID-19 vaccines, and suggests that misinformation could be causally responsible for vaccine hesitancy.
6 Broader Impact and Limitations
The predicted ITE scores of our model can bring impacts from two perspectives: misinformation mitigation and misinformation research. First, the predicted ITE scores help platforms allocate resources better for more efficient and effective misinformation mitigation. Here, resources include a wide range of specific concepts, including but limited the efforts of human verifiers, users' capacity to accept and spread the contents for clarification, and so on. Second, our proposed model provides researchers with a data-driven algorithmic tool to bridge the research in user behavior modeling and misinformation. This tool can help researchers in different ways, e.g. providing researchers a set of potential misinformation factors that could influence user behaviours, understanding misinformation campaigns, which spread misinformation with specific topics or narratives to influence public opinions, and designing better evaluation metrics for fake news detection
The proposed model also has some limitations. First, it mainly focus on the causal effect of engage- ment on posting. However, in real-world social media, there could be other impacts of misinformation, such as changing the user's 's preference of engagement, topics of interest and community identity 47 Also, due to the limitation of synthetic algorithm and the meta data in the real-world data, we did not consider that different types of engagement may have different impact strength. In addition, the real-world dataset experiment only consider one dataset related to COVID-19 19, which single topic dataset. Although our model does not prohibit from being generalized onto multi-topic datasets, e.g.. PolitiFact
40 41 how to verify the model performance and reliability on a multi-topic dataset is still questionable. These limitations provides strong motivation for further exploration on this paper's s topic in future works. Potential directions may include how to verify the model's is reliability on multi-topic datasets, how to generate synthetic data with more details and how to model more causal effect in real-world social media.
45 and GossipCop
7 Conclusion
In this paper, we propose framework to describe the causal structure model and causal effect about how misinformation influence online user behaviours. We further design a neural temporal point process model to conduct unbiased estimation on the causal effect in a data-driven approach. Experiments on synthetic dataset verify the effectiveness and efficiency of our model. We further apply our model on real-world dataset from Twitter and recognize identifiable causal effect of misinformation. The experiment results suggests that the misinformation about COVID-19 vaccine is hurting people's subjective attitudes toward vaccines. However, also noticeable that our model is a statistical machine learning model. Consequently. all of its estimation can only be regarded as reference rather than judgement. Also, misinformation campaigns could use the proposed approach to direct their editors to write more impactful fake news. A probable strategy to address this potential problem is to require social media platforms to raise necessary alerts to those suspicious articles that seems to be optimized. We discussed the strategy to detect such articles in the Checklist.
8 Acknowledgement and Funding Disclosure
This work is supported by NSF Research Grant IIS-2226087. Views and conclusions are of the authors and should not be interpreted as representing the social policies of the funding agency, or U.S. Government, Yizhou Zhang and Defu Cao are also partly supported by the Annenberg Fellowship of the University of Southern California. We sincerely appreciate the feedback, comments and suggestions from our anonymous reviewers.
We are thankful to anonymous reviewers for the discussion this issue.
10

---

References
[1] Odd O Aalen, Mats J Stensrud, Vanessa Didelez, Rhian Daniel, Kjetil Røysland, and Susanne Strohmaier. Time-dependent mediators in survival analysis: Modeling direct and indirect effects with the additive hazards model. Biometrical Journal, 62(3):532-544 -4999, 2020.
[2] Mohammad Taha Bahadori and David Heckerman. Debiasing concept-based explanations with causal analysis. In International Conference on Learning Representations, 2021.
[3] Alexis Bellot and Mihaela van der Schaar. Policy analysis using symthetic controls in continuous- time. In ICML, 2021.
[4] loana Bica, Ahmed M Alaa, James Jordon, and Mihaela van der Schaar. Estimating counterfac- tual treatment outcomes over time through adversarially balanced representations. International Conference on Learning Representations, 2020.
[5] N. Campbell. The study of discontinuous phenomena. Proc Cambr. Phil. Soc, vol. 15:pp. 117-136, 1909.
[6] Defu Cao, Yujing Wang, Juanyong Duan, Ce Zhang, Xia Zhu, Congrui Huang, Yunhai Tong, Bixiong Xu, Jing Bai, Jie Tong, et al. Spectral temporal graph neural network for multivariate time-series forecasting. Advances in neural information processing systems, 33:1 33:17766- 2020.
[7] Ricky TQ Chen, Brandon Amos, and Maximilian Nickel. Neural spatio-temporal point processes. arXiv preprint arXiv:2011.04583, 2020.
[8] Lu Cheng, Ruocheng Guo, Kai Shu, and Huan Liu. Causal understanding of fake news dissemination social media. In Proceedings of the 27th ACM SIGKDD Conference Knowledge Discovery & Data Mining, pages 148-157, 2021.
[9] Lu Cheng, Ahmadreza Mosallanezhad, Paras Sheth, and Huan Liu. Causal learning for socially responsible ai. In 30th International Joint Conference on Artificial Intelligence, IJCAI 2021, pages 4374 4381. International Joint Conferences Artificial Intelligence, 2021.
[10] Abir De, Utkarsh Upadhyay, and Manuel Gomez-Rodriguez. Temporal point processes. Techni- cal report, Technical report, Saarland University, 2019.
[11] Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Confer- ence of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume (Long and Short Papers), pages 4171-4186, Minneapolis, Minnesota, June 2019. Association for Computational Linguistics.
[12] Nan Du, Hanjun Dai, Rakshit Trivedi, Utkarsh Upadhyay, Manuel Rodrez-R, and Le Song. Recurrent marked temporal point processes: Embedding event history to vector. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 1555-1564 2016.
[13] Yaroslav Ganin and Victor Lempitsky. Unsupervised domain adaptation by backpropagation. In International conference on machine learning, pages 1 1180-1189. PMLR, 2015.
[14] Tian Gao, Dharmashankar Subramanian, Debarun Bhattacharjya, Xiao Shou, Nicholas Mattei, and Kristin Bennett. Causal inference for event pairs in multivariate point processes. In A. Beygelzimer, Y. Dauphin, P. Liang, and Wortman Vaughan, editors, Advances in Neural Information Processing Systems, 2021.
[15] Daniel Jolley and Karen M Douglas. The effeets of anti-vaccine conspiracy theories vaccination intentions. PloS one, 9()):89177, 2014.
[16] Nitin Kamra, Yizhou Zhang, Sirisha Rambhatla, Chuizheng Meng, and Yan Liu. Polsird: Modeling epidemic spread under intervention policies. Journal of Healthcare Informatics Research, 5(3):231 2021.

---

[17] D. P. Kingma and M. Welling, Auto-encoding variational bayes. In International Conference on Learning Representations (ICLR), 2014.
[18] Katherine Kricorian, Rachel Civen, and Ozlem Equils. Covid-19 vaccine hesitancy: Misinforma- tion and perceptions of vaccine safety. Human Vaccines & Immunothhraapeutics, 18(1)11500044 2022.
[19] Steven Loria et al. textblob documentation. Release 0.15, 2:269, 2018.
[20] Luca Luceri, Silvia Giordano, and Emilio Ferrara. Detecting troll behavior via inverse rein- forcement learning: A case study of russian trolls in the 2016 us election. In Proceedings of the International AAAI Conference on Web and Social Media, volume 14, pages 417-427, 2020.
[21] Aleix M Martinez and Avinash C Kak. Pca versus Ida. IEEE transactions on pattern analysis and machine intelligence, 23(2):228-233, 2001.
[22] Hongyuan Mei and Jason M Eisner. The neural hawkes process: A neurally self-modulating multivariate point process. In Advances in Neural Information Processing Systems, pages 6754-6764, 2017.
[23] Chuizheng Meng, Sungyong Seo, Defu Cao, Sam Griesemer, and Yan Liu. When physics meets machine learning: A survey of physics-informed machine learning. arXiv preprint IIXI:22003.6679,, 2022.
[24] Kimia Noorbakhsh and Manuel Gomez Rodriguez. Counterfactual temporal point processes. arXiv preprint arXiv:2111.07600) 2021.
[25] Takahiro Omi, Kazuyuki Aihara, al. Fully neural network based model for general temporal point processes. Advances in neural information processing systems, 32, 2019.
[26] Judea Pearl. Causality. Cambridge university press, 2009.
[27] Verónica Pérez-Rosas, Bennett Kleinberg, Alexandra Lefevre, and Rada Mihalcea. Automatic detection of fake news. In Proceedings of the 27th International Conference on Computational Linguistics, pages 3391-3401, Santa Fe, New Mexico, USA, August 2018. Association for Computational Linguistics.
[28] Edward Raff and Jared Sylvester. Gradient reversal against discrimination: A fair neural network learning approach. In 2018 IEEE 5th International Conference on Data Science and Advanced Analytics (DSAA). pages 189-198, 2018.
[29] Aijaz Ahmad Reshi, Furqan Rustam, Wajdi Aljedaani, Shabana Shafi, Abdulaziz Alhossan, Ziyad Alrabiah, Ajaz Ahmad, Hessa Alsuwailem, Thamer A Almangour, Musaad A Alsham- mari, et al. Covid-19 vaccination-related sentiments analysis: A case study using worldwide twitter dataset. In Healthcare, volume 10, page 411. MDPI, 2022.
[30] Kjetil Røysland. Counterfactual analyses with graphical models based on local independence. The Annals of Statistics, 10(4):2162-2194 2012.
[31] Pål Christie Ryalen, Mats Julius Stensrud. Sophie Fosså, and Kjetil Røysland, Causal inference in continuous time: an example on prostate cancer therapy. Biostatistics, 21(1):172-185, 2020.
[32] Fatima K Abu Salem, Roaa Al Feel, Shady Elbassuoni, Mohamad Jaber, and May Farah. Fa-kes: a fake news dataset around the syrian war. In Proceedings of the International AAAI Conference on Web and Social Media, volume 13, pages 573-582, 2019.
[33] Karan Samel, Zelin Zhao, Binghong Chen, Shuang Li, Dharmashankar Subramanian, Irfan Essa, and Le Song. Learning temporal rules from noisy timeseries data. arXiv preprint 20Xi::2202.05443,, 2022.
[34] AR Sanaullah, Anupam Das, Anik Das, Muhammad Ashad Kabir, and Kai Shu. Applications of machine learning for covid-19 19 misinformation: a systematic review. Social Network Analysis and Mining, 12(1):1-34, 2022.
12

---

[35] Karishma Sharma, Feng Qian, He Jiang, Natali Ruchansky, Ming Zhang, and Yan Liu. Com- bating fake news: A survey on identification and mitigation techniques. ACM Transcations on Intelligent Systems and TEchnology, 2019.
[36] Karishma Sharma, Sungyong Seo, Chuizheng Meng, Sirisha Rambhatla, Aastha Dua, and Yan Liu. Coronavirus social media: Analyzing misinformation in twitter conversations. arXiv preprint 3rXiv:2003./23000 2020.
[37] Karishma Sharma, Yizhou Zhang, Emilio Ferrara, and Yan Liu. Identifying coordinated accounts on social media through hidden influence and group behaviours. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, pages 1441-1451, 2021.
[38] Karishma Sharma, Yizhou Zhang, and Yan Liu. Covid-19 vaccine misinformation campaigns and social media narratives. In Proceedings of the International AAAI Conference on Web and Social Media, volume 16, pages 920-931, 2022.
[39] Kai Shu, Deepak Mahudeswaran, Suhang Wang, Dongwon Lee, and Huan Liu. Fakenewsnet: A data repository with news content, social context, and spatiotemporal information for studying fake news on social media. Big data, 8 3:171-188, 2020.
[40] Kai Shu, Amy Sliva, Suhang Wang, Jiliang Tang, and Huan Liu. Fake news detection on social media: A data mining perspective. ACM SIGKDD Explorations Newsletter, 19(1):22-36, 2017.
[41] Kai Shu, Suhang Wang, and Huan Liu. Exploiting tri-relationship for fake news detection. arXiv preprint arXiv: 17 12.07709, 2017.
[42] Sander Van der Linden. The conspiracy-effect: Exposure to conspiracy theories (about global warming) decreases pro-social behavior and science acceptance. Personality and Individual Differences, 87:171-173, 2015.
[43] Nguyen Vo and Kyumin Lee. Where are the facts? searching for fact-checked information to alleviate the spread of fake news. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP 2020). 2020.
[44] Qi Wang, Guangyin Jin, Xia Zhao, Yanghe Feng, and Jincai Huang. Csan: A neural network benchmark model for crime forecasting in spatio-temporal scale. Knowledge- Baased Systems, 189: 105100, 120, 2020.
[45] Steven Lloyd Wilson and Charles Wiysonge. Social media and vaccine hesitancy, BMJ Global Health, 5(10):e004206, 2020.
[46] Assaf Zeevi and Ronny Meir. Density estimation through convex combinations of densities: Approximation and estimation bounds. Neural Networks, 10(1):99-109, 1997.
[47] Junzhe Zhang, Jin Tian, and Elias Bareinboim. Partial counterfactual identification from observational and experimental data. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvari, Gang Niu, and Sivan Sabato. editors, Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research, pages 26548-26558 PMLR, 17-23 Jul 2022.
[48] Qiang Zhang, Aldo Lipani, Omer Kirnap, and Emine Yilmaz. Self-attentive hawkes process. ICML, 2020.
[49] Yizhou Zhang, Karishma Sharma, and Yan Liu. Vigdet: Knowledge informed neural temporal point process for coordination detection on social media. Advances in Neural Information Processing Systems, 34, 2021.
[50] Frederik Zuiderveen Borgesius, Damian Trilling, Judith Möller, Balázs Bodó, Claes H De Vreese, and Natali Helberger. Should worry about filter bubbles? Internet Policy Review. Journal on Internet Regulation, 5(1), 2016.
[51] Simiao Zuo, Haoming Jiang, Zichong Li, Tuo Zhao, and Hongyuan Zha. Transformer hawkes process. NeurIPS, 2020.
[52] Simiao Zuo, Tianyi Liu, Tuo Zhao, and Hongyuan Zha. Differentially private estimation of hawkes process. arXiv preprint arXiv:2209.077 07300, 2022.
13


Reviews:
[{"review_id": "QxJqHYu9niP", "reviewer": "Reviewer_SxLq", "summary": "The paper describes a novel method of tackling the impact of misinformation in social media through the lens of causal analysis via a neural temporal point process model to estimate the effects using generated synthetic data and real-world data in the domain of healthcare (re: COVID-19 vaccines). The proposed method draws motivation from how misinformation is spread on social media sites and how related previous works have only focused on correlational studies and randomized trials without exploring causality-based studies. To properly evaluate the performance of the causal inference model, the authors used synthetic data. Results show that the proposed method slightly outperforms neural network models without causal modeling and autoencoder-based methods in accuracy and inference time. In the real-world data experiment, the authors draw summarized observations on how sentiments affect the behavior of users in spreading (mis)information and differentiation from the two using estimated ITE.", "questions": "1. Is the BERT embeddings concatenated together with the subjectivity score and sentiment score? How were the latter two obtained? Is there a reliability rate for these scores?\n2. In Section 5, there is a mention that previous models used synthetic datasets for evaluating proposals on causality analysis. Just out of plain curiosity, were the authors able to retrieve the generation parameters from previous works and apply the synthetic data generation close if not equal to the parameters used for potential comparison? Discussion on this is encouraged.\n3. What do/does the author/s infer from the overlap of (mis)information based on ITE scores in Figure 3B?\n4. In the experiment setup with both synthetic and realistic data, can the engagement actions (like, retweet, comment) be ranked or weighted according to how likely can trigger the user to spread (mis)information? For example, a user may have a higher chance of spreading a misinformed post after liking it *without* commenting on it as we don’t always assume that both actions are done together all the time. In this case, the *like* has more *weight* than the comment.", "limitations": "The paper poses no serious negative societal impact as long as the results, as mentioned by the authors, are only taken into consideration as additional reference and not an assumption of absolute truth in a decision-making scenario. In addition, as mentioned in the sections above, I suggest the authors include a Limitations section to cover discussion on the limitations of the study including the breadth of the domain used for measuring the impact and effect of (mis)information, data generation settings, and evaluation.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "I appreciate the paper’s sufficient and balanced level of technicality on treatment effects, adversarial training, and the proposed causal structure model. I find the motivation for applying causal analysis in alleviating fake news on social media solid. I also commend the authors for structuring the paper in such a way that it’s very easy to follow, read, and understand. I have a few points, however, that can be considered and discussed to further improve the quality of the work:\n\n1. The dataset used (both real-world and synthetic) should be at least described sufficiently in the paper including mentions of limitations of the domain used (health-related topics only), language specifications, and search parameters such as geolocation used to name a few. The same way goes for the evaluation metrics used: MatDis and LinCor. I believe these are essential information that should be present in the paper itself without being directed to the Appendix. In case of acceptance, I highly recommend including this information for completeness. \n2. Discussion on comparison between the proposed method vs. the CRN-VAE in Section 5.1 for causal effect inference and estimation residuals can be improved to add more depth. What is it with the sampling method of CRN-VAE that makes it relatively at par with the proposed method but lags behind inference time? Moreover, the note on heavily revising CRN-VAE confuses the reader. Aside from getting the sampled and approximated outputs, what other changes were made to the framework?\n3. If there are no statistical tests done, I suggest switching the term significantly to substantially as the word is already overused in literature without support.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "70L3qA3q11K", "reviewer": "Reviewer_5Zb3", "summary": "This paper proposes a model for the causal structure of the misinformation influence on users. It proposes one of the first studies of this kind building on temporal point processes and modelling users' engagement and posting behaviour as events while misinformation is considered as treatment. A treatment effect evaluation is proposed which projects the intensity function to a vector to measure the individual treatment effect. This is then calculated more efficiently through reformulations and then estimated with a neural temporal point process model. In the experiments, synthetic data is used to show the accuracy of the model, whereas a real-world covid-vaccine Twitter dataset is discussed as a case study to show how misinformation influences users' behaviour differently compared to information, among other findings. ", "questions": "Can you explain the main intuition of how this model works? \n\nWhat is the problem definition?\n\nCan you explain why the subjectivity score is always zero in fig 3a? what is the accuracy of the misinformation labels used in 5.2. as well as the sentiment and subjectivity scores? have you done any evaluation on how reliable these are? what is the range for sentiment scores?\n\nCan you explain how your data is different from the references cited? the data sizes do not match. ", "limitations": "Not discussed in the paper in depth. \n\nChecklist 4d -> using Twitter data is common in research but still, the question of consent is relevant and can be discussed. \n", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- well-motivated paper, and interesting problem \n- paper is generally easy to follow and well structured\n\nWeaknesses:\n- presentation and writing needs improvement\n- many key details are given only in the appendix and the paper is hard to understand as is \n- results are not convincing, and hard to interpret\n\n\nMinor comments: \n\"This is not applicable for social media post because most meaningful features of the posts, such as geographical information, sentiment score and subjective score, are naturally continuous. Forcibly discretizing them will lose meaningful information.\" =>  geographical information usually is at the city level, sentiment is often a classification. I am not sure what is the subjective score but overall the argument is weak.\n\nWhat is S_{in} in line 122?\n\nMany language/grammar errors, examples:\n\n\"Consequently, an effective and efficient automatic methodology to estimate the impact of the misinformation on user beliefs and activities.\"\n\n\"to study the relation between misinformation on people’s behaviours.\"\n\n\"Unless getting heavily revision, such previous work can not be simply transferred to our problem scenario.\"\n\n\"the treatment effect can be trivially computed by compare the difference of the outcomes from real world and counterfactual world\"\n\n\"we are lack of well-established baselines for this specific task.\"", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "5YoWHqdwens", "reviewer": "Reviewer_vdd1", "summary": "This paper proposes to estimate the impact of misinformation on real-world social networks. It constructs a causal framework and leverages techniques such as neural temporal point process and gaussian mixture models for this task. Experiments were conducted on synthetic and real-world datasets to demonstrate the proposed approach's effectiveness.", "questions": "1. What does ITE stand for? This might be domain-specific jargon and one could infer from line 156 that it denotes \"individual treatment effect\", while I feel that the authors could explain this at the first occurrence of ITE. This would help people like me to understand your work better :)\n\n2. There are a lot of details regarding the synthetic dataset: \"hidden vector $u_i$\", \"topic vector $v_{topic}(n)$\", \"influence vector $v_{in}(n)$\", \"a post of original contents\". (line 265-268) Maybe I am missing something, but Section A.3 in the appendix does not seem to describe every single detail of the synthetic dataset. Since the authors only compared the proposed approach with existing methods on the synthetic dataset, these dataset details are crucial to validate that the synthetic dataset is a good indicator of model performance.\n\n3. In checklist 3(c), the authors state that error bars are provided \"in the appendix\". In Section A.4, the authors do provide the error bars of inference time on the ITE estimation task. However, the error bars of performance metrics, such as Accuracy, RAE, and RRSE in Table 1 as well as MatDis and LinCor in Table 2, are perhaps more important, especially given that the proposed approach only slightly outperforms CRN-VAE (Sampling). I hope the authors could provide the error bars for these evaluation metrics.\n\n4. For the real-world dataset, the authors simply point to [31, 25], while I would like to see more explanation. Why this dataset? Why not existing datasets on fake news detection such as PolitiFact or GossipCop? This real-world dataset seems to only focus on COVID, so maybe a multi-topic misinformation dataset would better support the evaluation?\n\n5. I personally believe that Figure 3(b) could be stronger if the authors also plot this figure for other baselines. Yes, it seems that the estimated ITE vectors could differentiate between real and fake news, but perhaps a language model encoding of news text could achieve the same effect? What about baseline methods in Tables 1 and 2? I would hope for a quantitative comparison (using clustering metrics such as homogeneity score or V-measure) between the proposed approach of baseline methods in the format of Figure 3(b).\n\n6. I am curious about the results in Figure 3(a). It seems that all pieces of news result in almost 0 change in user subjectivity. In light of this, should the subjectivity score remain a factor in the causal framework? I would like to hear about the authors' remarks on this issue.\n\n7. I am excited about the potential of this approach to understanding the impact of misinformation on social media. However, I feel like this paper could better position itself in the context of misinformation research. Specifically, what are the implications of the predicted ITEs towards mitigating online information? How can other researchers use this model to advance misinformation research and address the infodemic?\n\n8. In checklist 1(b), the authors describe this work's limitations as \"certain baselines outperform us\". I would like to see more discussion about the limitations of the proposed method itself. For example, the framework only models two user activities: post and engage, while there are other user features and there are different types of online engagement such as like and block, which signals vastly different user attitudes. Besides, the method is only evaluated on one real-world dataset that is COVID-related, while whether the approach would generalize to multi-topic real-world social media discourse is questionable.\n\n9. In checklist 1(c) and Section 6, the discussion about potential negative societal impact could be enhanced. For example, operators of misinformation campaigns could use the proposed approach to evaluate how different pieces of misinfo perform in deceiving people and thus design highly misleading misinformation strategies. I would suggest a little more discussion about the societal impact.", "limitations": "Please refer to the questions section. I will definitely read the authors' follow-up comments and update my rating accordingly. :)", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- proposes a new and interesting direction of misinformation research\n- the proposed approach is well motivated and clearly grounded yet improved from previous works\n- superior experiment results on both synthetic and real-world datasets\n\nWeaknesses:\n- important details about datasets and experiments are missing\n- experiments could be improved to better prove the author's claims\n- underexplored implications for misinformation research and societal impact", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept