Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 1: Reject
review_content: Three strengths:
1. This paper has been well written and easy to follow. Adequate details have been provided to help easily reproduce the experimental results.
2. The technical part is sound - the authors apply GAN for rumor detection and propose to use model-where and model-replace to extend conventional GAN models.
3. Experiments are conducted on real-world data.

Weaknesses:
1. Contributions of novelty are limited. The idea of using GAN to detect misinformation such as rumors and fake news has been studied in the literature several times, and the proposed method does not differ from them significantly. The problem of explainable rumor and fake news detection has also been well studied. Therefore, this piece of work is more a marginal extension of existing solutions.
2. The technical solution can be very limited. The generator can only manipulate content by replacing something from a true statement. The hidden assumption that misinformation is mostly generated by replacing some word definitely underestimates the complicated nature of fake news/rumor detection problem. If the assumption holds, the rumor detection problem can be easily done by collecting and comparing against true statements.
3. The limited experimental results cannot resolve my concerns. The rumor dataset is very small for a typical deep learning model. I am also curious about how many rumors in the dataset are generated by replacing words.

review_title: Official Blind Review #3

final_score: rating: 8: Accept
review_content: In this paper, the authors proposed an interesting model to solve rumor detection problem. The LEX-GAN model takes the advantage of GAN in generating high-quality fake examples by substitute few works in original tweets.  It achieved excellent performance on two kinds of dataset. 

The term ‘layered’ was a little confusing to me at the very beginning, though it is strengthened in many places around the paper. Maybe the author could use some other word to better summarize the two layers.

Another question is about the extended dataset with generated data, are they generated using the same distribution from G of the final model? What the result would it be if we use real, out-of-domain data?

I would like to see this paper accepted to motivate future works on fake news detection and rumor detection..

review_title: Official Blind Review #4

final_score: rating: 1: Reject
review_content: The paper presents a method for detecting rumours in text. Early on in the paper, the authors claim that this method:
1) is more accurate in rumour detection than prior work, 
2) can provide explainability, and
3) does not need labelled data because it is trained on synthetic "non-rumour looking rumours".

All three of these statements are problematic.
The experimental evaluation uses a small dataset of rumour classification (about 15000 tweet related to 14 news topics) and an even smaller dataset of gene classification. The rationale is to use the gene classification task as a proxy for rumour detection. This is not valid. The gene classification task does not contribute to the evaluation of the rumour detection method. The rumour classification dataset is relatively small, but even more importantly, the experimental results on that dataset are not thoroughly analysed, for instance through an ablation test. 

Explainability is not evaluated experimentally, nor formally proven. 

The claim that the method does not need labelled data because it is trained on synthetic "non-rumour looking rumours" is shaky, because 1) one could train the method on labelled data, and 2) it is not clear how "non-rumour looking rumours"  are guaranteed in the synthesis phase (how are they defined? how are they evaluated to be "non-rumour looking rumours"? etc).

Note that there is no definition of what sort of data representation corresponds to a "rumour" in the paper. 

review_title: Official Blind Review #2

final_score: rating: 3: Weak Reject
review_content: In the paper, authors proposed a generative adversarial network-based rumor detection model that can label short text like Twitter posts as rumor or not. The model can further highlight the words that are responsible for the rumor accusation. 

Proposed model consists of 4 sub models: a G_Where model finds the word to replace so to create an artificial rumor; a G_replace model decides what the replacement word should be; a D_classify model detects if a sequence is a rumor; a final D_explain model pinpoints the word of concern. D_ models and G_ models are trained in an adversarial competing way.

Experiments showed that the LEX-GAN model outperforms other non-GAN models by a large margin on a previously published rumor dataset (PHEME) and in a gene classification task.

My questions:

1) The task modeled is essentially a word replacement detection problem. Is this equivalent to rumor detection? Even if it performs really well on a static dataset, it could be very vulnerable to attackers. Various previous works mentioned in the paper, including the PHEME paper by Kochkina et al, used supporting evidence for detection, which sounds like a more robust approach.

2) Authors didn't explain the rationale behind the choice of model structure, e.g. GRU vs LSTM vs Conv. The different structures have been used in mix in the paper. Are those choices irrelevant or critical?

3) I would like to see more discussion on the nature of errors from those models, but it's lacking in the paper. This could be critical to understand the model’s ability and limitation, esp given that it’s not looking at supporting evidences from other sequences.

Small errors noticed: The citation for PHEME paper (Kochkina et al) points to a preprint version, while an ACL Anthology published version exists.
review_title: Official Blind Review #1

Meta-review: The paper is well-written and presents an extensive set of experiments. The architecture is a simple yet interesting attempt at learning explainable rumour detection models. Some reviewers worry about the novelty of the approach, and whether the explainability of the model is in fact properly evaluated. The authors responded to the reviews and provided detailed feedback. A major limitation of this work is that explanations are at the level of input words. This is common in interpretability (LIME, etc), but it is not clear that explanations/interpretations are best provided at this level and not, say, at the level of training instances or at a more abstract level. It is also not clear that this approach would scale to languages that are morphologically rich and/or harder to segment into words. Since modern approaches to this problem would likely include pretrained language models, it is an interesting problem to make such architectures interpretable. 
Decision: Reject