Towards Trustworthy Deception Detection: Benchmarking Model Robustness across Domains, Modalities, and Languages (2020)

Abstract: Evaluating model robustness is critical when developing trustworthy models not only to gain deeper understanding of model behavior, strengths, and weaknesses, but also to develop future models that are generalizable and robust across expected environments a model may encounter in deployment. In this paper, we present a framework for measuring model robustness for an important but difficult text classification task – deceptive news detection. We evaluate model robustness to out-of-domain data, modality-specific features, and languages other than English. Our investigation focuses on three type of models: LSTM models trained on multiple datasets (Cross-Domain), several fusion LSTM models trained with images and text and evaluated with three state-of-the-art embeddings, BERT ELMo, and GloVe (Cross-Modality), and character-level CNN models trained on multiple languages (Cross-Language). Our analyses reveal a significant drop in performance when testing neural models on out-of-domain data and non-English languages that may be mitigated using diverse training data. We find that with additional image content as input, ELMo embeddings yield significantly fewer errors compared to BERT or GLoVe. Most importantly, this work not only carefully analyzes deception model robustness but also provides a framework of these analyses that can be applied to new models or extended datasets in the future.

## introduction detection of deceptive content online is an extremely important but challenging task and there have been significant efforts to apply machine learning and deep learning to solve this problem @xcite @xcite @xcite @xcite . a plethora of models exist that rely on a range of data mined from various social platforms -facebook, twitter, reddit -and modalities -text, images, or both. these existing models rely on different text features (e.g., linguistic structure, lexical, psycholinguistic features, biased and subjective language), image features, and user interaction features (e.g., engagement, network structure, temporal patterns). however, current literature lacks details for how these models transfer, e.g., to out-of-domain data, across platforms, or across languages. moreover, there is a gap in understanding the underlying behavior of the decision-making processes behind model output. it is not clear why models for deception detection (particularly neural or deep learning based models) are making certain predictions (e.g., why one item is predicted as deceptive versus not). evaluation of model performance with one metric such as accuracy or f1 score is not enough. for such complex prediction tasks like deception detection for which humans often disagree @xcite @xcite , we need more rigorous evaluation of neural model behavior and a cohesive system for comparing model results across bodies of research. evaluations need to explicitly measure the extent to which model performance is affected when new data is supplied, e.g., data with a different topic distribution or how well-performing models on english data will perform on non-english inputs. there is also a critical need for evaluations that highlight when a model is correct, which examples have the ability to explain why, and, most importantly, conclude why a user should trust a given model in an interpretable manner. arguments in favor of the above requirements to model performance are well-aligned with recent work on machine learning interpretability, trust, fairness, accountability, and reliability @xcite @xcite . another key argument in favor of rigorous evaluation is the lack of benchmark datasets and the need for reproducibility. often, researchers do not make models or datasets publicly available and the details to reconstruct their models and experimental setup are not sufficient, which prevents other researchers from performing rigorous comparisons. to reproduce our key findings and experiments, we will make our framework available through interactive jupyter notebooks available at publication. the focus of the current work and our main contribution is an extensive evaluation of neural model robustness across frequently encountered factors in real-world applications of digital deception models such as new domains, languages, modalities, and upgrades to new state-of-the-art text embeddings. ## related work deep learning systems have been adopted in many areas from medicine to autonomous driving @xcite and as these algorithms are incorporated, the need for explainable and transparent models becomes more urgent. one approach researchers use to overcome the inherent ambiguity of these black-box methods is to develop additional models to learn and explain the decisions of existing models, analyze when these models fail, and introduce a human-in-the-loop component to improve performance @xcite @xcite . another way to tackle this challenge is to design models with a goal of interpretability in place when development starts @xcite @xcite @xcite . model performance beyond accuracy, precision, and recall measures are essential in order to build trustworthy and reliable models tasked with making decisions capable of significantly affecting the end users @xcite @xcite . in this study, we do not focus on developing new visualization techniques to explain deception detection models using intrinsic or post-hoc explanations @xcite @xcite or measure their interpretability @xcite . instead, we thoroughly evaluate model performance using error analysis under several real-world conditions e.g., across domains and input types to understand reliability and the underlying behavior of decision making processes that will in turn increase model explainability and interpretability. deception detection detecting suspicious or deceptive news online is a well explored area of study. current research efforts focus on broadly classifying between suspicious and trustworthy content @xcite to more specific distinctions such as between propaganda, hoax, satire, and trustworthy news @xcite , to examining the behavior of malicious users and bots @xcite @xcite , or analyzing misinformation and rumor spread patterns over time @xcite . methods for classification vary from random forests to deep neural networks and the addition of enriched features such as images, temporal and structural attributes, and linguistic features have been shown to boost model performance over relying on textual characteristics alone @xcite @xcite . with the high impact of digital deception on offline, real-world events, effective detection models are a critical concern. however, interpretable evaluations of model performance beyond traditional precision, recall, or f score measures are essential to building trustworthy and reliable systems when deep learning or machine learning models are tasked with making decisions capable of significantly affecting end users @xcite @xcite . many studies have relied on recurrent neural networks (rnns) or variations, particularly long short term memory (lstm) layers, for deception detection tasks @xcite @xcite @xcite . others have used convolutional neural networks (cnn) @xcite , or variations of lstm architectures such as including attention mechanisms @xcite which are typically dependent on specific tasks or parameter tuning of state-of-the-art deception detection models. for the purposes of consistency across our experiments, we rely on standard lstm models that underpin much of the recent work rather than tuning each architecture by task and data or comparing the wider range of recent state-of-the-art architectures. this enables us to make more accurate comparisons of the factors related to robustness that we vary in our experiments. developing novel models or comparing all state-of-the-art architectures is beyond the scope of this paper. evaluating model robustness measuring robustness of a model goes beyond fortifying against intentional manipulations. corrupted data inputs are possible in real-world scenarios without harboring the intent to fool a system and models should behave as expected in such cases, e.g., simple image transformations, missing or misspelled text. several key studies have endeavored to develop methods to evaluate model robustness @xcite @xcite . in this study we focus on evaluating model robustness to out-of-domain data, multilingual data, and multimodal inputs combined with conceptually different text embedding techniques to gain deeper insights into how much "learning and understanding" neural network models actually have. different studies have separately examined the relationships between extracted features across modalities, across languages, and across domains @xcite @xcite . to the best of our knowledge, this extensive evaluation of deception detection model robustness across multiple tasks has not been presented before in one unified work. ## methodology in this section we discuss the state-of-the-art neural models for deception detection used in this paper and describe our approach for evaluating model robustness across domains, modalities, and languages, a summary of which is presented in ## experimental results in this section, we highlight the results of our experiments regarding model robustness on out-of-domain data (cross-domain analyses), across multimodal inputs using various text embeddings (cross-modality analyses), and multilingual inputs (cross-language analyses). predicted class trustworthy deceptive trustworthy deceptive train on trustworthy 46% 54% 89% 11% (1.72k) (2.02k) (4.77k) (573) twitter deceptive 19% 81% 90% 10% (1.38k) (5.79k) (4.72k) (517) train on true class trustworthy 87% 13% 62% 38% (3.27k) (469) (3.28k) (2.06k) reddit deceptive 89% 11% 51% 49% (6.41k) (757) (2.69k) (2.55k) train on trustworthy 60% 40% 59% 41% (2.26k) (1.49k) (3.22k) (2.21k) twitter deceptive 20% 80% 28% 72% + reddit (1.43k) (5.75k) (1.55k) (3.95k) test on test on twitter reddit before we do so, we manually validate the models qualitatively by examining incorrectly classified tweets with high confidences. for example, the 4-way classification model, m 4 , that relies on pretrained elmo embeddings shows the best performance, but makes errors that resemble human mistakes. the tweet shown in figure 1 is an example of a high-confidence misclassification that could intuitively be similarly made by a human annotator. from the text or image alone the true classification is not immediately clear or intuitive from a human standpoint. the text implies conspiracy, but it is not obvious from the image -a stock image of a glass of water being poured -to which class it should belong. this analysis of high-confidence incorrect classifications provides a better understanding of how high confidence misclassifications (especially high-confidence which can serve as a proxy for where endusers should -or would feel that they should -trust the model) occur and under what conditions. ## discussion and future work we have presented an extensive evaluation of the robustness of digital deception models across frequently encountered real-world scenarios that would be necessary to benchmark against to identify reasonable performance for models considered for deployment. our analyses have identified several trends in behavior across multiple robustness tasks and granularities of deception detection tasks (from binary to 4-way classification). to the best of our knowledge, we are the first to present this type of evaluation of deception detection model robustness. in our robustness analyses detailed above, we have illustrated the danger of relying on single performance measurement, metrics, or analyses by showcasing how a model achieving optimal performance or significant performance increases on a specific task, domain, or context (e.g., multilingual, multimodal) may significantly under-perform with slight alterations in scope or context and other frequently encountered factors in real-world applications. further, we have shown several ways that out-of-domain, multilingual, and multiple modality inputs affect model performance and potential methods to mitigate the impact. for example, the out-of-domain analyses showed that a domain-diverse set of training examples led to higher performance on both domains in held out test sets compared to models trained on domain-specific examples. additionally, the cross-modality analyses illustrated how frequently humanlike classification mistakes can be made and which areas of the data distribution are not well represented, e.g., clickbait tweets with images containing non-personal related objects. results highlighted in the current work open several avenues of future work to develop, evaluate, and understand neural deception detection models in the context of real-world applications. future work will investigate the performance of a variety of such domain diverse training data compared to domain-specific models to identify if this hypothesis holds or if there may be other confounding factors related to specific platforms. in the current work, we have focused on quantifying the performance using a simplified, shared architecture that has been frequently used in deception detection models. however, future work can leverage our interactive jupyter notebooks that will be made available at publication for reproducible extensions using our analysis framework to benchmark the performance of more complex, or newly developed state-of-the-art neural architectures.