Exploring Deceptive Domain Transfer Strategies: Mitigating the Differences among Deceptive Domains (2023)

Abstract: Deceptive text poses a significant threat to users, resulting in widespread misinformation and disorder. While researchers have created numerous cutting-edge techniques for detecting deception in domain-specific settings, whether there is a generic deception pattern so that deception-related knowledge in one domain can be transferred to the other remains mostly unexplored. Moreover, the disparities in textual expression across these many mediums pose an additional obstacle for generalization. To this end, we present a Multi-Task Learning (MTL)-based deception generalization strategy to reduce the domain-specific noise and facilitate a better understanding of deception via a generalized training. As deceptive domains, we use News (fake news), Tweets (rumors), and Reviews (fake reviews) and employ LSTM and BERT model to incorporate domain transfer techniques. Our proposed architecture for the combined approach of domain-independent and domain-specific training improves the deception detection performance by up to 5.28% in F1-score.

## introduction with the advent of the digital age came a deluge of textual content online, which also contains an enormous amount of deceptive text. the deceptive text uses a variety of strategies to trick readers based on the information delivery medium. fake news, for instance, spreads false information about a person or organization in order to harm their reputation, while fake reviews intentionally exaggerate the positive or negative aspects of a product or service in order to gain attention. despite the technical differences in deception, all deceptive texts have the same objective of deceiving people; hence, a generic pattern of deception may exist @xcite . identifying the underlying generic pattern of deception may unravel useful information about textual deception. furthermore, such a system will enable a more effective detection approach through the intermingling of multiple deception domains. there has been a good number of work done to combat textual deception in domain-specific situations, but a powerful detection system requires a lot of labeled data, which is dependent on things like trustworthy annotators, resources, time, and money. consequently, the learning paradigm wherein multiple domains may support each other can be a promising solution in deception detection. in this paper, we explore the feasibility of generalizing deception across domains such as news, reviews, and tweets, and we present a multi-task learning (mtl) based deceptive domain transfer strategy to mitigate the domain differences and improve deception detection capacity beyond a standard single domain learning approach with lstm and bert models. researchers dealt with deceptive domain adaptation problems in cross-dataset learning settings, e.g., opinion spam on different entities @xcite , and fake news on different topics @xcite . although attempts have been made to generalize the deception for better detection, these efforts have been constrained so far @xcite . @xcite explored the problem of holistic deception detection, where they used single deep learning networks to detect deception from a holistic perspective @xcite . while this approach provides a domain-agnostic system, it is possible that the intrinsic variations between deception domains mean that a single network cannot give an effective solution. a feature-augmentationbased soft domain transfer approach using the last layer of learned models was proposed in @xcite . however, the last layers are prone to capturing the domain-specific noise which may have adverse effects in deceptive transfer. consequently, the lack of a sufficiently robust system that can account for domain differences and leverage the generic deception signal constitutes a significant research gap. considering the above aspects, we formulate the research problem as: given a set of deceptive domains {d i } n @xmath0, how to construct a generic feature set f g which can help improve detecting deception in all n domains, rather than the domain-specific feature set {f i } n @xmath1. to address this problem, we use a multi-task learning (mtl) approach with the lstm and bert model, where the part of the model is shared across all domains to capture the generic information, and multiple branches downstream to account for domain-specific information. we compare our approach with an all for one (ofa) mode of deception generalization and intermediate layer concatenation (ilc) mode of domain transfer @xcite . this study has a wide range of implications. at the outset, this study seeks to characterize the interconnectedness of various forms of deception and to identify the underlying generic pattern. learning deception across different domains together allows for the development of a more robust system. it will also take into account the labeled data shortage issue in many deception domains. on top of that, the appearance of a new event can frequently lead to more deceptive data in one domain than the other. in such instances, the generalization of deception studies can be extremely valuable. finally, the mtl-based simultaneous learning of generic and domain-specific deception will incorporate fewer parameters to be trained than separately learning from the domains. our research shows that for all domain-transfer and generalization experiments, mtl outperforms the ilc and ofa mode. our main contribution can be summarized as follows: for the three domains, we use six datasets for this paper. for the news domain deception, liar dataset contains data from politifact, and each data is labeled with one of them: true, mostly-true, half-true, mostly-false, false, pants-on-fire false. following the work of upadhayay and behzadan 2020, we label the first two as non-fake and the latter four as fake news. another news dataset, nela-gt-2021 (nela) is a source-based labeled news dataset collected from january 2021 to december 2021 and labeled by media bias fact check (mbfc) @xcite . we labeled the news with mbfc factuality score 0 as fake and 5 as non-fake, and we collect the news sources from the us only. the news domain contains 43,168 news with 64.29% as fake. in the tweets domain of deception, data comes from pheme and a collection of newly emerged rumors in twitter (nert) from 2016 to 2018 @xcite . in total we have 20,893 tweets with 49.77% as rumors. for the reviews domain, we use the yelp restaurant (res) and hotel (res) dataset, which 67,395 reviews, where 13.19% of them are labeled as fake @xcite . ## methodology as the baseline text classification models, we use attention-based lstm, and bert models, followed by a fc layer and a softmax layer @xcite . we explore intermediate layer concatenation (ilc) and multi-task learning (mtl) for deceptive domain transfer strategies. ## experiments and results we use 80-20 split for train and test, and 20% from the train set as validation with three random splits. for the reviews domain, we train with a balanced proportion of fake and non-fake reviews. we compare accuracy and binary f1-score for performance. algorithm 1 multi task learning for domain tranfer 1: input: deceptive target domain t , deceptive helper domain h, loss weight of helper domain λ 2: output: hard-shared parameters θ s , target domain paramters θ t , and helper domain parameters θ h 3: compute loss for target domain l t (t ; θ t , θ s ) 4: compute loss for helper domain l h (h; θ h , θ s ) 5: combine the losses @xmath2: update θ s based on combined loss l 7: update θ t based on loss l t 8: update θ h based on loss λl h the batch size, learning rate, hidden layers and epochs are chosen by validation set performance. ## result analysis and discussion overall, the ilc mode of domain transfer exhibits less improvement than the mtl mode. this is because, in the mtl mode, the deception domains share a latent space in the upstream layers and are only distinguished by the domain-specific layers on the later levels. on the contrary, the ilc mode can access high-level representations only. hence mtl mode has a higher chance of learning underlying representations than ilc modes by leveraging information from other domains. we investigate the performance improvement of the mtl-based lstm model by plotting the validation loss in the first 10 epochs. figure 2 shows that the mtl mode achieves better generalizability for all three deceptive domains, whereas self-domain modes tend to overfit quickly. thus, the improved performance of mtl mode might be attributed to better generalization. we further explore how deception generalization is conducted on the attention head level with mtl-based bert model. ## related works most of the previous works in domain transfer dealt with cross-dataset knowledge transfer on different topics from similar information sources. for example, fake news from different news sources and topics are shown to have different word usage and propagation pattern @xcite ). janicka et al. showed that stylometric and psycholinguistic features in different fake news varies widely and results in the performance drop to 20% when train and test sources are different @xcite . silva et al. addressed the challenge by storing domain-specific and cross-domain knowledge in embedding representation. @xcite . sicilia et al. explored how the differences in topics between train and test set affect the performance in rumor detection in the health domain @xcite . ren et al. linearly combined a set of vector representations on different topics with the textual features and obtained an attention network-based cross-topic solution for rumor detection @xcite . in the field of fake review detection, hernández-castañeda et al. performed a cross-domain fake review detection using three opinion datasets with lda, svn, and wsm-based features @xcite . they also measured the domain association by training on one domain and testing on the other. sànchez-junquera et al. proposed a model where they performed a filtering approach for masking domain-specific terms and transformed the original text to a domain-agnostic form @xcite . similar works in cross-domain fake review detection was done in @xcite and @xcite . the existing works on the cross-dataset domain transfer technique suggests that a robust model should exploit both domain-aware and domainindependent attributes for a successful deception detection task. our proposed method of mtlbased domain transfer technique builds up on shared and domain-specific layers to account for the aforementioned strategy. nevertheless, the comparative study of deceptive medium-based domain transfer was not explored in previous work to the best of our knowledge. hence, our method is the first one to address this problem. ## conclusion although distinct deception domains have their own methods and characteristics for disseminating deceptive information, they all have the same objective: to deceive individuals. hence, the generalized detection approach can be immensely useful for addressing labeled data shortage issues in numerous domains. here, we compare state-of-the-art domain transfer strategies and present an mtlbased method for transferring information across deceptive domains for enhanced deception detection. our experiments demonstrate that learning deception in multiple domains simultaneously results in improved generalization and performance. in any case, with mtl-based architecture showing promise as a possible option for universal deception detection, we can investigate different hybrid structures of textual parameter sharing and weightedloss methods for deception detection. furthermore, the continual learning approach of deception detection can be a promising research direction due to its capability of catastrophic forgetting prevention and knowledge transfer @xcite . in addition, we plan to incorporate email and facebook post deceptions into our future research.