  In 2019, outbreaks of vaccine-preventable diseases reached the highest number
in the US since 1992. Medical misinformation, such as antivaccine content
propagating through social media, is associated with increases in vaccine delay
and refusal. Our overall goal is to develop an automatic detector for
antivaccine messages to counteract the negative impact that antivaccine
messages have on the public health. Very few extant detection systems have
considered multimodality of social media posts (images, texts, and hashtags),
and instead focus on textual components, despite the rapid growth of
photo-sharing applications (e.g., Instagram). As a result, existing systems are
not sufficient for detecting antivaccine messages with heavy visual components
(e.g., images) posted on these newer platforms. To solve this problem, we
propose a deep learning network that leverages both visual and textual
information. A new semantic- and task-level attention mechanism was created to
help our model to focus on the essential contents of a post that signal
antivaccine messages. The proposed model, which consists of three branches, can
generate comprehensive fused features for predictions. Moreover, an ensemble
method is proposed to further improve the final prediction accuracy. To
evaluate the proposed model's performance, a real-world social media dataset
that consists of more than 30,000 samples was collected from Instagram between
January 2016 and October 2019. Our 30 experiment results demonstrate that the
final network achieves above 97% testing accuracy and outperforms other
relevant models, demonstrating that it can detect a large amount of antivaccine
messages posted daily. The implementation code is available at
https://github.com/wzhings/antivaccine_detection.
