Edited Media Understanding Frames: Reasoning About the Intent and Implications of Visual Misinformation (2021)

Abstract: Understanding manipulated media, from automatically generated ‘deepfakes’ to manually edited ones, raises novel research challenges. Because the vast majority of edited or manipulated images are benign, such as photoshopped images for visual enhancements, the key challenge is to understand the complex layers of underlying intents of media edits and their implications with respect to disinformation. In this paper, we study Edited Media Frames, a new formalism to understand visual media manipulation as structured annotations with respect to the intents, emotional reactions, attacks on individuals, and the overall implications of disinformation. We introduce a dataset for our task, EMU, with 56k question-answer pairs written in rich natural language. We evaluate a wide variety of vision-and-language models for our task, and introduce a new model PELICAN, which builds upon recent progress in pretrained multimodal representations. Our model obtains promising results on our dataset, with humans rating its answers as accurate 48.2% of the time. At the same time, there is still much work to be done – and we provide analysis that highlights areas for further progress.

## introduction the modern ubiquity of powerful image-editing software has led to a variety of new disinformation threats. from ai-enabled "deepfakes" to lowskilled "cheapfakes," attackers edit media to engage in a variety of harmful behaviors, such as spreading disinformation, creating revenge porn, and committing fraud @xcite @xcite . accordingly, we argue that it is important to develop systems to help spot harmful manipulated media. the rapid growth and virality of social i want to suggest that subject2 has the support of subject1. subject1 subject2 i feel mad, because we're definitely not friends... how subject1 might feel it seems like subject2 and subject1 are allies! original edited background changed person introduced horizontally flipped previous work: understanding the structure of edits @xcite this paper: understanding the intent and implications of edits why did the editor create this? how the community might respond frame: emotion frame: disinformation frame: intent given a manipulated image and its source, a model must generate natural language answers to a set of open-ended questions. our questions test the understanding of the what and why behind important changes in the image -like that subject1 appears to be on good terms with subject2. identifying whether an image or video has been digitally altered (i.e., "digital forgery detection") has been a long-standing problem in the computer vision and media forensics communities. this has enabled the development of a suite of detection approaches, such as analyzing pixel-level statistics and compression artifacts @xcite @xcite or identifying "what" the edit was @xcite . however, little work has been done on "why" an edit is made, which is necessary for identifying harm. darkening someone's skin in a family photo because background light made them seem quite pale is generally harmless. while such color rebalancing is common, darkening barack obama's (or rafael warnock's) skin in campaign ads was clearly meant as a harmful edit by the editor that did it. 1 we choose to focus on the "why" -we define a schema for approaching the problem of intent and provide a rich set of natural language responses. we also make a significant contribution towards the "what:" we include a physical-change question, provide rationales based in physical changes, and give structured annotations (bounding boxes) on what was changed in the edit. we introduce edited media understanding frames (emu), a new conceptual formalism that captures the notions of "why" and "what" in image editing for language and vision systems (figure 1 ). following literature on pragmatic frames @xcite @xcite )-derived from frame semantics @xcite -we formalize emu frames along six dimensions that cover a diverse range of inferences necessary to fully capture the scope of visual disinformation. we delve into the concept of intention as discussed by the fake news literature @xcite @xcite to capture editor's intent such as motivation for edit and intent to deceive, as well as the resulting implications of the edited content. for every dimension we collect both a classification label and a free-form text explanation. for example, for frame intent, a model must classify the intent of the edit, and describe why this classification is selected. we then introduce a new dataset for our task, emu, with 56k annotations over 8k image pairs. to kickstart progress on our task, we introduce a new language and vision model, pelican, that leverages recent progress in pretrained multimodal representations of images and text @xcite @xcite . we compare our model to a suite of strong baselines, including a standard vlp model @xcite , and show key improvement in terms of ability to reason about co-referent subjects in the edit. nevertheless, our task is far from solved: a significant gap remains our contributions are thus as follows. first, we introduce a new task of edited media understanding frames, which requires a deep understanding of why an image was edited, and a corresponding dataset, emu, with 56k captions that cover diverse inferences. in addition, we introduce a new model, pelican, improving over competitive languageand-vision transformer baselines. our empirical study demonstrates promising results, but significant headroom remains. we release our dataset at jeffda.com/edited-media-understanding to encourage further study in discovering pragmatic markers of disinformation. ## defining edited media understanding frames through an edit e on source image i (e.g. "@xmath0, an editor can cause harm to the subject x's mental state (mental state: "x is angry about e") and effect x's image (effect: "e makes x seem dishonest") @xcite . the editor does this through the intention of the edit (intent: "e intends to harm x's image") and changing the implications of the image (implication: "e frames x as a drug cartel member") @xcite @xcite . to this end, we collect edits e and source images i from reddit's r/photoshopbattles community. there is no readily available (large) central database of harmful image edits, but r/photoshopbattles is replete with suitable complex and culturally implicative edits (e.g., reference to politics or pop culture). this provides us with relevant image edits at a reasonable cost without advocation for dangerous training on real harmful image edits. keeping the source image i in the task allows us to sustain the tractability of the image edit problem @xcite . ## emu: a corpus of edited media understanding frames sourcing image edits we source our image edits from the r/photoshopbattles community on reddit which hosts regular photoshop competitions, where given a source photo, members submit a comment with their own edited photo. we collect 8k image edit pairs (source and edited photo pairs) from this community by, first, manually curating a list of more than 100 terms describing people frequently appearing in photoshop battles posts. then, we screen over 100k posts for titles that contain one or more of these search terms resulting in 20k collected image pairs. additionally, we run an object detector @xcite to ensure that is at least one person present in each image as a means for ensuring that annotators do not see image pairs without any subjects. ## modeling edited media understanding frames in this section, we present a new model for edited media understanding frames, with a goal of kickstarting research on this challenging problem. as described in section 2, our task differs from many standard vision-and-language tasks both in terms of format and required reasoning: a model must take as input two images (a source image and its edit), with a significant change of implication added by the editor. a model must be able to answer questions, grounded in the main subjects of the image, describing these changes. the answers are either boolean labels, or open-ended natural languageincluding explainable rationales. ## experimental results on emu in this section, we evaluate a variety of strong vision-and-language generators on emu. similar to past work on vqa, we rebalance our test set split ensuring a 50/50 split per question type of maliciously labeled captions. we provide two human evaluation metrics -head-to-head, in which generated responses are compared to human responses, and accuracy, in which humans are asked to label if generated responses are accurate in regards to the given edit. ## conclusion we present edited media understanding framesa language-and-vision task requiring models to answer open-ended questions that capture the intent and implications of an image edit. our model, pel-ican, kickstarts progress on our dataset -beating all previous models and with humans rating its answers as accurate 48.2% of the time. at the same time, there is still much work to be done -and we provide analysis that highlights areas for further progress.