Multi-Modal Framing Analysis of News
Authors: Srishti Yadav, Arnav Arora, Maria Antoniak, Serge Belongie, Isabelle Augenstein
Accepted at: acl25


Reviewer paper summaries: This paper presents the a large-scale study of media framing that jointly analyzes both news text and images. The authors collected 500K news articles from 28 U.S. outlets. Using large models (Mistral for text and Pixtral for images), they automatically tagged each article with media frames from the Media Frames Corpus. Unlike prior work, they allow multiple frames per article or image and generate freeform, issue-specific frame summaries to capture more nuanced framing.

They validated their annotations: for text, the model matched human labels in ~89% of cases, and for images, it got at least one correct frame ~84% of the time. The paper also looks at how framing varies by political leaning, confirming that left-leaning and right-leaning outlets frame the same issues in notably different ways.

This paper addresses the challenge of automated frame analysis in political communication, particularly in news media. Traditionally, framing analysis has focused on predefined textual frames, often overlooking the visual context that accompanies news articles. This paper proposes a novel approach to conducting multi-modal, multi-label framing analysis using large language and vision-language models, which incorporates both text and images. The authors ground their methodology in framing theory, extracting latent meanings from images and comparing them with textual frames to provide a more complete understanding of how media frames topics.
Summary of strengths: 1. The dataset and guidelines will be a great resource for both computational and social science communities. 500K articles from 28 U.S. news outlets over a full year, each annotated with topics, subjects, multiple generic frames, and open-ended issue-specific summaries. Compared to prior datasets like Media Frames Corpus, this one is massive and more diverse. The inclusion of left-leaning, center-leaning, and right-leaning sources makes it particularly valuable for studying partisan framing. 

2. Methodologically, the paper is scalable. It uses LLMs (Mistral) and a custom vision-language model (Pixtral) to annotate framing at scale—without needing massive manual labeling. The authors validated their results thoroughly: 89% overlap with gold-standard frames for text, and strong human alignment for images (~84%).

1. The paper uses both generic framing and issue-specific frames, and presents the label distribution of the dataset with detailed and specific visuals.
2. The paper compares the differences between text framing and image framing in the same article, which is an innovative approach.
3. The paper also incorporates the perspective of political leaning, showing how different political orientations use distinct framing strategies on various issues.
Summary of weaknesses: 1. One limitation here is the reliance on automated labels. The model seems to miss about half the gold-standard text frames, and for images, it fails in ~15% of evaluated cases. That level of noise could affect the trustworthiness of finer-grained insights. What kinds of frames get missed or misclassified? It would be helpful to see error analysis or concrete examples to judge how much weight we can put on the conclusions. 

2. The paper introduces a custom vision-language model, “Pixtral,” but details are thin. It’s unclear how the model was trained or how it outputs frame labels—was it fine-tuned? Prompted? 

3. Multi-label framing is more realistic than single-label setups, but it introduces interpretation challenges. When a model assigns five frames to an article, what does that mean in terms of narrative? Is one dominant? Are some spurious? it’s not clear how those free-text outputs of issue specific frames were analyzed—clustered, categorized, anything else? 
 
4. If framing doesn’t always apply (some articles or images just aren’t frame-rich I can think of some generic scene photos), and the method assumes every item has one or more frames, which might lead to forced or meaningless labels. It would be helpful to have discussion of how often images actually contribute substantive framing versus just being filler.

A major issue is the lack of coherence in the paper. For example:
1. Table 1 states that one target to extract is the caption of the image. However, it does not clarify whether the caption is model-generated or extracted (many images in the dataset lack an original caption), and the paper does not explain the usage of image captions.
2. In the paragraph “Lexical Comparison” on page 6, the authors raise an interesting question: “Why might the same article include a crime frame for its text and a political frame for its image?” Unfortunately, this question is not addressed further in the paper.

Another significant issue is an inaccuracy in the factual description:
In section 5.1, the description of Figure 4 states that the three most frequently occurring generic frames in the text are policy, quality of life, and legality, with the least frequent being cap & res and morality. 

In addition, when comparing text and image, it is difficult to understand the preferences of different modalities for the same frame because the vertical axes of the two charts in Figure 4 are not consistent. It is recommended to convert this figure from frequency to proportion for comparison.

Additionally, I have some questions regarding "images with logos" mentioned in line 231:
1. What is the definition of logos? Does it refer to watermarks on images or organizational trademarks? If it refers to organizational trademarks, what is the reason for excluding these logos? Could you offer some examples?
2. In Appendix B, it is mentioned: “There was no clear way to remove the images with logos. To approximate this, we remove all figures with a size of less than 5000 bytes.” What is the basis for this approach?
Comments, suggestions and typos: typo: 
line 513: possible reach

line 306: compare calculate

line 216: W e

line 249: Morality. Fairness&....


Comments:
1. Clarify pixtral in main text. 
2. Consider showing one article—headline, image/caption, text/image frame predictions, and the issue-specific frame. A short discussion of how these align or differ would help readers grasp what multimodal framing looks like in practice.
3. It would help to mention common mistakes and whether certain frames are systematically confused. 
4. Does dataset includes non-political content like sports or lifestyle? It would be helpful to see frame frequency patterns to topic distribution for better context.

The following details can be adjusted:
1. Standardize the capitalization of subsection titles (2.1 and 4.1).
2. Line 216: "W e" --> "We"
3. Line 306: "compare calculate"
4. Figure 8 should show up earlier than Figure 7
Overall assessment scores: 3

2