# Beyond Manual Media Coding: Evaluating Large Language Models and Agents for News Content Analysis
## 1 Introduction  
Processing and systematically extracting insights from media content is increasingly becoming a critical challenge for organizations, governments, and individuals. The volume of media data is growing exponentially with news articles from both traditional and online media serving as vital information channels for society [1]. However, the combination of extensive information, time limitations, and constrained cognitive processing capacity can substantially impair decision-making accuracy [2]. Understanding, analyzing, and leveraging this vast stream of media data are crucial for effective and accurate decision making [3], public relations [4], crisis management [5, 6], and policy development [7]. These demands and constraints highlight the importance of integrating technological advancements in content analysis. Content analysis can be defined as the \"systematic, objective, quantitative analysis of message characteristics\" [8]. While the need to understand information dates back to the dawn of human civilization, the first systematic efforts in content analysis emerged in the 17 th and 18 th centuries, particularly in the context of religious document interpretation. This was followed by a significant expansion of interest in information understanding during periods of economic crisis and in pre-, post-, and wartime environments [9]. Today, content analysis is carried out by a range of actors, including media monitoring organizations, public relations' professionals, analysts, and intelligence agencies, who are responsible for developing workflows for the structured coding of news stories. News story coding is a systematic method that involves defining and operationalizing variables to measure message content, developing coding schemes or dictionaries (codebooks), sampling and coding the content with reliability testing, and finally analyzing the results to establish relationships between content characteristics and other variables [8]. In practice, the creation of a codebook, both for human and automated annotation, involves the selection of questions and a specified answering schema. Codebooks are typically tailored to specific domains, organizations, or semantic topics. They comprise a list of domain-specific variables and possible values (codes), coding instructions, examples, and additional guidelines. This structure ensures consistent annotation and supports replicable, data-driven content analysis. They serve as a standardized framework for coders, reducing subjectivity and enabling comparison across studies [8; 9]. Manual codebook-based annotation of news stories presents several limitations, including inter-annotator reliability issues, biases, and, most critically, scalability constraints due to the labor-intensive nature of the task. Although technological advancements have enabled the rapid creation and dissemination of media content, content understanding and analysis remain limited by human cognitive capacity throughput and the financial resources required for either in-house or outsourced analysis. Processing and coding a typical web news article, generally ranging between 500 and 2500 words, can require approximately 10 to 35 min, depending on article length and the complexity of the coding schema. Recent technological advances in generative artificial intelligence (AI) [10] have enabled the creation of large language models (LLMs) capable of unprecedented contextual understanding without the need for domain-specific training [11; 12; 13]. Many research works explore the application of LLMs to automate and augment qualitative research tasks. Studies have shown that LLMs can be effective in generating thematic analyses of textual data and producing high-quality annotations, often rivaling human performance while significantly reducing time and effort [14]. However, structured media codebook analysis remains under-explored across diverse LLMs, particularly when leveraging state-of-the-art prompting techniques and coordinated expert roles within multi-agent systems.