# Beyond Manual Media Coding: Evaluating Large Language Models and Agents for News Content Analysis
## 6 Conclusions  
This paper explored the capabilities of LLMs in performing structured media content analysis, guided by two primary research questions. The key contributions of this work are as follows:  
* A curated dataset of 200 news articles, annotated with 26 codebook-derived questions and 122 codes related to US tariffs, is introduced, supporting reproducible evaluation and future research on media content analysis systems. * A systematic evaluation of seven state-of-the-art LLMs is conducted using structured prompting strategies. Notably, Claude-3-7-Sonnet achieved a global F1-score of 0.822, demonstrating strong performance in direct prompting settings. * An expert-based agentic architecture built on Meta's Llama 4 Maverick is proposed and benchmarked, demonstrating how role-specialized reasoning can systematically improve structured codebook-based content analysis performance (raising the F1-score from 0.757 to 0.805) while remaining cost-effective relative to commercial models. While the proposed approach demonstrates strong performance, several limitations remain. Our LLM-based and agentic approaches may ultimately aid large-scale audits of media data or, potentially, global coverage, which would be infeasible to scale with purely human coders. However, the scalability of our method must be further validated on larger and more diverse datasets. Evaluating this framework with additional languages and an even more varied collection of sources, while investigating different thematic categories, will increase its generalizability. Second, although structured prompting and agentic reasoning improve accuracy, human-in-the-loop validation may be essential to ensure reliability and provide explainability insights. Our agentic system is designed to support human-in-the-loop validation for future research. Future work could also extend our agentic approach to other architectures and models to further generalize our findings. It is worth noting that an evaluation of earlier LLM releases, traditional NLP techniques, and shallow ML models could also provide valuable insights into the relative performance and impact of the proposed approach [84]. While the focus of this study was to evaluate the performance of LLMs and LLM-based agents using a human-expert-created codebook, future work could explore codebook update phases during the agentic workflow. Finally, future work could address systematic errors in subjective sentiment or multi-label extraction, potentially through hybrid or ensemble approaches, as our results show important differences by question type and context across models. Our agentic approach could leverage different models per question or category of questions. This study has implications for scalable automations in media monitoring, public relation analysis, political communication, and journalistic accountability audits. While this study employed a tariff-focused codebook, the methodology is applicable to other policy or issue-specific content analyses. However, such analyses often demand near-perfect accuracy to prevent misinformation, inaccurate decision making, and biases that, depending on the domain, could have significant consequences. As LLM-based systems become increasingly integrated into content analysis workflows, attention must be given to interpretability, accountability, ethics, and error auditing to ensure responsible deployment in both academic and applied settings. Conceptualization, S.D. and S.V. ; methodology, S.D., E.K. and P.C. ; software, S.D. and E.K. ; validation, S.D. and S.K. ; formal analysis, S.D. and E.K. ; investigation, S.D. and S.K. ; resources, P.C. ; data curation, S.K., E.K. and P.C. ; writing--original draft preparation, S.D. ; writing--review and editing, S.D., E.K. and P.C. ; visualization, S.D. and E.K. ; supervision, S.V. ; project administration, S.D. and S.V. All authors have read and agreed to the published version of the manuscript. This research received no external funding. Not applicable.