# Beyond Manual Media Coding: Evaluating Large Language Models and Agents for News Content Analysis
## 3 Materials and Methods
### Large Language Model-Based Question Answering
#### 3.4.2 Prompt Formulation  
LLMs, built on Transformer architectures with self-attention mechanisms, excel at natural language tasks due to their ability to process long-range dependencies and understand context, meaning and intent behind text sequences [68; 69; 70]. Trained on vast, diverse datasets, they grasp nuanced syntax, semantics and pragmatics across multiple domains [71]. Their applications range from summarization and classification to advanced discourse analysis and content generation. However, LLMs are constrained by fixed knowledge cut-offs [72], limiting their awareness of post-training developments and retraining them to stay current, especially in fast-evolving fields like news content analysis, which is resource-intensive and impractical [73; 74]. To address this, dynamic augmentation techniques allow LLMs to incorporate up-to-date external information at inference time [75; 76]. In our study, this involved supplementing the model with news articles and a detailed codebook to answer questions that require contextual understanding. Thus, rather than relying solely on static knowledge, the model synthesizes real-time information using its deep linguistic capabilities, making this approach scalable and practical for media monitoring and journalism. Consequently, our research prioritized prompt engineering [77], including the development of a global system prompt to ensure analytical consistency and a structured codebook schema featuring specific questions, instructions, tagging values and data extraction protocols, all of which are detailed below. In the first part, we developed a concise, comprehensive global system prompt for the selected LLMs, designed to serve as a foundational guide throughout the content analysis process. This system prompt outlined the model's role, established its responsibilities in handling news content, emphasized the primary focus on the topic of [PERSON]'s tariffs and offered general guidelines to shape its reasoning process. Importantly, the prompt was designed to function independently of any specific news story, as it provided only high-level instructions intended to align the model's behavior without introducing bias from example-based conditioning. Our prompting strategy followed a zero-shot approach, meaning no examples of the desired output format were included. Instead, we relied exclusively on clear, direct suggestions that the model had to interpret and apply autonomously in order to reason and generate its final output. Another central component of our approach was role prompting. We instructed the LLMs to adopt the persona of a professional journalist, an expert tasked with evaluating, interpreting and tagging news content with accuracy, impartiality, and editorial insights. This role-based framing was reinforced with elements of emotional/motivational prompting aimed at enhancing the model's steerability by encouraging a measured and professional responsibility within the role-based context, encouraging the model to take its task more seriously and generate thoughtful, deliberate responses. Additionally, to strengthen the LLM's grasp of the full analytical context, we introduced a meta-cognitive step. Before proceeding with tagging individual questions from the codebook, the model was instructed to briefly summarize its approach as to re-evaluate its understanding of the given schema. This reflective prompt was implemented to encourage the model to internalize the logic and structure of the tagging system and general guidelines, thereby improving the relevance and accuracy of its output. In the second part of our prompt engineering process, after designing the initial version of the codebook, including the core questions, legible tagging values and their descriptions, we shifted our focus toward refining its clarity, structure, and usability. The questions were carefully reviewed and paired, when necessary, with distinct, unambiguous instructions for every available response option.