# Beyond Manual Media Coding: Evaluating Large Language Models and Agents for News Content Analysis
## 3 Materials and Methods
### Large Language Model-Based Question Answering  
#### 3.4.1 Model Selection  
To gain a comprehensive understanding of the potential and limitations of using LLMs for news content analysis, we conducted a thorough evaluation of various models across different performance tiers in an attempt to assess their performance in the task of tagging news based on a predefined list of questions [52]. This included testing LLMs classified as low-, mid- and high-level based on their complexity, capabilities and resource requirements. Table 2 shows the different models tested in our experiments. Specifically, for the lower-tier models, we opted for the quantized version of Meta's Llama 3.1 [53] and the distilled version of DeepSeek-R1 [54], which is based on the Qwen 2.5 [55] architecture. Both models support a similar context window (128 K) in terms of the volume of data they can process at once. However, they differ significantly in size, with Llama 3.1 utilizing approximately 8 billion parameters and DeepSeek-R1 employing around 32 billion. While a direct comparison between these two models may not be entirely equitable, given their distinct optimization techniques (quantization versus distillation) [56], training pipelines and intended use cases, it remains valuable to examine how they perform relative to each other, particularly when benchmarked against more advanced and resource-intensive LLMs. This comparison could also provide an additional insight into the trade-offs between model efficiency and performance, especially in resource-constrained environments [57; 58]. For the mid-tier category, we selected Meta's Llama 3.3 [53] and a quantized version of Qwen 3 [59]. These two models exhibit notable differences in both architecture and configuration, most significantly in their parameter counts and context window sizes. Llama 3.3 represents a balanced model with a moderate number of parameters and a reasonably large context window, making it well suited for a variety of general-purpose tasks [12]. In contrast, Qwen 3, despite being quantized for efficiency, features 235 billion parameters, placing it near the boundary of what might typically be considered a high-tier model [59]. However, its relatively limited context window of just 32 K tokens may constrain its effectiveness in tasks that require broader document comprehension or long-range reasoning, thus justifying its inclusion in the mid-tier category for this analysis. Mid-tier models often strike a favorable balance between computational efficiency and task performance. They are especially valuable in real-world applications where resource constraints exist but where the complexity of tasks still demands a reasonable level of model sophistication [60]. Evaluating models like Llama 3.3 and Qwen 3 in this tier provides \begin{table}
\begin{tabular}{l c c c c} \hline \hline
**Model** & **Quantization** & **Parameters** & **Context** & **Weights** \\ \hline Meta-Llama-3-1 & FP8 & 8 B & 128 K & Available \\ DeepSeek-R1-Distill-Qwen & - & 32 B & 128 K & Available \\ Meta-Llama-3-3 & - & 70 B & 128 K & Available \\ Qwen3 & FP8 & 235 B & 32 K & Available \\ Meta-Llama-4-Maverick & FP8 & 402 B & 1 M & Available \\ DeepSeek-R1 & - & 685 B & 160 K & Available \\ Claude-3-7-Sonnet & - & - & 200 K & Not Available \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Models selected for the evaluation process. insights into how architectural trade-offs, such as parameter scaling versus context length, affect practical outcomes across different types of news content analysis tasks. For the high-tier category, we selected two of the most advanced open-weight models available, Meta's Llama 4 Maverick [61] (in its quantized form) and the latest iteration of DeepSeek-R1, identified as version 0528. Both models significantly exceed the capabilities of the low- and middle-tier selections in terms of parameter count, context window size and computational demands. They represent the cutting edge of open-weight LLM development and are recognized for their advanced performance [62].