TAG – EQA : Text–And–Graph for Event Question Answering via Structured Prompting Strategies (2025)

Abstract: Large language models (LLMs) excel at general language tasks but often struggle with event-based questions—especially those requiring causal or temporal reasoning. We introduce TAG-EQA (Text-And-Graph for Event Question Answering), a prompting framework that injects causal event graphs into LLM inputs by converting structured relations into natural-language statements. TAG-EQA spans nine prompting configurations, combining three strategies (zero-shot, few-shot, chain-of-thought) with three input modalities (text-only, graph-only, text+graph), enabling a systematic analysis of when and how structured knowledge aids inference. On the TORQUESTRA benchmark, TAG-EQA improves accuracy by ~5% on average over text-only baselines, with gains up to ~12% in zero-shot settings and ~18% when graph-augmented CoT prompting is effective. While performance varies by model and configuration, our findings show that causal graphs can enhance event reasoning in LLMs without fine-tuning, offering a flexible way to encode structure in prompt-based QA.

## introduction consider the text in figure 1 : "organizers state the two days of music, dancing, and speeches is expected to draw two million people. but as supporters gathered... riot police deployed...". when asked, "did the protesters gather while the organizers made a statement?" , answering correctly requires chaining events: @xmath0, while recognizing that riot_police_deployed ⊣ organizers_state, @xmath1. such questions require structured event reasoning, where causal graphs make dependencies explicit by surfacing enable and block relations that 1 code and data available at https://github.com/ maithilikadam4/tag-eqa text : organizers state the two days of music, dancing, and speeches is expected to draw some two million people. but as supporters of the military leader gathered in the north, riot police deployed in nigeria's southern commercial capital lagos, to break up a protest rally called by the political opposition. question : did "gathered" happen while the organizers made a statement? ## related work prior work on event modeling, causal reasoning, and prompt engineering has independently ad-vanced narrative qa. we synthesize these strands by embedding structured causal graphs into prompt formats to guide event-centric inference in llms. ## evaluation we evaluate tag-eqa using binary classification accuracy: the percentage of questions answered correctly as "yes" or "no." each model is tested across all nine configurations-three prompting strategies (zero, few, cot) × three input modalities (text, graphs, tag). for cot prompts, we extract the final answer using a regex targeting phrases like "therefore, the final answer is: yes." if absent, we fall back to the first standalone yes/no token 3 . this ensures consistent evaluation across models with variable output formats. we report results on both the full torques-tra test set (full, 477k examples) and a 1,024- to analyze how structure and reasoning affect performance across reasoning types, we group questions into thirteen semantically grounded clusters derived from torquestra annotations. these extend the original eight-category taxonomy to include finer-grained types such as positive, negative, existential, and counterfactual. accuracy is reported per cluster and per configuration. see appendix a.5 for full cluster definitions and results. ## results we evaluate how prompting strategy and input modality affect event-based qa performance across three instruction-tuned llms: t5-xxl, qwen-32b (qwq), and gpt models (gpt-3.5 and gpt-4o). each model is tested under nine prompting configurations (zero/few/cot × text/graphs/tag). t5 and qwen are evaluated on both the full torquestra test set (full) and a 1,024-example subset (small). gpt-3.5 and gpt-4o are evaluated only on the small subset: gpt-3.5 for zero and few (non-reasoning), and gpt-4o for cot (reasoning), due to api cost and throughput constraints. across models, few-shot prompting consistently outperforms zero-shot in text-only settings. cot prompting yields mixed results: qwq achieves the highest overall accuracy (74.8%) with tag-cot, while t5 performs best with few-text. for t5, accuracy drops when cot is combined with structured input, suggesting difficulty integrating reasoning traces and graph content. graphs inputs significantly enhance zero-shot and cot performance for qwq, sometimes outperforming tag inputs. however, modality fusion does not always help: tag configurations often underperform compared to single-modality prompts, particularly for t5. gpt results remain relatively flat across input types, with zero-text (58.7%) performing best for gpt-3.5, and modest gains from cot in gpt-4o. these findings highlight the importance of modelaware prompt design: performance gains depend not just on adding structure or reasoning, but on whether a given model can effectively integrate them. we evaluate the effect of input modality-text, graphs, and tag-under both zero and few prompting. as shown in we now examine the effect of input modality under cot prompting. as shown in qwq achieves its highest accuracy (74.8%) with tag, while also showing strong performance with graphs-only input (72.7%). t5 shows modest gains from graphs input: on full, accuracy rises from 55.2% (text) to 56.9% (graphs), but drops to 50.4% with tag, suggesting that reasoning traces may conflict with multimodal inputs for models not tuned for integration. this trend persists on the small subset. gpt, evaluated only on small, shows a slight drop in performance with tag (70.6%) compared to text-only input (72.3%), while graphs-only input yields comparable performance (71.1%). this suggests that gpt-4o does not consistently benefit from structured input when combined with reasoning traces in zero-shot settings. overall, these results suggest that graph-augmented reasoning is most effective when the model can exploit structure natively-qwq benefits most-while other models struggle to integrate multiple information sources effectively under cot prompting. ## conclusion we introduced tag-eqa, a systematic framework for evaluating event-based question answering (qa) in large language models (llms) using structured causal graphs and reasoning-driven prompting. our experiments covered nine prompting configurations-three strategies (zero, few, cot) crossed with three input modalities (text, graphs, tag)-evaluated on three instructiontuned llms: t5, qwen (qwq), and gpt models. causal graphs consistently improved accuracy on event-centric questions, particularly for relational categories such as causal, past, and tempo-ral_conflict. qwq achieved the highest overall performance when combining structure and reasoning (tag+cot), while t5 performed best with few+text and showed limited gains from structured input. gpt models, evaluated only on a smaller subset, showed moderate benefits from cot prompting but little sensitivity to input modality. ambiguous or underspecified categories-such as possible and unknown-remained challenging across models and prompting styles. these findings highlight both the strengths and limitations of using structured causal input to guide reasoning in llms.