Title: Beneficial Reasoning Behaviors in Agentic Search and Effective Post-training to Obtain Them
Primary area: foundation or frontier models, including LLMs
Keywords: Agentic Search, Reasoning, Reinforcement Learning, LLM, Agent, Supervised Fine-Tuning, Midtrain

Abstract: Agentic search leverages large language models (LLMs) to interpret complex user information needs and execute a multi-step process of planning queries, executing searches, and adapting its strategy based on retrieved documents. This introduces unique challenges for LLMs’ reasoning and agentic capabilities in interacting with retrieval systems and the broader web. In this paper, we design a reasoning-driven LLM-based pipeline to study effective behaviors in agentic search. Using this pipeline, we analyze successful agentic search trajectories and extract four beneficial reasoning behaviors: Information Verification, Authority Evaluation, Adaptive Search, and Error Recovery. We then propose a new behavior priming technique to train more effective agentic search models. It synthesizes agentic search trajectories with target behaviors and integrates them into the agentic search model through supervised finetuning (SFT), followed by standard reinforcement learning (RL). Experiments on three benchmarks—GAIA, WebWalker, and HLE—demonstrate that behavior priming significantly improves agentic search accuracy, yielding over 35\% gains in Llama3.2-3B and Qwen3-1.7B after RL. Ablation studies further show that priming effective behaviors is more beneficial than emphasizing result correctness: finetuning on trajectories with desirable reasoning behaviors but incorrect results outperforms training on trajectories with correct answers. Additional analysis reveals that reasoning patterns introduced by behavior priming provide a strong foundation for RL, enabling more effective exploration (higher pass@k and entropy) and test-time scaling (longer trajectories). Our code and models will be released as open source.

TL;DR: We identified essential behaviors for effective improvement in RL for search agents, and proposed Behavior Priming, a method that instills these reasoning behaviors to help LLM search agents improve more effectively with reinforcement learning.