Pushing Test-Time Scaling Limits of Deep Search with Asymmetric Verification
Authors: Weihao Zeng
Keqing He
Chuqiao Kuang
Xiaoguang Li
Junxian He
Venue: ICLR 2026 Poster (2026)
Keywords: agentic search
test-time scaling
asymmetric verification
Primary area: other topics in machine learning (i.e., none of the above)

Abstract: Test-time compute can be scaled both sequentially and in parallel. Sequential scaling involves lengthening the generation process, while parallel scaling involves verifying and selecting among multiple candidate outputs. Combining these two strategies has led to the most powerful AI systems, such as Grok 4 Heavy, GPT-5 Pro, and Gemini-2.5 Pro Deep Think. A key observation is that, in certain contexts (e.g., solving Sudoku puzzles), verifying responses can be substantially easier than generating them. This property, referred to as \emph{asymmetric verification}, highlights the strong potential of test-time scaling. In this work, we study both sequential and parallel test-time scaling of deep search agents, motivated by the intuition that verification in this setting is often much easier than generation. In experiments, we first show that sequential scaling methods, such as budget forcing, can be effective initially but eventually degrade performance when over-applied in agentic search. Due to asymmetric verification, however, we are able to achieve substantial improvements by allocating only a modest amount of compute to the verifier. We conduct experiments with flagship open-source models, including GLM-4.5, K2, Qwen3-2507 and Tongyi-DeepResearch, and extend them to their ``Heavy'' variants through test-time scaling. These deep research agents achieve improvements of up to 20 absolute points on benchmarks such as BrowseComp. Remarkably, as an open-source alternative, GLM-4.5 Heavy reaches accuracy of {\bf 54.0\%} on BrowseComp, {\bf 66.0\%} on GAIA, and {\bf 68.0\%} on xbench-DeepSearch, placing it on par with the best proprietary choices such as OpenAI Deep Research and o3. Tongyi-DeepResearch Heavy pushes performance even further, attaining {\bf 69.0\%} accuracy on BrowseComp.

PUSHING TEST-TIME SCALING LIMITS OF DEEP SEARCH WITH ASYMMETRIC VERIFICATION
Weihao Zeng' Keqing He Chuqiao Kuang² Xiaoguang Li
Junxian He'
'The Hong Kong University of Science and Technology BUPT (wzengak, junxianh]e ose. csse
Huawei Noah's 's Ark Lab
ust. .kk
7 jikust-alpddeepparchhttts
ABSTRACT
Test-time compute can be scaled both sequentially and in parallel. Sequential scaling involves lengthening the generation process, while parallel scaling involves verifying and selecting among multiple candidate outputs. Combining these two strategies has led to the most powerful AI systems, such as Grok Heavy, GPT- 5 Pro, and Gemini-2.5 Pro Deep Think. A key observation is that, in certain contexts (e.g., solving Sudoku puzzles), verifying responses can be substantially easier than generating them. This property, referred as asymmetric verification, highlights the strong potential of test-time scaling. In this work, we study both sequential and parallel test-time scaling of deep search agents, motivated by the intuition that verification in this setting is often much easier than generation. In experiments, we first show that sequential scaling methods, such budget forcing, can be effective initially but eventually degrade performance when over-applied in agentic search. Due asymmetric verification, however, we able to achieve substantial improvements by allocating only modest amount of compute to the verifier. We conduct experiments with flagship open-source models, including GLM-4.5, K2, Qwen3-2507 and Tongyi-DeepResearche and extend them to their "Heavy" variants through test-time scaling. These deep research agents achieve improvements of up to 20 absolute points on benchmarks such as BrowseComp Remarkably, as an open-source alternative, GLM-4.5 Heavy reaches accuracy 54.0% % BrowseComp, 66.0% GAIA, and 68.0% % xbench-Deeph Secp-eeehh,, placing it on par with the best proprietary choices such as OpenAl AI Deep Research and o3. Mongyi-DeepReseaun 69.0 % accuracy on BrowseComp.
Heavy pushes performance even further, attaining
INTRODUCTION
Test-time scaling underpins the progress of most advanced Al systems, such Grok 4 Heavy (xAI
2025b), GPT-5- GPT-5-Thinking- Thinking-Pro (OpenAl (025b), and Gemini 2.5 Pro (DeepMind, (025). It operates through two complementary strategies (Snell al. 2024): sequential scaling and parallel scaling. Sequential scaling deepens reasoning by extending chains of thought, as in OpenAl-ol (Jaech al. 2024)., DeepSeek-R1 (DeepSeek-Al et al. 2025), and Kimi-k1.5 (Team et al., 2025b). where reinforcement learning encourages reflective reasoning that can unravel complex tasks. Parallel scaling, by contrast, broadens the exploration space by producing multiple candidate outputs and selecting among them through aggregation methods such as Best-of-K selection. This strategy widely used advanced closed-source systems such as GPT-5-Thinking- Pro (OpenAI, 2025b) and Gemini-2.5 Pro (DeepMind) 2025). While expanding exploration space greatly increases the likelihood of capturing the correct solution, it also imposes steep computational costs.
Compared to spending test-time compute endlessly expanding the number of candidate outputs, allocating compute to verification to evaluate and select outputs introduces a new and orthogonal scaling dimension. Many tasks, such Sudoku, exhibit an Asymmetry of Verification (Weil 2025): they are far easier to verify than to solve, meaning that allocating compute to verifiers may yield disproportionate benefits. Many applications of deep search are prime examples of this property. In deep search, the models are often required to search deeply over the web through multiple recursive searching operations, browse hundreds or even thousands of pages, and identify information that

---

80
BrowseComp
GAIA
69.0
72.0
20.5
60
55.0
54.0
51.5
68.3
66.0
43.4
40
60
5K.3
58.3
27.0
31.5
20
50
Tonget De Proffrorrds
Hoaey
GLM14455 Hewvy
Oal Deguternii
Tongyy Depptarrd
GEM4
Olral -009
Heary
Oread Jap
40
Deratimaa repattrard
Tongo
Herry" Tenger
Deppi Resptecarar
Droplavant uaunn
Clasde
GLM-4 Heary
ate
Honey
82*
Accuracy vs Tool Calls
GLM-4.5 Heavy
50
51
40
Sequential Scaling of Search Agent
40
30
Asymmetric Verification
30 27
20
Scaling Verifier Compute
Parallel Scaling of Search Agent
20
10
10
200
400
600
800
1,000 Asymmetaic Vertiication
250
500
750
1,000
1,250
1,500
1,750
---
GLMM-4.5
-O
-
Gwen3-2507
Figure 1: Top part shows accuracy BrowseComp and GAIA, Results marked with are from our test runs; * Heavy denotes the accuracy after test-time scaling. Bottom left shows how accuracy on BrowseComp varies with tool calls. Solid lines indicate scaling the search agent's compute, dashed lines indicate allocating compute to a verifier. Bottom right shows strategies for extending GLM-4.5 to GLM-4.5 Heavy on BrowseComp.
satisfies the users' needs. In such scenarios, forward search must navigate an enormous, sparsely informative space, while backward verification begins with candidate answer and checks whether satisfies well-defined conditions (Wei et al. 2025). drastically shrinking the search space and making far more efficient of compute. Research on test-time scaling for deep search, particularly leveraging this asymmetry, therefore opens the path to building powerful and cost-effective agentic
search systems such Deep Research (OpenAl 2025a Google 2025 XAI 2025a).
In this work, we use the deep search capabilities of Deep Research systems as representative case to study how scaling test-time compute affects complex cetemmatiionseekingg behavior. Under simple agentic scaffold, we first investigate strategies that encourage models to increase search tool usage within a single trajectory. For example, we adopt the budget forcing strategy Muennighoff et al.) (2025). By forcing models to make additional tool calls after premature termination, we incentivize exploration of alternative reasoning paths. This substantially boosts tool usage and Pass @ accuracy: GLM-4.5 (Zeng et al. 2025) improves from 19% to 27%, and Qwen3-0527 from 8% to 18% on the challenging BrowseComp benchmark (Wei et al. 2025). However, excessive budget forcing eventually degrades performance. To address this, turn to parallel scaling, where expanding compute in parallel further strengthens exploration: GLM-4.5 achieves 67% Pass 32 accuracy on BrowseComp. However, these gains expose a critical bottleneck: scaling compute broadens the search space but does not help the search agent effectively identify the best candidates- to the need shift compute from search toward verification.
Deep search benefits from this shift because of the Asymmetry of Verification, where verifying an answer may be much easier than generating one. For instance, on BrowseComp. GLM-4.5 needs about 75 search tool calls on average to solve a problem, but verifying a candidate answer requires only about 18 tool calls. We leverage this asymmetry by adapting the search agent into verifier agent with minimal modifications, which produces substantial efficiency gains (see Figure bottom left). Specifically, we rollout multiple trajectories and answers from the search agent, use the verifier agent to evaluate the search agent's 's predicted answers by conducting several rounds of search operations, and then select candidates using strategies such as Best-of-K, which uses the highest verifier's confidence score among the K sampled trajectories. On BrowseComp, scaling search agent alone is costly: for GLM-4.5,, raising accuracy from 30% to 40% requires roughly 500 additional tool calls. In contrast, adding verifier achieves the same 10- 10-point accuracy increase with only about 100 extra calls. Moreover, verifier performance scales well with additional compute, where budget forcing and parallel sampling further improve the accuracy on filtering correct answers.
2

---

These results highlight compute-optimin paradigm for test-time scaling. Rather than allocating all resources to exploration, dedicating a substantial share of compute to verification yields dispro- portionately larger gains. By striking the right balance between search agents and verifier agents, selecting scaling strategies appropriate to the task (sequential or parallel), and applying effective aggregation metries, transform open-source models such GL.M-4.5, K2, Qwen3-2507 and Sungyi-DeepResearde into their "Heavy" variants through test-time scaling as exemplified in Figure [ bottom right. In this way, open-source models achieve performance levels comparable leading commercial systems as exemplified in Figure top part. Particularly, GLM-4.5 Heavy, an open-source system, performs on par with commercial leaders such as OpenAl DeepResearch and o3 across most benchmarks, achieving 54.0% accuracy BrowseComp, 49. 000%% on BrowseComp-zh, 66.0% on GAIA, and 68.0% on xbench- Deeph-h. Congyi-DeepResennt information- further boosts performance, with the "Heavy" variant reaching 69% accuracy on BrowseComp.
designed for long-horizon
SCALING SEARCH COMPUTE
In this section, we begin by presenting the framework of the search agent and methods for scaling compute. We then outline the experimental setup, specifying the benchmarks, models, and evaluation metries employed. Lastly, we report the experimental results on scaling the search agent's 'ss test-time compute within the proposed framework.
2.1 OVERVIEW OF THE SEARCH AGENT FRAMEWORK
To ensure scalability and generality, we design a streamlined framework based on ReAct (Yao et al. 2023). In this framework, the search agent handles user problems by iteratively reasoning, generating and executing actions, and processing observations received from real-world web environments. The agent's 's action space includes either generating a final answer invoking search tools. The search tool itself combines the web search and browsing functionalities introduced in WebThinker (Li et al., 2025b): it takes input search query and explicit search intent, and returns organized information retrieved from the web. Within this tool, an auxiliary model determines how to organize retrieved content or whether additional browsing is needed. To study test-time compute scaling in a controlled setting, we select K2 as the auxiliary model across the entire paper and do not change (see Appendix for details on this choice). Similar frameworks with searching and browsing functionalities based on ReAct have been widely adopted to develop web searching agents (Li et al.
2025a
Liu al. 2025
Gao al. 2025).
2.2
TEST-TIME SCALING APPROACHES
We categorize methods for scaling the search agent's compute into two types:
Sequential Scaling increases computational resources along a single agentic trajectory for each individual problem. We introduce two straightforward yet effective methods: (1) Max # Tool Call: It is common practice to establish a maximum number of tool calls allowed for the search agent, which is explicitly defined in the agent's 's system prompt. Once the agent reaches this limit, additional results are provided from the search tool. We experiment with this approach to encourage more tool callings, and details of the system prompt configuration are available in Figure 13 (2) Budget Forcing: Our experiments revealed that agents often terminate tasks prematurely, even when the system prompt permits sufficient tool calls. Inspired by the approach in Muennighoff et al. (2025). after the agent produces an initial final answer, we allocate additional budget for further tool usage and instruct the agent to continue exploring alternative solution paths or search strategies. Examples of budget forcing scenarios can be found in the Figure 15
Parallel Scaling expands computation by generating multiple independent solution trajectories for the same problem in parallel and then applying aggregation methods to determine the final output. We classify these aggregation methods into two types, depending on whether they rely on an external verifier: (1) Majority Voting: This approach follows the self-consistency principle (Wang et al., 2022). where the model independently produces multiple trajectories, and the most frequently predicted answer selected as the final output. However, majority voting has limited practicality, as it cannot
3

---

Qwen3-2507
GLM-4.5
14
16
25
12
23
12
10
18
6
12
-O- Max Tool Call
Budget Forcing
Figure 2: Sequential scaling of compute, with the x-axis representing the actual number of tool calls, and the y-axis representing @ass@1 accuracy BrowseComp. For Max Tool Call, search tool limits 30, and 50 for Qwen3-2507 and K2, and 15, 30, 50, and for GLM-4.5. Budget Forcing begins from each model's peak Max # Tool Call setting and expands until saturation: Qwen3-2500 starts at 15, adding 15 tools per step (7 expansions); K2 starts at 30, adding 30 per step (2 expansions); GLM-4.5 starts at 50, adding 50 per step expansions).
15,
easily extend to open-ended generation tasks where answers are diverse and not strictly comparable. (2) Verifier-Based These approaches employ an external verifier to assign scores to the sampled trajectories. Based these scores, can either adopt Be which selects the trajectory with the highest score, or Weighted Voting, which aggregates candidate trajectories by weighting them according to their verifier-assignee scores. In what follows, we first report results obtained using Majority Voting where we focus on analyzing the search agent alone, while in $ we present experiments incorporating verifier-based strategies.
2.3 EXPERIMENTAL SETUP
Benchmarks and Models To rigorously evaluate the agent's information-seeking capabilities, we use benchmarks that are substantially more challenging than standard QA tasks. For this section and ] we use BrowseComp (Wei et al., 2025) to study test-time scaling in series of controlled experiments, where the search queries are complex and require navigation through a vast space of potential answers. Since the complete BrowseComp includes about 1200 questions, completing the
full test consumes substantial computational resources. Therefore, a random sample of 100 is used to reduce costs while still being representative enough to reflect performance on the full dataset. In we will include more benchmarks to present the final test-time scaling results. We select three open-source models with strong reasoning and agentic capabilities, GLM-4.5 (Zeng et al. 2025). K2 (Team et al.) 2025a), and Qwen3-2333 235B-A22 222B-Instruct2555 (Yang et al. 2025 as search agents to evaluate their effectiveness when scaling test-time compute.
Metrics Following prior work (Gao et al. 2025; Li et al, 2025a), we approximate the test-time compute of search agents by counting the actual number of tool calls they make to solve each problem. This choice reflects the nature of deep research tasks, which often require interacting with external tools gather information. We also compare total number of tokens and the number of tool calls in Appendix[] [F To evaluate the performance of the search agent, we use two metrics: Pass@K which measures whether the search agent finds the correct answer at least once among K independently sampled trajectories; and Maj @ K, which measures whether the correct answer matches the majority vote among the answers from K independently sampled trajectories.
ways of measuring the scaling compute budget: the
2.4 EXPERIMENTAL RESULTS
The Max # Tool Call method underutilizes compute, while Budget Forcing boosts tool use and performance. Figure 2 illustrates how different models scale with test-time compute on BrowseComp under two methods: Max # Tool Call and Budget Forcing, Under the Max # Tool Call setting, even when models are allotted generous tool call quotas, they often terminate early. For instance, Qwen3-2507 shows diminishing marginal increases in tool usage and even performance drops. In contrast, Budget Forcing actively drives models to exploit available compute. After single application of budget forcing, GL.--45's
caturns-increasnng its quota from 15 to 50 results in only
4

---

Qwen3-2507
GLM-4.5
40
60
30
50
25
41
40
20
15
22
10
-
Pass
Maj0sK
Figure 3: Parallel scaling of compute, where the x-axis shows K and the y-axis shows Pass K and Maj @ K accuracy BrowseComp. For K2 and GLM-4.5, we first apply the Max Tool Call strategy, setting maximum tool usage to 30 and 50 respectively, then perform parallel scaling by independently sampling = 1, 24. 32 trajectories. For Qwen3-2500 we apply the Max Tool Call strategy with a maximum of 15 tools, then apply Budget Forcing add 15 more, followed by parallel scaling with K 1, 2, 4. 8, 16, 24, 32 trajectories.
8, 16,
tool usage more than doubles, and its Pass@1 accuracy rises from 19% to 27%. Qwen3-2507 exhibits an even more pronounced jump. with its performance ceiling increasing from 8% to 18% across multiple rounds. However, observe diminishing gains as the number of tool calls within single trajectory increases. This saturation suggests that long trajectories challenge models to perform coherent long-range reasoning.
Models exhibit strong exploration capabilities but struggle with exploitation. As shown in Figure 3 surprisingly, increasing compute via parallel scaling leads to a rapid rise in Pass@ K accuracy, indicating that with more sampled trajectories, models are increasingly likely find at least one correct answer. For instance, GLM-4.5's Pass@ K accuracy increases from 16% to 67% % when scaling K from 1 to 32 on the extremely challenging BrowseComp benchmark. Figure X in Appendix fuuthhrr shows that combining Budget Forcing with parallel scaling enhances exploration even more. However, this does not translate into efficient exploitation: while K2 reaches a Pass 16 of about 34%, its Maj @ 16 accuracy -eeuracy-rellecting the ability to identify the correct answer through majority voting- only around 12%. This gap suggests that simply encouraging broader exploration insufficient. Instead, improving the model's ability to recognize and select high-quality answers is critical. This suggests that instead of simply promoting more diverse exploration, we should focus on improving the model's 's ability exploit promising candidates. In the following sections, explore how incorporating external verification signals can improve exploitation efficiency in $ and how balancing exploration with exploitation ZZeng et al. 2024 Liu al. 2024) leads to better overall performance in $4
SCALING VERIFICATION COMPUTE
In this section, we begin by introducing the asymmetric verification, followed by the framework of the verifier we implemented. We then outline the experimental setup and present the results.
3.1 ASYMMETRIC VERIFICATION
Asymmetric Verification has become central concept in AI, highlighting tasks where verifying solution is far easier than generating one. Many problems have this "easy to verify" property. For instance, in Sudoku or the Eight Queens problem, generating a valid solution requires extensive search, but once a candidate solution is given, verification is trivial by simply checking whether all constraints are satisfied. Some tasks, however, show near-symmetry of verification, where checking correctness takes almost much effort as solving. A classic example is matrix multiplication: confirming the product of two large matrices is essentially as costly as performing the multiplication itself. At the other extreme, certain problems are hard to verify: writing code may be relatively straightforward, but rigorously verifying that it is free from security vulnerabilities can demand exhaustive formal verification full security audits. This "easy to verify" property allows the construction robust reward systems at relatively low cost. By leveraging inexpensive verification, Al performance can be significantly boosted through scaling during training at test time.
5

---

Table 1: Average number of tool calls for searching and verification models and benchmarks. Since Beench-DeepSeach has relatively simple search requirements and balanced difficulty between search and verification, we report only the tool calls needed for search.
Model
BrowseComp
BrowseComp-zh
GAIA
Chench-DeepSeane
Solve
Verify
Solve
Verify
Solve
Verify
Solve
Qwen3-2507
32,4
11.3
18.8
9.7
9.7
7.9
13.4
K2
27.8
11.2
38.2
6.3
6,2
5.8
2.9
GLM-4.5
75.3
18.0
48.3
9.0
17.4
10.6
6.1
The deep search task similarly exhibits this asymmetry. While forward search requires models to navigate a vast space and retrieve hard-to-find information- through extensive explo- ration vntion-verification only needs minimal effort to confirm whether a predicted answer satisfies the necessary conditions. Consider the following example from BrowseComp:
Please provide the name of video game released in 1992. The series' first game was featured in a globally recognized record book, and it earned spot in an Australian photographer publication. The third game inspired legacy sequel. The fifth installment was the first to have a Chinese dub.
Given video game, one can easily verify its validity via simple web searches by checking its release year, whether the first game was the record book, and so on. We also evaluate different models on search and verification tasks across multiple benchmarks, measuring the number of tool calls each requires in T The results show that most deep research tasks exhibit asymmetric verification, and this asymmetry grows with task difficulty. For instance, in BrowseComp. GL.M-.55 uses about 75.3 tool calls to search for a candidate answer but only 18 tool calls to verify In this section, we aim to harness this asymmetry by introducing verifiers filter candidate answers, thereby improving the efficiency of test-time compute usage.
3.2 OVERVIEW OF THE VERIFIER FRAMEWORK
Our verifier shares the same design principles, framework, and search tools as the search agent in differing only in the system prompt. the verifier's prompt, the model instructed to use search tools specifically to check the correctness of predicted answer and to assign confidence score based on whether the answer meets the necessary conditions confirmed by the search results. The full system prompt is provided in Figure 14 in the Appendix.
22T
3.3 EXPERIMENTAL SETUP
We use the same open-source models introduced in 22.3 and evaluate them on the most challenging benchmark, BrowseComp. To fairly compare the computational efficiency of scaling search agents versus scaling verifier agents, we start from the models that have already undergone parallel scaling. From this baseline, we explore two directions: (1) further scaling the search agent's 's computation, continuing to generate more trajectories through parallel sampling (2) introducing a verifier agent to evaluate the predicted answers from the existing trajectories. For the verifier-based approach, we filter candidates using the verifier's ' confidence scores, testing strategies such as Best-of-K and Weighted Voting, We note that verifier computation can also be scaled sequentially (c.g.. Budget Forcing) or in parallel, where multiple verification trajectories are generated for each candidate answer. The final confidence score the average across trajectories, reducing bias from any single sample. We provide the details of the experiments introducing the verifier agent (i.e., Figure 4) and the experiments on scaling verifier computation (i.e., Figure 5) in the Appendix D
3.4 EXPERIMENTAL RESULTS
illustrates that the asymmetry property of verification consistently appears across different models when scaling test compute. In other words, adding a verifier allows accuracy to grow more quickly than simply increasing the computation the search agent. For example, with GL GLM-4.5, increasing search computation alone requires about
Verifier yields superior accuracy-cost trade-off. Figure 4

---

Qwen3-2507
K2
GLM-4.5
24
20
50
22
40
20
15
18
30
10
14
20
12
5
10
0.00
100
200
300
400
500
0.00
100
200
300
400
500
0.00
250
500
750
1,000
1,250
Owen K2
GLM-4.5
-O-
MajaKK
+0
Best-ofK
- Weigh Weighted Voling
Asymmetric Verifrantion
Figure Parallel scaling results of different models on BrowseComp. The x x-axis shows the number of tool calls counting both the searcher and the verifier. The solid lines represent the growth of Maj @ K corresponding to scaling the searcher's compute, while the dashed lines represent the growth of Best-of K and Weighted Voting after introducing verifier.
additional 560 tool calls just to raise accuracy from 35.7% with Maj@ 16 to 40.8% with Maj@ 32. By contrast, introducing a verifier achieves larger gain with far less cost: only about 100 additional tool calls are needed to improve from 35.7% with Maj @ 8 to 45% with Weighted Voting. A similar pattern emerges with K2. Without verifier, raising performance from around 10% with Maj@ to 15.2% with Maj @ 32 requires increase of about 370 tool calls. With verifier, however, the same starting point improves to 18% with Best-of-8 or Weighted Voting after only about 77 additional tool calls.
Scaling the verifier can further raise the performance ceiling, though the gains depend on the model and strategy used. We apply the scaling methods like Max # Tool Call, Budget Forcing and Parallel Scaling to the verifier, with results shown in Figure These methods lead to notable improvements. For example, in the K2 case, applying Budget Forcing increases actual tool usage and boosts accuracy from 10% to 20%. Scaling methods such as budget forcing or parallel scaling, and answer aggregation schemes such Best-of-N Weighted Voting produce distinct improvement patterns. For example, with GLM-4.5, parallel scaling paired with the Best-of-8 approach achieves the largest gain, reaching 42% on BrowseComp. contrast, K2 attains better Best-of-8 accuracy with lower compute using budget forcing. These results show that, in practical deployments, the choice of scaling target (e.g., search agent or verifier agent), scaling strategy, and answer aggregation metric should be guided by both the model's 's characteristies and the available compute budget, to maximize performance per unit of cost.
4 PUSHING TEST-TIME SCALING LIMITS OF OPEN MODELS
4.1 EXPERIMENTAL SETUP
Building on the results from $2 and $ we can conceptualize test-time compute scaling through three orthogonal dimensions: scaling target, scaling strategy, and aggregation metric. Scaling target defines what part of the system is being expanded- either the search agent, which explores candidate solutions, or the verifier agent, which evaluates and filters them. Scaling strategy specifies how the additional compute is applied, encompassing approaches such as Max # Tool Call (adjusting the maximum number of tool calls allowed), Budget Forcing (actively increasing tool invocation), and Parallel Scaling (running multiple trajectories concurrently.. Aggregation metric determines how the outputs are combined and assessed, using criteria like Pass@ (single-best answer), Maj @K (majority voting), Weighted Voting (confidence-weighted voting), Best-of-K (selecting the highest-scorine candidate). This unified view helps disentangle where allocate extra computation, how to deploy it, and how measure its benefits, clarifying trade-offs and guiding compute-optimal scaling decisions.
Benchmarks and Models Compared to evaluating the agent on QA tasks that relatively simple, we aim to assess its information-seekinn capabilities using significantly more challenging benchmarks. Specifically, we evaluate the following benchmarks: (1) BrowseComp (Wei al. 2025). which
7

---

Qwen3-2507
GLM-4.5
20.0
20.5
18.0
60.
29.0
29.3
16.9
15.0
13.2
50-
Atcura
12.3
10.0-
20-
5.0
3.2
2.5
vonsa
Bustget Paeclep
Pareiil Scaling
Vaeilla
Max foo cal
Budget norang
Peralel trating
Vanssaa
Has e Yoat call
Betget Furning
Parella Staling
Mas
361.1
461.8
498.1
250
500
433.7
Comsute
300
250-
200
190.2
-163.2%
402
302
331.5
+274.0%
310
336.3
+75.2%
200
130
137.2
+38.6%
102
123.3
+74.9%
168.9%
+52.6%
50
Ha wont Cal
Binget Foriing
Stating
Homila
Star tast
Budgee Forrng
ole Statng
That
Burget Foring
Paratol Scating
anilla
Max Thol Call
Budget Forcing
Paralee Scaling
Figure 5: Different strategies for scaling verifier computation across models. The top panel shows accuracy. and the bottom panel shows the corresponding number of tool calls. Along the x-axis, vanilla indicates the Maj @8 accuracy achieved by scaling search agent without verification, while Max Tool Call, Budget Forcing, and Parallel Scaling show the Best-of-8 results when applying these strategies to increase verifier compute.
Table 2: Accuracy (%) of proprictary and open-souree agentic systems across multiple benchmarks. Results marked with from our test runs, indicating the maximum Pass@) that models can achieve when provided with sufficient calls without budget forcing. For Tongyi DeepResearch, we report both the results of our test runs and the officially reported results
BrowseComp
BrowseComp-zh
GAIA
sbench-DeepSearch
Proprietary Resp-Research Systems Models
Gemini 2.5 Pro
27.3
OpenAl
29.1
OpenAl o3*
55,0
59.0
68.0
68.0
OpenAl GPT-5
19.8
34.3
30.0
Grok3 DeepResearch
12.9
50+
Doubuo DeepResearch
26.0
50+
OpenAl DeepResearch
51.5
42.9
70.5
66.7
Open-Source Models
DeepScek-RI
23.2
16.5
32.7
Qwen3-2507'
23.0
44.7
45.5
K2'
11.0
22.0
50.0
54.0
GLM-4.5'
19.0
27.0
58.0
58.0
DeepSeek-V3.1
30.0
49.2
71.2
Tongyi D
43.0
39.0
70.6
68.0
Tongyi DeepResearch
43.4
46.7
70.9
75.0
Open-Source Models (Heavy Version)
Qwen3-2507 Heavy
29.0 (+21.0)
42.0 (+19.0)
53.4 (+8.7)
63,0 (+17.5)
K2 Heavy
(+13.0)
36.0 (+14.0)
58.3 (+8.3)
57.0 (+3.0)
GLM-4.5 Heavy
54.0 (+35.0)
49.0 (+22.0)
66.0 (+8.0)
68.0 (+10.0)
Tongyi DeepResearch Heavy
69.0 (+26.0)
55.0 (+16.0)
72.8 (+2.2)
80.0 (+12.0)
includes complex search problems that require navigating through a large space of potential answers. where relevant information difficult to locate. (2) BrowseComp-zh (Zhou et al. 2025), the Chinese
counterpart of BrowseComp to evaluate LLM agents on the Chinese web. (3) GAIA (Mialon
et al. 2023), which evaluates multi-modalitt and tool-use capabilities; following previous work (Li et al. 2025ar, we focus its text-only validation set. (4) xbench- DeepSearch (Chen al. 2025). another challenging benchmark designed to assess tool usage capabilities specifically in search and information retrieval contexts. The models used remain consistent with those in and 3 We
8

---

Qwen3-2507
K2
GLM-4.5
29
24
54
22
50
Sequential Scaling
Search Agent
25
20
40
20 18
14
Scaling Verifier Compute
30
27
15
10
Parallel Scaling
20
Agent
8
10
5
10
0.00
200
1,000
0.00
200
400
600
0.00
500
1,000
1,500
Budget Forcing
Scaling
Asymmetric Verification
Pass@1
Best of
*
Heavy Variant
Figure 6: Scaling test time compute for different models on BrowseComp, where the x-axis represents the actual number of tool calls, and the y-axis represents accuracy.
also include Tongyi-DeepRessi Cceppesearchh300-AA5 (eam 2025), model that is specifically designed for long-horizon, deep information-sen seeking tasks. Our evaluation uses 100 randomly sampled tasks from BrowseComp and BrowseComp-zhh since running the full test is computationally expensive. This sampling balances cost efficiency with representativeness. To ensure that our randomly selected subsets are representative, conduct robustness analyses three mutually exclusive random subsets in Appendix In contrast, evaluations for other benchmarks are conducted on their full datasets.
Scaling Details We scale performance by combining Max # Tool Call scaling with Budget Forcing to reach the Pass@1 peaks of different models. To broaden the exploration space of the search agents, apply parallel scaling on top of sequential scaling. For example, BrowseComp. GLM-4.5 generates 32 parallel samples, K2 generates 24, and Qwen3-2507 generates 16, while on other benchmarks every model samples 8 times in parallel. Once these trajectories are produced, verifier selects the answers, after which parallel scaling or budget forcing is applied again to the verifier to increase the verifier's test-time compute. These scaling approaches lead to the heavy variant of different models. We document the specific scaling setup for GLM-4.5 Heavy, K2 Heavy, and Qwen3-25t5 2507 Heavy Appendix
4.2 EXPERIMENTAL RESULTS
Table presents the experimental results of GLM-4.5 Heavy, K2 Heavy, and Qwen3-2507 Heavy. The three models after test-time scaling achieve substantial performance gains across all benchmarks. For example, Qwen3-2507 Heavy improves accuracy over Qwen3-2507 by about absolute points on BrowseComp. BrowseComp-zl and xbench-Deeepeene DeepSearch. Notably, GLM-4.5 Heavy reaches 54.0% accuracy on BrowseComp, 66.0% on GAIA, and 68.0% on xbench- Deeehh--eeSearchh putting on par with the best commercial agents such o3 and OpenAl Deep Research these three benchmarks. We also introduce a heavy variant of Tongyi Deep Research, which achieves 69% accuracy on BrowseComp. 55% accuracy on BrowseComp-zih 72.8% on GAIA and 80.0% xbench- -eepeh- shown Figure Full implementation details are provided in the Appendix
Figure presents the results of scaling test-time compute for different models BrowseComp. Sustained compute expansion can bring open-source systems close to closed-source performance. Verifier introduction further accelerates performance gains. For K2, adding a verifier increases Best-of-16 to 23%, with a slope much steeper than simply expanding the search agent. Figure I0 in Appendix H shows that performance patterns on BrowseComp-zll closely mirror those on BrowseC- omp. On BrowseComp-zhh GLM-4.5 attains Best-of-8 score of 49% exceeding the level of OpenAl Deep Research (see Table p Notably, parallel scaling of the verifier yields consistent accuracy gains: for K2, Best-of-8 rises from 31% to 36%. This is because parallel sampling mitigates variance in individual confidence scores. making the selection process more robust to noise by reducing the impact of extreme values. Figure [1 in Appendix presents the results of scaling test-time compute for different models on GAIA. Across models, different scaling strategies still yield notable performance gains for both search agents and verifier agents. We report xhench- Deaahh results in

---

Figure 12 of the Appendix. Because solving and verifying tasks in this benchmark are nearly equally difficult (as detailed in Appendix(]. we only evaluate search-agent scaling in Figure 12 and Table 2
RELATED WORK
Previous work on test-time scaling has mostly focused narrow reasoning domains such coding and mathematics (Muennighoff' et al. 2025 Luo et al. 2025). where increasing the number of "thinking tokens" reliably improves performance. Existing methods for scaling test- time compute can be organized along two axes: parallel and sequential (Muennighoff et al. [205 Snell et al. 2024). In parallel methods, the model independently generates multiple candidate solutions via repeated sampling (Brown et al. 2024), and final answer is chosen either by majority voting or by an external verifier using Best-of-N strategies (Brown et al., 2024, Irvine et al. [023]] Levi, 2024). In scquential methods, the model repeatedly revises and refines its own previous outputs to produce better solutions (Hou et al. 2025 Lee et al. 2025).
Verification is central to test-time scaling, but fully automated verification is only available in few settings (Brown et al.) 2024). such as proof checkers for formal theorems (Zheng et al.] [022]) or unit tests for code (Jain et al. 2025). As result, most settings rely LLM-as-judge (i.e., reward models) for verification. Current reward models are typically either outcome- maase (Xin et al. 2024 Ankner et al. 20244), which assign single global score each candidate solution and often used in Best-of-N Selection methods, or process-based (Lightman et al., 2023) Wang et al. 2024 Wu al. 2024 Wang 2025), which score individual reasoming steps and are commonly used to guide generation in tree-search methods (Gandhi et al. 20244) Wu et al. 2024) that combine both parallel and sequential exploration (Liu et al.,| 2003] Xie et al. 2023 Wu al., 2024).
Different from previous studies in which the verifier is not "agentic" (Brown et al. 2024 Jam
et al. 2025), our work investigates a verifier agent capable of invoking tools. Because deep research operates in open-ended, real-world environments where solving problem requires not only reasoning but also active information gathering via web tools -etri-netii,ving. assessing, and integrating external
evidence that cannot be inferred from the model's s internal knowledge alone (Wei et al. 2025 Li
al. 2025a). In prior works (Lightman et al. 2023; Wang et al. 2024| Brown et al.] 2024), each verification step requires a roughly fixed amount of computation, and test-time compute scaling primarily occurs along the search dimension. For the first time, however, our work explores verifier compute scaling in the context of an agentic verifier and quantitatively analyzes the allocation of computational resources between search and verification within asymmetric verification (Wei) 2025) situations.
6 DISCUSSION
In this paper, we present systematic study of test-time scaling for deep research agents, examining both sequential and parallel scaling. Our approach primarily exploits the properties of asymmetric verification, where we apply test-time scaling to range of flagship open-source models, extending them into "heavy variants" that achieve substantially higher performance on deep research tasks. Although current use of asymmetric verification is relatively simple by employing verifier to select final answers, it nevertheless proves highly effective, significantly improving the efficiency of test-time scaling. Looking ahead, we aim to adopt more flexible strategies. For example, verifiers could be applied at each step along the search trajectory, or even used to actively guide the search process. In addition, we envision incorporating asymmetric verification more deeply into training, enabling search agents to internalize verification capabilities and using them directly during inference.
ACKNOWLEDGMENTS
This project is partially supported by NSFC Grant 62306177, Hong Kong RGC ECS Grant 26218122 125, and Hong Kong RGC CRF Grant C6003-24Y.
10

---

REFERENCES
Zachary Ankner, Mansheej Paul, Brandon Cui, Jonathan D Chang, and Prithviraj Ammanabrolu. Critique-out-loud reward models. arxiv preprint arXiv:2408. 1179/, 2024.
Bradley Brown, Jordan Juravsky, Ryan Ehrlich, Ronald Clark, Quoe V Le, Christopher Ré, and Azalia Mirhoseini. Large language monkeys: Scaling inference compute with repeated sampling. arXiv preprint arXiv:2407.2 1787, 2024.
Kaiyuan Chen, Yixin Ren, Yang Liu, Xiaobo Hu, Haotong Tian, Tianbao Xie, Fangfu Liu, Haoye Zhang, Hongzhang Liu, Yuan Gong, et al. xbench: Tracking agents productivity scaling with profession-aligned real-world evaluations. arXiv preprint wrXiv:2506.1365/ 2025.
Google DeepMind. Gemini 2.5 pro system card. httpss://eeppe www.pmmmiinoooeleeeemmmmm haele////oeee modelo s gemini/pro/ 2025. General availability model card providing specifications, performance benchmarks, modalities, context window, pricing, and reasoning capabilities.
DeepSeck-AI, Daya Guo, Dejian Yang. Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, Bing Xue, Bingxuan Wang, Bochao Wu, Bei Feng. Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai Dai, Deli Chen, Dongjie Ji. Erhang Li, Fangyun Lin, Fucong Dai, Fuli Luo, Guangbo Hao, Guanting Chen, Guowei Li, H. Zhang, Han Bao, Hanwei Xu, Haocheng Wang. Honghui Ding, Huajian Xin, Huazuo Gao, Hui Qu, Hui Li, Jianzhong Guo, Jiashi Li, Jiawei Wang, Jingchang Chen, Jingyang Yuan, Junjie Qiu, Junlong Li. J. L. Cai, Jiaqi Ni, Jian Liang, Jin Chen, Kai Dong, Kai Hu, Kaige Gao, Kang Guan, Kexin Huang, Kuai Yu, Lean Wang. Lecong Zhang, Liang Zhao, Litong Wang. Liyue Zhang, Lei Xu, Leyi Xia, Mingchuan Zhang, Minghua Zhang, Minghui Tang. Meng Li, Miaojun Wang, Mingming Li, Ning Tian, Panpan Huang, Peng Zhang, Qiancheng Wang. Qinyu Chen, Qiushi Du, Ruiqi Ge, Ruisong Zhang, Ruizhe Pan, Runji Wang. R. Chen, R. Jin, Ruyi Chen, Shanghao Lu, Shangyan Zhou, Shanhuang Chen, Shengfeng Ye, Shiyu Wang, Shuiping Yu, Shunfeng Zhou, Shuting Pan, S. Li, Shuang Zhou, Shaoqing Wu, Shengfeng Ye. Tao Yun, Tian Pei. Tianyu Sun, T. Wang, Wangding Zeng, Wanjia Zhao, Wen Liu, Wenfeng Liang. Wenjun Gao, Wenqin Yu, Wentao Zhang, W. L. Xiao, Wei An, Xiaodong Liu, Xiaohan Wang, Xiaokang Chen, Xiaotao Nie, Xin Cheng, Xin Liu, Xin Xie, Xingchao Liu, Xinyu Yang, Xinyuan Li, Xuecheng Su, Xuheng Lin, X. Q. Li, Xiangyue Jin, Xiaojin Shen, Xiaosha Chen, Xiaowen Sun, Xiaoxiang Wang, Xinnan Song, Xinyi Zhou, Xianzu Wang. Xinxia Shan, Y. K. Li, Y. Q. Wang, Y. X. Wei, Yang Zhang, Yanhong Xu, Yao Li, Yao Zhao, Yaofeng Sun, Yaohui Wang, Yi Yu, Yichao Zhang, Yifan Shi, Yiliang Xiong, Ying He, Yishi Piao, Yisong Wang, Yixuan Tan, Yiyang Ma. Yiyuan Liu, Yongqiang Guo. Yuan Ou, Yuduan Wang, Yue Gong, Yuheng Zou, Yujia He. Yunfan Xiong. Yuxiang Luo, Yuxiang You, Yuxuan Liu, Yuyang Zhou, Y. X. Zhu, Yanhong Xu, Yanping Huang, Yaohui Li, Yi Zheng, Yuchen Zhu, Yunxian Ma, Ying Tang, Yukun Zha, Yuting Yan, Z. Z. Ren, Zehui Ren, Zhangli Sha, Zhe Fu, Zhean Xu, Zhenda Xie, Zhengyan Zhang, Zhewen Hao. Zhicheng Ma. Zhigang Yan, Zhiyu Wu, Zihui Gu, Zijia Zhu, Zijun Liu, Zilin Li, Ziwei Xie, Ziyang Song, Zizheng Pan, Zhen Huang, Zhipeng Xu, Zhongyu Zhang, and Zhen Zhang. Deepseek-rl: Incentivizing reasoning capability in llms via reinforcement learning, 2025.
URL http:tttps//aaxx.o/ org/ 2abb./201..22.
Kanishk Gandhi, Denise Lee, Gabriel Grand, Muxin Liu, Winson Cheng, Archit Sharma, and Noah D Goodman. Stream of search (sos): Learning to search in language. arXiv preprint arXiv:2404.038 033683 2024.
Jiaxuan Gao, Wei Fu, Minyang Xie, Shusheng Xu, Chuyi He, Zhiyu Mei, Banghua Zhu, and Yi Wu. Beyond ten turns: Unlocking long-horizon agentic search with large-scale asynchronous rl. arXiv preprint wwXiv:2508.07977 2025.
Google. Gemini deep research overview.
httpsr//gemihi..oogl///vvvvvrvee
Seep-researchh 2025.
Zhenyu Hou, Xin Lv, Rui Lu, Jiajie Zhang, Yujiang Li, Zijun Yao, Juanzi Li, Jie Tang, and Yuxiao Dong. Advancing language model reasoning through reinforcement learning and inference scaling. arXiv preprint 3rXiv:2501./1651 2025.
11

---

Robert Irvine, Douglas Boubert, Vyas Raina, Adian Liusie, Ziyi Zhu, Vineet Mudupalli, Aliaksei Korshuk, Zongyi Liu, Fritz Cremer, Valentin Assassi, et al. Rewarding chatbots for real-world engagement with millions of users. arXiv preprint arXiv:2303.061.3 2023.
Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Camey, et al. Openai ol system card. arXiv preprint arXiv:2412.16720, 2024.
Naman Jain, Jaskirat Singh, Manish Shetty, Liang Zheng, Koushik Sen, and lon Stoica. R2e- gym: Procedural environments and hybrid verifiers for scaling open-weights swe agents. arXiv preprint arXiv:2504.07164, 2025.
Kuang-Huei Lee, Ian Fischer, Yueh-Hua Wu, Dave Marwood, Shumeet Baluja, Dale Schuurmans, and Xinyun Chen. Evolving deeper IIm thinking. arXiv preprint arXiv:2501. 10991,, 2025.
Noam Levi. A simple model of inference scaling laws. arXiv preprint arXiv:2410. 163777 2024.
Kuan Li, Zhongwang Zhang, Huifeng Yin, Liwen Zhang, Litu Ou, Jialong Wu, Wenbiao Yin, Baixuan Li, Zhengwei Tao, Xinyu Wang, et al. Websailor: Navigating super-human reasoning for web agent. arXiv preprint 250i::2500.0292,, 2025a.
Xiaoxi Li, Jiajie Jin, Guanting Dong, Hongjin Qian, Yutao Zhu, Yongkang Wu, Ji-Rong Wen, and Zhicheng Dou. Webthinker: Empowering large reasoning models with deep research capability. arXiv preprint arXlv:2504.21776 2025b.
Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let's verify step by step. In The Twelfth International Conference on Learning Representations, 2023.
Jiacheng Liu, Andrew Cohen, Ramakanth Pasunuru, Yejin Choi, Hannaneh Hajishirzi, and Asli Celikyilmaz. Don't throw away your value model! generating more preferable text with value- guided monte-carlo tree search decoding. arXiv preprint arXiv:2309.15028 2023.
Junteng Liu, Yunji Li, Chi Zhang, Jingyang Li, Aili Chen, Ke Ji, Weiyu Cheng, Zijia Wu, Chengyu Du, Qidi Xu, et al. Webexplorer: Explore and evolve for training long- hong-horioon web agents. arXiv preprint arXiv:2509.0653
2025.
Wei Liu, Juniong Li, Xiwen Zhang, Fan Zhou, Yu Cheng, and Junxian He. Diving into self-evolving training for multimodal reasoning, arXiv preprint 2rXiv:2412.14411,2000 2024.
Michael Luo, Sijun Tan, Justin Wong, Xiaoxiang Shi, William Y Tangg Manan Roongta, Colin Cai, Jeffrey Luo, Tianjun Zhang, Li Erran Li, et al. Deepscaler: Surpassing ol -prevvew with 1.5 b model by scaling rl. Notion Blog, 2025.
Grégoire Mialon, Clémentine Fourrier, Thomas Wolf, Yann LeCun, and Thomas Scialom. Gaia: benchmark for general ai assistants. In The Twelfth International Conference on Learning Representations, 2023.
Niklas Muennighoff, Zitong Yang. Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannanch Hajishirzi, Luke Zettlemoyer, Percy Liang. Emmanuel Candès, and Tatsunori Hashimoto. sl: Simple test-time scaling. arXiv preprint 1xXi:2550111999 2025.
OpenAl
Deep research system card.
https://cch.opena..
card. pdf 2025a.
OpenAl.
https://cch.openai.com//// 8124a3ce ab78 4206 96eb www.449eaaaaa.... sysptts-syttmmmaaaaa www.eaarrdaug.pppc.
Gpt-5 system
card.
aug 2025b.
Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. Scaling Ilm test-time compute optimally can be more effective than scaling model parameters. arXiv preprint arXiv:2408.033// 2024.
Kimi Team, Yifan Bai, Yiping Bao, Guanduo Chen, Jiahao Chen, Ningxin Chen, Ruijue Chen, Yanru Chen, Yuankun Chen, Yutian Chen, et al. Kimi k2: Open agentic intelligence. arXiv preprint 2xxi:2507720534 2025a
12

---

Kimi Team, Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, et al. Kimi kl. Scaling reinforcement learning with Ilms. arXiv preprint arXiv:2501.12599 2025b.
Tongyi DeepResearch Team. Tongyi-deepresearch. http:://gtthus..cmm....b..o.... DeepResearch 2025.
Peiyi Wang, Lei Li, Zhihong Shao, Runxin Xu, Damai Dai, Yifei Li, Deli Chen, Yu Wu, and Zhifang Sui. Math-shepherd: Verify and reinforce Ilms step-by-step without human annotations. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume
I: Long Papers). pp. 9426 9439, 2024.
Xingyao Wang.
Sota on swe-bench verified with inference-time scaling and model.
critic
2025. All
Hands AI Blog.
Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang. Aakanksha Chowdh- cry, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. arXiv preprint arXiv:2203.111771 2022.
Jason Wei. Asymmetry of verification and verifiers' law. https: Www. jasonwei.neg net/ b blog/as /llog/ssymmet:tmm ry- - fies--laf 2025. Accessed: 2025-
08-07.
Jason Wei, Zhiqing Sun, Spencer Papay, Scott McKinney, Jeffrey Han, Isa Fulford, Hyung Won Chung, Alex Tachard Passos, William Fedus, and Amelia Glaese. Browsecomp: A simple yet challenging benchmark for browsing agents. arXiv preprint arXiv:2504.12511 2025.
Yangzhen Wu, Zhiqing Sun, Shanda Li, Sean Welleck, and Yiming Yang. An empirical analysis of compute-optine optimal inference for problem-solving with language models. 2024.
xAI. Grok 3 beta - the age of reasoning agents. 2025a.
gro.// February 19
XAI. Grok 4 the most intelligent model in the world. http::://...e//e//////ggoooe July 09 2025b.
Yuxi Xie, Kenji Kawaguchi, Yiran Zhao, James Xu Zhao, Min-Yen Kan, Junxian He, and Michael Xie. Self-evaluation guided beam search for reasoning. Advances Neural Information Processing Systems, 36:41618-4118 -16500 2023.
Huajian Xin, Daya Guo, Zhihong Shao, Zhizhou Ren, Qihao Zhu, Bo Liu, Chong Ruan, Wenda Li, and Xiaodan Liang. Deepseek-prover Advancing theorem proving in llms through large-scale synthetic data. arXiv preprini arXiv:2405. 14333, 2024.
An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report. arXiv preprint TrxXi::5005.03880 2025.
Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models. In International Conference on Learning Representations (ICLR), 2023.
Aohan Zeng, Xin Lv, Qinkai Zheng, Zhenyu Hou, Bin Chen, Chengxing Xie, Cunxiang Wang, Da Yin, Hao Zeng, Jiajie Zhang, al. Glm-4.5: Agentic, reasoning, and coding (arc) foundation models. arXiv preprint arXiv:2508.0647 2025.
Weihao Zeng. Yuzhen Huang, Lulu Zhao, Yijun Wang, Zifei Shan, and Junxian He. B-star: Monitoring and balancing exploration and exploitation in self-taught reasoners. arXiv preprint arXiv:2412.17256 2024.
Kunhao Zheng. Jesse Michael Han, and Stanislas Polu. Minif2f: a cross cross-system benchmark for formal olympiad-level mathematics. arXiv preprint arXiv:2109.00/ 10, 2021.
13

---

Peilin Zhou, Bruce Leon, Xiang Ying, Can Zhang, Yifan Shao, Qichen Ye, Dading Chong, Zhiling Jin, Chenxuan Xie, Meng Cao, et al. Browsecomp-zh zh: Benchmarking web browsing ability of large language models in chinese. arXiv preprint arXiv:2504.19314 2025.
14

---

Tool Calls
19.0
20
Tool Calls
20
12.0
Passog
1.5
K2
GPT-4o-mini
Figure 7: @ and the average number of actual tool uses per problem for different auxiliary models, under maximum limit of 50 search tool calls.
Pass@ K
Maj@K
35
30
14
22
12
20
11
Max # Tool Call
Budget Forcing
Figure 8: A comparison of Qwen3-2507's performance after applying Max Tool Call and Budget Forcing followed by parallel scaling. The x-axis represents K; the y-axis shows Pass@ K and Maj@K, K. Max # Tool Call limits tool usage to 15 calls and applies Parallel Scaling with K 1, independent trajectories. Budget Forcing increases this limit by 15 calls and uses the same values.
A THE USE OF LARGE LANGUAGE MODELS
Large language models (LLMs) were used exclusively to refine the writing of this paper. All aspects of the research process-including ideation, design, analysis, and interpretation-weet conducted entirely without the assistance L.MMs.
B IMPACT OF AUXILIARY MODEL ON SEARCH TOOL
The auxiliary model in the search tool is responsible for extracting information from retrieved content based on the search intent, initiating follow-up queries, and selectively accessing relevant URLs. We assess how its capabilities affect search tool performance by using GLM-4.5 as the search agent and comparing a stronger auxiliary model (K2) with a weaker one (GPT-4o-mini) with a maximum of 50 tool calls. Figure shoow performance on BrowseComp along with the actual number of tool calls. While both auxiliary models make similar number of calls, their accuracy diverges sharply: GPT-4o-mini reaches only 12%, whereas K2 attains 19%. This gap demonstrates that auxiliary model strength directly impacts the quality of search results and, consequently, the search agent's final performance. For this reason, we use K2 as the default auxiliary model in our experiments.
C COMBINING BUDGET FORCING WITH PARALLEL SCALING
In Figure compare Qwen3-2507's performance after applying Max # Tool Call and Budget Forcing, followed by parallel scaling. The results show that combining Budget Forcing with parallel scaling further enhances exploration.
15

---

Table 3: GLM-4.5's Pass@8, Maj@8, and Best-of-8 three mutually exclusive subsets BrowseComp and BrowseComp-zh.
Dataset
Metric
Run
Run
Run 3
Mean + Std
P
46.4
48.0
50.0
48.1 1.8
BrowseComp
Maj@8
30.1
30,9
32.0
31.0 + 0.9
Best-of-8
39.0
38.0
40.0
39.0 1.0
P
64.0
67.0
65.9
65.6 1.5
BrowseComp-zh
Maj@8
39.8
38,2
42.6
40.2 + 2.2
Best-of-8
42.0
41.0
46.6
43.2 + 3.0
D SCALING DETAILS OF THE VERIFIER
We introduce verifier agent that evaluates predicted answers derived from search agent's trajectories. In this verifier-based approach, we filter candidate answers using the verifier's confidence scores, exploring strategies such as Best-of-K K and Weighted Voting. The search agent's computation is scaled in exactly the same way as in S2 each model samples up to 32 trajectories, from which we extract the predicted answers. The verifier then evaluates each answer, producing confidence scores that guide downstream aggregation. Different aggregation strategies can be used. In the Best-of-K setting, select the answer with the highest verifier score among the K sampled trajectories, In the Weighted Voting setting, we aggregate the candidate answers by weighting each one according to its perifier-assinned confidence. Results for these strategies are shown Figure Verifier computation itself can also be scaled in different ways, either sequentially or in parallel. In the sequential case, the verifier can be given additional tool usage after providing initial score. This Budget Forcing setup encourages the verifier to explore alternative verification paths, refining its confidence estimate before updated score used for filtering. In the parallel case, the verifier runs multiple independent evaluations of the same candidate answer. The confidence of these evaluations. We report scaling verfier's compute results in Figure
used for filtering is then the average
E ROBUSTNESS ANALYSIS VIA RANDOM SAMPLING
To verify that the originally selected subsets representative, conduct additional experiments by randomly sampling two new sets of 100 mutually exclusive samples from BrowseComp. For BrowseComp-zh, since the full dataset contains fewer than 300 samples, split the remaining data into two groups, one with 100 samples and the other with 89 samples, and evaluate both. For each group, measure GLM 4.5's Pass@8, @8, Maj @8, and Best-of-8 (using the verifier) and compute the mean and standard deviation over the three runs. The results are shown in Table 3 where Run I corresponds the subset used in 2 $ and $4 These results show that across all three mutually exclusive subsets, the Pass @ 8, Maj Φ 8, and Best-of-8 scores remain consistent, indicating that the subsets used Run provide reliable reflection of model performance on the full datasets.
F DIFFERENT METHODS FOR MEASURING SCALING BUDGET
We compare two ways of measuring the scaling compute budget: the total number of tokens and the number tool calls. this comparison, we replace the x-axis metric for GLM-4.5 Figure 4 with the total number of tokens and present the resulting comparison in Figure It shows that the asymmetry verification phenomenon remains clear and consistent even when the scaling budget is measured by total tokens rather than by tool calls.
G SCALING DETAILS OF THE HEAVY VARIANT
We scale performance by combining Max # Tool Call scaling with Budget Forcing to reach the Pass @ peaks of different models. In BrowseComp. Budget Forcing is applied Qwen3-2507 by expanding it five times, with each expansion adding 15 additional tool calls. For K2, the expansion is performed once with 30 additional tool calls, while GL.-4.55 is expanded with 50 additional
16

---

GLM-4.5 (Tool Calls)
GLM-4.5 (Tokens)
50
Accuracy (%)
Accuracy (%)
40
40
3.0
35
30
10
0.00
200
400
600
800
1,000
1,200
50%
100k
150k
200k
250%
300%
350k
Number Tool Calls
Number Tokens
-O- GLM-45 (Mae
-O- of
+ Weighted Voting
Asymmetrie Verification
Figure 9: The parallel scaling results of GLM-4.5 on BrowscComp are evaluated in two ways: by the number of tool calls and by the total number of tokens used, In the left figure, the horizontal axis represents the number of tool calls; in the right figure, it represents the total token count. Both measurements include contributions from the searcher and verifier. The solid curves show how Maj@ K improves as the searcher's compute increases, while the dashed curves show how Best-of-K and Weighted Voting improve when a verifier is added.
tool calls. In BrowscComp-zh. Budget Foreing also applied to Qwen3-2500 this time expanding three times with 30 additional tool calls at each step. K2 achieves its highest Pass @ 1 performance when the Max Tool Call is set to 15, whereas GLM-4.5 expanded once with 50 additional tool calls. In GAIA, Budget Forcing is again used to expand Qwen3-2507, adding 15 additional tool calls at each of three expansion steps. K2 is expanded once with 30 additional tool calls, while GLM-4.5 achieves its peak Pass 1 score when the Max # Tool Call is set to 30. Finally, in sbbench-DeeSSeie Qwen3-2500 is expanded five times, with each step adding 15 additional tool calls. K2 reaches its highest Pass @ score when the maximum number of tool calls is set to 100, whereas GLM-4.5 reaches its peak with the maximum number of tool calls set to 15.
To broaden the exploration space of the search agents, apply parallel scaling top of sequential scaling. On BrowseComp. for instance, GLM-4.5 generates 32 parallel samples, K2 generates 16, and Qwen3-2507 generates 24, while on other benchmarks every model produces 8 parallel samples. Once these trajectories are generated, a verifier evaluates the answers, after which we apply either parallel scaling or budget forcing again to the verifier in order to increase its test-time compute. On BrowseComp with Qwen3-25t5 2507, each trajectory from the search agent is verified four times in parallel, and the verifier's average score is taken as the trajectory score, yielding Best-of-24 accuracy. With K2, the same process produces Best-of- 16 accuracy, and with GL.M-.55 it yields a Best-of-32 accuracy. On BrowseComp-zhh, the procedure similar: Qwen3-2507, K2, and GLM-4.55 all rely on four parallel verifications per trajectory, with the average score determining the outcome, leading in each case to a Best-of-8 accuracy. On GAIA, Qwen3-2507 and K2 follow the same approach and also obtain Best-of-8 accuracy. For GLM-4.5 on GAIA, however, we used budget forcing to expand the verifier's 's compute once, with the resulting verification score taken as the trajectory score, again producing a Best-of-8 result. Finally, since solving and verifying tasks in xbench- Beench-DeepSeach are of nearly equal difficulty, we did not employ a verifier on that benchmark.
We apply test-time extension to Tongyi Deep Research BrowseComp. Because this model does not perform well as a verifier, we rely on GLM 4.5 for verification. The process works as follows: Tongyi Deep Research first generates 16 parallel samples, and GLM 4.5 then extracts answers from these responses. Next, GLM 4.5 serves as a verifier to check the extracted answers. Finally, through weighted voting, this approach achieves an accuracy 69%. For other benchmarks such as BrowseComp-zil zh, GAIA and Abench-DeepSeeee first generates parallel samples, and GL.M 4.5 then extracts answers from these responses. Next, GLM 4.5 serves verifier to check the extracted answers. Finally, through weighted voting, this approach achieves 55.0%. 72.8%, and 80.0% on BrowseComp-zh, GAIA and xbench- Seeechh respectively.
we take similar steps, where Tongyi Deep Research
H SCALING RESULTS ON BROWSECOMP-ZH
Figure T0 presents the results of scaling test-time compute for various models on Browsecomp-zh.
17

---

Qwen3-2507
K2
GLM-4.5
42
36
50
49
40
Sequential Scalling
45 of Search Agent
35
30
Scaling Verifier Compute
30
35
33
30
Parallel Scaling of Search Agent
20
23.
22
25
15
20
0.00
200
0.00
50
100
150
0.00
100
300
Budget Forcing
Scaling
Asymmetric Verification
Pass@1
Best of
*
Heavy Variant
Figure 10: Scaling test time compute for different models on Browsecomp-zh where the x-axis represents the actual number of tool calls, and the y-axis represents accuracy.
Qwen3-2507
K2
GLM-4.5
53
58
66
58
66
Sequential Scaling
52
64
Agent
62
50 50
52
58
Scaling Verifler Compute
50
58"
48
56
Parallel Scaling of Search Agent
46
46
54
52
44
0.00
100
Max # Tool Call
Budget Forcing
Parallel Scaling
Asymmetric Verification
O
Pass@1
Weighted Voting
*
Heavy Variant
Figure 11: Scaling test time compute for different models on GAIA, where the x-axis represents the actual number of tool calls, and the y-axis represents accuracy.
I SCALING RESULTS ON GAIA
Figure II presents the results of scaling test-time compute for various models on GAIA.
J SCALING RESULTS ON EBENCH-DEEPSEARE
Figure 12 presents the results of scaling test-time compute for various models on abench-DeepSeee Scaling the search agent's 's compute consistently yields substantial performance gains. In contrast, verifier scaling offers little improvement. For example, with GLM-4.5, Maj @8 reaches 67.61% 61,, yet adding a verifier only raises weighted voting 68. 68.0%. This minimal gain arises because, on xbench- DeepSec,, solving a problem and verifying it require comparable effort. For instance, consider the query: As of May 18 2025, what the maximum context token count supported by OpenAl's codex-1 agent? The correct answer is 192k. but confirming this demands nearly the same search intensity as finding it.
18

---

Qwen3-2507
K2
GLM-4.5
57
68
68
60
56
58
54
64
58
54
52
60
52
50
56
52
0.00
20
40
60
10
15
20
25
10
20
30
40
50
Max # Tool Call
Budget Forcing
Paraliel Scaling
Asymmetric Verification
Pass@1
Maj@k
*
Heavy Variant
Figure 12: Scaling test time compute for different models hhench-DeepSearch, where the x-axis represents the actual number of tool calls, and the y-axis represents accuracy.
You are a reasoning assistant equipped with web search capabilities to help solve problems. Use the "deep_websearch" tool whenever you need additional information or evidence. You may use this tool multiple times, to maximum of imax_search_colls, times.
answer must be provided within boxed environment like this:
W
NboxeefECyour final answer>}3
W
Figure Details the Max Tool Call's system prompt for search agent.
19

---

You reasoning assistant equipped with web search capabilities to fact-check and verify the accuracy of predicted answer to user's question. Your sole task is to verify the predicted answer, not to answer the question yourself. To ensure rigorous and reliable verification, actively use the "deep_websearch tool whenever supporting evidence needed. You may use tool multiple times, up
of {nox_search_cally searches.
Follow these steps carefully:
Carefully read the user's question and extract all conditions that correct answer must satisfy. are the requirements imposed by the question properties the predicted
alone.
For each condition, check whether the predicted answer satisfies it. Use deep_websearch to find evidence. A condition is considered verified only if:
is explicitly supported by search results obtained through weep_websearch, and The search results confirm that the predicted answer does indeed satisfy the condition.
Compute the confidence score as the ratio of conditions that are both
(1) verified using webe_websaa
(11) confirmed be satisfied, the total number of identified conditions.
Finally, present the confidence score in the following format:
WC Nbbeef(hheer<Kconttteee..ceeell
WJ
Here, <confidence_scorte scoree (number of conditions both verified and satisfied via deep_websearch) / (total number of conditions from the question).
Figure 14: Details the Max Tool Call's system prompt for verfier agent.
20

---

System:
You are reasoning assistant equipped with web search capabilities to help solve problens. Use the "deep_ seppwwebeerrhh tool whenever you need additional information or evidence. You may use this tool multiple times, up to maxinum of *+15 times*x. answer provided within boxed NDNn\\boxedECyour final answer>]\n\\]
like this:
User:
I'n looking for the name cartoon based on following details, as of December 2023:
on during the 1998s of characters had multiple who appeared in Soap opera One of the creators of the show was given particular award in particular decade and was also gåven another particular award in different decade One of the creators of the show has had their work appear on an album cover by band founded in the 1980s One the companies produced carteon was founded the 1970s, they made multiple cartoons featuring aninal characters"
voiced by
Assistant:
("search_intent:: "Identiify 1990s cartoon based character and voice actor details', "search_query" "1990s cartoon multiple heads character voiced by soap opera actor")"
After analyzing the details provided and conducting multiple searches, the cartoon fits criteria is **Animaniacsex.
User: (Budget Forcing)
will give you up to a maximum of 15 additional chances to use the 'ddep_. deep_websearcch solve the problem. Trying other solution paths search strategies encouraged.
Assistant:
("search_intents "Identify 1990s cartoon with multi-headed character voiced soop opera actor and produced by studio founded in the 1970s", seerrc *1990s cartoon multi-head multi-headed character voice actor soap opera studio founded 1970a*}
Figure 15: Examples of budget forcing scenarios.
21


Reviews:
[{"review_id": "qRWtjyuADK", "reviewer": "Reviewer_S1Wg", "summary": "This paper explores test-time compute scaling for deep search agents through both sequential and parallel strategies. Sequential scaling extends generation, while parallel scaling verifies and selects among multiple outputs. Building on the idea of asymmetric verification, where verification is substantially easier than generation. The authors show that allocating limited compute to verification significantly boosts performance. Experiments with several open-source models demonstrate consistent improvements across multiple benchmarks.", "questions": "1. Could the authors clarify how the random subset of 100 samples from BrowseComp and BrowseComp-zh was selected? Was any stratified or difficulty-based sampling strategy used to ensure representativeness?\n\n2. When the same base model architecture is used for both search and verification, how do the authors ensure that the verifier's decision is independent and not overly correlated with the search agent's outputs?\n\n3. Since Tongyi Deep Research uses GLM-4.5 as its verifier, could the authors discuss potential fairness concerns and whether a unified verification framework could address this issue?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper is well-motivated, clearly structured, and supported by comprehensive experiments that effectively validate its key claims:\n\n1. This paper demonstrates the inherent asymmetry of deep search tasks, where generation is substantially harder than verification, and shows that allocating compute to the verifier during test-time scaling (TTS) yields a superior accuracy–cost trade-off.\n\n2. It systematically decomposes TTS into a three-dimensional design space of scaling target × scaling strategy × aggregation metric, providing a principled framework for compute-optimal allocation.\n\n3. It achieves \"heavy\" variants of open-source models through TTS, significantly narrowing the performance gap between open and top proprietary systems—and even surpassing closed-source models in certain benchmarks.", "weaknesses": "1. External Validity Risk from Benchmark Sampling: On BrowseComp and BrowseComp-zh, the authors evaluate only a random subset of 100 samples instead of the full benchmark of approximately 1,200 questions. This limited sample size increases sampling variance and may reduce the robustness of observed performance differences across methods. Although the paper mentions that this decision was made \"to reduce computational cost,\" it does not report confidence intervals or conduct resampling-based robustness analyses to support the reliability of the results.\n\n2. The paper's insight that verification is inherently easier than search is novel; however, its implementation exhibits notable simplifications and potential biases. The verification process still relies on prompt-based judgment by the large language model agent, meaning its reliability ultimately depends on the model's underlying capabilities and biases.\n\n3. In addition, the verifier shares the same base architecture as the search agent (except for Tongyi Deep Research), differing only in prompts and objectives. This design introduces a risk of self-verification bias, as the verifier is not fully independent. Moreover, using GLM-4.5 as the verifier for Tongyi-DeepResearch raises fairness concerns, as this hybrid setup makes direct comparison difficult. A more objective evaluation would benefit from incorporating an external or model-agnostic verifier applicable across different search agents.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "WMsCsxG7VW", "reviewer": "Reviewer_A9rG", "summary": "This paper studies how previous test-time compute scaling, including sequential scaling and parallel scaling, affects deep search agents. It identifies a critical property of deep search agents, which is referred to as asymmetric verification, that verifying an answer is easier than generating it. Based on that, this paper proposes a new test-time scaling (TTS) method by constructing a verification agent and allocating a modest amount of compute to the verifier to verify answers retrieved from the search agent. Experimental results demonstrate that the proposed TTS method yields better accuracy–cost trade-offs than scaling computation on the search agent alone. Then, this paper extends flagship open-source models (GLM-4.5, K2, Qwen3-2507, Tongyi-DeepResearch) into “Heavy” variants using the proposed TTS method, and these variants achieve or surpass the performance of proprietary systems like OpenAI o3 and OpenAI DeepResearch.", "questions": "1. Please analyze the differences between the TTS method proposed in the paper and the previous verifier-based TTS methods.\n2. Please present the experimental results of the proposed TTS framework on the full dataset of the BrowseComp and BrowseComp-zh benchmarks using flagship open-source models, or provide an analysis demonstrating that the randomly sampled sub-sets used in the paper can represent the full datasets.\n3. Please analyze and categorize the strength of asymmetric verification across different datasets, and analyze its impact on parameter selection for the proposed TTS framework (e.g., how to allocate computation when dealing with datasets with different strengths of asymmetric verification). This would provide explicit guidelines when using the TTS framework proposed in the paper.\n4. For Table 2, please provide the performance of flagship open-source models using either the previous TTS methods or the baseline TTS methods in the paper when adding the same or similar computational resources as the “heavy version”.\n5. Please review the experimental results. Mismatch in experimental results: the Maj@8 accuracy for GLM-4.5 reported in line 312 is 35.7%, whereas the Maj@8 accuracy for GLM-4.5 displayed in the upper right corner of Figure 5 is 30.6%. This discrepancy impacts certain experimental conclusions.\n6. Please check typos. For example, \"heavey\" in line 430.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper is well written and structured, with clear motivation and logical flow. Figures effectively summarize trends and highlight how the proposed TTS method boosts the performance of deep search agents using the property of asymmetric verification.\n2. Discovering the asymmetric verification property of deep search agents and designing a verifier-based TTS method based on this discovery is highly insightful.\n3. Experimental results on multiple benchmarks show that the proposed TTS method achieves a better accuracy–cost trade-off than previous TTS methods (e.g., sequential scaling, parallel scaling). More importantly, the proposed verifier-based TTS method is orthogonal to previous TTS methods, so they can be combined to form a more powerful TTS method.\n4. The “Heavy” variants of flagship open-source models, by using the proposed TTS method, achieve on par or better performance than commercial systems, further demonstrating the effectiveness of the proposed TTS method.", "weaknesses": "1. The paper does not explicitly discuss related works, such as previous verifier-based test-time scaling (TTS) methods, making it difficult to assess the contributions and innovations of this work.\n2. In the experimental section, the authors randomly selected 100 tasks from the BrowseComp and BrowseComp-zh benchmarks for comparative experiments. They state that “This sampling balances cost efficiency with representativeness”. However, the paper does not analyze the representativeness of the selected subsets. Consequently, it is difficult to determine whether the improvements achieved by the proposed TTS framework on the BrowseComp and BrowseComp-zh benchmarks are representative.\n3. The authors propose that the deep search scenario exhibits asymmetric verification, meaning verification is easier than generation. However, in the xbench-DeepSearch benchmark, verification and generation present similar levels of difficulty. The TTS method proposed in this paper does not demonstrate significant improvement over previous TTS methods. This demonstrates that the strength of asymmetric verification in specific datasets or scenarios impacts the applicability of the proposed TTS method. But the paper does not provide a detailed analysis in this aspect.\n4. The TTS framework proposed in this paper employs different parameter settings when applied to various benchmarks, but it does not provide an in-depth discussion on parameter selection.\n5. The experimental section of the paper claims that the proposed TTS framework yields a superior accuracy-cost trade-off. The figure captions in the experimental section also demonstrate this. However, the comparative results in Table 2 do not reflect this claim.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "JFKXH0tAcb", "reviewer": "Reviewer_1D6P", "summary": "The paper studies the problem of test time scaling from a perspective of assymetric verification. The overall intuition is straightforward: verification is easier then generation. e.g. verifying if an agent or llm solved a task e..g sudoku is much easier then actually solving the puzzle. For test time scaling the paper considers two approaches: sequential scaling which is simply budget forcing (simple scaling paper). For parallel scaling, the authors consider parallel sampling with a verifier. The agent generates multiple parallel responses which are aggreagated through majority voting or external verifier. Overall, the paper shows that each of these approaches have limitations (fig. 2, 3), which can be solved to some extent with test time scaling with assymetric verification.", "questions": "* The agentic variant of budget forcing from simple scaling paper is nice as showin in fig. 14. Have the authors tried other budget forcing prompts for the same?\n\nPlease also see the weaknesses section for some additional questions.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "* The paper studies an important problem of test time scaling whe verification is easier then generation.\n\n* Fig. 2 and Fig. 3 provide a nice visualization on limitation of budget forcing and parallel scaling, applied alone.\n\n* The authors  show efficiacy of their method with performance gains on BrowseComp and GAIA.", "weaknesses": "* Some of the statements in the paper are unsubstantiated. For instance, the paper says that \"in forward search must navigate an enormous, sparsely\ninformative space, while backward verification ... drastically shrinking the search space and making far more efficient use of compute.\" While it intuitively makes sense it needs supporting evidence or references showing that backward verification or per-step verification is shrinking the search space for test-time scaling or forward search.\n\n* The introduction provides some overview of the approach but its not very clear until later sections on how exactly assymetric verification is used for test-time scaling.\n\n* The idea that \"verification is easier then generation\" is not very novel and already used in prior works [1,2]. Thus, I feel its important to provide detailed discussion on the how the proposed approach differs from prior works using verification for better tts (test time scaling)\n\n* For fig. 2, the authors use number of tool calls for specifying the budget. Have the authors considered standard number of tokens for measuring scaling of performance with the used budget?\n\n* Minor question, Maj@K in Line 196 is same as standard Best@K right with majority verifier?\n\n* Some of the details in Sec. 3 are not clear. Sec. 3 does a good job of providing high level details and intuition but it should have concrete details for how exactly assymetric verification is used for test-time scaling.\n\n* Have the authors tried comparing their approach to recent test-time scaling approaches like hybrid scaling[1] which also studies the verification for test-time scaling?\n\n* Also I would be interested in how this approach differs and compares to \"Large Language Monkeys: Scaling Inference Compute with Repeated Sampling\"[2] which also combines use of verifiers with parallel sampling for test-time scaling?\n    - the main difference seems to be that here the verifier is based on majority voting while in the large language monkeys paper the verifier could be both based on majority or as an exteernal verifier?\n    - Thus, it will be important to compare the proposed approach with the \"large language monkeys\" paper in terms of both conceptual difference as well as empirical performance under different verifiers.\n\n* Some of the future work mentioned in the Conclusion section \"Looking ahead, we aim to adopt more flexible strategies. For example, verifiers could be applied at each step along the search trajectory, ... \" has already been explored in context of coding agents[3]. Thus, I feel the paper will benefit from thorgh discussion with prior works in this area.\n\n\nReferences:\n[1] R2E-Gym: Procedural Environments and Hybrid Verifiers for Scaling Open-Weights SWE Agents, COLM 2025\n\n[2] Large Language Monkeys: Scaling Inference Compute with Repeated Sampling, 2024\n\n[3] SOTA on SWE-Bench Verified with Inference-Time Scaling and Critic Model, 2025", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "f0mRShRzqB", "reviewer": "Reviewer_VZiR", "summary": "This paper investigates test-time compute scaling for deep search agents through the lens of \"asymmetric verification\" - the observation that verifying answers is often cheaper than finding them. The authors experiment with sequential scaling (budget forcing, max tool calls) and parallel scaling strategies across search and verification agents. Testing on BrowseComp, GAIA, and other benchmarks with models like GLM-4.5, K2, and Qwen3-2507, they show that allocating compute to verification yields better cost-efficiency than pure search expansion. For instance, GLM-4.5 improves from 19% to 54% on BrowseComp when extended to a \"Heavy\" variant through their scaling approach.", "questions": "1. **Can you provide a decision framework for task suitability?** \n\n    What specific features of a task predict whether verification optimization will help? Could you test a simple classifier or heuristic rules on held-out tasks?\n\n2. **What's the optimal allocation strategy?** \n\n    Given compute budget B and measured costs for search (75 calls) and verification (18 calls), what's the theoretical optimal split? Even a simple analytical model would strengthen the contribution.\n\n3. **How does this differ from standard reward modeling?** \n\n    The paper should explicitly acknowledge that the \"verifier\" is functionally equivalent to a reward model. What, if anything, makes this application special beyond the domain?\n\n4. **Why not adaptive strategies?** \n\n    Rather than fixed configurations, why not dynamically adjust the search/verification balance based on observed success rates? This could lead to more efficient resource use.\n\n5. **Generalization beyond deep search?** \n    \n    The asymmetric verification property likely applies to many tasks (math problems, code generation, fact-checking). Can you demonstrate broader applicability or is this fundamentally limited to web search?\n\n6. **Statistical significance?** \n\n    The experiments use only 100-sample subsets. Do you have confidence intervals or significance tests for the reported improvements? How stable are these results across different samples?\n\n7. **Failure mode analysis?** \n\n    When verification disagrees across multiple runs for the same answer, what's happening? Understanding these cases could provide insights into the limits of the approach.", "limitations": "", "rating": 2, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 1, "strengths": "**Empirical rigor**: The experimental setup is comprehensive, testing 4 models across 4 benchmarks with systematic ablations. The scaling curves and trade-off analyses are well-executed.\n\n**Clear presentation**: The paper tells a coherent story. Figure 1 effectively summarizes the key findings, and the concept of asymmetric verification is intuitive and well-motivated through concrete examples.", "weaknesses": "**Limited technical novelty**: The core approach is standard Best-of-K selection with a reward model (here called \"verifier\"). This is well-established in RLHF and LLM literature. The authors essentially apply existing techniques to a new domain without methodological innovation.\n\n**Lack of theoretical framework**: The paper doesn't provide any optimal resource allocation strategy between search and verification. What's the right balance given a compute budget? The paper only offers empirical observations from grid search, not principled solutions. No complexity analysis, convergence guarantees, or formal optimization framework.\n\n**Missing predictive framework**: A critical limitation is the absence of any method to predict whether a task will benefit from verification optimization. The paper only provides post-hoc measurements (Table 1) without actionable criteria. Users can't determine a priori whether to apply this approach to new tasks.\n\n**Case study rather than research contribution**: The authors acknowledge this is studying deep search \"as a representative case\" but don't provide generalizable insights beyond this specific application. The findings largely confirm expected behavior (verification is cheaper than search) without revealing new phenomena.\n\n**Incomplete analysis**: Why does xbench-DeepSearch show minimal improvement? The paper mentions verification and search have similar difficulty but doesn't deeply analyze when and why asymmetric verification breaks down.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)