surfPRM-qwen3.5-4b

A process reward model (PRM) for web agents, built by LoRA SFT on Qwen3.5-4B and merged into full weights. It evaluates candidate next actions of web agents at intermediate steps, given the task intent, the current page's accessibility tree (AXTree), and the previous action trajectory.

Training data

Trained on the surfPRM-data dataset: 9,821 pairwise step-preference samples for web agents. Each sample contains the task intent, the current page's accessibility tree (AXTree), the previous action trajectory, the start/current URLs, and two candidate next actions grounded on the current page; the model outputs structured <State>/<Criteria>/<Analysis>/<Answer> reasoning and selects the better action.

Training details

Item	Value
Base model	Qwen3.5-4B
Method	LoRA SFT (merged)
Batch size	256
Learning rate	1.0e-4
LoRA rank	128
Thinking mode	disabled
Precision	bfloat16

Prompt template

System prompt:

You are a skilled expert at evaluating assistant responses. You should evaluate given responses based on the judging criteria.
 Given the context of the conversation and two responses from the Assistant, you need to refer to determine the better response. Provide an overall comprehensive comparison upon them.

User input template:

#### Intent ####
{instruction}

#### AXTREE ####
Note: [bid] is the unique alpha-numeric identifier at the beginning of lines for each element in the AXTree. Always use bid to refer to elements in your actions.
{accessibility tree}

#### Trajectory ####
Note: The trajectory contains the sequence of previous actions performed by the agent.
{action history}

#### start url ####
{start url}

#### current url ####
The URL provides clues about the user's position in the application flow. Use both the path and query parameters to infer page type (e.g., homepage, search results, product detail, cart, checkout).
{current url}

#### Assistant Responses ####
[The Begin of Response 1]
ACTION: {action 1}
[The End of Response 1]

[The Begin of Response 2]
ACTION: {action 2}
[The End of Response 2]

### Output Instructions ###
Format your output strictly using the following XML-style tags:
<State>Summarize the current state based on the URL, AXTree, and previous actions. Include what page the user is currently on, and what relevant UI elements or information are visible.</State>
<Criteria>Other potential criteria specific to the query and the context, and the weights of each criteria.</Criteria>
<Analysis>Compare Response 1 and Response 2 in detail according to the <State> and <Criteria>.</Analysis>
<Answer>Response 1 or Response 2</Answer>

Rules for <Answer>:
- If Response 1 is better, output exactly: <Answer>Response 1</Answer>
- If Response 2 is better, output exactly: <Answer>Response 2</Answer>

Important Notes:
- Be objective and base your evaluation strictly on the content of the responses.
- Do not let the response order, length bias your judgment.

Usage

Requires transformers >= 5.6. Load as a standard Qwen3.5 (multimodal) checkpoint with AutoProcessor / AutoModelForImageTextToText, format inputs with the prompt template above, and parse the preferred action from the <Answer> tag of the output.