PTCS-RL-1.5B

PTCS-RL: Progressive Tool Call Scheduling Reinforcement Learning for Mathematical Reasoning

PTCS-RL is a 1.5B-parameter mathematical reasoning model based on Qwen2.5-Math-1.5B and trained with tool-augmented reinforcement learning.

The key idea is to progressively relax the tool-call budget during training, allowing the model to first learn stable tool use and then gradually develop multi-step tool interaction, verification, and self-correction capabilities.

Method

PTCS-RL integrates natural-language reasoning with external Python execution.

During training, the maximum number of tool calls is scheduled as:

Stage	Training Steps	Tool Budget
Early	0–400	1
Middle	401–500	2
Late	501–600	4

The model is optimized using GSPO, with external execution results treated as environmental feedback for subsequent reasoning.

This progressive strategy reduces interaction noise in the early stage while enabling richer multi-step tool use later in training.

Results

We evaluate PTCS-RL on four mathematical reasoning benchmarks: GSM8K, AMC23, AIME24, and AIME25. All models are evaluated using greedy decoding with temperature = 0.

Model	Code	GSM8K	AMC23	AIME24	AIME25	Avg. acc
Base & Instruction-Tuned Models
Qwen2.5-Math-1.5B	✗	74.6	28.0	6.7	4.6	28.48
Qwen2.5-Math-1.5B-Instruct	✗	84.8	62.5	10.0	10.0	41.75
Qwen2.5-Math-1.5B-Instruct	✓	83.7	55.0	13.3	13.3	41.33
RL Reasoning Models
SimpleRL-Zoo-7B	✗	91.7	62.5	20.0	16.7	47.73
Eurus-2-7B-PRIME	✗	92.0	50.6	20.0	6.7	42.32
Tool-Augmented RL Models
ToRL-1.5B	✓	85.3	67.5	26.7	26.7	51.55
PTCS-RL-1.5B (Ours)	✓	88.6	70.0	36.7	30.0	56.33

PTCS-RL achieves the best overall average accuracy of 56.33% among the compared methods.

Compared with the strong tool-augmented RL baseline ToRL-1.5B, PTCS-RL improves performance on all four benchmarks:

• GSM8K: 85.3 → 88.6
• AMC23: 67.5 → 70.0
• AIME24: 26.7 → 36.7
• AIME25: 26.7 → 30.0
• Average accuracy: 51.55 → 56.33

The improvement is especially pronounced on the more challenging AIME benchmarks, with a 10.0-point gain on AIME24.

Tool-Augmented Reasoning

PTCS-RL can use execution feedback to support both computation and self-correction.

A typical reasoning process is:

Problem
  ↓
Reasoning
  ↓
Generate Python code
  ↓
Execute code
  ↓
Receive execution feedback
  ↓
Continue / revise reasoning
  ↓
Final answer

The model itself does not directly execute Python code. To reproduce the full tool-augmented reasoning behavior, an external execution environment is required.

Our experiments use SandboxFusion for code execution.

Usage

from transformers import AutoTokenizer, AutoModelForCausalLM

model_name = "yinheyiebi/PTCS-RL"

tokenizer = AutoTokenizer.from_pretrained(model_name)

model = AutoModelForCausalLM.from_pretrained(
    model_name,
    device_map="auto",
    torch_dtype="auto"
)

For full PTCS-RL inference, the generation process should be connected to an external Python sandbox that executes generated tool calls and returns the results to the model context.

Training Setup

• Base model: Qwen2.5-Math-1.5B
• RL algorithm: GSPO
• RL framework: verl
• Execution environment: SandboxFusion
• Batch size: 128
• Generations per problem: 16
• Training temperature: 1.0
• Evaluation temperature: 0

Citation

If you find this model useful, please cite:

@misc{liang2026ptcsrl,
  title  = {PTCS-RL: Progressive Tool Call Scheduling Reinforcement Learning for Mathematical Reasoning},
  author = {Liang, Jukun and Zhou, Mingtao and Gan, Jianhou and Zhou, Juxiang and Shi, Huibang},
  year   = {2026}
}

Citation information will be updated after the paper is formally published.

Authors

Jukun Liang, Mingtao Zhou, Jianhou Gan, Juxiang Zhou, and Huibang Shi

Yunnan Normal University