ToolEnv-14B

ToolEnv-14B is a full-parameter fine-tuned Qwen3-14B model for API response simulation. Given API documentation and input parameters, it generates a JSON response with error and response fields.

It is part of ParaAgent: Reinforcing Parallel Acting in Open-World Tool Environments.

Paper · Code

This repository contains the complete BF16 model weights. Use non-thinking mode (enable_thinking=False). Responses are simulated; the model does not execute APIs.

Usage

pip install "transformers>=4.57.6,<5" "accelerate>=0.34" torch

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "ShengbinYue/ToolEnv-14B"  # Or the local model directory.
system_prompt = """You are an API simulator acting as a backend server. Your task is to handle API requests and return realistic, logically consistent responses that strictly follow the API documentation and provided input parameters.

### RESPONSE RULES

1. **Output Format**
   - Only return valid, well-formed JSON (no markdown, no explanations, no comments, no extra text).
   - Response schema:
     {
       "error": "none" | { "type": "<error type>", "msg": "<error message>" },
       "response": <object | string | number | array | "none">
     }

2. **Error Handling**
   - Use "error": "none" for successful executions.
   - For failures, return:
     {
       "error": { "type": "<error type>", "msg": "<brief message>" },
       "response": "none"
     }
   - Possible error types include (but are not limited to): InvalidRequestError, NetworkError, NotFoundError, PermissionError, ToolExecutionError.

3. **Data Generation**
   - Generate realistic, type-correct, domain-appropriate data fully aligned with the API documentation and parameters (e.g., timestamps, valid URLs, unique numeric IDs, correct currency codes, usable email formats).

4. **Logical Consistency**
   - Maintain meaningful and coherent relationships between fields. Avoid contradictions or obviously artificial data.

5. **Quality Requirements**
   - Do not use placeholders, meaningless filler data, or repetitive patterns.
   - Ensure outputs appear production-grade and believable.

6. **Final Output Restriction**
   - Return only the JSON object — no additional formatting, commentary, explanation, or wrapping."""
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    dtype=torch.bfloat16,
    device_map="auto",
)

messages = [
    {"role": "system", "content": system_prompt},
    {
        "role": "user",
        "content": (
            '## API Documentation:\nAPI: add\n'
            'Compute the sum of the input numbers. The response is {"sum": integer}.\n\n'
            '## Input Parameters:\n{"numbers": [2, 3]}'
        ),
    },
]
prompt = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
    enable_thinking=False,
)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
with torch.inference_mode():
    outputs = model.generate(
        **inputs, max_new_tokens=512, do_sample=False, use_cache=True
    )
print(tokenizer.decode(outputs[0, inputs.input_ids.shape[1]:], skip_special_tokens=True))

The full system prompt above matches the standard training and evaluation prompt and is also included in system_prompt.txt. Successful responses use "error": "none". Allow enough output tokens for the requested response to avoid truncated JSON. BF16 weights alone require approximately 28 GiB; inference needs additional memory.

Deploy with vLLM

The examples below assume the model directory is ./ToolEnv-14B. Use a separate environment for vLLM.

pip install "vllm==0.17.1" openai

vllm serve ./ToolEnv-14B \
  --api-key EMPTY \
  --port 12345 \
  --served-model-name Toolenv-14B

Check the service:

curl http://127.0.0.1:12345/v1/models \
  -H "Authorization: Bearer EMPTY"

Send a request with the full system prompt and non-thinking mode:

from pathlib import Path
from openai import OpenAI

system_prompt = Path("./ToolEnv-14B/system_prompt.txt").read_text(encoding="utf-8")
client = OpenAI(base_url="http://127.0.0.1:12345/v1", api_key="EMPTY")
result = client.chat.completions.create(
    model="Toolenv-14B",
    messages=[
        {"role": "system", "content": system_prompt},
        {
            "role": "user",
            "content": (
                '## API Documentation:\nAPI: add\n'
                'Compute the sum of the input numbers. The response is {"sum": integer}.\n\n'
                '## Input Parameters:\n{"numbers": [2, 3]}'
            ),
        },
    ],
    temperature=0,
    top_p=1,
    max_tokens=4096,
    extra_body={"chat_template_kwargs": {"enable_thinking": False}},
)
print(result.choices[0].message.content)

The API model name is Toolenv-14B, matching --served-model-name. See the vLLM serving documentation for additional deployment options.

License

Apache-2.0. See LICENSE and NOTICE.

Citation

@article{yue2026paraagent,
  title={ParaAgent: Reinforcing Parallel Acting in Open-World Tool Environments},
  author={Yue, Shengbin and Wang, Hongru and Wang, Siyuan and Chen, Xiaoxin and Chen, Wei and Wei, Zhongyu},
  journal={arXiv preprint arXiv:2609.33618},
  year={2026}
}