Qwen3-8B-Seeker

Qwen3-8B-Seeker is Qwen3-8B fine-tuned to act as a Wikipedia multi-hop question-answering agent. Given a factual question, it plans sub-questions, issues Wikipedia search queries, reflects on the retrieved text, and produces a final answer.

Important notice

This is a task-specific model, not a general-purpose assistant or agent. It was trained for one workflow with one tool (Wikipedia search). It has not been trained to use other tools, and using it as a general agentic model may produce confident but incorrect or misleading output. It has not been formally evaluated. Verify its answers independently.

Intended use

• Multi-hop factual question answering over Wikipedia-style passages, in an agent loop with an external retriever.
• Research and experimentation with tag-based agent protocols.

Out of scope: general chat, coding, math, medical or legal advice, or any tool other than Wikipedia search.

Agent protocol

The model follows a tag-based protocol inherited from the training data:

Tag	Role
<think>	Reasoning before each step
<plan>	Sub-questions and steps
<wiki_search>	Wikipedia search query (the only tool)
<observation>	Retrieved text, supplied by your code, not generated by the model
<reflection>	Self-check after results arrive
<answer>	Final answer

This is not native Qwen tool calling. The model does not search by itself. You must run the loop: generate until </wiki_search>, run your own retriever, append the results inside <observation> ... </observation>, and continue until </answer>. Use the system prompt from the dataset card and set enable_thinking=False in the chat template.

Usage

Illustrative agent loop (you supply search_fn, which takes a query string and returns passage text):

from transformers import AutoTokenizer, AutoModelForCausalLM

repo = "Rumiii/Qwen3-8B-Seeker"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo, dtype="auto", device_map="auto")

def run_agent(question, system_prompt, search_fn, max_turns=6):
    msgs = [{"role": "system", "content": system_prompt},
            {"role": "user", "content": f"Question: {question}"}]
    prompt = tok.apply_chat_template(msgs, tokenize=False,
                                     add_generation_prompt=True, enable_thinking=False)
    text = ""
    for _ in range(max_turns):
        inputs = tok(prompt + text, return_tensors="pt").to(model.device)
        out = model.generate(**inputs, max_new_tokens=512, do_sample=False,
                             stop_strings=["</wiki_search>", "</answer>"], tokenizer=tok)
        new = tok.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)
        text += new
        if "</answer>" in new:
            break
        if "</wiki_search>" in new:
            query = new.split("<wiki_search>")[-1].split("</wiki_search>")[0].strip()
            text += f"\n\n<observation> {search_fn(query)} </observation>\n\n"
    return text

Plain chat also works for simple prompts (for example, asking who the model is), using enable_thinking=False.

Training

Item	Detail
Base model	Qwen/Qwen3-8B
Method	QLoRA (4-bit base, LoRA r=32, alpha=32, all attention and MLP projections), merged into fp16 weights
Data	3,000 randomly sampled trajectories from PersonalAILab/AFM-MHQA-Agent-SFT-Dataset, plus 72 identity samples (3,072 total)
Length handling	Samples over 4,096 tokens were excluded, none were truncated; incomplete trajectories (no <answer>) were removed
Loss masking	Only the assistant turn is trained; retrieved <observation> text is masked so the model does not learn to generate search results
Chat format	Non-thinking mode (enable_thinking=False); the model's reasoning is expressed through the protocol's <think> tags
Schedule	1 epoch, 192 optimizer steps, effective batch size 16, learning rate 1e-4 (cosine), 8-bit AdamW, fp16
Hardware	One NVIDIA T4, about 5.4 hours
Final training loss	About 0.71 to 0.74

The identity samples teach the model to introduce itself as Qwen3-8B-Seeker, a model fine-tuned from Qwen3-8B.

Limitations

• No benchmark evaluation has been run; only qualitative checks were performed. Answer accuracy is unknown.
• Trained on a subset (3,000 trajectories) of one dataset, with the longest trajectories (over 4,096 tokens) excluded. Performance on questions needing many search steps may be weaker.
• Wikipedia is the only supported tool. Behavior with other tools or retrievers is untested.
• Self-introduction works in plain chat but is less reliable when the agent system prompt is active.
• It inherits the limitations and biases of Qwen3-8B and of the training data.

Acknowledgements

Base model: Qwen3-8B by the Qwen team. Training data: AFM-MHQA-Agent-SFT-Dataset by OPPO Personal AI Lab (Chain-of-Agents, arXiv:2508.13167). Fine-tuned with Unsloth.