MissionCtrl Overseer (Qwen2.5-0.5B LoRA)

  • Developed by: Jit-fnc
  • Model type: LoRA adapter (PEFT) for Qwen2.5-0.5B-Instruct (4-bit Unsloth base)
  • License: Apache-2.0
  • Finetuned from: unsloth/qwen2.5-0.5b-instruct-unsloth-bnb-4bit
  • Training framework: Unsloth + TRL (GRPO)

This model is a task-specific overseer policy trained for the MissionCtrl environment: a multi-agent software simulation where the overseer must detect and manage hallucination risk (approve/reject/flag/redelegate/escalate/synthesize actions).


Model description

This adapter is optimized for structured oversight actions in MissionCtrl episodes, not generic chat quality.
Training emphasizes:

  • Hallucination detection under curriculum difficulty (easy -> medium -> hard)
  • Low false-positive behavior
  • Action validity and episode-level reward optimization

Training details

Base + method

  • Base: unsloth/qwen2.5-0.5b-instruct-unsloth-bnb-4bit
  • Finetuning: LoRA/QLoRA via Unsloth
  • RL algorithm: GRPO (TRL)

Run snapshot (from training logs)

  • Hardware: 1x T4 (Colab run snapshot)
  • Curriculum steps: easy 200, medium 220, hard 180
  • Reward ceiling reference: 0.85

Post-phase eval (10 episodes each)

  • Easy: reward 0.289 ± 0.137, detect 70.0%, fp 0.0%
  • Medium: reward 0.067 ± 0.107, detect 10.0%, fp 0.0%
  • Hard: reward 0.172 ± 0.024, detect 0.0%, fp 0.0%

Rounded curriculum means often shown in docs: 0.29 / 0.07 / 0.17.


Intended use

Recommended

  • MissionCtrl/OpenEnv-style oversight simulation
  • Research on hallucination containment and supervisory policies
  • Benchmarking action policies in multi-agent pipelines

Not recommended

  • General-purpose assistant/chatbot usage
  • Factual QA without environment constraints
  • Safety-critical deployment without independent validation

Prompt / output format expectations

Best results are obtained when prompts follow MissionCtrl-style action grammar and environment context.
The policy is tuned to produce short structured actions such as:

  • APPROVE(task_id)
  • REJECT(task_id, reason)
  • FLAG(task_id, evidence)
  • REDELEGATE(task_id, agent)
  • ESCALATE(task_id)
  • SYNTHESIZE_REPORT()
  • NOOP

Limitations

  • Performance drops on higher difficulty tiers (medium/hard) in current snapshot.
  • Curriculum reward and full greedy rollout eval can diverge during early RL stages.
  • Domain transfer outside MissionCtrl may be poor.
  • Small model capacity (0.5B) limits robustness to long-horizon reasoning.

Bias, risks, and safety

This model is specialized for synthetic oversight tasks and may:

  • Overfit to simulator-specific patterns
  • Underperform on unseen hallucination styles
  • Produce confident but incorrect actions outside its trained setting

Use with human monitoring and external evaluation before broader use.


Usage (PEFT)

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_id = "unsloth/qwen2.5-0.5b-instruct-unsloth-bnb-4bit"
adapter_id = "Jit-fnc/missionctrl_env"  # replace if repo name differs

tokenizer = AutoTokenizer.from_pretrained(base_id)
base_model = AutoModelForCausalLM.from_pretrained(base_id)
model = PeftModel.from_pretrained(base_model, adapter_id)

prompt = "You are the MissionCtrl Overseer. Return one valid action."
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=64)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Hugging Face Inference Endpoints (task=custom)

This repo includes handler.py and requirements.txt so a dedicated endpoint configured with custom task can load the LoRA adapter.

  • Base weights at runtime: Qwen/Qwen2.5-0.5B-Instruct (non-quantized). Override with env BASE_MODEL if needed.
  • Adapter + tokenizer: loaded from the endpoint’s /repository (this model repo).

Deploy: commit and push handler.py and requirements.txt to the Hugging Face model repo backing the endpoint, then restart or redeploy the endpoint.

Clients (e.g. MissionCtrl inference.py):

  • Set MISSIONCTRL_HF_LLM_STRATEGY=native_only so requests use the root URL with {"inputs": "...", "parameters": {...}} instead of relying on /v1/chat/completions (custom handlers do not expose OpenAI routes by default).
  • The handler returns [{"generated_text": "..."}], matching native HF generation parsing.
Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support