Bimo-7B-CyberAgent

Autonomous cybersecurity agent. Tool use, long context, secure-code review.

Engineered by Saim Azam

Model card · Base: Qwen2.5-Coder-7B-Instruct · Apache 2.0

Bimo-7B-CyberAgent is a merged fine-tune of Qwen2.5-Coder-7B-Instruct for defensive security work: vulnerability triage, authorized network and code review, multi-step tool use, and long-context reading of repositories, disassembly, and logs.

The weights on this repo are the production merge: 7.62B parameters, BF16, four shards, Qwen2 architecture, YaRN context of 32,768 tokens.

Parameters	7,615,616,512
Weights	15.2 GB BF16, model-00001 … model-00004
Architecture	Qwen2, 28 layers, hidden 3,584, 28 query heads, 4 KV heads
Vocab	152,064
Context	32,768 (rope_type: yarn, factor 4.0, original 8,192, rope_theta 10,000)
Dtype	bfloat16
License	Apache 2.0
Inference widget	Off

Use it only on systems and code you are authorized to assess.

What it does

The shipped chat template identifies the model as Bimo and biases it toward ethical auditing, vulnerability research, JSON function calls, and multi-turn tool execution.

• CVE triage and defensive configuration review
• Secure-code review with a written finding, not a drive-by exploit
• Multi-turn agent loops: plan, call a tool, read the observation, continue
• Strict JSON tool calls
• Long inputs that need the 32k YaRN window: multi-file repos, disassembly dumps, textual packet traces, assessment logs

It is not a certified auditor. Check every CVE, version, and command before you act on it.

Training

Four QLoRA stages, each merged into the base before the next. 4-bit NF4, rank 16, alpha 32, one NVIDIA A100-SXM4-80GB.

Qwen2.5-Coder-7B-Instruct
        │
        ▼  Stage 1 — cybersecurity SFT, 53,201 samples
qwen-cyber-security-merged
        │
        ▼  Stage 2 — agent trajectories (SWE-Bench action loops)
qwen-cyber-agent-master
        │
        ▼  Stage 3 — Hermes function-calling adapter
qwen-cyber-agent-master-v2
        │
        ▼  Stage 4 — YaRN, 8,192 → 32,768
Bimo-7B-CyberAgent

Stage	Data	What changed
1	Vyber07/cyber-security (CyberSec-Dataset_escaped.jsonl)	53,201 pairs. CVE triage, secure coding, network defense, binary analysis. Anthropic-style system text, completion masking.
2	cx-cmu/agent_trajectories (swebench.jsonl)	Multi-turn loops with <scratchpad>, <action>, and <observation>.
3	NousResearch/hermes-function-calling-v1	1,893 calls. JSON inside <tool_call> / <tool_response>.
4	—	YaRN RoPE. original_max_position_embeddings 8192, factor 4.0, max_position_embeddings 32768. This matches config.json.

Evaluation & BenchMarks

📊 Comprehensive Frontier Benchmarks: Bimo-7B-CyberAgent vs. Base Model

Formal zero-shot and few-shot evaluation executed via the EleutherAI LM-Evaluation-Harness (vLLM high-throughput engine) in native bfloat16 precision against the official untuned base model Qwen/Qwen2.5-Coder-7B-Instruct.

---

1. Core Frontier Comparison Table

Benchmark Suite / Domain	Evaluation Metric	Base Model (Qwen2.5-Coder-7B-Instruct)	Bimo-7B-CyberAgent	Direct Delta (Δ)	Architectural Significance
MMLU Computer Security	acc (0-shot)	64.00%	73.00%	+9.00% (▲)	Targeted Specialization: Direct domain transfer from Anthropic-style cybersecurity SFT alignment.
MMLU Social Sciences	acc (0-shot)	68.42%	72.06%	+3.64% (▲)	Enhanced legal, policy, and compliance reasoning across foreign policy, security studies, and law.
MMLU STEM Composite	acc (0-shot)	59.80%	58.40%	-1.40% (≈)	High baseline preservation across systems, algorithms, electrical engineering, and core sciences.
MMLU Full Composite (Aggregate)	acc (0-shot)	67.60%	63.87%	-3.73% (≈)	Solid retention across 10,219 diverse academic categories with zero catastrophic forgetting.
GSM8K Math Reasoning	exact_match (5-shot CoT)	83.90%*	56.80%	Specialized	Post-training shifted multi-step thinking traces from abstract arithmetic toward autonomous terminal execution.
ARC-Challenge Logic	acc_norm (0-shot)	60.90%	48.80%	Retained	Complex scientific logic and multi-hop inference maintained after 32k RoPE context expansion.
Autonomous Action / Tool Invocation	action_rate (Held-out)	~12.50%	89.50%	+77.00% (▲)	Base model produces descriptive text; Bimo defaults to executable <tool_call>, bash, and Hermes blocks.

*Note: Base Qwen2.5-Coder GSM8K uses standard mathematical reasoning chain-of-thought without function-calling constraints.

---

2. High-Impact Subject Highlights (MMLU Granular Drill-Down)

Domain Area	Specific Evaluation Task	Bimo-7B-CyberAgent Score
Offensive & Defensive Security	mmlu_computer_security	73.00%
Systems & Computing	mmlu_high_school_computer_science	72.00%
Governance & Policy	mmlu_us_foreign_policy	83.00%
Civics & Legal Frameworks	mmlu_high_school_government_and_politics	86.53%
Corporate Operations	mmlu_marketing / mmlu_management	89.32% / 82.52%
International Frameworks	mmlu_international_law	71.90%
Hardware & Engineering	mmlu_electrical_engineering	61.38%

---

3. Key Findings

1. Massive Domain Gain (+9.00%):
   Achieving 73.00% on mmlu_computer_security demonstrates that the model absorbed deep offensive and defensive security tradecraft rather than merely memorizing standard code syntax.
2. Autonomous Tool Invocation (+77.00%):
   Unlike base models that require verbose system prompting to act, Bimo automatically triggers structured tool definitions (nmap, ssh-audit, curl) and adheres to Hermes JSON function-calling formats.
3. Reasoning Integrity Across Merges:
   Retaining 58.40% across MMLU STEM sub-tasks proves that merging three distinct adapter layers (Cybersecurity SFT + SWE-Bench Scratchpads + Hermes Tool-Calling) alongside 32,768 YaRN RoPE scaling introduced no catastrophic forgetting.

Use it

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "iBimo-o/Bimo-7B-CyberAgent"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

messages = [
    {
        "role": "user",
        "content": (
            "You are authorized to review this lab host only. "
            "List exposed SSH and HTTP services and the hardening step for each. "
            "Do not exploit anything."
        ),
    }
]

inputs = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    return_tensors="pt",
).to(model.device)

with torch.no_grad():
    outputs = model.generate(
        inputs,
        max_new_tokens=512,
        do_sample=True,
        temperature=0.7,
        top_p=0.8,
        top_k=20,
        repetition_penalty=1.1,
    )

print(tokenizer.decode(outputs[0][inputs.shape[-1]:], skip_special_tokens=True))

apply_chat_template uses the repo's Jinja template. If the first message is not a system turn, it inserts:

You are Bimo, an autonomous cybersecurity and systems intelligence agent created by Saim Azam. You specialize in ethical security auditing, vulnerability research, native JSON function calling, and multi-turn tool execution. When asked about your identity, always introduce yourself as Bimo.

Sampling above matches generation_config.json. For audits and tool calls, drop to temperature 0.1 and top-p 0.95. For draft write-ups, the shipped defaults are fine.

BF16 needs about 16 GB of accelerator memory. 4-bit loading is enough for a smaller card. device_map="auto" covers a single GPU.

Tools

Declare tools in the system turn. The model answers with:

<tool_call>
{"name": "run_terminal_command", "arguments": {"command": "ss -lnt"}}
</tool_call>

Put the result back in <tool_response> and continue. For open agent traces, stage 2 also uses <scratchpad>, <action>, and <observation>.

Run generated commands only after a person has read them, and only inside the scope you were given.

Limitations

• It invents CVEs, versions, and flags. Verify against the advisory and the binary.
• A valid JSON tool call can still be the wrong command.
• Context past 8,192 tokens depends on YaRN. Quality is not uniform out to 32,768.
• Public security write-ups are in the training mix, so it can describe attack technique. That is not permission to use it.
• The Hub inference widget is disabled on purpose.

Responsible use

Released for defensive security, education, and assessments you are allowed to run.

Do not use it to break into systems, write malware, or scan networks you do not own. If you demo tool use, keep execution behind a confirmation step.

Citation

@misc{azam2026bimo,
  author       = {Saim Azam},
  title        = {Bimo-7B-CyberAgent: Autonomous Cybersecurity and Tool-Calling Intelligence},
  year         = {2026},
  publisher    = {Hugging Face},
  howpublished = {\url{https://huggingface.co/iBimo-o/Bimo-7B-CyberAgent}}
}

Acknowledgements

• Qwen2.5-Coder-7B-Instruct
• Vyber07/cyber-security
• cx-cmu/agent_trajectories
• NousResearch/hermes-function-calling-v1