BrainboxAI/cyber-analyst-4B-verifier-pilot

Cyber-Analyst 4B: Verifier Pilot

A LoRA fine-tune of cyber-analyst-4B that adds a strict finding-verifier behavior. Given the evidence for a suspected web vulnerability, it decides whether the evidence actually proves the bug, and rejects the ones that only look real.

This is a pilot. It was trained on a small (~100 example) balanced set to check the recipe before scaling. It lives in its own repo and does not replace the production cyber-analyst-4B.

What it does

You give it the evidence for a suspected vulnerability (tool, URL, payload, response snippet, PoC). It returns a JSON verdict:

{"verdict": "true_positive | false_positive | inconclusive", "reason": "..."}

It judges the evidence, not the scary-looking payload. That is the difference between a verifier and a scanner. It is trained to reject the look-alikes that fool most tools:

Looks like a finding	Why it is a false positive
SQLi, got a 500	a generic 500 with no DB error and no extracted data proves nothing
blind boolean SQLi	the true and false conditions returned identical responses
reflected XSS	the payload came back HTML-encoded or JS-escaped, so it cannot run
SSRF to a metadata IP	ECONNREFUSED. the request was blocked, no internal data returned
IDOR by enumerating IDs	every non-owned ID returned 403; authorization is enforced
path traversal with a null byte	404, no /etc/passwd contents; the bypass read no sensitive file
JWT with alg=none	the server rejected it with 401

When something is genuinely suspicious but not confirmed, it answers inconclusive and says what evidence is still missing (a differential test, a decoded token, an extracted row). It also knows who it is (Cyber-Analyst 4B, by BrainboxAI), in Hebrew and English.

Lineage

Google Gemma-4 E4B
  -> cyber-analyst-4B          (1.27M security examples, BrainboxAI)
       -> this pilot           (verifier + identity LoRA delta)

Training

Base	BrainboxAI/cyber-analyst-4B (weights kept; this only adds a delta)
Method	bf16 LoRA (Unsloth), r=16 / alpha=32, LR 1e-4, 3 epochs, cosine, seq 2048
Data	~100 balanced verifier examples (true / false / inconclusive, with adversarial hard negatives across 19 vuln classes) plus a small share of bilingual identity
Eval	held-out split, eval_loss around 1.13
Hardware	one RTX 4090, ~16GB VRAM

Benchmark: does the delta actually reject false positives?

The whole point of the verifier delta is to stop the model from confirming vulnerabilities that only look real. So the benchmark measures exactly that.

17 held-out examples (never seen in training): 8 true positives, 6 false positives, 3 inconclusive. Each is a suspected finding with its evidence. The model returns a verdict. Thinking mode is disabled with an assistant prefill so the model answers directly. Same test, same prompt, run against the base and the pilot.

The base model confirms everything. It says true_positive to all 8 real bugs, but also to all 6 look-alikes: a 500 with no DB error, an SSRF that got ECONNREFUSED, an IDOR where every non-owned ID returned 403. It cannot tell a finding from a decoy. That is the scanner bias a verifier is supposed to remove.

After 100 verifier examples the pilot rejects 4 of those 6 false positives, and overall accuracy goes from 47% to 65%. The cost was one over-correction: it flagged a genuine Twig SSTI as a false positive. A fair trade for a delta this small, and the direction is the one we wanted.

What it still gets wrong: neither model abstains. Both answer 0 of 3 inconclusive cases, because the pilot inherited the base model's habit of always committing to a verdict, and 100 examples was not enough to install "say inconclusive when the confirming evidence is missing." That is the first thing the next round fixes, with more inconclusive training data.

Read this as what it is: a pilot proving the recipe works. A 100-example delta moved false-positive rejection from 0/6 to 4/6 without breaking true-positive detection. Scale the data and the abstention gap should close too.

Files

File	What	Use
cyber-analyst-4B-verifier-pilot.Q4_K_M.gguf	quantized model, ~5.3 GB	LM Studio / Ollama / llama.cpp
cyber-analyst-4B-verifier-pilot.mmproj-BF16.gguf	vision projector	only for image input (Gemma-4 is multimodal)

The LoRA adapter lives in a separate repo, cyber-analyst-4B-verifier-pilot-lora, so this one stays a clean GGUF repo.

Note: Gemma-4 is a new architecture. Loading the GGUF needs a recent llama.cpp, so update LM Studio or Ollama to the latest. In LM Studio, make sure the format selector is set to GGUF.

Quick start (LM Studio)

Download gguf/cyber-analyst-4B-safetensors.Q4_K_M.gguf and load it. For the verifier task, use a system prompt like:

You are a strict security-finding verifier. Given the evidence for a suspected vulnerability, decide true_positive (the evidence proves exploitation), false_positive (disproven or not exploitable), or inconclusive (suspicious but the confirming evidence is missing; say what is missing). Judge from the evidence, not the title. Respond with only JSON: {"verdict":"…","reason":"…"}.

Continue training from here

Load the base cyber-analyst-4B and attach the adapter from cyber-analyst-4B-verifier-pilot-lora to keep training. The cleaner option is to train from the original cyber-analyst-4B with the full accumulated dataset, which avoids stacking one delta on top of another. A merged 16-bit model can be regenerated from the base plus the adapter in one step when you need it.

Limitations

• Pilot scale, about 100 examples. It checks the format and basic discrimination. It is not a production verifier yet. Scale the dataset (600+) before relying on it.
• It inherits the base model's limits (public-data bias, temporal cutoff).
• Not for autonomous security decisions without human review.

Citation

@misc{elyasi2026cyberverifierpilot,
  title  = {Cyber-Analyst 4B: Verifier Pilot},
  author = {Elyasi, Netanel},
  year   = {2026},
  publisher = {BrainboxAI},
  howpublished = {\url{https://huggingface.co/BrainboxAI/cyber-analyst-4B-verifier-pilot}}
}

Author

Built by Netanel Elyasi, founder of BrainboxAI. Contact: netanele@brainboxai.io