HAWQ-SEC-RE-v1 (updated to v1.1)

Reverse-engineering + crypto/exploit-audit analysis model. A LoRA SFT fine-tune of lancejames221b/HAWQ-v1, trained to produce faithful, structured analysis of x86-64 assembly and decompiled C (purpose, inputs/outputs, algorithm, control flow, security-relevant behavior) and, as of v1.1, cryptographic-primitive identification, misuse enumeration, and exploit-path reasoning over arbitrary source code. Merged into the base weights (no adapter required at inference time) and shipped as GGUF (llama.cpp / LM Studio) and MLX (Apple Silicon) quantizations. An Ollama registry listing is planned but not live yet - see "Ollama" under Usage.

This is the HAWQ-SEC-RE line's public repo ("v1" — internal training iteration v3 for the original RE release; iteration v4 for the v1.1 crypto/exploit-audit update below). Same repo, same links, updated in place; a v1.0 tag on this repo preserves the exact prior commit.

Known limitations (read before using for audit triage)

v1.1 over-flags clean code as vulnerable. On a held-out probe of correct, textbook AES-256-GCM (no injected flaw), this model claims a security flaw in 9/9 sampled runs (probe_clean_control, eval_crypto_audit.py, k=9, temperature=0). This is not an edge case — it reproduces on straightforward clean-code review and should be expected as the default behavior whenever this model is pointed at code that has no actual vulnerability.

Cause: the v1.1 crypto/exploit-audit training data (build_sec_audit.py) generated each example as a chosen/rejected pair for a planned DPO pass, but the SFT stage that actually shipped in this release trained on the rejected (i.e. vulnerable) variant of every sample unconditionally - the model never saw a "this code is actually fine" example during SFT. It pattern-matches "review this code" to "find something wrong," independent of whether anything is actually wrong.

Mitigation for now: treat any "vulnerability found" or "critical issue" output from this model as a candidate requiring independent confirmation, not a verdict - especially on code you already believe is correct. Do not use v1.1 as an unattended pass/fail gate in a CI security-audit pipeline.

Fix status: a DPO pass (chosen=clean-verdict / rejected=the over-flagging behavior above, mined from real correction data plus a dedicated clean-code counterweight set) is designed and the training pipeline is written and verified (loss math checked bit-for-bit against a reference implementation; corpus contamination-gated at <0.4% overlap with eval probes), but the training run itself has not completed as of this release - deferred, tracked as follow-up work. Every other capability this model has (crypto-primitive ID, misuse enumeration, exploit-path reasoning on genuinely vulnerable code, RE/tool-calling/long-CoT behavior) is unaffected and gate-verified below.

Changelog

v1.1 (this update): added crypto/exploit-audit SFT domain (primitive identification from constants alone, misuse-pattern enumeration, and stack-overflow exploit-path reasoning) on top of the v1.0 RE-analysis model. Same base, same merge methodology. Ships with the clean-code over-flagging regression documented above; a DPO-based fix is designed but not yet trained into this release.

v1.0: original public release - RE-analysis SFT only (x86-64 disassembly / decompiled-C structured analysis). Preserved at the v1.0 git tag on this repo.

Model family

• Base: lancejames221b/HAWQ-v1 — a Holo3 + Qwopus + AgentWorld merge on Qwen3.6-35B-A3B: hybrid linear-attention/SSM MoE, 40 text layers (10 full self-attention, 30 linear-attention/SSM), 256 experts, moe_intermediate_size=512, hidden_size=2048, num_attention_heads=16, num_key_value_heads=2, head_dim=256, vocab_size=248320. Text-only Qwen3_5MoeForCausalLM (693 tensors) — no vision tower.
• Fine-tune: HAWQ-SEC-RE — LoRA SFT (r=64, alpha=128) over attention (q_proj/k_proj/v_proj/o_proj) and linear-attention/SSM projection modules (in_proj_qkv/in_proj_a/in_proj_b/in_proj_z/out_proj), merged into the base with PeftModel.merge_and_unload().

Training

• Method: LoRA SFT via transformers + peft, response-only prompt-prefix masking (no TRL, to avoid a transformers-v5 compatibility risk on this architecture).
• Data: hawq-re-v3 — RE-analysis + decompile families sourced from LLM4Binary/decompile-bench, with frontier-model-generated gold analyses as the SFT targets.
• Compute: 4x NVIDIA A100 (GCE a2-highgpu-4g), FSDP, 2 epochs, MAXLEN=4096.
• Hyperparameters: r=64, lora_alpha=128, target modules as above.

v1.1 addition: crypto/exploit-audit SFT domain (build_sec_audit.py) layered on top of the same base via a further LoRA SFT pass (targets and methodology unchanged from above), merged the same way. See "Known limitations" above for the one confirmed regression this introduced.

Validation

Gate 1 — coherence probes (eval_peft_direct.py, run on a Colab G4 session against the merged checkpoint vs. the unmodified HAWQ-v1 base):

• probe_tool_loop — native Qwen tool-call convention (no CALL text, no vacuous calls that never fire): PASS
• probe_error_recovery — matches the canonical eval_loop_recovery.py gate methodology, requires finish reason stop (no forced-loop artifacts): PASS
• probe_long_cot — respects the card's 81,920-token hard-math cap, classifies length-cap hits as PASS/FAIL/TRUNCATE by deconfounding sentence-repetition tokens from the raw base rate: PASS
• Adapter-binding assertion (added to the eval harness specifically for this run) confirmed the LoRA delta was actually applied — not a no-op merge silently shipping unmodified base weights under a new name.

Gate 2 — win-rate judge (frontier-judged base-vs-tuned comparison on held-out RE-analysis tasks, eval_re_analysis.py, gate threshold 0.55) was not completed for this release — base-side and tuned-side generation passes were run partway (dozens of tasks each) before the formal run was deprioritized in favor of shipping the already-working deployment. Treat this release as coherence-validated, not yet win-rate-validated against the base model. Re-running Gate 2 to completion is tracked as follow-up work.

Gate 3 — crypto/exploit-audit probes (eval_crypto_audit.py, HTTP against the deployed v1.1 GGUF/MLX artifact, k=9 samples per case, temperature=0, majority vote):

• probe_crypto_id (identify AES/SHA-256/TEA/Blowfish from constants alone, 5 cases): PASS (5/5)
• probe_misuse_enum (ECB mode + time-seeded key + reused static IV, one function, three planted misuses): PASS (found all three, 9/9 runs)
• probe_clean_control (correct AES-256-GCM, should NOT be flagged): FAIL (9/9 runs incorrectly claim a flaw) — this is the regression documented under "Known limitations" above.
• probe_exploit_path (stack overflow with attacker-controlled length, requires overflow diagnosis + concrete hijack consequence): PASS (9/9)

Gate 4 — RE/tool-use regression check (eval_re_v2_http_probes.py, same HTTP-deployed-artifact methodology, confirms the v1.1 crypto/exploit SFT pass didn't regress the v1.0 RE/agentic behavior):

• tool_loop (4 distinct real tool calls, no repeats/vacuous calls): PASS
• error_recovery (clean finish_reason: stop, no forced-loop artifacts): PASS
• long_cot (terminates on its own on hard math, no runaway repetition): PASS

Related repositories

Repo	Format	Modality
this repo	GGUF (IQ4_XS)	text-only
lancejames221b/hawq-sec-re-v1-mlx-4bit	MLX 4-bit	multimodal (vision)

MLX is published as its own repo because format detection (LM Studio, HF filters) keys off files at the repository root — a repo whose root holds a .gguf is classified GGUF, and a nested MLX folder is not discoverable regardless of tags. The copy under hawq-sec-re-v1-mlx-4bit/ here is retained for continuity; prefer the dedicated repo.

Files in this repository

File	Format	Size	Notes
hawq-sec-re-v1-IQ4_XS.gguf	GGUF, IQ4_XS	≈18.9 GB (≈4.4 bpw)	Text-only. llama.cpp / LM Studio. Fits a 24GB GPU (e.g. RTX 4090) with room for q8_0 KV cache at long context. Deployed and smoke-tested (coherent generation on both x86-64 disassembly and RE-analysis prompts). Built from the merged text-only checkpoint (no vision tower).
hawq-sec-re-v1-mlx-4bit/	MLX, mixed 4-bit/8-bit	≈19 GB (4.649 bpw average)	Multimodal (vision-enabled). Apple Silicon, via mlx-vlm (mlx_vlm.models.qwen3_5_moe) - NOT plain mlx-lm, which silently drops the vision tower for this architecture. Vision encoder grafted back from the original Qwen/Qwen3.6-35B-A3B multimodal checkpoint (same lineage this model's text backbone was extracted from); text weights are this model's fine-tuned weights. MoE router (mlp.gate) and shared_expert_gate tensors kept at 8-bit per-layer (architecture's built-in quantization predicate), everything else 4-bit at group size 64. Verified end-to-end on an M4 Max: text and vision paths both load and generate (≈95-100 tok/s, ≈21 GB peak RAM). File integrity byte-exact against the source build; vision tensor count matches the donor exactly (333/333).

Ollama registry listing (lancejames221b/hawq-sec-re-v1) is in progress but push has stalled repeatedly on a registry-side network error; retry pending. The GGUF above is the reliable distribution path for now.

Both quantizations were built fresh from bf16 checkpoints and carry the model's native chat_template.jinja (Qwen-style, thinking-mode-capable) embedded (verified byte-identical to source on both).

Known GGUF conversion gotcha (already fixed in these builds)

Earlier GGUF builds of this architecture family assumed IQ4_XS was structurally broken on this MoE/hybrid-attention layout. That was misdiagnosed: the real defect was an off-by-one block_count (MTP head counted as a regular transformer block) in the f16 intermediate GGUF. Converting with --no-mtp and block_count=40 (the correct text layer count) produces a working IQ4_XS quant, confirmed by smoke-testing with llama-cli and by a verified Ollama deployment on an RTX 4090 (24GB) producing coherent output on x86-64 disassembly and RE-analysis prompts.

Verified deployment numbers (Ollama 0.32.x + llama.cpp backend, single RTX 4090):

offloaded 41/41 layers to GPU          -> 100% GPU, ~22 GB resident
n_ctx = n_ctx_seq = n_ctx_slot = 262144
--cache-type-k q8_0 --cache-type-v q8_0
flash_attn = enabled
llama_kv_cache: 2720 MiB (262144 cells, 10 layers) K q8_0 1360 + V q8_0 1360
llama_memory_recurrent: 62.81 MiB (40 layers)
truncated = 0

Note the KV cache is only ≈2.7 GB at the full 262k window because just 10 of 40 layers use KV attention - the other 30 are linear-attention/SSM layers holding a small recurrent state (≈63 MiB). That's what makes 262k context co-exist with 18.9 GB of weights on a 24 GB card.

Context vs. parallel slots is runtime-specific — don't assume the divide. Under Ollama / the llama.cpp server, n_ctx is split across concurrent slots (n_ctx_seq = n_ctx / n_parallel), so the full 262k per request needs --parallel 1 / OLLAMA_NUM_PARALLEL=1; serving 8 slots at n_ctx=262144 leaves ≈32k per request.

LM Studio does not behave this way. Measured against this exact IQ4_XS build loaded at --context-length 262144 --parallel 4 on a single RTX 4090: an 80,010-token prompt and then a 200,010-token prompt were both accepted and returned normally. LM Studio treats the window as a shared pool that a single request may consume in full, not a statically pre-divided one — so you get 4-way concurrency and the full context per request. Concurrent large requests still contend for the one pool, so peak concurrency at near-full context is bounded by KV memory, not by a fixed per-slot cap.

Usage

llama.cpp / LM Studio

llama-cli -m hawq-sec-re-v1-IQ4_XS.gguf -p "Analyze this function: ..." \
  --temp 0.6 --top-p 0.95 --top-k 20 --min-p 0.0

Sampler defaults match the Qwen3.6-35B-A3B thinking-mode card (temperature=0.6, top_p=0.95, top_k=20, min_p=0.0). Recommended --ctx-size 262144 if VRAM allows; the model was validated at this context length on a single RTX 4090 (24GB) with q8_0 KV cache.

MLX (Apple Silicon)

python3 -m mlx_vlm.generate --model hawq-sec-re-v1-mlx-4bit \
  --prompt "Analyze this function: ..." --temperature 0.6

Requires mlx-vlm (not mlx-lm alone) for the vision tower to load. Or point LM Studio's MLX engine at the hawq-sec-re-v1-mlx-4bit/ folder.

Pass an image with --image:

python3 -m mlx_vlm.generate --model hawq-sec-re-v1-mlx-4bit \
  --image disassembly.png --temperature 0.6 --max-tokens 200 \
  --prompt "Read the assembly in this image and explain what it computes."

MLX verification

Measured on an M4 Max (mlx-vlm 0.6.8, mlx 0.32.0):

Path	Prompt tokens	Generation	Peak RAM	Result
Text-only	43	120 tok @ 94.6 tok/s	20.95 GB	Coherent RE analysis
Image + text	168	58 tok @ 100.8 tok/s	21.07 GB	Transcribed all 7 rendered instructions verbatim, then analyzed them

The prompt-token jump (43 -> 168) is the image patch tokens, confirming the vision encoder is actually in the graph rather than being silently skipped - the specific failure mode that occurs if this checkpoint is converted with plain mlx-lm, whose qwen3_5_moe handler strips vision_tower/model.visual keys during sanitization.

Ollama

Not published to the ollama.com registry yet (the push repeatedly fails partway through a multi-gigabyte multipart upload and restarts from zero; unresolved). Running it locally from the GGUF works and is verified - build it yourself:

cat > Modelfile <<'EOF'
FROM ./hawq-sec-re-v1-IQ4_XS.gguf
PARAMETER num_ctx 262144
PARAMETER temperature 0.6
PARAMETER top_p 0.95
PARAMETER top_k 20
PARAMETER min_p 0.0
EOF

ollama create hawq-sec-re-v1 -f Modelfile
ollama run hawq-sec-re-v1

Do not add a hand-written TEMPLATE block - the GGUF already carries the model's native tokenizer.chat_template, and Ollama derives the correct template from it (verified byte-identical to the source chat_template.jinja). A hand-rolled template risks diverging from what the model was trained against.

For the full 262k window and q8_0 KV cache, set these on the Ollama server (q8_0 KV is silently ignored unless flash attention is on):

OLLAMA_FLASH_ATTENTION=1
OLLAMA_KV_CACHE_TYPE=q8_0
OLLAMA_NUM_PARALLEL=1
OLLAMA_KEEP_ALIVE=-1

Tool calling

The model is tool-capable and thinking-enabled at the same time, which has two practical consequences worth getting right - both verified against the IQ4_XS GGUF served by LM Studio (server-side Qwen tool-call parsing) on an RTX 4090:

1. Give it enough output budget, or you get no tool call at all. Reasoning is emitted before the tool call. If max_tokens cuts the response off mid-thought you get finish_reason: "length", an empty content, a populated reasoning field, and no tool_calls - which resembles "the model can't call tools" but is purely a budget problem:

max_tokens	Result
400	finish_reason: length, content: "", tool_calls: None
8000	finish_reason: tool_calls, correct call, only 359 tokens used

It needed just 359 completion tokens once allowed to finish - the cap needs headroom for the reasoning pass, not for the final answer.

2. Handle the reasoning field separately from content. Reasoning arrives in its own field (reasoning_content), not inline in content. A client that doesn't model that separately can drop or mangle the tool call once the model starts thinking. This is a client/harness configuration concern, not a property of any particular server.

Whatever client you use, the requirement is the same: read the reasoning field from its own key and keep it associated with the turn that carries the tool call. The keys below are specific to the omp harness and are shown only as a concrete example - they are not a standard, and other runtimes (vLLM, LiteLLM, llama.cpp server, etc.) expose their own reasoning-parser options under different names:

reasoning: true
compat:
  thinkingFormat: qwen-chat-template
  reasoningContentField: reasoning_content
  requiresReasoningContentForToolCalls: true

A correct response looks like:

{
  "finish_reason": "tool_calls",
  "message": {
    "content": "",
    "tool_calls": [{"type": "function", "id": "...",
      "function": {"name": "bash",
                   "arguments": "{\"command\":\"unzip -P infected -d /tmp/out /tmp/sample.zip\"}"}}]
  }
}

FAQ

Which architecture is this, exactly? The model_type string is qwen3_5_moe, but the model is Qwen3.6-35B-A3B — the architecture identifier lags the release name, which trips people up. It's a hybrid MoE: 40 text layers of which 10 are full self-attention and 30 are linear-attention/SSM, 256 experts, ~3B active params per token. That layer split is why a 262k context costs only ~2.7 GB of KV cache.

Why build on an abliterated / uncensored base? Because refusal gates are actively counterproductive here. RE work means staring at malware, packers, exploit code and obfuscated binaries — a safety-tuned model refuses or hedges on exactly the inputs you care about, and you spend your time fighting the model instead of analyzing the sample. Starting from weights where those gates are already gone means the SFT budget goes into analysis quality rather than clawing back refused behavior.

Lineage specifics, since two lines came out of the same repo and are easy to conflate: an abliterated donor (huihui-ai/...Claude-4.7-Opus-abliterated) was used explicitly in razorstrike-v1, an earlier DARE-TIES merge that is now superseded. This model's chain is nightmedia/Qwen3.6-35B-A3B-Holo3-Qwopus-AgentWorld-qx64-hi-mlx (a third-party merge, dequantized to bf16) → HAWQ-v1 → this SFT. I did not apply abliteration myself anywhere in the HAWQ chain, and I don't document the internal composition of that third-party merge.

What tooling did you use for training?

• LoRA SFT with plain transformers + peft — deliberately not TRL, to avoid a transformers-v5 compatibility risk on this architecture.
• Response-only prompt-prefix masking.
• 4x A100 80GB (GCE a2-highgpu-4g), FSDP, 2 epochs, MAXLEN=4096.
• r=64, alpha=128. Target modules cover both attention families: q_proj/k_proj/v_proj/o_proj plus the linear-attention/SSM projections in_proj_qkv / in_proj_a / in_proj_b / in_proj_z / out_proj. Ground these against the real weight map rather than reusing a dense-model recipe — on this arch a Qwen-standard target list silently misses 30 of the 40 layers.
• Merged with PeftModel.merge_and_unload().

What about inference / serving?

• GGUF: convert_hf_to_gguf.py + llama-quantize (IQ4_XS), served via LM Studio and Ollama. Fits one RTX 4090 with all 41/41 layers offloaded.
• MLX: mlx-vlm 0.6.8 on an M4 Max, ~95-100 tok/s.

Any non-obvious gotchas on this architecture? Three that cost real time:

1. Don't convert the multimodal checkpoint with mlx-lm. Its qwen3_5_moe handler strips vision_tower / model.visual during sanitization, so you get a silently text-only model wearing a multimodal label. Use mlx-vlm.
2. The MTP head breaks GGUF conversion. The multi-token-prediction head gets counted as a transformer block, yielding block_count = 41 instead of 40. This was long misattributed to "IQ4_XS is broken on this arch" — it isn't. Convert with --no-mtp.
3. Whether n_ctx divides across parallel slots depends on the runtime — verify, don't assume. Under Ollama / the llama.cpp server it does (n_ctx_seq = n_ctx / n_parallel; 262144 across 8 slots gives each request ~32k), so use --parallel 1 / OLLAMA_NUM_PARALLEL=1 there for the full window. LM Studio does not divide it — measured on this build at --parallel 4, single requests of 80,010 and 200,010 tokens both succeeded, so the window is a shared pool one request can use in full. Relatedly, on both runtimes JIT-loading applies the default context, not your configured one.

Why LoRA instead of a full fine-tune? This is a behavioral/format adaptation — produce faithful structured RE analysis — not knowledge injection. LoRA at r=64 across both attention families was sufficient and leaves the base model's general capability intact. The adapter is ~260 MB against a ~70 GB bf16 base.

Limitations

• GGUF is text-only. It's built from the merged text-only checkpoint (no vision tower - despite vestigial vision_config metadata inherited from the original Qwen3.6-35B-A3B lineage in that checkpoint's config.json). Use the MLX build for multimodal.
• MLX multimodal is verified (Apple Silicon, M4 Max). Both the text and vision paths load and generate correctly - see "MLX verification" below. Note the vision tower is the stock Qwen3.6-35B-A3B encoder grafted back on; it was not fine-tuned by this model's RE-analysis SFT, so image understanding is base-Qwen quality, while the text backbone carries the RE tuning.
• Win-rate not benchmarked. See "Gate 2" above — this release has not been formally scored against the base model on held-out RE tasks.
• Long chain-of-thought on hard math/reasoning tasks can approach the 81,920-token cap; the eval harness treats this as expected behavior for the thinking-mode preset, not a defect, but downstream callers setting a lower max_tokens should budget accordingly.

License

Released under Apache 2.0, matching the lancejames221b/HAWQ-v1 base model.