ahmedandaloes/CyberStrike-OffSec-35B-MLX-8bit

CyberStrike-OffSec-35B — MLX 8-bit

8-bit quantized MLX conversion of oyildirim/CyberStrike-OffSec-35B, for fast local inference on Apple Silicon.

• Architecture: qwen3_5_moe (Qwen3.5 MoE) — base Qwen/Qwen3.6-35B-A3B
• Precision: 8-bit affine, group size 64. MoE router gates (mlp.gate, mlp.shared_expert_gate) kept at 8-bit to preserve routing quality.
• Size: ~35 GB (fp16 source ~70 GB). Estimated quality retention: ~99.9% (near-lossless).
• Converted with: mlx-lm 0.31.3, mlx, transformers 5.13.

Weights unchanged from the source model — format + precision conversion only, no extra fine-tuning.

Benchmarks (from the source model)

Benchmark	Task	Score
SecEval	Cybersecurity knowledge	81.39
CyberMetric-10000	Cybersecurity knowledge	86.61
SECURE-MAET	MITRE ATT&CK extraction	93.94
SECURE-CWET	CWE extraction	93.05
MMLU	General knowledge	76.94

Scores are the base model's; this MLX build preserves the weights (quantization may cause a small deviation at lower bit-widths).

Other builds

Build	Bits	Size	Quality
MLX-4bit	4	~18 GB	~95-98%
MLX-6bit	6	~27 GB	~99%
MLX-8bit	8	~35 GB	~99.9%
MLX-bf16	bf16	~67 GB	100%

Usage

Note: this model uses the transformers-5 TokenizersBackend tokenizer. mlx-lm 0.31.3 (current pypi release) crashes at import under transformers 5 due to an unrelated AutoTokenizer.register(...) call.

Cleanest fix — install mlx-lm from git (already patched on main, see ml-explore/mlx-lm#1458):

pip install "git+https://github.com/ml-explore/mlx-lm" "transformers>=5"

Then the snippet below works without the runtime patch.

Or, staying on the 0.31.3 release, patch it at runtime before importing mlx_lm (harmless — registers an mlx-lm helper this model does not use):

from transformers import AutoTokenizer
_orig = AutoTokenizer.register
def _safe(*a, **k):
    try:
        return _orig(*a, **k)
    except Exception:
        pass
AutoTokenizer.register = staticmethod(_safe)

from mlx_lm import load, generate

model, tokenizer = load("ahmedandaloes/CyberStrike-OffSec-35B-MLX-8bit")
messages = [{"role": "user", "content": "What is SQL injection?"}]
prompt = tokenizer.apply_chat_template(messages, add_generation_prompt=True)
print(generate(model, tokenizer, prompt=prompt, max_tokens=256, verbose=True))

pip install -U mlx-lm "transformers>=5"

Intended use

Authorized offensive-security work: penetration testing, red-team engagements, CTF, security research, and education. Use responsibly and only on systems you are authorized to test.

Attribution & license

• Source model: oyildirim/CyberStrike-OffSec-35B
• Foundation: Qwen/Qwen3.6-35B-A3B
• License: Apache-2.0 (inherited from the source model)

Credit for the model and its training belongs to the original author. This repository provides an MLX 8-bit build for the Apple Silicon community.