tsinghua-sigs-robot-lab/VeriLoop-E2-GGUF

VeriLoop E2 — GGUF

Official llama.cpp distribution of VeriLoop E2
Reference-grade BF16 target + MTP speculative-decoding draft

27B post-trained model for code, mathematics, and physics · 262K native context · Apache License 2.0
Developed by Tsinghua SIGS Robot Lab · Libo Wang

Parent Model · Technical Report · Evaluation Evidence · Riemann ζ Artifact

---

Overview

VeriLoop E2 GGUF is the official llama.cpp-oriented distribution of VeriLoop E2, a 27B post-trained model built on Qwen3.8-27B for code, software engineering, mathematics, and physics.

This repository is intentionally organized as a GGUF release repository rather than a duplicate of the parent model card. The parent repository remains the authoritative source for model methodology, training, system architecture, benchmark protocols, scientific demonstrations, and release boundaries. This repository focuses on the artifacts and information that matter for local GGUF deployment:

• canonical GGUF files and their roles;
• exact conversion provenance and integrity checks;
• llama.cpp loading and serving instructions;
• MTP speculative decoding;
• hardware and context guidance;
• quantization-quality policy for lower-bit releases;
• a concise, evidence-linked snapshot of the parent model's public benchmark record.

The first release establishes BF16 as the canonical GGUF reference point. Lower-bit variants are published only after they are compared against this reference under a fixed quality protocol.

---

Release Files

File	Role	Precision	Tensor count	Approx. local size	SHA256	Status
VeriLoop-E2-BF16.gguf	Canonical target model	BF16 / selected F32 metadata-sensitive tensors	851	~51 GiB	11bf5defde1a256b7582bc34fd2c4a85a61615ed88e7a422dcd24d814ea6d35d	Verified
mtp-VeriLoop-E2-BF16.gguf	MTP draft for speculative decoding	BF16	18	~5.6 GiB	10e36ca18d508f3321f976262cc2e05b01e807f3a797605696a8b6a3f490429f	Verified

BF16 is the reference, not a low-bit quantization. It is provided as the high-fidelity GGUF baseline against which subsequent Q8/Q6/Q5/Q4/IQ4 releases can be measured.

Hugging Face may display file sizes in decimal GB while local Unix tools commonly display binary GiB; small differences in displayed size are therefore expected.

---

Conversion Provenance and Quality Gates

This release was converted with the public llama.cpp GGUF toolchain and validated before promotion.

Check	Result
Parent checkpoint	tsinghua-sigs-robot-lab/VeriLoop-E2
Base model family	Qwen3.8-27B
HF architecture	Qwen3_5ForConditionalGeneration
Native model context	262,144 tokens
Conversion tool	convert_hf_to_gguf.py
llama.cpp source revision	bddf8263c31c3dce3212263b00ebd2d98c1a752b
Target export	BF16, separate from MTP
MTP export	BF16 draft GGUF
Target conversion return code	0
MTP conversion return code	0
Target tensor audit	851 tensors / PASS
MTP tensor audit	18 tensors / PASS
Non-zero tensor audit	PASS
Source-checkpoint immutability gate	PASS
CUDA llama-server build	PASS
llama-server model load	PASS
OpenAI-compatible /v1/models endpoint	PASS
Reference GGUF runtime context tested in release gate	32,768 tokens

The native 262K context length is inherited from the parent model configuration. The 32K figure above is the context length used in this GGUF release's runtime validation, not a redefinition of the model's native context. Larger llama.cpp contexts require appropriate VRAM/RAM and KV-cache planning and should be validated on the target hardware.

Integrity policy

The release process treats conversion as an auditable transformation rather than a successful file write:

Parent checkpoint
    ↓
tensor-name / shard consistency checks
    ↓
BF16 target export
    ↓
separate MTP export
    ↓
GGUF tensor-count audit
    ↓
llama-server build + load gate
    ↓
runtime endpoint gate
    ↓
release manifest + SHA256

The SHA256 values above are the frozen identities for the initial BF16 reference release.

---

Parent-Model Benchmark Record

The following numbers are the frozen public VeriLoop E2 parent-model release results. They are included here so local users can identify the model they are downloading without having to reconstruct its evaluation record from multiple pages.

These scores are not presented as GGUF-specific re-runs. BF16 is a format-conversion reference; each future lower-bit variant will be evaluated for degradation relative to this BF16 reference before being recommended.

Benchmark	VeriLoop E2 release score	Public evidence
SWE-bench Pro	76.2	Evidence
Terminal-Bench 2.1	88.8	Evidence
DeepSWE v1.1	64.6	Evidence
Terminal-Bench 3.0	29.7	Evidence
Terminal-Bench 4.0	37.9	Evidence
SWE-Marathon v1.1	45.0	Release source
AIME 2026	98.3	Evidence
GPQA Diamond	93.9	Evidence
Apex 2025	89.6	Evidence

For exact benchmark protocols, task-level outputs, evaluator receipts, and provenance, use the VeriLoop E2 Evaluation Evidence repository.

---

Quick Start with llama.cpp

Use a recent llama.cpp build with Qwen3.8 support.

Install

curl -LsSf https://llama.app/install.sh | sh

Or build from source:

git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
cmake -B build -DGGML_CUDA=ON
cmake --build build -j --target llama-server llama-cli

Download the BF16 reference

hf download \
  tsinghua-sigs-robot-lab/VeriLoop-E2-GGUF \
  VeriLoop-E2-BF16.gguf \
  --local-dir .

Run the target model

llama-server \
  -m ./VeriLoop-E2-BF16.gguf \
  -ngl all \
  -c 32768 \
  --host 127.0.0.1 \
  --port 8080

The server exposes an OpenAI-compatible API at:

http://127.0.0.1:8080/v1

Direct CLI inference

llama-cli \
  -m ./VeriLoop-E2-BF16.gguf \
  -ngl all \
  -c 32768

-ngl all is appropriate when the full model fits in available VRAM. llama.cpp also supports automatic or partial GPU offload; users with less VRAM should allow the runtime to place part of the model in system memory.

---

MTP Speculative Decoding

The release includes a separate BF16 MTP draft GGUF. In llama.cpp, MTP can be used as a speculative draft model:

hf download \
  tsinghua-sigs-robot-lab/VeriLoop-E2-GGUF \
  mtp-VeriLoop-E2-BF16.gguf \
  --local-dir .

llama-server \
  -m ./VeriLoop-E2-BF16.gguf \
  --model-draft ./mtp-VeriLoop-E2-BF16.gguf \
  --spec-type draft-mtp \
  -ngl all \
  --n-gpu-layers-draft all \
  -c 32768 \
  --host 127.0.0.1 \
  --port 8080

MTP is an optional acceleration path. Throughput gains depend on hardware, prompt distribution, draft acceptance, context length, and llama.cpp version; this repository does not claim a universal speedup factor.

---

Memory Guidance

The BF16 reference is intentionally large.

Component	Approx. file size	Practical implication
Main BF16 GGUF	~51 GiB	Requires substantial GPU VRAM or CPU RAM
BF16 MTP draft	~5.6 GiB	Additional memory when speculative decoding is enabled
KV cache / runtime workspace	Workload-dependent	Increases with context length, batch size, cache dtype, and parallelism

A full-GPU BF16 deployment therefore needs materially more memory than the model file alone. For constrained systems, wait for or use a validated lower-bit release rather than forcing an unsuitable BF16 configuration.

---

Quantization Release Policy

This repository follows a reference-first, measurement-driven quantization policy.

The BF16 GGUF above is the baseline. Lower-bit releases are not labeled as recommended solely because they are common quant types. Each candidate is evaluated against the BF16 reference before recommendation.

The release program targets the following families:

• Q8_0 — high-fidelity local deployment;
• Q6_K — quality-focused compression;
• Q5_K_M / Q5_K_S — high-quality balanced tiers;
• Q4_K_M / Q4_K_S — broad local-deployment tiers;
• IQ4_NL / IQ4_XS — compact 4-bit-class alternatives where supported.

For each lower-bit release, the quality card is intended to report:

Dimension	Measurement
File size	GB / GiB
Effective density	bits per weight where applicable
Language-model drift	fixed-corpus perplexity delta vs BF16
Distribution drift	mean KL divergence vs BF16
Decision stability	same-top-p / token agreement metrics
Code capability	selected software-engineering regression gates
Mathematics	selected mathematical-reasoning regression gates
Science	selected scientific-reasoning regression gates
Long-context behavior	fixed-context smoke / stability checks
llama.cpp runtime	load, generation, API and MTP compatibility

No cross-repository PPL or KLD comparison is claimed unless the corpus, tokenizer path, chunking, context, and command line are identical. Absolute perplexity numbers from different GGUF repositories are generally not directly comparable.

---

Model Identity

Property	Value
Model	VeriLoop E2
Distribution	VeriLoop E2 GGUF
Parent model	tsinghua-sigs-robot-lab/VeriLoop-E2
Base model	Qwen3.8-27B
Parameter class	27B
Training stage	Post-Training
Primary domains	Code, software engineering, mathematics, physics
HF architecture	Qwen3_5ForConditionalGeneration
Native context	262,144 tokens
Languages	English, Chinese
GGUF runtime	llama.cpp
Initial GGUF reference	BF16
Speculative decoding	MTP draft supplied separately
License	Apache License 2.0

---

Prompt and Chat Protocol

Use the chat template embedded in the GGUF / inherited from the VeriLoop E2 release. Do not casually replace role delimiters, reasoning delimiters, stop conditions, or tool-call formatting: prompt-template changes can materially alter observed behavior even when the model weights are identical.

For the full model protocol and system-level behavior, see the VeriLoop E2 parent model card.

---

Release Boundaries

• This repository distributes the GGUF deployment form of VeriLoop E2.
• The BF16 file is a reference conversion, not a claim of low-bit compression.
• Parent-model benchmark numbers above are provided for model identity and provenance; lower-bit GGUFs require their own degradation measurements.
• The model may still produce incorrect code, invalid mathematical reasoning, unsupported scientific arguments, or unsafe commands.
• Native context length does not guarantee that every local hardware configuration can serve the full context efficiently.
• MTP acceleration is runtime- and workload-dependent.
• Community-modified templates, samplers, stop rules, cache settings, or parsers can materially change observed behavior.

---

Source Model and Evidence

Resource	Link
Parent model	VeriLoop E2
GGUF repository	VeriLoop E2 GGUF
Technical report	OpenReview
Evaluation evidence	VeriLoop E2 Evaluation Evidence
Riemann ζ artifact	Public artifact
llama.cpp	ggml-org/llama.cpp

---

License

The VeriLoop E2 model weights and this GGUF distribution are released under the Apache License 2.0, consistent with the parent model release.

Third-party software, benchmarks, and separately published scientific artifacts retain their own terms.

---

Citation

If you use VeriLoop E2 or this GGUF distribution in research, please cite the parent model release:

@misc{wang2026veriloope2,
  title        = {VeriLoop E2: A 27B Post-Trained Model for Code, Mathematics, and Scientific Reasoning},
  author       = {Wang, Libo},
  year         = {2026},
  note         = {Tsinghua Shenzhen International Graduate School (SIGS)},
  howpublished = {Open model release},
  url          = {https://huggingface.co/tsinghua-sigs-robot-lab/VeriLoop-E2}
}

For quantization-specific comparisons, identify the exact GGUF filename and SHA256 in addition to the parent-model citation.

---

Acknowledgements

VeriLoop E2 builds on Qwen3.8-27B and the open-source model-serving and evaluation ecosystem. We thank the communities behind Qwen, llama.cpp, Transformers, Hugging Face, software-engineering benchmarks, mathematical evaluation suites, and reproducible scientific computing.

This GGUF repository is maintained as a reproducible deployment release, with conversion provenance, immutable file identities, and quantization-quality measurements treated as first-class release artifacts.