summerMC/Longbook-Looped-8-4-pre

Longbook — Indexed Looped HQQ 8/4-bit

Delivery status: uploaded to summerMC/Longbook-Looped-8-4 (private) on 2026-10-20. Local HF AutoModel checks passed and remote weight-file sizes were verified. No local/Drive model backup was created. Remote model reload has not been tested.

Frozen experimental inference checkpoint. One physical decoder executes 64 logical depths by selecting depth-specific HQQ operators. All 64 layers' parameter information remains in the quantized banks. This is not a weight-tied one-layer recurrent model. Core reuse itself does not reduce parameter storage.

This derivative uses 93 four-bit modules and 530 eight-bit modules. Safetensor payload: 26,334,960,363 bytes (~26.33 GB), vs the source's 55,066,031,584 bytes. No model optimizer training or LoRA distillation was performed for this checkpoint. HQQ performs its own weight quantization optimization.

Install and load

Install CUDA-compatible PyTorch first, then the included requirements. Tested with PyTorch 2.11.0+cu130, Transformers 5.20.1, HQQ 0.2.8.post1, safetensors 0.8.0, and flash-linear-attention 0.5.2 on an A100 80GB.

# pip install transformers==5.20.1 hqq==0.2.8.post1 accelerate safetensors \
#   huggingface_hub flash-linear-attention==0.5.2
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

REPO = "summerMC/Longbook-Looped-8-4-pre"  # or this local export directory
model = AutoModelForCausalLM.from_pretrained(
    REPO, trust_remote_code=True, dtype=torch.bfloat16, device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained(REPO, trust_remote_code=True)
print(len(model.model.language_model.layers))  # 1 physical decoder
print(model.model.language_model.logical_depth)  # 64 logical depths

text = tokenizer.apply_chat_template(
    [{"role": "user", "content": "What is 2 + 2? Answer briefly."}],
    tokenize=False, add_generation_prompt=True, enable_thinking=False,
)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
with torch.inference_mode():
    output = model.generate(**inputs, max_new_tokens=64, do_sample=False, num_beams=1)
print(tokenizer.decode(output[0, inputs.input_ids.shape[1]:], skip_special_tokens=True))

AutoModelForImageTextToText and AutoModelForMultimodalLM aliases are also provided, but only text inference was validated. Review all remote code before trusting it.

Format and runtime limitations

• Custom from_pretrained loads per-operator HQQ safetensors and the dense tensors. It deliberately bypasses Transformers' native HQQ quantizer.
• No standard model.safetensors is provided: use AutoModel with trusted custom code, not a native backend that ignores the custom loader.
• Single CUDA device, BF16 compute only. device_map="auto" means CUDA:0 here, not multi-GPU automatic placement. CPU/disk offload is unsupported.
• Native quantized execution used ~24.7 GiB resident GPU allocation and ~32.0 GiB generation peak in short tests. One-core resident allocation was ~24.9 GiB. These figures do not guarantee fit on a 32GB GPU. A100 80GB was tested.
• HQQ PyTorch forward dequantizes weights for matmul; speedup is NOT promised.
• Do not call model.to(), model.half() or move devices after loading.
• num_beams=1 only; cache beam/batch selection is explicitly unsupported.
• Image/video quality, padding batches, long contexts, vLLM/TGI/other serving engines, multi-GPU and fine-tuning are unvalidated.
• save_pretrained copies the immutable source checkpoint; it does NOT serialize fine-tuning changes. Do not mutate weights. It rejects explicit training state. Use upload_hf.py rather than save_pretrained(push_to_hub=True).

Limited quality evaluation

Eight English/Japanese development documents and eight document-disjoint test documents were used, with at most 96 input tokens and eight sampled prediction positions per document. Test results against the BF16 source:

Metric	Result
Mean full-vocabulary teacher KL	0.02923067
Maximum per-document KL	0.05432044
Mean sampled-token CE difference	-0.00208880

These are small development/test probes, not a general language-quality benchmark. The slightly negative CE difference is not evidence of improved model quality. Simple substring checks for Tokyo/Paris/4 are weak checks, not semantic accuracy evaluation. The source model itself sometimes answers oddly; short Japanese answers may remain verbose or fail to reach a conclusion within a token budget.

One-core restructuring matched the native quantized model with zero tested logit error, identical cached generation tokens, and 64 separate recurrent states. This is a smoke-test equivalence result, not zero error relative to the BF16 source.

Repository contents and cleanup

The inference checkpoint retains its custom model/runtime code, configuration, tokenizer assets, float_weights.safetensors, and the 623 quantized weight files referenced by looped_hqq_manifest.json.

The following example and report artifacts were removed from the current revision:

• example_hf_chat.py
• hf_load_validation.json
• package_code_checksums.json
• package_status.json
• quality_report.json

The loading example above replaces the removed standalone chat example. Evaluation figures in this model card are historical results; the original reports remain available in the repository history before the cleanup commit. No model weights were deleted by that cleanup. Inference was not re-tested after cleanup. upload_hf.py and the optional speedx_1t_runtime.py remain available; this cleanup does not establish validation of optional runtime features.

Source, license and attribution

Source: https://huggingface.co/summerMC/Qwen3.8-27B-Longbook27-VL-GDN64 Pinned revision: a579208838a5ef31718bd031babfbf859d377a05. The source model card declares Apache-2.0. Original custom model Python files are preserved. New packaging/runtime code is under Apache-2.0. See LICENSE and UPSTREAM.md and verify all upstream license obligations before redistribution.