Instructions to use brain-lab/Llama-2-70b-QuIP-HARP-2Bit with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use brain-lab/Llama-2-70b-QuIP-HARP-2Bit with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="brain-lab/Llama-2-70b-QuIP-HARP-2Bit")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("brain-lab/Llama-2-70b-QuIP-HARP-2Bit")
model = AutoModelForCausalLM.from_pretrained("brain-lab/Llama-2-70b-QuIP-HARP-2Bit", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use brain-lab/Llama-2-70b-QuIP-HARP-2Bit with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "brain-lab/Llama-2-70b-QuIP-HARP-2Bit"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "brain-lab/Llama-2-70b-QuIP-HARP-2Bit",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/brain-lab/Llama-2-70b-QuIP-HARP-2Bit

SGLang

How to use brain-lab/Llama-2-70b-QuIP-HARP-2Bit with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "brain-lab/Llama-2-70b-QuIP-HARP-2Bit" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "brain-lab/Llama-2-70b-QuIP-HARP-2Bit",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "brain-lab/Llama-2-70b-QuIP-HARP-2Bit" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "brain-lab/Llama-2-70b-QuIP-HARP-2Bit",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use brain-lab/Llama-2-70b-QuIP-HARP-2Bit with Docker Model Runner:
```
docker model run hf.co/brain-lab/Llama-2-70b-QuIP-HARP-2Bit
```

Llama-2-70b-QuIP-HARP-2Bit

2-bit quantized version of meta-llama/Llama-2-70b-hf, produced with HARP (Hadamard-Preconditioned Adaptive Rotations), a learnable structured orthogonal incoherence processor for extreme LLM quantization.

Paper: HARP: Hadamard-Preconditioned Adaptive Rotation Processor for Extreme LLM Quantization (EMNLP 2026)
Code: brain-lab-research/HARP
Backend: QuIP# (harp_quip/), HARP as a drop-in replacement for the fixed randomized Hadamard/RHT incoherence-processing stage
Collection: brain-lab/harp-quantized-models

What is HARP?

Ultra-low-bit PTQ methods such as QuIP# rely on a randomized Hadamard transform (RHT) to make weights more incoherent before quantization. HARP replaces this fixed transform with a learned structured orthogonal processor. It starts from a Hadamard-style initialization and is fit only on calibration data, so no model retraining is needed. Stored in int8, it adds negligible bits-per-parameter overhead. Everything else in the QuIP# pipeline (codebooks, LDLQ solver, HF export, inference) stays unchanged.

Results

WikiText2 / C4 perplexity at ~2 bits, context length 4096 (BPP includes HARP processor storage):

Method	BPP↓	Wiki2↓	C4↓
QuIP# (fixed RHT)	2.00	4.16	6.01
QuIP# + HARP (this model)	2.02	4.01	5.82

Usage

This checkpoint uses QuIP#-style packed weights and requires the harp_quip inference code (a fork of QuIP#) rather than plain transformers.AutoModelForCausalLM:

git clone https://github.com/brain-lab-research/HARP
cd HARP/harp_quip
pip install -r requirements.txt
cd quiptools && python setup.py install && cd ..

from lib.utils.unsafe_import import model_from_hf_path

model, model_str = model_from_hf_path("brain-lab/Llama-2-70b-QuIP-HARP-2Bit")

See harp_quip/eval/eval_ppl.py and harp_quip/eval/eval_zeroshot.py for perplexity / zero-shot evaluation scripts, and the main repo README for the full quick-start guide.

License

Governed by the Llama 2 license. This is a derivative of Meta's Llama 2 70B; use is subject to Meta's Llama 2 Community License Agreement.

Citation

@article{zagitov2026harp,
  title={HARP: Hadamard-Preconditioned Adaptive Rotation Processor for Extreme LLM Quantization},
  author={Artur Zagitov and Gleb Molodtsov and Aleksandr Beznosikov},
  year={2026},
  eprint={2605.29843},
  archivePrefix={arXiv},
  primaryClass={cs.LG},
  url={https://arxiv.org/abs/2605.29843},
}