llama-2-13b-hf-reordered

This model has QEFT Offline Global Reordering (OGR) applied. No quantization has been applied.

Reordering Configuration

Parameter	Value
Base model	meta-llama/Llama-2-13b-hf
Method	QEFT OGR (Offline Global Reordering)
Outlier channels (k)	128
Quantization	None (reordering only)

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("jsyeom/reordered_models/llama-2-13b-hf-reordered")
tokenizer = AutoTokenizer.from_pretrained("jsyeom/reordered_models/llama-2-13b-hf-reordered")

Important Notes

• o_proj: columns (input dim) are NOT reordered due to the multi-head attention structure. Only rows are reordered. (See QEFT paper Section 3.2 and Limitations.)

Reference

• QEFT: Quantization for Efficient Fine-Tuning of LLMs