llama-3.1-8b-awq

Optimized llama-3.1-8b-awq for efficient inference.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("gulf-inference/llama-3.1-8b-awq", trust_remote_code=True)
tokenizer = AutoTokenizer.from_pretrained("gulf-inference/llama-3.1-8b-awq", trust_remote_code=True)