Nexesenex
/

llama-3-70B-Instruct-abliterated-CQ-GGUF

Inference Endpoints

Model card Files Files and versions Community

llama-3-70B-Instruct-abliterated-CQ-GGUF / README.md

Nexesenex's picture

Update README.md

60a5567 verified 5 months ago

|

183 Bytes

Custom GGUF quant for : failspy/llama-3-70B-Instruct-abliterated-GGUF

IQ4_SR is Optimal (8k ctx) for 36GB VRAM with an IGP displaying the OS.

Without an IGP, the IQ4_XSR is for you.