Hugging Face's logo

gradientai
/
Llama-3-8B-Instruct-262k

Text Generation
Transformers
Safetensors
English
llama
meta
llama-3
conversational
Inference Endpoints
text-generation-inference
Model card Files Community
20
bullerwins commited on
Commit
b2a03cc
•
1 Parent(s): d18ddbb

Update README.md

Browse files

Added Exl2 quantized versions

Files changed (1) hide show
  1. README.md +7 -0
README.md CHANGED
@@ -29,6 +29,13 @@ We build on top of the EasyContext Blockwise RingAttention library [3] to scalab
29
 
30
  GGUF is available on on Crusoe's huggingface account. Check it out here: [crusoeai/Llama-3-8B-Instruct-262k-GGUF](https://huggingface.co/crusoeai/Llama-3-8B-Instruct-262k-GGUF)
31
 
 
 
 
 
 
 
 
32
  **Data:**
33
 
34
  For training data, we generate long contexts by augmenting [SlimPajama](https://huggingface.co/datasets/cerebras/SlimPajama-627B).
 
29
 
30
  GGUF is available on on Crusoe's huggingface account. Check it out here: [crusoeai/Llama-3-8B-Instruct-262k-GGUF](https://huggingface.co/crusoeai/Llama-3-8B-Instruct-262k-GGUF)
31
 
32
+ **Exl2 quantized versions**
33
+
34
+ Exl2 is available on Bullerwins's huggingface account. Check it out here:
35
+ [8.0bpw exl2](https://huggingface.co/bullerwins/gradientai_Llama-3-8B-Instruct-262k_exl2_8.0bpw)
36
+ [6.0bpw exl2](https://huggingface.co/bullerwins/gradientai_Llama-3-8B-Instruct-262k_exl2_6.0bpw)
37
+ [5.0bpw exl2](https://huggingface.co/bullerwins/gradientai_Llama-3-8B-Instruct-262k_exl2_5.0bpw)
38
+
39
  **Data:**
40
 
41
  For training data, we generate long contexts by augmenting [SlimPajama](https://huggingface.co/datasets/cerebras/SlimPajama-627B).