Update README.md

README.md

@@ -50,6 +50,7 @@ For training data, we generate long contexts by augmenting [SlimPajama](https://
 | GPU Type               | NVIDIA L40S | NVIDIA L40S | NVIDIA L40S | NVIDIA L40S |
 | Minutes to Train (Wall)| 202       | 555       | 61        | 87        |
 
+
 **Evaluation:**
 
 ![image/png](https://cdn-uploads.huggingface.co/production/uploads/6585dc9be92bc5f258156bd6/mWxIGZNi3ejlmeIDWafKu.png)
@@ -75,8 +76,11 @@ HAYSTACK3:
 All boxes not pictured for Haystack 1 and 3 are 100% accurate. Haystacks 1,2 and 3 are further detailed in this [blog post](https://gradient.ai/blog/the-haystack-matters-for-niah-evals).
 
 **Quants:**
-- [GGUF](https://huggingface.co/crusoeai/Llama-3-8B-Instruct-1048k-GGUF)
+- [GGUF by Crusoe](https://huggingface.co/crusoeai/Llama-3-8B-Instruct-1048k-GGUF). Note that you need to add 128009 as [special token with llama.cpp](https://huggingface.co/gradientai/Llama-3-8B-Instruct-262k/discussions/13).
 - [MLX-4bit](https://huggingface.co/mlx-community/Llama-3-8B-Instruct-1048k-4bit)
+- [Ollama](https://ollama.com/library/llama3-gradient)
+- vLLM docker image, recommended to load via `--max-model-len 32768`
+- If you are interested in a hosted version, drop us a mail below.
 
 ## The Gradient AI Team