Update README.md
Browse filesFixed ROPE and Conversion using latest llama.cpp and Transformers dev build.
README.md
CHANGED
@@ -6,7 +6,12 @@ tags:
|
|
6 |
- text-generation-inference
|
7 |
---
|
8 |
> [!WARNING]
|
9 |
-
> At the time of this release, llama.cpp did not support the rope scaling required for full context (limit is 8192). Soon this will be updated for full 128K functionality.
|
|
|
|
|
|
|
|
|
|
|
10 |
|
11 |
![image/png](https://cdn-uploads.huggingface.co/production/uploads/645e666bb5c9a8666d0d99c5/9T9q6k90ZGa5EJKeSMbru.png)
|
12 |
|
|
|
6 |
- text-generation-inference
|
7 |
---
|
8 |
> [!WARNING]
|
9 |
+
> At the time of this release, llama.cpp did not support the rope scaling required for full context (limit is 8192). Soon this will be updated for full 128K functionality.
|
10 |
+
> Depriciated models still listed do not have 128k mark.
|
11 |
+
|
12 |
+
> [!NOTE]
|
13 |
+
> The new release of llama.cpp and transformers have been applied and the gguf was tested.
|
14 |
+
> [Meta-Llama-3.1-8B-Instruct-128k](https://huggingface.co/3Simplex/Meta-Llama-3.1-8B-Instruct-gguf/blob/main/Meta-Llama-3.1-8B-Instruct-128k-Q4_0.gguf)
|
15 |
|
16 |
![image/png](https://cdn-uploads.huggingface.co/production/uploads/645e666bb5c9a8666d0d99c5/9T9q6k90ZGa5EJKeSMbru.png)
|
17 |
|