leo-pekelis-gradient commited on
Commit
eadf5aa
1 Parent(s): 1c075c4

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +25 -1
README.md CHANGED
@@ -50,7 +50,31 @@ For training data, we generate long contexts by augmenting [SlimPajama](https://
50
  | GPU Type | NVIDIA L40S | NVIDIA L40S | NVIDIA L40S | NVIDIA L40S |
51
  | Minutes to Train (Wall)| 202 | 555 | 61 | 87 |
52
 
53
- **Quants**:
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
54
  - [GGUF](https://huggingface.co/crusoeai/Llama-3-8B-Instruct-1048k-GGUF)
55
  - [MLX-4bit](https://huggingface.co/mlx-community/Llama-3-8B-Instruct-1048k-4bit)
56
 
 
50
  | GPU Type | NVIDIA L40S | NVIDIA L40S | NVIDIA L40S | NVIDIA L40S |
51
  | Minutes to Train (Wall)| 202 | 555 | 61 | 87 |
52
 
53
+ **Evaluation:**
54
+
55
+ ![image/png](https://cdn-uploads.huggingface.co/production/uploads/6585dc9be92bc5f258156bd6/mWxIGZNi3ejlmeIDWafKu.png)
56
+
57
+ ```
58
+ EVAL_MAX_CONTEXT_LENGTH=1040200
59
+ EVAL_MIN_CONTEXT_LENGTH=100
60
+ EVAL_CONTEXT_INTERVAL=86675
61
+ EVAL_DEPTH_INTERVAL=0.2
62
+ EVAL_RND_NUMBER_DIGITS=8
63
+
64
+ HAYSTACK1:
65
+ EVAL_GENERATOR_TOKENS=25
66
+
67
+ HAYSTACK2:
68
+ EVAL_CONTEXT_INTERVAL=173350
69
+ EVAL_GENERATOR_TOKENS=150000
70
+
71
+ HAYSTACK3:
72
+ EVAL_GENERATOR_TOKENS=925000
73
+ ```
74
+
75
+ All boxes not pictured for Haystack 1 and 3 are 100% accurate. Haystacks 1,2 and 3 are further detailed in this [blog post](https://gradient.ai/blog/the-haystack-matters-for-niah-evals).
76
+
77
+ **Quants:**
78
  - [GGUF](https://huggingface.co/crusoeai/Llama-3-8B-Instruct-1048k-GGUF)
79
  - [MLX-4bit](https://huggingface.co/mlx-community/Llama-3-8B-Instruct-1048k-4bit)
80