Hugging Face's logo

nm-testing
/
Llama-2-70b-chat-hf-FP8

Text Generation
Transformers
Safetensors
llama
fp8
vllm
conversational
text-generation-inference
Inference Endpoints
Model card Files Community
mgoin commited on
Commit
db718e9
•
1 Parent(s): d8ac34d

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +6 -2
README.md CHANGED
@@ -34,12 +34,16 @@ model.save_quantized(quantized_model_dir)
34
 
35
  Evaluation:
36
 
37
-
38
  ```
 
 
 
 
 
 
39
  vllm (pretrained=Llama-2-70b-fp8-4096,tensor_parallel_size=2,distributed_executor_backend=ray), gen_kwargs: (None), limit: None, num_fewshot: None, batch_size: auto
40
  |Tasks|Version| Filter |n-shot| Metric | |Value | |Stderr|
41
  |-----|------:|----------------|-----:|-----------|---|-----:|---|-----:|
42
  |gsm8k| 3|flexible-extract| 5|exact_match|↑ |0.5625|± |0.0137|
43
  | | |strict-match | 5|exact_match|↑ |0.5428|± |0.0137|
44
-
45
  ```
 
34
 
35
  Evaluation:
36
 
 
37
  ```
38
+ vllm (pretrained=meta-llama/Llama-2-70b-chat-hf,tensor_parallel_size=2,distributed_executor_backend=ray), gen_kwargs: (None), limit: None, num_fewshot: None, batch_size: auto
39
+ |Tasks|Version| Filter |n-shot| Metric | |Value | |Stderr|
40
+ |-----|------:|----------------|-----:|-----------|---|-----:|---|-----:|
41
+ |gsm8k| 3|flexible-extract| 5|exact_match|↑ |0.5307|± |0.0137|
42
+ | | |strict-match | 5|exact_match|↑ |0.5064|± |0.0138|
43
+
44
  vllm (pretrained=Llama-2-70b-fp8-4096,tensor_parallel_size=2,distributed_executor_backend=ray), gen_kwargs: (None), limit: None, num_fewshot: None, batch_size: auto
45
  |Tasks|Version| Filter |n-shot| Metric | |Value | |Stderr|
46
  |-----|------:|----------------|-----:|-----------|---|-----:|---|-----:|
47
  |gsm8k| 3|flexible-extract| 5|exact_match|↑ |0.5625|± |0.0137|
48
  | | |strict-match | 5|exact_match|↑ |0.5428|± |0.0137|
 
49
  ```