nm-testing
/

Llama-2-70b-chat-hf-FP8

Text Generation

text-generation-inference

Inference Endpoints

Model card Files Files and versions Community

mgoin commited on Jul 16

Commit

e521dee

•

1 Parent(s): 33cf10b

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -46,7 +46,7 @@ vllm (pretrained=meta-llama/Llama-2-70b-chat-hf,tensor_parallel_size=2,distribut
 |gsm8k|      3|flexible-extract|     5|exact_match|↑  |0.5307|±  |0.0137|
 |     |       |strict-match    |     5|exact_match|↑  |0.5064|±  |0.0138|
-vllm (pretrained=Llama-2-70b-fp8-4096,tensor_parallel_size=2,distributed_executor_backend=ray), gen_kwargs: (None), limit: None, num_fewshot: None, batch_size: auto
 |Tasks|Version|     Filter     |n-shot|  Metric   |   |Value |   |Stderr|
 |-----|------:|----------------|-----:|-----------|---|-----:|---|-----:|
 |gsm8k|      3|flexible-extract|     5|exact_match|↑  |0.5625|±  |0.0137|

 |gsm8k|      3|flexible-extract|     5|exact_match|↑  |0.5307|±  |0.0137|
 |     |       |strict-match    |     5|exact_match|↑  |0.5064|±  |0.0138|
+vllm (pretrained=nm-testing/Llama-2-70b-chat-hf-FP8,tensor_parallel_size=2,distributed_executor_backend=ray), gen_kwargs: (None), limit: None, num_fewshot: None, batch_size: auto
 |Tasks|Version|     Filter     |n-shot|  Metric   |   |Value |   |Stderr|
 |-----|------:|----------------|-----:|-----------|---|-----:|---|-----:|
 |gsm8k|      3|flexible-extract|     5|exact_match|↑  |0.5625|±  |0.0137|