mgoin
/

Meta-Llama-3-8B-Instruct-Marlin

Text Generation

text-generation-inference

Inference Endpoints

4-bit precision

Model card Files Files and versions Community

mgoin commited on Apr 18

Commit

4b3e6a7

•

1 Parent(s): 5d3c06f

Update README.md

Files changed (1) hide show

README.md +0 -1

README.md CHANGED Viewed

@@ -18,7 +18,6 @@ license_link: LICENSE
 > python -m vllm.entrypoints.openai.api_server --model mgoin/Meta-Llama-3-8B-Instruct-Marlin
 > OPENAI_API_KEY=dummy lm-eval --model local-completions --model_args model=mgoin/Meta-Llama-3-8B-Instruct-Marlin,tokenizer_backend=huggingface,base_url=http://0.0.0.0:8000/v1 --task gsm8k --batch_size 128
 local-completions (model=mgoin/Meta-Llama-3-8B-Instruct-Marlin,tokenizer_backend=huggingface,base_url=http://0.0.0.0:8000/v1), gen_kwargs: (None), limit: None, num_fewshot: None, batch_size: 128
 |Tasks|Version|     Filter     |n-shot|  Metric   |Value |   |Stderr|
 |-----|------:|----------------|-----:|-----------|-----:|---|-----:|

 > python -m vllm.entrypoints.openai.api_server --model mgoin/Meta-Llama-3-8B-Instruct-Marlin
 > OPENAI_API_KEY=dummy lm-eval --model local-completions --model_args model=mgoin/Meta-Llama-3-8B-Instruct-Marlin,tokenizer_backend=huggingface,base_url=http://0.0.0.0:8000/v1 --task gsm8k --batch_size 128
 local-completions (model=mgoin/Meta-Llama-3-8B-Instruct-Marlin,tokenizer_backend=huggingface,base_url=http://0.0.0.0:8000/v1), gen_kwargs: (None), limit: None, num_fewshot: None, batch_size: 128
 |Tasks|Version|     Filter     |n-shot|  Metric   |Value |   |Stderr|
 |-----|------:|----------------|-----:|-----------|-----:|---|-----:|