Create README.md

README.md

@@ -0,0 +1,15 @@
+---
+tags:
+- vllm
+---
+
+```
+lm_eval --model vllm --model_args pretrained=/home/mgoin/code/nemotron-3-8b-chat-4k-sft-HF --tasks gsm8k --num_fewshot 5 --batch_size a
+uto
+
+vllm (pretrained=/home/mgoin/code/nemotron-3-8b-chat-4k-sft-HF), gen_kwargs: (None), limit: None, num_fewshot: 5, batch_size: auto
+|Tasks|Version|     Filter     |n-shot|  Metric   |   |Value |   |Stderr|
+|-----|------:|----------------|-----:|-----------|---|-----:|---|-----:|
+|gsm8k|      3|flexible-extract|     5|exact_match|↑  |0.1031|±  |0.0084|
+|     |       |strict-match    |     5|exact_match|↑  |0.1016|±  |0.0083|
+```