ibm-fms
/

llama3-8b-accelerator

Inference Endpoints

Model card Files Files and versions Community

JRosenkranz commited on Apr 29

Commit

510bfdb

•

1 Parent(s): 8ac38c5

Update README.md

Files changed (1) hide show

README.md +3 -3

README.md CHANGED Viewed

@@ -135,7 +135,7 @@ python fms-extras/scripts/paged_speculative_inference.py \
     --model_path=$MODEL_PATH \
     --model_source=hf \
     --tokenizer=$MODEL_PATH \
-    --speculator_path=ibm-fms/codellama-13b-accelerator \
     --speculator_source=hf \
     --speculator_variant=3_2b \
     --top_k_tokens_per_head=4,3,2,2 \
@@ -153,7 +153,7 @@ python fms-extras/scripts/paged_speculative_inference.py \
     --model_path=$MODEL_PATH \
     --model_source=hf \
     --tokenizer=$MODEL_PATH \
-    --speculator_path=ibm-fms/codellama-13b-accelerator \
     --speculator_source=hf \
     --speculator_variant=3_2b \
     --top_k_tokens_per_head=4,3,2,2 \
@@ -170,7 +170,7 @@ python fms-extras/scripts/paged_speculative_inference.py \
     --model_path=$MODEL_PATH \
     --model_source=hf \
     --tokenizer=$MODEL_PATH \
-    --speculator_path=ibm-fms/codellama-13b-accelerator \
     --speculator_source=hf \
     --speculator_variant=3_2b \
     --top_k_tokens_per_head=4,3,2,2 \

     --model_path=$MODEL_PATH \
     --model_source=hf \
     --tokenizer=$MODEL_PATH \
+    --speculator_path=ibm-fms/llama3-8b-accelerator \
     --speculator_source=hf \
     --speculator_variant=3_2b \
     --top_k_tokens_per_head=4,3,2,2 \
     --model_path=$MODEL_PATH \
     --model_source=hf \
     --tokenizer=$MODEL_PATH \
+    --speculator_path=ibm-fms/llama3-8b-accelerator \
     --speculator_source=hf \
     --speculator_variant=3_2b \
     --top_k_tokens_per_head=4,3,2,2 \
     --model_path=$MODEL_PATH \
     --model_source=hf \
     --tokenizer=$MODEL_PATH \
+    --speculator_path=ibm-fms/llama3-8b-accelerator \
     --speculator_source=hf \
     --speculator_variant=3_2b \
     --top_k_tokens_per_head=4,3,2,2 \