How to use from
vLLM
Install from pip and serve model
# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Locutusque/Orca-2-13b-SFT_v5"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Locutusque/Orca-2-13b-SFT_v5",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'
Use Docker
docker model run hf.co/Locutusque/Orca-2-13b-SFT_v5
Quick Links

The "microsoft/Orca-2-13b" model fully fine-tuned on HuggingFaceH4/no_robots, totally-not-an-llm/EverythingLM-data-V3, mlabonne/guanaco-llama2-1k, OpenAssistant/oasst_top1_2023-08-25, and garage-bAInd/Open-Platypus. This model achieved a test loss of 0.38 on garage-bAInd/Open-Platypus.

Make sure to comply with the microsoft research license. Please read it before using this model.

This model was trained on the ChatML prompt template.

The responses seen in the inference API were generated using the following sampling parameters:

temperature = 0.1

top_p = 0.14

top_k = 41

repetition_penalty = 1.176

Downloads last month
25
Safetensors
Model size
13B params
Tensor type
BF16
·
Inference Examples
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for Locutusque/Orca-2-13b-SFT_v5

Finetuned
(5)
this model
Quantizations
4 models

Datasets used to train Locutusque/Orca-2-13b-SFT_v5