Instructions to use hamishivi/OLMo-1B-0724-SFT-hf with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use hamishivi/OLMo-1B-0724-SFT-hf with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="hamishivi/OLMo-1B-0724-SFT-hf")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("hamishivi/OLMo-1B-0724-SFT-hf")
model = AutoModelForCausalLM.from_pretrained("hamishivi/OLMo-1B-0724-SFT-hf")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use hamishivi/OLMo-1B-0724-SFT-hf with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "hamishivi/OLMo-1B-0724-SFT-hf"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "hamishivi/OLMo-1B-0724-SFT-hf",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/hamishivi/OLMo-1B-0724-SFT-hf

SGLang

How to use hamishivi/OLMo-1B-0724-SFT-hf with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "hamishivi/OLMo-1B-0724-SFT-hf" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "hamishivi/OLMo-1B-0724-SFT-hf",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "hamishivi/OLMo-1B-0724-SFT-hf" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "hamishivi/OLMo-1B-0724-SFT-hf",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use hamishivi/OLMo-1B-0724-SFT-hf with Docker Model Runner:
```
docker model run hf.co/hamishivi/OLMo-1B-0724-SFT-hf
```

OLMo-1B-0724-SFT-hf / README.md

hamishivi

Update README.md

2f6fb76 verified almost 2 years ago

preview code

raw

history blame contribute delete

2.42 kB

metadata

license: apache-2.0
datasets:
  - allenai/dolma
  - allenai/tulu-v2-sft-mixture-olmo-4096
language:
  - en

OLMo-1B-0724 SFT

OLMo-1B-0724-hf finetuned for 5 epochs with a learning rate of 1e-5 on the Tulu 2 dataset - specifically this version. I used a batch size of 1, 128 grad accumulation steps. Linear warmup for the first 3% of training then linear decay to 0.

I've additionally released an 'instruct' version which has additionally gone through DPO training. This model is generally more performant (see the metrics below), so check it out!

Evals are as follows:

Metric	OLMo-1B-0724-hf	OLMo-1B-0724-SFT-hf (this model!)	OLMo-1B-0724-Instruct-hf
MMLU 0-shot	25.0	36.0	36.7
GSM8k CoT 8-shot	7.0	12.5	12.5
BBH CoT 3-shot	22.5	27.2	30.6
HumanEval P@10	16.0	21.2	22.0
AlpacaEval 1	-	41.5	50.9
AlpacaEval 2 LC	-	2.7	2.5
Toxigen % Toxic	80.3	59.7	14.1
TruthfulQA %Info+True	23.0	40.9	42.2
IFEval Loose Acc	20.5	26.1	24.2
XSTest F1	67.6	81.9	79.8
Average of above metrics	25.2	33.0	38.7

Model training and evaluation was performed using Open-instruct, so check that out for more details on evaluation.