theSOL1/llama-3.1-8b-oracle-rm-helpsteer2-helpfulness

Reward model trained for the helpfulness preference on HH-RLHF helpful-base.

Model

• Type: oracle
• Base checkpoint: meta-llama/Llama-3.1-8B
• Label type: hard
• Architecture: LlamaForSequenceClassification
• Tokenizer: LlamaTokenizer
• Transformers compatibility: transformers>=4,<5

Evaluation

Validation metrics are computed on the temporary eval split created by the training sweep.

metric	value
Raw validation accuracy	0.566879
Eval loss	0.689909
Eval margin	0.008876
Mean reward	-0.414209

For proxy models, the validation labels are produced by the corresponding oracle reward model.

Loading

from transformers import AutoModelForSequenceClassification, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("theSOL1/llama-3.1-8b-oracle-rm-helpsteer2-helpfulness")
model = AutoModelForSequenceClassification.from_pretrained("theSOL1/llama-3.1-8b-oracle-rm-helpsteer2-helpfulness", num_labels=1)

This repository intentionally contains only model, tokenizer, and configuration files. Optimizer states, scheduler states, checkpoints, and trainer state files are not uploaded.