This variant of the model has undergone reinforcement learning (RL) fine-tuning and is based on teknium/OpenHermes-2.5-Mistral-7B. The fine-tuning process utilized a preference dataset derived from HuggingFace's no robots dataset, incorporating Differential Privacy Optimization (DPO) techniques.

Downloads last month
49
Safetensors
Model size
7B params
Tensor type
F32
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for jae24/openhermes_dpo_norobot_0201

Quantizations
1 model