this is a DPO fine-tuned MoE model with 60B parameter.

DPO Trainer
TRL supports the DPO Trainer for training language models from preference data, as described in the paper Direct Preference Optimization: Your Language Model is Secretly a Reward Model by Rafailov et al., 2023.

GGUF format is ready at cloudyu/Phoenix_DPO_60B_gguf

Downloads last month: 457

Safetensors

Model size

60.8B params

Tensor type

BF16

Inference Examples

Text Generation

This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Model tree for cloudyu/Phoenix_DPO_60B

Quantizations

2 models