Trainer
Source: trl

# TrainerAt TRL we support PPO (Proximal Policy Optimisation) with an implementation that largely follows  the structure introduced in the paper "Fine-Tuning Language Models from Human Preferences" by D. Ziegler et al. [[paper](https://huggingface.co/papers/1909.08593), [code](https://github.com/openai/lm-human-preferences)].The Trainer and model classes are largely inspired from `transformers.Trainer` and `transformers.AutoModel` classes and adapted for RL.We also support a `RewardTrainer` that can be used to train a reward model.## CPOConfig[[autodoc]] CPOConfig## CPOTrainer[[autodoc]] CPOTrainer## DDPOConfig[[autodoc]] DDPOConfig## DDPOTrainer[[autodoc]] DDPOTrainer## DPOTrainer[[autodoc]] DPOTrainer## IterativeSFTTrainer[[autodoc]] IterativeSFTTrainer## KTOConfig[[autodoc]] KTOConfig## KTOTrainer[[autodoc]] KTOTrainer## ORPOConfig[[autodoc]] ORPOConfig## ORPOTrainer[[autodoc]] ORPOTrainer## PPOConfig[[autodoc]] PPOConfig## PPOTrainer[[autodoc]] PPOTrainer## RewardConfig[[autodoc]] RewardConfig## RewardTrainer[[autodoc]] RewardTrainer## SFTTrainer[[autodoc]] SFTTrainer## set_seed[[autodoc]] set_seed