ChatGPT's Reinforcement Learning with Human Feedback(RLHF) has 3 steps: Supervised fine-tuning, Rewording model training and Reinforcement Learning fine-tuning. The first two steps have humans involved. The last step uses Proximal Policy Optimization (PPO) algorithm in reinforcement learning.