tanu20/ppo-lunarlander

PPO LunarLander Agent

This repository contains a PPO (Proximal Policy Optimization) agent implemented in PyTorch and trained on the LunarLander environment as part of the Hugging Face Deep Reinforcement Learning Course — Unit 8 Part I.

Algorithm

• PPO (Proximal Policy Optimization)
• Actor-Critic architecture
• Generalized Advantage Estimation (GAE)
• PPO clipped objective
• PyTorch

Environment

• Environment: LunarLander-v3
• Observation space: 8
• Action space: 4

Training Hyperparameters

• Learning rate: 2.5e-4
• Gamma: 0.99
• GAE lambda: 0.95
• PPO clip coefficient: 0.2
• Number of environments: 8
• Rollout steps: 128
• PPO epochs: 4
• Minibatch size: 256
• Entropy coefficient: 0.01
• Value function coefficient: 0.5
• Gradient clipping: 0.5

Evaluation

Evaluation was performed over 10 episodes.

Mean reward: -132.37 +/- 43.43

Files

• model.pt — trained PPO model
• results.json — evaluation results
• replay.mp4 — agent gameplay replay

Course

Hugging Face Deep Reinforcement Learning Course
Unit 8 Part I — PPO with PyTorch.