asiful2/ppo-LunarLander-v3

PPO Agent playing LunarLander-v3

This is a trained model of a PPO agent playing LunarLander-v3 using the stable-baselines3 library.

Trained as part of the Hugging Face Deep RL Course, Unit 1.

Results

mean_reward = 278.80 +/- 19.13 over 50 evaluation episodes with deterministic=True.

The task is considered solved at 200.

Usage

from huggingface_sb3 import load_from_hub
from stable_baselines3 import PPO
from stable_baselines3.common.evaluation import evaluate_policy
from stable_baselines3.common.monitor import Monitor
import gymnasium

checkpoint = load_from_hub("asiful2/ppo-LunarLander-v3", "ppo-LunarLander-v3-2M.zip")
model = PPO.load(checkpoint)

eval_env = Monitor(gymnasium.make("LunarLander-v3"))
mean_reward, std_reward = evaluate_policy(model, eval_env, n_eval_episodes=10, deterministic=True)
print(f"{mean_reward:.2f} +/- {std_reward:.2f}")

Training

Trained for 2,000,000 timesteps across 16 parallel environments, in two stages of 1M each.

Hyperparameter	Value
policy	MlpPolicy
n_steps	1024
batch_size	64
n_epochs	4
gamma	0.999
gae_lambda	0.98
ent_coef	0.01
n_envs	16

Both checkpoints are in the repo:

File	Timesteps	mean_reward
ppo-LunarLander-v3.zip	1M	260.59 +/- 18.91
ppo-LunarLander-v3-2M.zip	2M	278.80 +/- 19.13

Final explained_variance was 0.969, up from 0.81 at 1M steps.