Shad0wKillar/ppo-LunarLander-v3

PPO Agent for LunarLander-v3

A Proximal Policy Optimization (PPO) agent trained with Stable-Baselines3 for Unit 1 of the Hugging Face Deep Reinforcement Learning Course.

The agent learns to land safely by choosing among four discrete actions using an eight-value observation vector.

Video replay

Watch or download the replay.

Training details

The uploaded checkpoint records a target of 1,000,000 timesteps and an actual count of 1,015,808 timesteps.

Stable-Baselines3 completes full rollouts before stopping, so the final count can exceed the target. With 16 environments and 1,024 steps per environment, each rollout contains 16,384 transitions.

Timesteps count interactions with the environment. Optimization epochs are separate: this model uses four passes over each collected rollout.

Setting	Saved value
Algorithm	PPO
Policy	MlpPolicy
Environment	LunarLander-v3
Requested training timesteps	1,000,000
Saved training timesteps	1,015,808
Parallel environments	16
Steps per environment per rollout	1,024
Batch size	64
Optimization epochs per rollout	4
Learning rate	0.0003
Discount factor (gamma)	0.999
GAE lambda	0.98
Entropy coefficient	0.01
PPO clip range	0.2
Random seed	Not set

These settings come from the saved checkpoint and config.json.

Evaluation results

Metric	Value
Mean episode reward	264.65
Standard deviation	21.34
Evaluation episodes	10
Action selection	Deterministic

Results are self-reported and stored in results.json.

Using the displayed model-card values, the course score is:

Mean reward − standard deviation = 264.65 − 21.34 = 243.31

This exceeds Unit 1's stated threshold of 200. It does not independently confirm leaderboard registration or certification status.

Scores may vary across evaluations because environment resets are stochastic. These results represent a sample of 10 episodes.

Installation

The saved training environment used Stable-Baselines3 2.9.0 and Gymnasium 1.3.0.

python -m pip install swig
python -m pip install "stable-baselines3==2.9.0" "gymnasium[box2d]==1.3.0" huggingface_hub

In Colab, prefix these commands with !, or use %pip install.

Load and evaluate the trained model

This downloads the saved checkpoint and evaluates it without retraining.

import gymnasium as gym
from huggingface_hub import hf_hub_download
from stable_baselines3 import PPO
from stable_baselines3.common.evaluation import evaluate_policy
from stable_baselines3.common.monitor import Monitor

checkpoint = hf_hub_download(
    repo_id="Shad0wKillar/ppo-LunarLander-v3",
    filename="ppo-LunarLander-v3.zip",
)

model = PPO.load(checkpoint, device="cpu")
print(f"Saved training timesteps: {model.num_timesteps:,}")

env = Monitor(gym.make("LunarLander-v3"))

try:
    mean_reward, std_reward = evaluate_policy(
        model,
        env,
        n_eval_episodes=10,
        deterministic=True,
    )
    print(f"Mean reward: {mean_reward:.2f} +/- {std_reward:.2f}")
finally:
    env.close()

Watch the agent locally

After loading model above, run this on a computer with a graphical display. In Colab, use the replay video or record with render_mode="rgb_array" and the course's virtual-display setup.

env = gym.make("LunarLander-v3", render_mode="human")

try:
    observation, info = env.reset()

    for _ in range(1000):
        action, _ = model.predict(observation, deterministic=True)
        observation, reward, terminated, truncated, info = env.step(
            int(action.item())
        )

        if terminated or truncated:
            break
finally:
    env.close()

Repository contents

File	Purpose
ppo-LunarLander-v3.zip	Loadable SB3 checkpoint, including model parameters, optimizer state, and training metadata
ppo-LunarLander-v3/	Extracted checkpoint contents
config.json	Saved model configuration
results.json	Evaluation metrics and settings
replay.mp4	Agent demonstration video

The recorded software environment is available in system_info.txt.

References

• Hugging Face Deep RL Course — Unit 1
• Stable-Baselines3 PPO documentation
• Gymnasium Lunar Lander documentation