huggsook/lunar-lander-v3-dqn

🚀 LunarLander-v3 Deep Q-Network (DQN) Agent

A Deep Q-Network (DQN) reinforcement learning agent trained to autonomously land the lunar module in the Gymnasium LunarLander-v3 environment.

📊 Performance & Evaluation

• Environment: LunarLander-v3
• Algorithm: Deep Q-Network (DQN) with Polyak Target Network Soft Update ($\tau=1e-3$) & Huber Loss
• Episodes to Solve: 288 episodes (Threshold: $\ge 200$ avg reward)
• Evaluation Reward: +282.25 (Clean landing on the landing pad between the flags)

🧠 Model Architecture

• State Space: 8 Continuous Dimensions ($x, y, v_x, v_y, \theta, \omega, leg_L, leg_R$)
• Action Space: 4 Discrete Actions (0: Idle, 1: Fire Left Engine, 2: Fire Main Engine, 3: Fire Right Engine)
• Neural Network: Multi-Layer Perceptron (MLP)
  • Linear(8, 128) -> ReLU
  • Linear(128, 128) -> ReLU
  • Linear(128, 4)

⚙️ Hyperparameters

{
  "gamma": 0.99,
  "learning_rate": 0.0005,
  "batch_size": 64,
  "buffer_size": 100000,
  "tau": 0.001,
  "epsilon_start": 1.0,
  "epsilon_end": 0.01,
  "epsilon_decay": 0.995,
  "loss_function": "SmoothL1Loss (Huber)"
}

🛠️ How to Use & Evaluate

import gymnasium as gym
import torch
import torch.nn as nn

# 1. Define Model
class QNetwork(nn.Module):
    def __init__(self, state_dim=8, action_dim=4):
        super().__init__()
        self.net = nn.Sequential(
            nn.Linear(state_dim, 128),
            nn.ReLU(),
            nn.Linear(128, 128),
            nn.ReLU(),
            nn.Linear(128, action_dim)
        )
    def forward(self, x):
        return self.net(x)

# 2. Load Weights
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model = QNetwork().to(device)
model.load_state_dict(torch.load("lunar_lander_dqn.pth", map_location=device))
model.eval()

# 3. Test Agent
env = gym.make("LunarLander-v3", render_mode="human")
state, _ = env.reset(seed=42)
total_reward = 0

for _ in range(1000):
    with torch.no_grad():
        s = torch.FloatTensor(state).unsqueeze(0).to(device)
        action = model(s).argmax().item()
    state, reward, terminated, truncated, _ = env.step(action)
    total_reward += reward
    if terminated or truncated:
        break

print(f"Test Score: {total_reward:.2f}")
env.close()