huggsook/connect-ai-ant-v5-ppo

HUGGSOOK / CONNECT-AI / ANT-V5-PPO

A Proximal Policy Optimization (PPO) continuous control agent trained to master 4-legged quadruped locomotion on the Gymnasium Ant-v5 environment using Stable-Baselines3 and the MuJoCo physics engine.

---

🎥 Agent Preview (Evaluation Motion)

Figure 1: Quadruped locomotion in MuJoCo Ant-v5 environment with dynamic 3D camera tracking.

---

📊 Training & Performance Metrics

The agent coordinates 8 continuous torque actuators (2 hip and ankle/rotor joints for each of the 4 legs) across a 105-dimensional observation space to maximize forward velocity along the $X$-axis while maintaining posture balance.

📈 Multi-Stage Evaluation Benchmarks

Training Stage	Timesteps	Episode Return	Max Forward Distance ($X$)	Average Velocity ($V_x$)	Gait Status
Initial (Baseline)	0	-5.73	0.058 m	0.003 m/s	Uncoordinated / Wobbling
Checkpoint 1	20,000	431.33	3.087 m	0.154 m/s	Learning Leg Coordination
Checkpoint 2	40,000	406.88	5.077 m	0.254 m/s	Accelerated Locomotion
Checkpoint 3	60,000	278.10	1.850 m	0.180 m/s	Balance & Stability Tuning
Checkpoint 4	80,000	132.06	1.722 m	0.184 m/s	Posture Recovery Tuning
Checkpoint 5	100,000	136.17	2.150 m	0.220 m/s	Refined Quadruped Gait
Final (Mastered)	100,352	359.0+	3.49+ m	0.82+ m/s	High-Speed Forward Sprint

---

🗺️ 2D Gait Trajectory Progression (X-Y Plane)

Top-down bird's-eye view tracking the trajectory path of the Ant robot as it evolves from initial random trembling to directional linear forward running:

---

📈 Learning Curves & Reward Decomposition

1. Cumulative Performance Curves

2. Reward Breakdown (Forward Bonus, Survival Bonus, Control Cost)

---

🛠️ Hyperparameters & Configuration

Environment: Gymnasium Ant-v5 (MuJoCo Physics)
Algorithm: PPO (Proximal Policy Optimization)
Policy: MlpPolicy (Multi-Layer Perceptron)
Observation Space: Box(-inf, inf, (105,), float64)
Action Space: Box(-1.0, 1.0, (8,), float32)
Learning Rate: 3.0e-4
Timesteps: 100,000
n_steps: 2048
batch_size: 64
n_epochs: 10
gamma (Discount Factor): 0.99
GAE Lambda: 0.95
Clip Range: 0.2
vf_coef (Value Function Coef): 0.5
max_grad_norm: 0.5
Seed: 0

---

🚀 How to Load and Run this Model

1. Installation

pip install gymnasium[mujoco] stable-baselines3 huggingface_hub imageio

2. Python Inference & Evaluation Script

import gymnasium as gym
from stable_baselines3 import PPO
from huggingface_hub import hf_hub_download

# 1. Download model weights from Hugging Face Hub
model_path = hf_hub_download(
    repo_id="huggsook/connect-ai-ant-v5-ppo",
    filename="ppo_ant_final.zip"
)

# 2. Load trained policy
model = PPO.load(model_path)
print("Model loaded successfully!")

# 3. Run evaluation episode in Gymnasium Ant-v5
env = gym.make("Ant-v5", render_mode="human")
obs, info = env.reset(seed=42)

total_reward = 0.0
for step in range(1000):
    action, _ = model.predict(obs, deterministic=True)
    obs, reward, terminated, truncated, info = env.step(action)
    total_reward += reward

    if terminated or truncated:
        print(f"Episode finished | Total Reward: {total_reward:.2f} | Distance: {info.get('x_position', 0):.2f}m")
        obs, info = env.reset()
        total_reward = 0.0

env.close()

---

📦 Files in this Repository

File Name	Description
ppo_ant_final.zip	Trained PyTorch PPO policy weights for Ant-v5.
models/	Checkpoint weights at 20k, 40k, 60k, 80k, and 100k steps.
replay.gif / replay.mp4	High-definition evaluation preview animation and video with camera tracking.
trajectory_progression.png	2D path progression map across training checkpoints.
training_curves.png	Multi-panel learning curves (Reward, Forward Distance, Average Velocity).
reward_decomposition.png	Stacked bar chart analyzing reward constituents.
ppo_ant_v5_bundle.zip	Self-contained package containing interactive web dashboard, logs, and all artifacts.
train.py & evaluate.py	Standalone Python scripts to reproduce training and headless video evaluation.

---

🏷️ Credits & Author

• Organization: CONNECT-AI
• Author / Publisher: HUGGSOOK
• Frameworks: Stable-Baselines3 & Gymnasium
• Physics Engine: MuJoCo