bestdive/ppo-Pyramids

PPO Pyramids

Trained from random initialization by Kay Zheng (bestdive), with AI coding and execution assistance, for the Hugging Face Deep RL course. No pretrained policy or fabricated evaluation metrics were used. Training used free Colab CPU, seed 42, ML-Agents 1.1.0, and 100096 environment steps.

This introductory policy has not solved the sparse-reward task; all evaluation returns are approximately -1. The course Unit 5 criterion requires submission of a genuinely trained model, without a minimum reward.

Independent evaluation

112 completed episodes, new environment seed 100002, deterministic ONNX actions. Mean reward -0.9999999310821295, population standard deviation 0.0. See evaluation.json for every episode return and evaluate_unity.py for the evaluation procedure. All agents terminating in the final vector batch were retained.

Reproduction

Python 3.10.12, mlagents==1.1.0, torch==2.2.2, onnx==1.15.0, setuptools<81, onnxruntime. Download the environment from the official course Unit 5 hands-on. Train with:

mlagents-learn training-config.yaml --env=PATH_TO_ENV --run-id=Kay-Pyramids --results-dir=results --no-graphics --seed=42 --torch-device=cpu

The configuration, training log, ONNX checkpoint, and independent evaluation are included. Model performance is limited to this simulated task.