bestdive/ppo-SnowballTarget

PPO SnowballTarget

Trained from random initialization by Kay Zheng (bestdive), with AI coding and execution assistance, for the Hugging Face Deep RL course. No pretrained policy or fabricated evaluation metrics were used. Training used free Colab CPU, seed 42, ML-Agents 1.1.0, and 100448 environment steps.

Training resumed from step 49936 after pinning PyTorch from 2.14.0 to 2.2.2 to resolve an ONNX export incompatibility. Final ONNX policy was independently evaluated successfully.

Independent evaluation

110 completed episodes, new environment seed 100001, deterministic ONNX actions. Mean reward 12.49090909090909, population standard deviation 5.403274183384224. See evaluation.json for every episode return and evaluate_unity.py for the evaluation procedure. All agents terminating in the final vector batch were retained.

Reproduction

Python 3.10.12, mlagents==1.1.0, torch==2.2.2, onnx==1.15.0, setuptools<81, onnxruntime. Download the environment from the official course Unit 5 hands-on. Train with:

mlagents-learn training-config.yaml --env=PATH_TO_ENV --run-id=Kay-SnowballTarget --results-dir=results --no-graphics --seed=42 --torch-device=cpu

The configuration, training log, ONNX checkpoint, and independent evaluation are included. Model performance is limited to this simulated task.