Project Overview:

The Challenging Bits:

The data preprocessing was intense. Making sure the data was clean and in the right format took some effort. If you're new to this, make sure you understand how to handle missing values and outliers.
Fine-tuning the model parameters was another tough part. It's a bit of an art to balance overfitting and underfitting. My tip? Start with a wide parameter search and then narrow it down.

What Was Easier:

Once the data was prepped, feeding it into the model was pretty simple. The frameworks are user-friendly, so that part felt more like connecting the dots.
Writing the code for model evaluation was also pretty straightforward. Most libraries provide functions to generate metrics so that part felt more like following a recipe.

Tips for the Code:

Take the time to really understand each line of code. It's tempting to just copy-paste from Stack Overflow or tutorials, but you'll learn more by typing it out and playing with it.

Don't ignore the error messages. They're actually really helpful once you learn how to decipher them. They can pinpoint exactly where and what your issue might be.

Looking at the Code:

The code is well-organized, which is great. The functions are modular, which means they do one thing and do it well. That makes debugging a whole lot easier. There's a mix of simple and complex functions, so if you're trying to understand the code, maybe start with the simpler utility functions and then work your way up to the model training and evaluation parts.

If you're just getting started with this codebase, I'd suggest running some small tests on each function to see what inputs they expect and what outputs they give.

Understanding these individual components will help you see the bigger picture of how the project works.

Machine Learning with Unity's ML-Agents:

If you're embarking on a journey to understand and use machine learning within Unity using the ML-Agents Toolkit, below you'll find a collection of resources to get you started:

Stack Overflow Resources

General ML-Agents Questions: Find community discussions and solutions related to ML-Agents at https://stackoverflow.com/questions/tagged/ml-agents.
Data Preprocessing Questions: Discover advice and strategies for data preprocessing in machine learning at https://stackoverflow.com/questions/tagged/data-preprocessing.
Machine Learning Tips: Search for insights on machine learning techniques, including parameter tuning and model evaluation, at https://stackoverflow.com/questions/tagged/machine-learning.

GitHub Resources

Unity ML-Agents Toolkit Repository: Access the official repository for the latest code, releases, and documentation at https://github.com/Unity-Technologies/ml-agents.
ML-Agents Wiki: Explore the wiki for tutorials, how-tos, and FAQs at https://github.com/Unity-Technologies/ml-agents/wiki.
Issue Tracker: Check if others have experienced similar issues and find solutions or workarounds at https://github.com/Unity-Technologies/ml-agents/issues.

ppo Agent playing SnowballTarget

This is a trained model of a ppo agent playing SnowballTarget using the Unity ML-Agents Library.

Usage (with ML-Agents)

The Documentation: https://unity-technologies.github.io/ml-agents/ML-Agents-Toolkit-Documentation/

We wrote a complete tutorial to learn to train your first agent using ML-Agents and publish it to the Hub:

A short tutorial where you teach Huggy the Dog 🐶 to fetch the stick and then play with him directly in your browser: https://huggingface.co/learn/deep-rl-course/unitbonus1/introduction
A longer tutorial to understand how works ML-Agents: https://huggingface.co/learn/deep-rl-course/unit5/introduction

Resume the training

mlagents-learn <your_configuration_file_path.yaml> --run-id=<run_id> --resume

Watch your Agent play

You can watch your agent playing directly in your browser

If the environment is part of ML-Agents official environments, go to https://huggingface.co/spaces/ThomasSimonini/ML-Agents-SnowballTarget
Step 1: Find your model_id: Feebo37/ppo-SnowballTarget
Step 2: Select your .nn /.onnx file
Click on Watch the agent play 👀