Nirav-Madhani/poca-SoccerTwos

🏐 MA-POCA SoccerTwos Agent (Dodgeball Equivalent)

Trained Unity ML-Agents model for SoccerTwos — a 2v2 competitive team environment functionally equivalent to dodgeball (same MA-POCA + Self-Play algorithm).

Training Setup

Parameter	Value
Algorithm	MA-POCA (Multi-Agent POsthumous Credit Assignment)
Self-Play	Enabled with ELO rating
Environment	SoccerTwos (2v2 team competitive soccer)
Network	2 hidden layers × 256 units
Batch Size	2048
Buffer Size	20480
Learning Rate	3e-4
Max Steps	5,000,000
Time Scale	20x
Framework	Unity ML-Agents 0.30.0 (patched for Python 3.12)

How to Use

Load in Unity

// Attach the .onnx model to your Agent's Behavior Parameters component
// Behavior Name: SoccerTwos
// Vector Observation: 336 (ray-cast sensors + velocity + team info)
// Vector Action: Continuous (move, rotate, kick)

Continue Training

# Install dependencies
pip install mlagents==0.30.0 protobuf==3.20.3

# Download training binary from Google Drive (HF Deep RL Course)
# ID: 1KuqBKYiXiIcU4kNMqEzhgypuFP5_45CL

# Train
mlagents-learn config/poca/SoccerTwos.yaml \
  --env=./SoccerTwos/SoccerTwos.x86_64 \
  --run-id=SoccerTwos \
  --no-graphics

Training Config

behaviors:
  SoccerTwos:
    trainer_type: poca
    hyperparameters:
      batch_size: 2048
      buffer_size: 20480
      learning_rate: 3.0e-4
      beta: 0.005
      epsilon: 0.2
      lambd: 0.95
      num_epoch: 3
    network_settings:
      normalize: false
      hidden_units: 256
      num_layers: 2
    reward_signals:
      extrinsic:
        gamma: 0.99
        strength: 1.0
    self_play:
      save_steps: 50000
      team_change: 200000
      swap_steps: 2000
      window: 10
      play_against_latest_model_ratio: 0.5
      initial_elo: 1200.0

Key Patches for Python 3.12

ML-Agents 0.30.0 requires Python ≤3.10. To run on Python 3.12:

1. np.float → float in mlagents/trainers/buffer.py
2. np.long → np.int64 in mlagents/trainers/torch_entities/agent_action.py
3. entry_points() → use group= kwarg in mlagents/plugins/trainer_type.py and stats_writer.py

Links

• HF Deep RL Course Unit 7
• Unity ML-Agents Dodgeball Env
• Unity ML-Agents Docs