simoswish/MAS_MultiDroneExploration

Multi-Drone Cooperative Search with Random Faults — trained policy

Trained weights for the Multi-Agent Systems course project (MSc Artificial Intelligence, University of Bologna, A.Y. 2025/26).

A team of N drones searches a 32×32 occupancy grid for one hidden target under partial observability, range-limited communication and a fixed time budget, while any drone can permanently break at any moment, with no central authority to announce the loss. A single Dueling Double DQN with parameter sharing drives every drone; per-episode domain randomization plus a compact context vector make that one policy a generalist across team size, sensing, communication range, obstacle density and fault rate.

• Code (GitHub mirror): https://github.com/simoswish02/Multi-Agent-Systems-Project
• Code (course repository, GitLab): https://dvcs.apice.unibo.it/pika-lab/courses/ai-ethics/projects/rimondi2526-mas
• Report (49 pp.): Rimondi-MultiDroneSearch-MAS.pdf
• Video walkthrough (~4 min): https://youtu.be/8dnRG6ZneP8

Files

File	Description
mas_50k_dr_faults_ep50000.pt	Final checkpoint of the 50,000-episode run. Every number in Section 7 of the report comes from this file. This is the one to download for normal use.
checkpoints/mas_50k_dr_faults_ep*.pt	The 20 periodic snapshots taken every 2,500 episodes, including a copy of the final one. Needed only to reproduce the learning curve (--axis epoch), which re-evaluates each of them; ~5.4 GB in total.

Usage

git clone https://github.com/simoswish02/Multi-Agent-Systems-Project
cd Multi-Agent-Systems-Project
conda env create -f environment.yaml && conda activate rl-drone

# download the checkpoint into checkpoints/
hf download simoswish/MAS_MultiDroneExploration \
   mas_50k_dr_faults_ep50000.pt --local-dir checkpoints/

# watch it fly
python main.py --mode eval --checkpoint checkpoints/mas_50k_dr_faults_ep50000.pt \
               --fault-prob 0.003

# reproduce the experiments of Section 7
python testing/evaluate_policy.py --axis all --seeds 500 \
       --checkpoint checkpoints/mas_50k_dr_faults_ep50000.pt

Loading it directly:

import torch
ckpt = torch.load("mas_50k_dr_faults_ep50000.pt", map_location="cpu")
state_dict = ckpt["q_net"]          # 22.4M parameters
# the file also carries optimizer / scheduler / epsilon state, so training
# can be resumed with `python main.py --mode train --resume <file>`

Model

Architecture (agents/networks.py, report Section 4.4):

• Global trunk over six 32×32 belief planes (Visited, Obstacle, Trajectory, Target, Own_Position, Broken): a ConvNeXt-style CNN, stages of width 96 → 192 → 384 at resolutions 32 → 16 → 8, with per-stage FiLM conditioning, followed by a 3-layer / 8-head Transformer over the 8×8 = 64 spatial tokens plus a CLS token and a context token.
• Local trunk over a 13×13 egocentric crop with five channels: three stride-1 ConvNeXt blocks at width 96 (the middle one dilated) → 384.
• Fusion and head: concat (1024 + 384 + 5) → FC 1536 → 768 → 384 → dueling head, Q = V + A − mean(A), over 4 actions (up / down / left / right).
• 22,445,183 parameters, float32.

Context vector (CTX_DIM = 5), rebuilt every round for every drone: [vision_radius, comm_range, n_agents, agent_id, n_alive_belief], each normalized by the domain-randomization maxima (6, 12, 4, 4, 4). agent_id breaks the parameter-sharing symmetry so co-located drones split up from step 0; n_alive_belief is that drone's own belief about how many teammates remain — decentralized, and possibly stale.

Training

Parameter-sharing Dueling Double DQN: one shared network trained on the pooled experience of the whole team, in a single phase of 50,000 episodes. Adam, weight decay 1e-6, gradient clip 1.0, γ = 0.97, 3-step returns, Huber loss, uniform replay of 2·10⁵ transitions, batch 16, one gradient step every 4 agent-turns, hard target sync every 500 steps. ε decays 1.0 → 0.05 (×0.9995/episode, floor at episode 5,990); the learning rate warms up over the first 10% of episodes to 5·10⁻⁴, then anneals cosine-wise to 10⁻⁶.

Per-episode domain randomization: vision radius ∈ [1, 6], communication range ∈ [2, 12], team size ∈ [1, 4], obstacle density ∈ [0.10, 0.30], per-turn fault probability ∈ [0, 0.003]. The grid size (32) is the one constant.

Results

At the reference operating point (N = 3, r_vis = 3, r_comm = 5, ρ = 0.2, T = 200, no faults), 500 seeded instances, greedy (ε = 0):

Success rate	97.6% (Wilson CI [95.9, 98.6])
Map coverage	53.4%
SPL	0.715
Time to find	53.9 rounds

Generalization (success %, endpoints of each sweep, 500 seeds per point):

Axis	In-distribution	OOD probe
Team size N	79.0 (N=1) → 97.6 (N=4)	98.8, 99.0 (N = 5, 6)
Vision r_vis	80.2 (r=1) → 96.6 (r=6)	96.4, 95.6 (r = 7, 8)
Comm. r_comm	96.8 (r=2) → 98.0 (r=12)	95.6, 98.2, 98.8 (r = 0, 14, 16)
Density ρ	99.0 (ρ=0.10) → 83.6 (ρ=0.30)	99.2, 80.0, 86.0 (ρ = 0, 0.35, 0.40)

No axis shows a cliff, in or out of distribution, including beyond the trained maxima of team size, vision and communication. Obstacle density is the one factor that genuinely governs difficulty, costing 15 points across its trained range alone.

Fault robustness — the project's headline result:

p_fault	0	0.001	0.002	0.003	0.005	0.010
Success [%]	97.6	96.0	93.6	91.0	85.4	72.6
SPL	0.715	0.695	0.682	0.666	0.631	0.536
Drones lost (mean)	0.00	0.16	0.30	0.48	0.78	1.37

Across the entire trained fault range (0 → 0.003) success falls by under seven points while the team sheds, on average, close to half a drone; at more than triple the trained maximum it still completes nearly three missions in four, with no cliff. Conditioning on survivors instead: a team that finishes intact succeeds 98.7% of the time, losing one drone costs six points (92.5%), and even a lone survivor still finds the target in 65.9% of episodes.

Limitations

• Simulation only: 4-connected grid moves, noiseless sensing, instantaneous range-limited fusion. No sim-to-real transfer is claimed or tested.
• Faults are i.i.d. across drones and turns and produce inert, non-blocking wrecks. Correlated failures, passage-blocking wrecks and an adversary choosing whom to disable are outside the model.
• Above ρ ≈ 0.30 the density sweep stops measuring the same task. The target is always drawn from the spawn's connected component, and at high clutter the free space fragments until that component holds under a quarter of the free cells, so the apparent rebound at ρ = 0.40 is a structurally shorter mission rather than a better policy (Section 7.3 of the report).
• Statistical, not formal, evidence: confidence intervals bound sampling error, not worst-case behaviour on an unseen instance.

Section 8 of the report discusses the dual-use profile of the capability, its treatment under the GDPR and the EU AI Act, and why this specific system would be unacceptable as an autonomous weapon.

Citation

@misc{rimondi2026multidrone,
  author = {Rimondi, Simone},
  title  = {Multi-Drone Cooperative Search with Random Faults},
  year   = {2026},
  note   = {Multi-Agent Systems project, MSc Artificial Intelligence,
            University of Bologna},
  url    = {https://github.com/simoswish02/Multi-Agent-Systems-Project}
}

License

MIT.